Vibe Code Bench
Vibe Code Bench, a component of the Vals Index.
No filters. Showing 38 of 38 rows.
1[1–4] AnthropicDossier | high | OpenHands | Fable 5.2, Accuracy: 93.8% (scale 0.0 to 100.0), 95% confidence interval 91.5 to 96.093.8%(91.5–96.0) | 2,437.3 s | $22.13 | Runs unstated | Cost kind unknown |
2[1–7] AnthropicDossierTied #1 | max | OpenHands | Sonnet 5.5, Accuracy: 92.4% (scale 0.0 to 100.0), 95% confidence interval 89.9 to 94.992.4%(89.9–94.9) | 3,672.1 s | $19.73 | Runs unstated | Cost kind unknown |
3[1–9] GoogleDossierTied #1 | high | OpenHands | Gemini 4 Argon, Accuracy: 91.9% (scale 0.0 to 100.0), 95% confidence interval 88.2 to 95.691.9%(88.2–95.6) | 2,247.7 s | $8.11 | Runs unstated | Cost kind unknown |
4[2–11] AnthropicDossierTied #1 | max | OpenHands | Haiku 5.5, Accuracy: 90.4% (scale 0.0 to 100.0), 95% confidence interval 87.1 to 93.890.4%(87.1–93.8) | 2,075.7 s | $6.07 | Runs unstated | Cost kind unknown |
5[2–12] AnthropicDossierTied #1 | max | OpenHands | Opus 5.5, Accuracy: 90.3% (scale 0.0 to 100.0), 95% confidence interval 87.3 to 93.390.3%(87.3–93.3) | 5,766.5 s | $35.77 | Runs unstated | Cost kind unknown |
6[2–15] OpenAIDossierTied #1 | max | OpenHands | GPT-6 Astra, Accuracy: 89.6% (scale 0.0 to 100.0), 95% confidence interval 85.3 to 93.989.6%(85.3–93.9) | 2,344.1 s | $38.51 | Runs unstated | Cost kind unknown |
7[1–17] GoogleDossierTied #1 | high | OpenHands | Gemini 4 Carbon, Accuracy: 89.1% (scale 0.0 to 100.0), 95% confidence interval 84.6 to 93.689.1%(84.6–93.6) | 2,172 s | $8.25 | Runs unstated | Cost kind unknown |
8[2–15] OpenAIDossier | max | OpenHands | GPT-6.1 Sol, Accuracy: 88.9% (scale 0.0 to 100.0), 95% confidence interval 85.2 to 92.688.9%(85.2–92.6) | 4,489.8 s | $6.23 | Runs unstated | Cost kind unknown |
9[3–19] GoogleDossier | xhigh | OpenHands | Gemini 4 Hydrogen, Accuracy: 87.8% (scale 0.0 to 100.0), 95% confidence interval 83.3 to 92.387.8%(83.3–92.3) | 864.6 s | $4.55 | Runs unstated | Cost kind unknown |
10[2–21] DeepSeekDossier | max | OpenHands | DeepSeek V4.1 Flash 1017, Accuracy: 87.7% (scale 0.0 to 100.0), 95% confidence interval 82.3 to 93.187.7%(82.3–93.1) | 973.8 s | $0.58 | Runs unstated | Cost kind unknown |
11[4–20] AlibabaDossier | xhigh | OpenHands | Qwen4 Max, Accuracy: 86.7% (scale 0.0 to 100.0), 95% confidence interval 82.2 to 91.186.7%(82.2–91.1) | 4,172.5 s | $41.40 | Runs unstated | Cost kind unknown |
12[6–21] xAIDossier | xhigh | OpenHands | Grok 4.7, Accuracy: 86.2% (scale 0.0 to 100.0), 95% confidence interval 81.9 to 90.486.2%(81.9–90.4) | 2,147.1 s | $15.83 | Runs unstated | Cost kind unknown |
13[7–21] MetaDossier | max | OpenHands | Muse Wonder Full 1.0, Accuracy: 86% (scale 0.0 to 100.0), 95% confidence interval 82.2 to 89.886%(82.2–89.8) | 648.3 s | $2.29 | Runs unstated | Cost kind unknown |
14[4–25] XiaomiDossier | No effort set | OpenHands | MiMo-V2.6-Pro, Accuracy: 85.2% (scale 0.0 to 100.0), 95% confidence interval 78.6 to 91.985.2%(78.6–91.9) | 3,646.6 s | $1.04 | Runs unstated | Cost kind unknown |
15[7–26] Moonshot AIDossier | max | OpenHands | Kimi K3, Accuracy: 85% (scale 0.0 to 100.0), 95% confidence interval 79.6 to 90.485%(79.6–90.4) | 999 s | $10.01 | Runs unstated | Cost kind unknown |
16[6–25] Moonshot AIDossier | max | OpenHands | Kimi K3.1 Pro, Accuracy: 84.9% (scale 0.0 to 100.0), 95% confidence interval 79.1 to 90.784.9%(79.1–90.7) | 3,028.9 s | $7.49 | Runs unstated | Cost kind unknown |
17[7–27] OpenAIDossier | max | OpenHands | GPT-6.1 Terra, Accuracy: 84% (scale 0.0 to 100.0), 95% confidence interval 78.4 to 89.684%(78.4–89.6) | 2,250 s | $6.00 | Runs unstated | Cost kind unknown |
18[8–27] MetaDossier | max | OpenHands | Muse Ember, Accuracy: 83.7% (scale 0.0 to 100.0), 95% confidence interval 78.3 to 89.183.7%(78.3–89.1) | 167.9 s | $0.054 | Runs unstated | Cost kind unknown |
19[10–26] MetaDossier | max | OpenHands | Muse Wonder 1.0, Accuracy: 83.1% (scale 0.0 to 100.0), 95% confidence interval 78.2 to 88.083.1%(78.2–88.0) | 350.1 s | $1.80 | Runs unstated | Cost kind unknown |
20[9–28] DeepSeekDossier | max | OpenHands | DeepSeek V4 Pro 0813, Accuracy: 82.3% (scale 0.0 to 100.0), 95% confidence interval 76.1 to 88.582.3%(76.1–88.5) | 3,982.6 s | $0.36 | Runs unstated | Cost kind unknown |
21[9–29] OpenAIDossier | max | OpenHands | GPT-6 Luna, Accuracy: 81.6% (scale 0.0 to 100.0), 95% confidence interval 75.0 to 88.381.6%(75.0–88.3) | 2,219.5 s | $1.35 | Runs unstated | Cost kind unknown |
22[10–29] MistralDossier | high | OpenHands | Mistral Large 4 OSS, Accuracy: 81.6% (scale 0.0 to 100.0), 95% confidence interval 75.1 to 88.081.6%(75.1–88.0) | 3,012.6 s | $11.72 | Runs unstated | Cost kind unknown |
23[10–30] Thinking MachinesDossier | No effort set | OpenHands | Inkling 2 Small, Accuracy: 81.5% (scale 0.0 to 100.0), 95% confidence interval 74.8 to 88.381.5%(74.8–88.3) | 1,537.4 s | $1.90 | Runs unstated | Cost kind unknown |
24[9–31] TencentDossier | No effort set | OpenHands | Hy4, Accuracy: 81% (scale 0.0 to 100.0), 95% confidence interval 74.1 to 88.081%(74.1–88.0) | 2,281.5 s | $2.39 | Runs unstated | Cost kind unknown |
25[11–32] XiaomiDossier | No effort set | OpenHands | MiMo-V2.6-Flash, Accuracy: 79% (scale 0.0 to 100.0), 95% confidence interval 71.0 to 86.979%(71.0–86.9) | 3,013 s | $0.56 | Runs unstated | Cost kind unknown |
26[14–32] GoogleDossier | high | OpenHands | Gemini 3.8 Flash, Accuracy: 78.7% (scale 0.0 to 100.0), 95% confidence interval 71.0 to 86.378.7%(71.0–86.3) | 519.1 s | $6.87 | Runs unstated | Cost kind unknown |
27[15–32] Z.aiDossier | max | OpenHands | GLM-5.3, Accuracy: 78.1% (scale 0.0 to 100.0), 95% confidence interval 70.4 to 85.878.1%(70.4–85.8) | 3,847.2 s | $12.45 | Runs unstated | Cost kind unknown |
28[16–32] TencentDossier | No effort set | OpenHands | Hy4 Preview, Accuracy: 77.5% (scale 0.0 to 100.0), 95% confidence interval 69.6 to 85.477.5%(69.6–85.4) | 2,493.1 s | $2.18 | Runs unstated | Cost kind unknown |
29[17–32] AlibabaDossier | No effort set | OpenHands | Qwen3.8 Max Internal, Accuracy: 77.1% (scale 0.0 to 100.0), 95% confidence interval 69.8 to 84.477.1%(69.8–84.4) | 5,079.4 s | $11.67 | Runs unstated | Cost kind unknown |
30[19–32] MetaDossier | xhigh | OpenHands | Muse Spark Open, Accuracy: 76.2% (scale 0.0 to 100.0), 95% confidence interval 69.7 to 82.776.2%(69.7–82.7) | 1,869.2 s | $1.66 | Runs unstated | Cost kind unknown |
31[21–33] MiniMaxDossier | max | OpenHands | MiniMax M3.1 Flash, Accuracy: 73.9% (scale 0.0 to 100.0), 95% confidence interval 65.6 to 82.373.9%(65.6–82.3) | 1,263.9 s | $1.15 | Runs unstated | Cost kind unknown |
32[27–34] StepFunDossier | No effort set | OpenHands | Step 5 Preview, Accuracy: 70.4% (scale 0.0 to 100.0), 95% confidence interval 62.0 to 78.870.4%(62.0–78.8) | 5,466.4 s | $5.86 | Runs unstated | Cost kind unknown |
33[29–34] ReflectionDossier | No effort set | OpenHands | Beam 1.0, Accuracy: 66.4% (scale 0.0 to 100.0), 95% confidence interval 58.5 to 74.466.4%(58.5–74.4) | 145.3 s | Not tested | Runs unstated | Cost kind unknown |
34[31–34] InceptionDossier | high | OpenHands | Mercury 3, Accuracy: 60.9% (scale 0.0 to 100.0), 95% confidence interval 50.4 to 71.560.9%(50.4–71.5) | 566.6 s | $0.63 | Runs unstated | Cost kind unknown |
35[35–36] Z.aiDossier | max | OpenHands | GLM-5.3 Flash, Accuracy: 30.8% (scale 0.0 to 100.0), 95% confidence interval 20.4 to 41.130.8%(20.4–41.1) | 5,027.3 s | $4.70 | Runs unstated | Cost kind unknown |
36[35–37] Thinking MachinesDossier | No effort set | OpenHands | Inkling, Accuracy: 19.2% (scale 0.0 to 100.0), 95% confidence interval 12.0 to 26.419.2%(12.0–26.4) | 1,105.5 s | $1.61 | Runs unstated | Cost kind unknown |
37[35–37] Thinking MachinesDossier | No effort set | OpenHands | Inkling Small, Accuracy: 19.1% (scale 0.0 to 100.0), 95% confidence interval 12.4 to 25.719.1%(12.4–25.7) | 2,949.5 s | $0.43 | Runs unstated | Cost kind unknown |
38 NVIDIADossier | high | OpenHands | Nemotron 3.5 Super VL, Accuracy: 3.7% (scale 0.0 to 100.0), 95% confidence interval 1.3 to 10.03.7%(1.3–10.0) | 262.9 s | $0.54 | Runs unstated | Cost kind unknown |
- 1AnthropicDossier
high · OpenHands
Accuracy
Fable 5.2, Accuracy: 93.8% (scale 0.0 to 100.0), 95% confidence interval 91.5 to 96.093.8%(91.5–96.0)- Duration per test
- 2,437.3 s
- Cost per test
- $22.13
Accuracy
Sonnet 5.5, Accuracy: 92.4% (scale 0.0 to 100.0), 95% confidence interval 89.9 to 94.992.4%(89.9–94.9)- Duration per test
- 3,672.1 s
- Cost per test
- $19.73
Accuracy
Gemini 4 Argon, Accuracy: 91.9% (scale 0.0 to 100.0), 95% confidence interval 88.2 to 95.691.9%(88.2–95.6)- Duration per test
- 2,247.7 s
- Cost per test
- $8.11
Accuracy
Haiku 5.5, Accuracy: 90.4% (scale 0.0 to 100.0), 95% confidence interval 87.1 to 93.890.4%(87.1–93.8)- Duration per test
- 2,075.7 s
- Cost per test
- $6.07
Accuracy
Opus 5.5, Accuracy: 90.3% (scale 0.0 to 100.0), 95% confidence interval 87.3 to 93.390.3%(87.3–93.3)- Duration per test
- 5,766.5 s
- Cost per test
- $35.77
Accuracy
GPT-6 Astra, Accuracy: 89.6% (scale 0.0 to 100.0), 95% confidence interval 85.3 to 93.989.6%(85.3–93.9)- Duration per test
- 2,344.1 s
- Cost per test
- $38.51
Accuracy
Gemini 4 Carbon, Accuracy: 89.1% (scale 0.0 to 100.0), 95% confidence interval 84.6 to 93.689.1%(84.6–93.6)- Duration per test
- 2,172 s
- Cost per test
- $8.25
- 8OpenAIDossier
max · OpenHands
Accuracy
GPT-6.1 Sol, Accuracy: 88.9% (scale 0.0 to 100.0), 95% confidence interval 85.2 to 92.688.9%(85.2–92.6)- Duration per test
- 4,489.8 s
- Cost per test
- $6.23
- 9GoogleDossier
xhigh · OpenHands
Accuracy
Gemini 4 Hydrogen, Accuracy: 87.8% (scale 0.0 to 100.0), 95% confidence interval 83.3 to 92.387.8%(83.3–92.3)- Duration per test
- 864.6 s
- Cost per test
- $4.55
- 10DeepSeekDossier
max · OpenHands
Accuracy
DeepSeek V4.1 Flash 1017, Accuracy: 87.7% (scale 0.0 to 100.0), 95% confidence interval 82.3 to 93.187.7%(82.3–93.1)- Duration per test
- 973.8 s
- Cost per test
- $0.58
- 11AlibabaDossier
xhigh · OpenHands
Accuracy
Qwen4 Max, Accuracy: 86.7% (scale 0.0 to 100.0), 95% confidence interval 82.2 to 91.186.7%(82.2–91.1)- Duration per test
- 4,172.5 s
- Cost per test
- $41.40
- 12xAIDossier
xhigh · OpenHands
Accuracy
Grok 4.7, Accuracy: 86.2% (scale 0.0 to 100.0), 95% confidence interval 81.9 to 90.486.2%(81.9–90.4)- Duration per test
- 2,147.1 s
- Cost per test
- $15.83
- 13MetaDossier
max · OpenHands
Accuracy
Muse Wonder Full 1.0, Accuracy: 86% (scale 0.0 to 100.0), 95% confidence interval 82.2 to 89.886%(82.2–89.8)- Duration per test
- 648.3 s
- Cost per test
- $2.29
- 14XiaomiDossier
OpenHands
Accuracy
MiMo-V2.6-Pro, Accuracy: 85.2% (scale 0.0 to 100.0), 95% confidence interval 78.6 to 91.985.2%(78.6–91.9)- Duration per test
- 3,646.6 s
- Cost per test
- $1.04
- 15Moonshot AIDossier
max · OpenHands
Accuracy
Kimi K3, Accuracy: 85% (scale 0.0 to 100.0), 95% confidence interval 79.6 to 90.485%(79.6–90.4)- Duration per test
- 999 s
- Cost per test
- $10.01
- 16Moonshot AIDossier
max · OpenHands
Accuracy
Kimi K3.1 Pro, Accuracy: 84.9% (scale 0.0 to 100.0), 95% confidence interval 79.1 to 90.784.9%(79.1–90.7)- Duration per test
- 3,028.9 s
- Cost per test
- $7.49
- 17OpenAIDossier
max · OpenHands
Accuracy
GPT-6.1 Terra, Accuracy: 84% (scale 0.0 to 100.0), 95% confidence interval 78.4 to 89.684%(78.4–89.6)- Duration per test
- 2,250 s
- Cost per test
- $6.00
- 18MetaDossier
max · OpenHands
Accuracy
Muse Ember, Accuracy: 83.7% (scale 0.0 to 100.0), 95% confidence interval 78.3 to 89.183.7%(78.3–89.1)- Duration per test
- 167.9 s
- Cost per test
- $0.054
- 19MetaDossier
max · OpenHands
Accuracy
Muse Wonder 1.0, Accuracy: 83.1% (scale 0.0 to 100.0), 95% confidence interval 78.2 to 88.083.1%(78.2–88.0)- Duration per test
- 350.1 s
- Cost per test
- $1.80
- 20DeepSeekDossier
max · OpenHands
Accuracy
DeepSeek V4 Pro 0813, Accuracy: 82.3% (scale 0.0 to 100.0), 95% confidence interval 76.1 to 88.582.3%(76.1–88.5)- Duration per test
- 3,982.6 s
- Cost per test
- $0.36
- 21OpenAIDossier
max · OpenHands
Accuracy
GPT-6 Luna, Accuracy: 81.6% (scale 0.0 to 100.0), 95% confidence interval 75.0 to 88.381.6%(75.0–88.3)- Duration per test
- 2,219.5 s
- Cost per test
- $1.35
- 22MistralDossier
high · OpenHands
Accuracy
Mistral Large 4 OSS, Accuracy: 81.6% (scale 0.0 to 100.0), 95% confidence interval 75.1 to 88.081.6%(75.1–88.0)- Duration per test
- 3,012.6 s
- Cost per test
- $11.72
- 23Thinking MachinesDossier
OpenHands
Accuracy
Inkling 2 Small, Accuracy: 81.5% (scale 0.0 to 100.0), 95% confidence interval 74.8 to 88.381.5%(74.8–88.3)- Duration per test
- 1,537.4 s
- Cost per test
- $1.90
- 24TencentDossier
OpenHands
Accuracy
Hy4, Accuracy: 81% (scale 0.0 to 100.0), 95% confidence interval 74.1 to 88.081%(74.1–88.0)- Duration per test
- 2,281.5 s
- Cost per test
- $2.39
- 25XiaomiDossier
OpenHands
Accuracy
MiMo-V2.6-Flash, Accuracy: 79% (scale 0.0 to 100.0), 95% confidence interval 71.0 to 86.979%(71.0–86.9)- Duration per test
- 3,013 s
- Cost per test
- $0.56
- 26GoogleDossier
high · OpenHands
Accuracy
Gemini 3.8 Flash, Accuracy: 78.7% (scale 0.0 to 100.0), 95% confidence interval 71.0 to 86.378.7%(71.0–86.3)- Duration per test
- 519.1 s
- Cost per test
- $6.87
- 27Z.aiDossier
max · OpenHands
Accuracy
GLM-5.3, Accuracy: 78.1% (scale 0.0 to 100.0), 95% confidence interval 70.4 to 85.878.1%(70.4–85.8)- Duration per test
- 3,847.2 s
- Cost per test
- $12.45
- 28TencentDossier
OpenHands
Accuracy
Hy4 Preview, Accuracy: 77.5% (scale 0.0 to 100.0), 95% confidence interval 69.6 to 85.477.5%(69.6–85.4)- Duration per test
- 2,493.1 s
- Cost per test
- $2.18
- 29AlibabaDossier
OpenHands
Accuracy
Qwen3.8 Max Internal, Accuracy: 77.1% (scale 0.0 to 100.0), 95% confidence interval 69.8 to 84.477.1%(69.8–84.4)- Duration per test
- 5,079.4 s
- Cost per test
- $11.67
- 30MetaDossier
xhigh · OpenHands
Accuracy
Muse Spark Open, Accuracy: 76.2% (scale 0.0 to 100.0), 95% confidence interval 69.7 to 82.776.2%(69.7–82.7)- Duration per test
- 1,869.2 s
- Cost per test
- $1.66
- 31MiniMaxDossier
max · OpenHands
Accuracy
MiniMax M3.1 Flash, Accuracy: 73.9% (scale 0.0 to 100.0), 95% confidence interval 65.6 to 82.373.9%(65.6–82.3)- Duration per test
- 1,263.9 s
- Cost per test
- $1.15
- 32StepFunDossier
OpenHands
Accuracy
Step 5 Preview, Accuracy: 70.4% (scale 0.0 to 100.0), 95% confidence interval 62.0 to 78.870.4%(62.0–78.8)- Duration per test
- 5,466.4 s
- Cost per test
- $5.86
- 33ReflectionDossier
OpenHands
Accuracy
Beam 1.0, Accuracy: 66.4% (scale 0.0 to 100.0), 95% confidence interval 58.5 to 74.466.4%(58.5–74.4)- Duration per test
- 145.3 s
- Cost per test
- Not tested
- 34InceptionDossier
high · OpenHands
Accuracy
Mercury 3, Accuracy: 60.9% (scale 0.0 to 100.0), 95% confidence interval 50.4 to 71.560.9%(50.4–71.5)- Duration per test
- 566.6 s
- Cost per test
- $0.63
- 35Z.aiDossier
max · OpenHands
Accuracy
GLM-5.3 Flash, Accuracy: 30.8% (scale 0.0 to 100.0), 95% confidence interval 20.4 to 41.130.8%(20.4–41.1)- Duration per test
- 5,027.3 s
- Cost per test
- $4.70
- 36Thinking MachinesDossier
OpenHands
Accuracy
Inkling, Accuracy: 19.2% (scale 0.0 to 100.0), 95% confidence interval 12.0 to 26.419.2%(12.0–26.4)- Duration per test
- 1,105.5 s
- Cost per test
- $1.61
- 37Thinking MachinesDossier
OpenHands
Accuracy
Inkling Small, Accuracy: 19.1% (scale 0.0 to 100.0), 95% confidence interval 12.4 to 25.719.1%(12.4–25.7)- Duration per test
- 2,949.5 s
- Cost per test
- $0.43
- 38NVIDIADossier
high · OpenHands
Accuracy
Nemotron 3.5 Super VL, Accuracy: 3.7% (scale 0.0 to 100.0), 95% confidence interval 1.3 to 10.03.7%(1.3–10.0)- Duration per test
- 262.9 s
- Cost per test
- $0.54