Legal Agent Benchmark (HLAB-Vals)
Legal Agent Benchmark (HLAB-Vals), a component of the Vals Index.
No filters. Showing 38 of 38 rows.
1[1–7] GoogleDossier | high | No harness | Gemini 4 Argon, Accuracy: 27.1% (scale 0.0 to 100.0), 95% confidence interval 19.8 to 34.427.1%(19.8–34.4) | 1,755.4 s | $10.55 | Runs unstated | Cost kind unknown |
2[1–9] MetaDossierTied #1 | max | No harness | Muse Ember, Accuracy: 25% (scale 0.0 to 100.0), 95% confidence interval 17.8 to 32.225%(17.8–32.2) | 353.6 s | $0.12 | Runs unstated | Cost kind unknown |
2[1–9] MetaDossierTied #1 | xhigh | No harness | Muse Spark Open, Accuracy: 25% (scale 0.0 to 100.0), 95% confidence interval 17.9 to 32.125%(17.9–32.1) | 14,355.7 s | $13.63 | Runs unstated | Cost kind unknown |
4[1–10] MetaDossierTied #1 | max | No harness | Muse Wonder 1.0, Accuracy: 23.8% (scale 0.0 to 100.0), 95% confidence interval 16.6 to 30.923.8%(16.6–30.9) | 906.1 s | $7.08 | Runs unstated | Cost kind unknown |
5[1–10] MetaDossierTied #1 | max | No harness | Muse Wonder Full 1.0, Accuracy: 23.3% (scale 0.0 to 100.0), 95% confidence interval 16.4 to 30.323.3%(16.4–30.3) | 2,649.4 s | $5.41 | Runs unstated | Cost kind unknown |
6[2–12] Thinking MachinesDossierTied #1 | No effort set | No harness | Inkling 2 Small, Accuracy: 21.7% (scale 0.0 to 100.0), 95% confidence interval 15.1 to 28.221.7%(15.1–28.2) | 1,290.9 s | $1.23 | Runs unstated | Cost kind unknown |
7[2–13] AlibabaDossierTied #1 | xhigh | No harness | Qwen4 Max, Accuracy: 21.3% (scale 0.0 to 100.0), 95% confidence interval 14.7 to 27.821.3%(14.7–27.8) | 4,454.1 s | $12.87 | Runs unstated | Cost kind unknown |
8[3–18] xAIDossierTied #1 | xhigh | No harness | Grok 4.7, Accuracy: 18.3% (scale 0.0 to 100.0), 95% confidence interval 12.1 to 24.618.3%(12.1–24.6) | 2,148.4 s | $10.75 | Runs unstated | Cost kind unknown |
9[5–19] GoogleDossier | high | No harness | Gemini 4 Carbon, Accuracy: 17.5% (scale 0.0 to 100.0), 95% confidence interval 11.6 to 23.417.5%(11.6–23.4) | 1,720.8 s | $6.52 | Runs unstated | Cost kind unknown |
10[4–23] MistralDossier | high | No harness | Mistral Large 4 OSS, Accuracy: 15.8% (scale 0.0 to 100.0), 95% confidence interval 9.7 to 22.015.8%(9.7–22.0) | 688.7 s | $0.73 | Runs unstated | Cost kind unknown |
11[6–23] AlibabaDossier | No effort set | No harness | Qwen3.8 Max Internal, Accuracy: 15.4% (scale 0.0 to 100.0), 95% confidence interval 9.4 to 21.415.4%(9.4–21.4) | 2,950.6 s | $3.36 | Runs unstated | Cost kind unknown |
12[7–25] Z.aiDossier | max | No harness | GLM-5.3, Accuracy: 14.6% (scale 0.0 to 100.0), 95% confidence interval 9.0 to 20.214.6%(9.0–20.2) | 1,911 s | $3.99 | Runs unstated | Cost kind unknown |
13[8–25] DeepSeekDossier | max | No harness | DeepSeek V4.1 Flash 1017, Accuracy: 14.2% (scale 0.0 to 100.0), 95% confidence interval 8.7 to 19.614.2%(8.7–19.6) | 1,328.1 s | $0.61 | Runs unstated | Cost kind unknown |
14[7–25] XiaomiDossier | No effort set | No harness | MiMo-V2.6-Pro, Accuracy: 13.8% (scale 0.0 to 100.0), 95% confidence interval 8.2 to 19.313.8%(8.2–19.3) | 1,861.8 s | $0.22 | Runs unstated | Cost kind unknown |
15[8–27] XiaomiDossier | No effort set | No harness | MiMo-V2.6-Flash, Accuracy: 13.3% (scale 0.0 to 100.0), 95% confidence interval 7.8 to 18.913.3%(7.8–18.9) | 1,342.6 s | $0.086 | Runs unstated | Cost kind unknown |
16[8–27] Moonshot AIDossier | max | No harness | Kimi K3, Accuracy: 12.9% (scale 0.0 to 100.0), 95% confidence interval 7.4 to 18.412.9%(7.4–18.4) | 2,556.3 s | $4.14 | Runs unstated | Cost kind unknown |
16[9–26] StepFunDossier | No effort set | No harness | Step 5 Preview, Accuracy: 12.9% (scale 0.0 to 100.0), 95% confidence interval 7.4 to 18.412.9%(7.4–18.4) | 993.2 s | $1.13 | Runs unstated | Cost kind unknown |
18[10–26] TencentDossier | No effort set | No harness | Hy4, Accuracy: 12.5% (scale 0.0 to 100.0), 95% confidence interval 7.3 to 17.712.5%(7.3–17.7) | 1,167.2 s | $1.04 | Runs unstated | Cost kind unknown |
19[9–26] GoogleDossier | high | No harness | Gemini 3.8 Flash, Accuracy: 12.1% (scale 0.0 to 100.0), 95% confidence interval 7.0 to 17.112.1%(7.0–17.1) | 1,161.7 s | $3.66 | Runs unstated | Cost kind unknown |
20[9–28] Moonshot AIDossier | max | No harness | Kimi K3.1 Pro, Accuracy: 11.7% (scale 0.0 to 100.0), 95% confidence interval 6.5 to 16.811.7%(6.5–16.8) | 4,373.4 s | $9.00 | Runs unstated | Cost kind unknown |
21[12–30] GoogleDossier | xhigh | No harness | Gemini 4 Hydrogen, Accuracy: 11.3% (scale 0.0 to 100.0), 95% confidence interval 6.0 to 16.511.3%(6.0–16.5) | 742.8 s | $3.28 | Runs unstated | Cost kind unknown |
22[12–31] DeepSeekDossier | max | No harness | DeepSeek V4 Pro 0813, Accuracy: 10.8% (scale 0.0 to 100.0), 95% confidence interval 5.5 to 16.210.8%(5.5–16.2) | 655.4 s | $0.80 | Runs unstated | Cost kind unknown |
23[11–30] Z.aiDossier | max | No harness | GLM-5.3 Flash, Accuracy: 10.4% (scale 0.0 to 100.0), 95% confidence interval 5.4 to 15.510.4%(5.4–15.5) | 744.8 s | $0.32 | Runs unstated | Cost kind unknown |
23[13–30] AnthropicDossier | max | No harness | Sonnet 5.5, Accuracy: 10.4% (scale 0.0 to 100.0), 95% confidence interval 5.4 to 15.510.4%(5.4–15.5) | 4,398 s | $17.71 | Runs unstated | Cost kind unknown |
25[16–32] AnthropicDossier | max | No harness | Opus 5.5, Accuracy: 9.2% (scale 0.0 to 100.0), 95% confidence interval 4.5 to 13.89.2%(4.5–13.8) | 3,507.4 s | $21.81 | Runs unstated | Cost kind unknown |
26[17–32] TencentDossier | No effort set | No harness | Hy4 Preview, Accuracy: 8.8% (scale 0.0 to 100.0), 95% confidence interval 4.5 to 13.08.8%(4.5–13.0) | 1,989.3 s | $0.98 | Runs unstated | Cost kind unknown |
27[19–34] MiniMaxDossier | max | No harness | MiniMax M3.1 Flash, Accuracy: 7.5% (scale 0.0 to 100.0), 95% confidence interval 3.1 to 11.97.5%(3.1–11.9) | 1,497.6 s | $0.47 | Runs unstated | Cost kind unknown |
28[20–35] OpenAIDossier | max | No harness | GPT-6 Astra, Accuracy: 7.1% (scale 0.0 to 100.0), 95% confidence interval 3.4 to 10.87.1%(3.4–10.8) | 1,953.4 s | $25.46 | Runs unstated | Cost kind unknown |
29[22–36] OpenAIDossier | max | No harness | GPT-6.1 Sol, Accuracy: 6.3% (scale 0.0 to 100.0), 95% confidence interval 2.7 to 9.86.3%(2.7–9.8) | 2,425.6 s | $3.54 | Runs unstated | Cost kind unknown |
30[25–37] AnthropicDossier | high | No harness | Fable 5.2, Accuracy: 5% (scale 0.0 to 100.0), 95% confidence interval 1.6 to 8.45%(1.6–8.4) | 1,954.2 s | $19.37 | Runs unstated | Cost kind unknown |
30[24–38] InceptionDossier | high | No harness | Mercury 3, Accuracy: 5% (scale 0.0 to 100.0), 95% confidence interval 1.8 to 8.25%(1.8–8.2) | 886.9 s | $0.19 | Runs unstated | Cost kind unknown |
32[25–38] ReflectionDossier | No effort set | No harness | Beam 1.0, Accuracy: 4.6% (scale 0.0 to 100.0), 95% confidence interval 1.1 to 8.14.6%(1.1–8.1) | 116.9 s | Not tested | Runs unstated | Cost kind unknown |
33[27–38] OpenAIDossier | max | No harness | GPT-6 Luna, Accuracy: 4.2% (scale 0.0 to 100.0), 95% confidence interval 1.0 to 7.44.2%(1.0–7.4) | 1,305.5 s | $0.30 | Runs unstated | Cost kind unknown |
33[28–38] AnthropicDossier | max | No harness | Haiku 5.5, Accuracy: 4.2% (scale 0.0 to 100.0), 95% confidence interval 1.2 to 7.14.2%(1.2–7.1) | 2,620.8 s | $4.26 | Runs unstated | Cost kind unknown |
35[28–38] Thinking MachinesDossier | No effort set | No harness | Inkling, Accuracy: 3.8% (scale 0.0 to 100.0), 95% confidence interval 0.9 to 6.63.8%(0.9–6.6) | 529.6 s | $1.25 | Runs unstated | Cost kind unknown |
35[29–38] NVIDIADossier | high | No harness | Nemotron 3.5 Super VL, Accuracy: 3.8% (scale 0.0 to 100.0), 95% confidence interval 1.1 to 6.43.8%(1.1–6.4) | 172.1 s | $0.37 | Runs unstated | Cost kind unknown |
37[29–38] Thinking MachinesDossier | No effort set | No harness | Inkling Small, Accuracy: 2.9% (scale 0.0 to 100.0), 95% confidence interval 0.2 to 5.62.9%(0.2–5.6) | 455.3 s | $0.18 | Runs unstated | Cost kind unknown |
38[33–38] OpenAIDossier | max | No harness | GPT-6.1 Terra, Accuracy: 1.7% (scale 0.0 to 100.0), 95% confidence interval 0.5 to 5.41.7%(0.5–5.4) | 821.5 s | $1.48 | Runs unstated | Cost kind unknown |
- 1GoogleDossier
high
Accuracy
Gemini 4 Argon, Accuracy: 27.1% (scale 0.0 to 100.0), 95% confidence interval 19.8 to 34.427.1%(19.8–34.4)- Duration per test
- 1,755.4 s
- Cost per test
- $10.55
Accuracy
Muse Ember, Accuracy: 25% (scale 0.0 to 100.0), 95% confidence interval 17.8 to 32.225%(17.8–32.2)- Duration per test
- 353.6 s
- Cost per test
- $0.12
Accuracy
Muse Spark Open, Accuracy: 25% (scale 0.0 to 100.0), 95% confidence interval 17.9 to 32.125%(17.9–32.1)- Duration per test
- 14,355.7 s
- Cost per test
- $13.63
Accuracy
Muse Wonder 1.0, Accuracy: 23.8% (scale 0.0 to 100.0), 95% confidence interval 16.6 to 30.923.8%(16.6–30.9)- Duration per test
- 906.1 s
- Cost per test
- $7.08
Accuracy
Muse Wonder Full 1.0, Accuracy: 23.3% (scale 0.0 to 100.0), 95% confidence interval 16.4 to 30.323.3%(16.4–30.3)- Duration per test
- 2,649.4 s
- Cost per test
- $5.41
Accuracy
Inkling 2 Small, Accuracy: 21.7% (scale 0.0 to 100.0), 95% confidence interval 15.1 to 28.221.7%(15.1–28.2)- Duration per test
- 1,290.9 s
- Cost per test
- $1.23
Accuracy
Qwen4 Max, Accuracy: 21.3% (scale 0.0 to 100.0), 95% confidence interval 14.7 to 27.821.3%(14.7–27.8)- Duration per test
- 4,454.1 s
- Cost per test
- $12.87
Accuracy
Grok 4.7, Accuracy: 18.3% (scale 0.0 to 100.0), 95% confidence interval 12.1 to 24.618.3%(12.1–24.6)- Duration per test
- 2,148.4 s
- Cost per test
- $10.75
- 9GoogleDossier
high
Accuracy
Gemini 4 Carbon, Accuracy: 17.5% (scale 0.0 to 100.0), 95% confidence interval 11.6 to 23.417.5%(11.6–23.4)- Duration per test
- 1,720.8 s
- Cost per test
- $6.52
- 10MistralDossier
high
Accuracy
Mistral Large 4 OSS, Accuracy: 15.8% (scale 0.0 to 100.0), 95% confidence interval 9.7 to 22.015.8%(9.7–22.0)- Duration per test
- 688.7 s
- Cost per test
- $0.73
- 11AlibabaDossier
Accuracy
Qwen3.8 Max Internal, Accuracy: 15.4% (scale 0.0 to 100.0), 95% confidence interval 9.4 to 21.415.4%(9.4–21.4)- Duration per test
- 2,950.6 s
- Cost per test
- $3.36
- 12Z.aiDossier
max
Accuracy
GLM-5.3, Accuracy: 14.6% (scale 0.0 to 100.0), 95% confidence interval 9.0 to 20.214.6%(9.0–20.2)- Duration per test
- 1,911 s
- Cost per test
- $3.99
- 13DeepSeekDossier
max
Accuracy
DeepSeek V4.1 Flash 1017, Accuracy: 14.2% (scale 0.0 to 100.0), 95% confidence interval 8.7 to 19.614.2%(8.7–19.6)- Duration per test
- 1,328.1 s
- Cost per test
- $0.61
- 14XiaomiDossier
Accuracy
MiMo-V2.6-Pro, Accuracy: 13.8% (scale 0.0 to 100.0), 95% confidence interval 8.2 to 19.313.8%(8.2–19.3)- Duration per test
- 1,861.8 s
- Cost per test
- $0.22
- 15XiaomiDossier
Accuracy
MiMo-V2.6-Flash, Accuracy: 13.3% (scale 0.0 to 100.0), 95% confidence interval 7.8 to 18.913.3%(7.8–18.9)- Duration per test
- 1,342.6 s
- Cost per test
- $0.086
- 16Moonshot AIDossier
max
Accuracy
Kimi K3, Accuracy: 12.9% (scale 0.0 to 100.0), 95% confidence interval 7.4 to 18.412.9%(7.4–18.4)- Duration per test
- 2,556.3 s
- Cost per test
- $4.14
- 16StepFunDossier
Accuracy
Step 5 Preview, Accuracy: 12.9% (scale 0.0 to 100.0), 95% confidence interval 7.4 to 18.412.9%(7.4–18.4)- Duration per test
- 993.2 s
- Cost per test
- $1.13
- 18TencentDossier
Accuracy
Hy4, Accuracy: 12.5% (scale 0.0 to 100.0), 95% confidence interval 7.3 to 17.712.5%(7.3–17.7)- Duration per test
- 1,167.2 s
- Cost per test
- $1.04
- 19GoogleDossier
high
Accuracy
Gemini 3.8 Flash, Accuracy: 12.1% (scale 0.0 to 100.0), 95% confidence interval 7.0 to 17.112.1%(7.0–17.1)- Duration per test
- 1,161.7 s
- Cost per test
- $3.66
- 20Moonshot AIDossier
max
Accuracy
Kimi K3.1 Pro, Accuracy: 11.7% (scale 0.0 to 100.0), 95% confidence interval 6.5 to 16.811.7%(6.5–16.8)- Duration per test
- 4,373.4 s
- Cost per test
- $9.00
- 21GoogleDossier
xhigh
Accuracy
Gemini 4 Hydrogen, Accuracy: 11.3% (scale 0.0 to 100.0), 95% confidence interval 6.0 to 16.511.3%(6.0–16.5)- Duration per test
- 742.8 s
- Cost per test
- $3.28
- 22DeepSeekDossier
max
Accuracy
DeepSeek V4 Pro 0813, Accuracy: 10.8% (scale 0.0 to 100.0), 95% confidence interval 5.5 to 16.210.8%(5.5–16.2)- Duration per test
- 655.4 s
- Cost per test
- $0.80
- 23Z.aiDossier
max
Accuracy
GLM-5.3 Flash, Accuracy: 10.4% (scale 0.0 to 100.0), 95% confidence interval 5.4 to 15.510.4%(5.4–15.5)- Duration per test
- 744.8 s
- Cost per test
- $0.32
- 23AnthropicDossier
max
Accuracy
Sonnet 5.5, Accuracy: 10.4% (scale 0.0 to 100.0), 95% confidence interval 5.4 to 15.510.4%(5.4–15.5)- Duration per test
- 4,398 s
- Cost per test
- $17.71
- 25AnthropicDossier
max
Accuracy
Opus 5.5, Accuracy: 9.2% (scale 0.0 to 100.0), 95% confidence interval 4.5 to 13.89.2%(4.5–13.8)- Duration per test
- 3,507.4 s
- Cost per test
- $21.81
- 26TencentDossier
Accuracy
Hy4 Preview, Accuracy: 8.8% (scale 0.0 to 100.0), 95% confidence interval 4.5 to 13.08.8%(4.5–13.0)- Duration per test
- 1,989.3 s
- Cost per test
- $0.98
- 27MiniMaxDossier
max
Accuracy
MiniMax M3.1 Flash, Accuracy: 7.5% (scale 0.0 to 100.0), 95% confidence interval 3.1 to 11.97.5%(3.1–11.9)- Duration per test
- 1,497.6 s
- Cost per test
- $0.47
- 28OpenAIDossier
max
Accuracy
GPT-6 Astra, Accuracy: 7.1% (scale 0.0 to 100.0), 95% confidence interval 3.4 to 10.87.1%(3.4–10.8)- Duration per test
- 1,953.4 s
- Cost per test
- $25.46
- 29OpenAIDossier
max
Accuracy
GPT-6.1 Sol, Accuracy: 6.3% (scale 0.0 to 100.0), 95% confidence interval 2.7 to 9.86.3%(2.7–9.8)- Duration per test
- 2,425.6 s
- Cost per test
- $3.54
- 30AnthropicDossier
high
Accuracy
Fable 5.2, Accuracy: 5% (scale 0.0 to 100.0), 95% confidence interval 1.6 to 8.45%(1.6–8.4)- Duration per test
- 1,954.2 s
- Cost per test
- $19.37
- 30InceptionDossier
high
Accuracy
Mercury 3, Accuracy: 5% (scale 0.0 to 100.0), 95% confidence interval 1.8 to 8.25%(1.8–8.2)- Duration per test
- 886.9 s
- Cost per test
- $0.19
- 32ReflectionDossier
Accuracy
Beam 1.0, Accuracy: 4.6% (scale 0.0 to 100.0), 95% confidence interval 1.1 to 8.14.6%(1.1–8.1)- Duration per test
- 116.9 s
- Cost per test
- Not tested
- 33OpenAIDossier
max
Accuracy
GPT-6 Luna, Accuracy: 4.2% (scale 0.0 to 100.0), 95% confidence interval 1.0 to 7.44.2%(1.0–7.4)- Duration per test
- 1,305.5 s
- Cost per test
- $0.30
- 33AnthropicDossier
max
Accuracy
Haiku 5.5, Accuracy: 4.2% (scale 0.0 to 100.0), 95% confidence interval 1.2 to 7.14.2%(1.2–7.1)- Duration per test
- 2,620.8 s
- Cost per test
- $4.26
- 35Thinking MachinesDossier
Accuracy
Inkling, Accuracy: 3.8% (scale 0.0 to 100.0), 95% confidence interval 0.9 to 6.63.8%(0.9–6.6)- Duration per test
- 529.6 s
- Cost per test
- $1.25
- 35NVIDIADossier
high
Accuracy
Nemotron 3.5 Super VL, Accuracy: 3.8% (scale 0.0 to 100.0), 95% confidence interval 1.1 to 6.43.8%(1.1–6.4)- Duration per test
- 172.1 s
- Cost per test
- $0.37
- 37Thinking MachinesDossier
Accuracy
Inkling Small, Accuracy: 2.9% (scale 0.0 to 100.0), 95% confidence interval 0.2 to 5.62.9%(0.2–5.6)- Duration per test
- 455.3 s
- Cost per test
- $0.18
- 38OpenAIDossier
max
Accuracy
GPT-6.1 Terra, Accuracy: 1.7% (scale 0.0 to 100.0), 95% confidence interval 0.5 to 5.41.7%(0.5–5.4)- Duration per test
- 821.5 s
- Cost per test
- $1.48