Models ranked
262
Paired-comparison Bradley-Terry standings across shared boards, the Arena preference gap, domain fingerprints, and Bug Hunt team coverage.
262
58
Percentile IQR spread >= 25 pts
0.85
104 models paired
| # | Model | Latent score (0–100) | Median pct | IQR spread | Agreement class |
|---|---|---|---|---|---|
| 1 | Gemini 4 Carbon 18 boards across 4 indep. sources | : 100.0 (scale 0.0 to 100.0), 95% confidence interval 93.4 to 100.0100.0(93.4–100.0) | 98th | 9.5 | consistent |
| 2 | Fable 5.2 15 boards across 4 indep. sources | : 95.3 (scale 0.0 to 100.0), 95% confidence interval 83.7 to 100.095.3(83.7–100.0) | 99th | 4.0 | consistent |
| 3 | Gemini 4 Argon 19 boards across 4 indep. sources | : 92.8 (scale 0.0 to 100.0), 95% confidence interval 88.1 to 95.592.8(88.1–95.5) | 92th | 13.9 | consistent |
| 4 | Opus 5.5 23 boards across 4 indep. sources | : 92.4 (scale 0.0 to 100.0), 95% confidence interval 84.8 to 99.092.4(84.8–99.0) | 92th | 10.8 | consistent |
| 5 | Sonnet 5.5 22 boards across 4 indep. sources | : 91.6 (scale 0.0 to 100.0), 95% confidence interval 83.7 to 96.891.6(83.7–96.8) | 92th | 12.7 | consistent |
| 6 | Gemini 4 Hydrogen 15 boards across 3 indep. sources | : 91.1 (scale 0.0 to 100.0), 95% confidence interval 84.1 to 97.391.1(84.1–97.3) | 89th | 17.6 | consistent |
| 7 | Muse Wonder Full 1.0 12 boards across 3 indep. sources | : 90.9 (scale 0.0 to 100.0), 95% confidence interval 85.0 to 94.690.9(85.0–94.6) | 84th | 7.8 | consistent |
| 8 | Fable 5.1 16 boards across 3 indep. sources | : 89.8 (scale 0.0 to 100.0), 95% confidence interval 81.4 to 94.289.8(81.4–94.2) | 92th | 10.3 | consistent |
| 9 | Muse Wonder 1.0 16 boards across 4 indep. sources | : 85.0 (scale 0.0 to 100.0), 95% confidence interval 78.7 to 88.385.0(78.7–88.3) | 76th | 19.0 | consistent |
| 10 | Fable 5 15 boards across 3 indep. sources | : 84.2 (scale 0.0 to 100.0), 95% confidence interval 76.6 to 90.884.2(76.6–90.8) | 92th | 17.7 | consistent |
| 11 | GPT-5.5 Pre-release 3 boards across 1 indep. sources | : 83.7 (scale 0.0 to 100.0), 95% confidence interval 72.0 to 89.783.7(72.0–89.7) | 97th | 2.6 | consistent |
| 12 | GPT-6 Astra 25 boards across 4 indep. sources | : 83.5 (scale 0.0 to 100.0), 95% confidence interval 74.5 to 90.283.5(74.5–90.2) | 86th | 18.2 | consistent |
| 13 | GPT-5.6 Sol 16 boards across 3 indep. sources | : 83.5 (scale 0.0 to 100.0), 95% confidence interval 76.6 to 90.383.5(76.6–90.3) | 90th | 13.2 | consistent |
| 14 | Opus 5 16 boards across 3 indep. sources | : 83.3 (scale 0.0 to 100.0), 95% confidence interval 77.2 to 91.283.3(77.2–91.2) | 92th | 11.1 | consistent |
| 15 | GPT-6.1 Sol 22 boards across 4 indep. sources | : 83.1 (scale 0.0 to 100.0), 95% confidence interval 73.3 to 89.683.1(73.3–89.6) | 85th | 17.4 | consistent |
| 16 | Qwen4 Max 9 boards across 1 indep. sources | : 82.7 (scale 0.0 to 100.0), 95% confidence interval 76.0 to 89.182.7(76.0–89.1) | 73th | 13.5 | consistent |
| 17 | DeepSeek V4.1 Flash 1017 15 boards across 4 indep. sources | : 82.4 (scale 0.0 to 100.0), 95% confidence interval 75.5 to 86.882.4(75.5–86.8) | 68th | 26.7 | consistent |
| 18 | Muse Ember 11 boards across 2 indep. sources | : 81.5 (scale 0.0 to 100.0), 95% confidence interval 73.8 to 87.481.5(73.8–87.4) | 67th | 12.2 | consistent |
| 19 | GPT-5.4 Pro 4 boards across 1 indep. sources | : 81.3 (scale 0.0 to 100.0), 95% confidence interval 71.5 to 85.381.3(71.5–85.3) | 93th | 9.7 | consistent |
| 20 | Opus 4.7 17 boards across 2 indep. sources | : 80.3 (scale 0.0 to 100.0), 95% confidence interval 73.8 to 87.480.3(73.8–87.4) | 90th | 17.8 | consistent |
| 21 | Opus 4.6 18 boards across 3 indep. sources | : 80.3 (scale 0.0 to 100.0), 95% confidence interval 71.7 to 87.480.3(71.7–87.4) | 91th | 17.0 | consistent |
| 22 | GPT-5.5 14 boards across 2 indep. sources | : 79.9 (scale 0.0 to 100.0), 95% confidence interval 74.3 to 84.879.9(74.3–84.8) | 87th | 10.3 | consistent |
| 23 | Qwen3.8-Max 10 boards across 3 indep. sources | : 79.1 (scale 0.0 to 100.0), 95% confidence interval 69.9 to 87.779.1(69.9–87.7) | 92th | 8.6 | consistent |
| 24 | Kimi K3.1 Pro 15 boards across 4 indep. sources | : 78.5 (scale 0.0 to 100.0), 95% confidence interval 69.5 to 84.278.5(69.5–84.2) | 70th | 39.9 | within-domain-dispute. Evidence grade C of A to D: thin. |
| 25 | Muse Spark 1.3 10 boards across 2 indep. sources | : 78.2 (scale 0.0 to 100.0), 95% confidence interval 72.4 to 84.778.2(72.4–84.7) | 79th | 24.1 | consistent |
| 26 | Qwen3.8 Max Internal 9 boards across 1 indep. sources | : 77.8 (scale 0.0 to 100.0), 95% confidence interval 64.5 to 86.677.8(64.5–86.6) | 59th | 13.5 | consistent |
| 27 | GPT-6 Sol 13 boards across 3 indep. sources | : 76.8 (scale 0.0 to 100.0), 95% confidence interval 69.5 to 84.976.8(69.5–84.9) | 82th | 27.0 | consistent |
| 28 | Grok 4.7 21 boards across 4 indep. sources | : 76.4 (scale 0.0 to 100.0), 95% confidence interval 70.6 to 80.576.4(70.6–80.5) | 68th | 22.5 | consistent |
| 29 | GPT-6.1 Terra 14 boards across 4 indep. sources | : 76.1 (scale 0.0 to 100.0), 95% confidence interval 63.8 to 86.276.1(63.8–86.2) | 75th | 57.3 | within-domain-dispute. Evidence grade C of A to D: thin. |
| 30 | Hy4 10 boards across 2 indep. sources | : 75.7 (scale 0.0 to 100.0), 95% confidence interval 66.6 to 81.275.7(66.6–81.2) | 59th | 15.5 | consistent |
| 31 | GLM-5.3 18 boards across 4 indep. sources | : 75.6 (scale 0.0 to 100.0), 95% confidence interval 69.0 to 79.975.6(69.0–79.9) | 64th | 35.4 | within-domain-dispute. Evidence grade C of A to D: thin. |
| 32 | Gemini 3.7 Flash 12 boards across 3 indep. sources | : 75.6 (scale 0.0 to 100.0), 95% confidence interval 65.9 to 82.175.6(65.9–82.1) | 82th | 34.7 | domain-specialist. Evidence grade C of A to D: thin. |
| 33 | Inkling 2 Small 14 boards across 3 indep. sources | : 74.8 (scale 0.0 to 100.0), 95% confidence interval 65.3 to 84.974.8(65.3–84.9) | 68th | 36.7 | within-domain-dispute. Evidence grade C of A to D: thin. |
| 34 | MiMo-V2.6-Pro 16 boards across 3 indep. sources | : 74.4 (scale 0.0 to 100.0), 95% confidence interval 67.1 to 79.974.4(67.1–79.9) | 65th | 28.6 | consistent |
| 35 | Gemini 3.8 Flash 23 boards across 4 indep. sources | : 73.8 (scale 0.0 to 100.0), 95% confidence interval 65.4 to 80.273.8(65.4–80.2) | 64th | 47.9 | within-domain-dispute. Evidence grade C of A to D: thin. |
| 36 | Muse Spark 9 boards across 2 indep. sources | : 73.3 (scale 0.0 to 100.0), 95% confidence interval 63.3 to 81.673.3(63.3–81.6) | 89th | 16.5 | consistent |
| 37 | GPT-5.6 Terra 14 boards across 3 indep. sources | : 73.0 (scale 0.0 to 100.0), 95% confidence interval 66.8 to 78.873.0(66.8–78.8) | 74th | 21.5 | consistent |
| 38 | Gemini 3.5 Flash 12 boards across 2 indep. sources | : 72.8 (scale 0.0 to 100.0), 95% confidence interval 63.9 to 78.672.8(63.9–78.6) | 87th | 37.0 | within-domain-dispute. Evidence grade C of A to D: thin. |
| 39 | Muse Spark 1.1 9 boards across 2 indep. sources | : 72.7 (scale 0.0 to 100.0), 95% confidence interval 63.5 to 79.372.7(63.5–79.3) | 79th | 22.9 | consistent |
| 40 | GPT-5.4 17 boards across 3 indep. sources | : 72.6 (scale 0.0 to 100.0), 95% confidence interval 65.2 to 77.572.6(65.2–77.5) | 77th | 24.1 | consistent |
Compares each model's percentile on human preference arenas against its median percentile across objective verifiable boards.
| Model | Arena pct | Objective median pct | Delta | Classification |
|---|---|---|---|---|
| GPT-5.3 Codex | 12.0 | 68.9 | -56.9 | capability-favored. Evidence grade C of A to D: thin. |
| Gemini 3.8 Flash | 85.2 | 43.8 | +41.4 | style-favored. Evidence grade C of A to D: thin. |
| Gemini 3.1 Pro Preview | 57.1 | 90.7 | -33.6 | capability-favored. Evidence grade C of A to D: thin. |
| GPT-6 Sol | 54.8 | 88.2 | -33.4 | capability-favored. Evidence grade C of A to D: thin. |
| GLM-5.3 Flash | 70.2 | 38.3 | +31.9 | style-favored. Evidence grade C of A to D: thin. |
| GPT-5.2 | 36.6 | 64.9 | -28.3 | capability-favored. Evidence grade C of A to D: thin. |
| GPT-4.5 Preview | 37.6 | 12.6 | +25.0 | style-favored. Evidence grade C of A to D: thin. |
| Inkling Small | 24.7 | 0.0 | +24.7 | style-favored. Evidence grade C of A to D: thin. |
| Kimi K3 | 80.9 | 56.3 | +24.6 | style-favored. Evidence grade C of A to D: thin. |
| Gemini 2.5 Flash | 35.5 | 11.1 | +24.4 | style-favored. Evidence grade C of A to D: thin. |
| Hy4 Preview | 52.5 | 28.1 | +24.4 | style-favored. Evidence grade C of A to D: thin. |
| Inkling | 27.2 | 3.1 | +24.1 | style-favored. Evidence grade C of A to D: thin. |
| Gemini 3 Flash Preview | 35.7 | 58.6 | -22.9 | capability-favored. Evidence grade C of A to D: thin. |
| Fable 5 | 90.2 | 68.5 | +21.7 | style-favored. Evidence grade C of A to D: thin. |
| Grok 4.6 | 48.7 | 70.4 | -21.7 | capability-favored. Evidence grade C of A to D: thin. |
| GPT-5 | 28.6 | 50.0 | -21.4 | capability-favored. Evidence grade C of A to D: thin. |
| Beam 1.0 | 27.9 | 7.8 | +20.1 | style-favored. Evidence grade C of A to D: thin. |
| Grok 4.5 | 60.2 | 40.4 | +19.8 | style-favored. Evidence grade C of A to D: thin. |
| Kimi K3.1 Pro | 75.8 | 56.3 | +19.5 | style-favored. Evidence grade C of A to D: thin. |
| Muse Spark Open | 44.0 | 25.0 | +19.0 | style-favored. Evidence grade C of A to D: thin. |
| Sonnet 5.5 | 72.8 | 91.7 | -18.9 | capability-favored. Evidence grade C of A to D: thin. |
| Haiku 5.5 | 29.7 | 48.4 | -18.7 | capability-favored. Evidence grade C of A to D: thin. |
| Opus 4.1 | 14.3 | 33.0 | -18.7 | capability-favored. Evidence grade C of A to D: thin. |
| GPT-4.1 | 29.0 | 10.3 | +18.7 | style-favored. Evidence grade C of A to D: thin. |
| GPT-6.1 Terra | 82.9 | 64.8 | +18.1 | style-favored. Evidence grade C of A to D: thin. |
| Grok 4.7 | 44.4 | 62.5 | -18.1 | capability-favored. Evidence grade C of A to D: thin. |
| Step 5 Preview | 48.7 | 31.3 | +17.4 | style-favored. Evidence grade C of A to D: thin. |
| GLM-4.7 | 38.7 | 21.3 | +17.4 | style-favored. Evidence grade C of A to D: thin. |
| DeepSeek V4 Pro 0813 | 57.0 | 40.6 | +16.4 | style-favored. Evidence grade C of A to D: thin. |
| Opus 4.6 | 93.3 | 77.8 | +15.5 | style-favored. Evidence grade C of A to D: thin. |
One median percentile per model and domain: flat profiles are generalists, jagged ones are specialists.
Each axis is a domain; a line is a model's median percentile on that domain's boards (100 is best). Domains need at least 12 models and models at least 4 domains, so 150 models and 9 domains are drawn. Neighbouring axes are the most correlated ones. Spread is the standard deviation of a model's domain values; the lists below keep models with a mean percentile of at least 60.
Top generalists
Flattest lines among strong models
Top specialists
Most jagged lines among strong models
Hover a line, or tab to one, to see its values. 30 of 150 lines highlighted; the rest are muted. Gaps are domains the model was not tested on.
Method rules
Known limits
Exclusions
Each model counts once, by its best current Bug Hunt row. A bug is as hard as the number of models that missed it, and a team covers a bug when any member fixed it. Rows averaged over several runs list the bugs of all their runs, so models tested more often look broader than they are.
24 of 105 bugs were fixed by no model, and 5 by exactly one (Gemini 4 Carbon, GPT-6.1 Terra, GPT-5.6 Sol, GPT-5.6 Luna, GPT-6 Sol).
Each step adds the model that fixes the most bugs the team has missed so far.
The 12 models that fix the most bugs on their own, paired.
Hover or focus a cell to compare two models. Arrow keys move between cells.
Method rules
Known limits
Exclusions
Method rules
Known limits
Exclusions