Acropolis

Bug Hunt Bench

105 real bugs planted in two production repos. Coding agents find and fix them; a blind judge grades the diff against a withheld answer key.

Upstream source
CodingAgentic105 tasks
Independent. Run by an operator independent of the model vendors.Bug Hunt Bench by Pawel HurynVersion 1Data as of 14 Nov 2026

No filters. Showing 145 of 145 rows.

  1. 1
    AnthropicDossier

    max · Claude Code 2.1.321

    Caveat

    Bugs fixed

    Fable 5.2, Bugs fixed: 58 (scale 0.0 to 105.0)58
    Fix rate
    55.2%
    False-positive fixes
    0
    Wall time
    142.8 min
    Cost per run
    $139.65
  2. 2
    GoogleDossier

    high · Antigravity CLI 1.5.5

    Caveat

    Bugs fixed

    Gemini 4 Carbon, Bugs fixed: 57.3 (scale 0.0 to 105.0)57.3
    Fix rate
    54.6%
    False-positive fixes
    0
    Wall time
    114 min
    Cost per run
    $31.32
  3. 3
    GoogleDossier

    medium · Antigravity CLI 1.5.5

    Caveat

    Bugs fixed

    Gemini 4 Carbon, Bugs fixed: 56 (scale 0.0 to 105.0)56
    Fix rate
    53.3%
    False-positive fixes
    0
    Wall time
    81.1 min
    Cost per run
    $24.95
  4. 4
    AnthropicDossier

    max · Claude Code 2.1.283

    Caveat

    Bugs fixed

    Sonnet 5.5, Bugs fixed: 51.3 (scale 0.0 to 105.0)51.3
    Fix rate
    48.9%
    False-positive fixes
    0
    Wall time
    287.1 min
    Cost per run
    $154.03
  5. 5
    AnthropicDossier

    xhigh · Claude Code 2.1.321

    Caveat

    Bugs fixed

    Fable 5.2, Bugs fixed: 49 (scale 0.0 to 105.0)49
    Fix rate
    46.7%
    False-positive fixes
    0
    Wall time
    57 min
    Cost per run
    $61.01
  6. 6
    GoogleDossier

    low · Antigravity CLI

    Caveat

    Bugs fixed

    Gemini 4 Carbon, Bugs fixed: 46 (scale 0.0 to 105.0)46
    Fix rate
    43.8%
    False-positive fixes
    0
    Wall time
    60.4 min
    Cost per run
    $7.67
  7. 7
    OpenAIDossier

    max · Codex CLI

    Caveat

    Bugs fixed

    GPT-6 Astra, Bugs fixed: 45 (scale 0.0 to 105.0)45
    Fix rate
    42.9%
    False-positive fixes
    0
    Wall time
    89.8 min
    Cost per run
    $33.03
  8. 8
    OpenAIDossier

    max · Codex CLI 0.159.0

    Caveat

    Bugs fixed

    GPT-6.1 Sol, Bugs fixed: 44.3 (scale 0.0 to 105.0)44.3
    Fix rate
    42.2%
    False-positive fixes
    0
    Wall time
    139.3 min
    Cost per run
    $5.88
  9. 9
    OpenAIDossier

    max · Codex CLI 0.154.0

    Caveat

    Bugs fixed

    GPT-5.6 Sol, Bugs fixed: 43.5 (scale 0.0 to 105.0)43.5
    Fix rate
    41.4%
    False-positive fixes
    0
    Wall time
    253.1 min
    Cost per run
    $95.35
  10. 10
    AnthropicDossier

    max · Claude Code 2.1.251

    1 seed. Based on one run, so no interval can be computed.

    Bugs fixed

    Fable 5.1, Bugs fixed: 43 (scale 0.0 to 105.0), single run, no repeated seeds43(1 seed)
    Fix rate
    41%
    False-positive fixes
    0
    Wall time
    73.1 min
    Cost per run
    $87.18
  11. 10
    AnthropicDossier

    high · Claude Code 2.1.321

    Caveat

    Bugs fixed

    Fable 5.2, Bugs fixed: 43 (scale 0.0 to 105.0)43
    Fix rate
    41%
    False-positive fixes
    0
    Wall time
    39.5 min
    Cost per run
    $46.67
  12. 10
    OpenAIDossier

    xhigh · Codex CLI

    1 seed. Based on one run, so no interval can be computed.

    Bugs fixed

    GPT-6 Astra, Bugs fixed: 43 (scale 0.0 to 105.0), single run, no repeated seeds43(1 seed)
    Fix rate
    41%
    False-positive fixes
    0
    Wall time
    58.8 min
    Cost per run
    $24.22
  13. 13
    DeepSeekDossier

    max · DeepSeek Harness 0.2.1-rc.2

    Caveat

    Bugs fixed

    DeepSeek V4.1 Flash 1017, Bugs fixed: 42.7 (scale 0.0 to 105.0)42.7
    Fix rate
    40.7%
    False-positive fixes
    0
    Wall time
    62.1 min
    Cost per run
    $1.84
  14. 13
    OpenAIDossier

    xhigh · Codex CLI 0.159.0

    Caveat

    Bugs fixed

    GPT-6.1 Sol, Bugs fixed: 42.7 (scale 0.0 to 105.0)42.7
    Fix rate
    40.7%
    False-positive fixes
    0
    Wall time
    98.7 min
    Cost per run
    $4.35
  15. 15
    DeepSeekDossier

    max · Pi 1.3.0

    Caveat

    Bugs fixed

    DeepSeek V4.1 Flash 1017, Bugs fixed: 42.2 (scale 0.0 to 105.0)42.2
    Fix rate
    40.2%
    False-positive fixes
    0
    Wall time
    54.1 min
    Cost per run
    $0.83
  16. 16

    max · Muse Code 1.4.1

    Caveat

    Bugs fixed

    Muse Wonder Full 1.0, Bugs fixed: 42 (scale 0.0 to 105.0)42
    Fix rate
    40%
    False-positive fixes
    0
    Wall time
    145.3 min
    Cost per run
    Not tested
  17. 17
    AnthropicDossier

    max · Claude Code 2.1.280

    Caveat

    Bugs fixed

    Opus 5.5, Bugs fixed: 41.7 (scale 0.0 to 105.0)41.7
    Fix rate
    39.7%
    False-positive fixes
    0
    Wall time
    66.9 min
    Cost per run
    $58.53
  18. 18
    AnthropicDossier

    medium · Claude Code 2.1.321

    Caveat

    Bugs fixed

    Fable 5.2, Bugs fixed: 41.5 (scale 0.0 to 105.0)41.5
    Fix rate
    39.5%
    False-positive fixes
    0
    Wall time
    19.8 min
    Cost per run
    $26.55
  19. 18

    xhigh · Muse Code 1.4.1

    Caveat

    Bugs fixed

    Muse Wonder Full 1.0, Bugs fixed: 41.5 (scale 0.0 to 105.0)41.5
    Fix rate
    39.5%
    False-positive fixes
    0
    Wall time
    90.5 min
    Cost per run
    Not tested
  20. 20
    AnthropicDossier

    low · Claude Code 2.1.321

    Caveat

    Bugs fixed

    Fable 5.2, Bugs fixed: 41 (scale 0.0 to 105.0)41
    Fix rate
    39%
    False-positive fixes
    0
    Wall time
    18.7 min
    Cost per run
    $24.32
  21. 21
    GoogleDossier

    high · Antigravity CLI 1.3.5

    Caveat

    Bugs fixed

    Gemini 4 Argon, Bugs fixed: 40.7 (scale 0.0 to 105.0)40.7
    Fix rate
    38.8%
    False-positive fixes
    0
    Wall time
    56.5 min
    Cost per run
    $7.16
  22. 21
    OpenAIDossier

    max · Codex CLI 0.159.0

    Caveat

    Bugs fixed

    GPT-6.1 Terra, Bugs fixed: 40.7 (scale 0.0 to 105.0)40.7
    Fix rate
    38.8%
    False-positive fixes
    0
    Wall time
    37.3 min
    Cost per run
    $3.88
  23. 23
    DeepSeekDossier

    high · Pi 1.3.0

    Caveat

    Bugs fixed

    DeepSeek V4.1 Flash 1017, Bugs fixed: 40.4 (scale 0.0 to 105.0)40.4
    Fix rate
    38.5%
    False-positive fixes
    0
    Wall time
    48.3 min
    Cost per run
    $0.64
  24. 24
    GoogleDossier

    medium · Antigravity CLI

    Caveat

    Bugs fixed

    Gemini 4 Argon, Bugs fixed: 39.3 (scale 0.0 to 105.0)39.3
    Fix rate
    37.4%
    False-positive fixes
    0
    Wall time
    59.9 min
    Cost per run
    $7.50
  25. 25
    DeepSeekDossier

    high · DeepSeek Harness 0.2.1-rc.2

    Caveat

    Bugs fixed

    DeepSeek V4.1 Flash 1017, Bugs fixed: 39.2 (scale 0.0 to 105.0)39.2
    Fix rate
    37.3%
    False-positive fixes
    0
    Wall time
    53 min
    Cost per run
    $1.27
  26. 26
    DeepSeekDossier

    max · Claude Code 2.1.302

    Caveat

    Bugs fixed

    DeepSeek V4.1 Flash 1017, Bugs fixed: 39.1 (scale 0.0 to 105.0)39.1
    Fix rate
    37.2%
    False-positive fixes
    0
    Wall time
    57.7 min
    Cost per run
    $1.01
  27. 27
    OpenAIDossier

    xhigh · Codex CLI

    1 seed. Based on one run, so no interval can be computed.

    Bugs fixed

    GPT-5.6 Sol, Bugs fixed: 39 (scale 0.0 to 105.0), single run, no repeated seeds39(1 seed)
    Fix rate
    37.1%
    False-positive fixes
    0
    Wall time
    126.2 min
    Cost per run
    $52.75
  28. 28
    DeepSeekDossier

    max · OpenCode 1.19.3

    Caveat

    Bugs fixed

    DeepSeek V4.1 Flash 1017, Bugs fixed: 37.6 (scale 0.0 to 105.0)37.6
    Fix rate
    35.8%
    False-positive fixes
    0
    Wall time
    55.7 min
    Cost per run
    $0.96
  29. 29
    DeepSeekDossier

    high · OpenCode 1.19.3

    Caveat

    Bugs fixed

    DeepSeek V4.1 Flash 1017, Bugs fixed: 36.6 (scale 0.0 to 105.0)36.6
    Fix rate
    34.9%
    False-positive fixes
    0
    Wall time
    41.6 min
    Cost per run
    $0.79
  30. 30
    OpenAIDossier

    high · Codex CLI 0.159.0

    Caveat

    Bugs fixed

    GPT-6.1 Sol, Bugs fixed: 36.5 (scale 0.0 to 105.0)36.5
    Fix rate
    34.8%
    False-positive fixes
    0
    Wall time
    70.3 min
    Cost per run
    $3.26
  31. 31
    DeepSeekDossier

    high · Claude Code 2.1.302

    Caveat

    Bugs fixed

    DeepSeek V4.1 Flash 1017, Bugs fixed: 36.4 (scale 0.0 to 105.0)36.4
    Fix rate
    34.7%
    False-positive fixes
    0
    Wall time
    54.5 min
    Cost per run
    $0.82
  32. 32
    Moonshot AIDossier

    max · Claude Code 2.1.309

    Caveat

    Bugs fixed

    Kimi K3.1 Pro, Bugs fixed: 36 (scale 0.0 to 105.0)36
    Fix rate
    34.3%
    False-positive fixes
    0
    Wall time
    64.8 min
    Cost per run
    $13.76
  33. 32
    AnthropicDossier

    xhigh · Claude Code 2.1.280

    Caveat

    Bugs fixed

    Opus 5.5, Bugs fixed: 36 (scale 0.0 to 105.0)36
    Fix rate
    34.3%
    False-positive fixes
    0
    Wall time
    42.5 min
    Cost per run
    $34.98
  34. 32
    AnthropicDossier

    xhigh · Claude Code 2.1.283

    Caveat

    Bugs fixed

    Sonnet 5.5, Bugs fixed: 36 (scale 0.0 to 105.0)36
    Fix rate
    34.3%
    False-positive fixes
    0
    Wall time
    72.7 min
    Cost per run
    $42.86
  35. 35
    GoogleDossier

    low · Antigravity CLI

    Caveat

    Bugs fixed

    Gemini 4 Argon, Bugs fixed: 35.3 (scale 0.0 to 105.0)35.3
    Fix rate
    33.6%
    False-positive fixes
    0
    Wall time
    59.6 min
    Cost per run
    $6.47
  36. 36
    OpenAIDossier

    high · Codex CLI

    1 seed. Based on one run, so no interval can be computed.

    Bugs fixed

    GPT-6 Astra, Bugs fixed: 35 (scale 0.0 to 105.0), single run, no repeated seeds35(1 seed)
    Fix rate
    33.3%
    False-positive fixes
    0
    Wall time
    40.5 min
    Cost per run
    $20.60
  37. 37
    DeepSeekDossier

    low · DeepSeek Harness 0.2.1-rc.2

    Caveat

    Bugs fixed

    DeepSeek V4.1 Flash 1017, Bugs fixed: 34.8 (scale 0.0 to 105.0)34.8
    Fix rate
    33.1%
    False-positive fixes
    0
    Wall time
    33.8 min
    Cost per run
    $1.06
  38. 38
    DeepSeekDossier

    low · Claude Code 2.1.302

    Caveat

    Bugs fixed

    DeepSeek V4.1 Flash 1017, Bugs fixed: 34.4 (scale 0.0 to 105.0)34.4
    Fix rate
    32.8%
    False-positive fixes
    0
    Wall time
    44.5 min
    Cost per run
    $0.71
  39. 39
    AnthropicDossier

    high · Claude Code 2.1.283

    Caveat

    Bugs fixed

    Sonnet 5.5, Bugs fixed: 34.3 (scale 0.0 to 105.0)34.3
    Fix rate
    32.7%
    False-positive fixes
    0
    Wall time
    27.9 min
    Cost per run
    $19.85
  40. 40
    OpenAIDossier

    high · Codex CLI

    1 seed. Based on one run, so no interval can be computed.

    Bugs fixed

    GPT-5.6 Sol, Bugs fixed: 34 (scale 0.0 to 105.0), single run, no repeated seeds34(1 seed)
    Fix rate
    32.4%
    False-positive fixes
    0
    Wall time
    66.7 min
    Cost per run
    $33.92
  41. 40
    OpenAIDossier

    medium · Codex CLI

    1 seed. Based on one run, so no interval can be computed.

    Bugs fixed

    GPT-6 Astra, Bugs fixed: 34 (scale 0.0 to 105.0), single run, no repeated seeds34(1 seed)
    Fix rate
    32.4%
    False-positive fixes
    0
    Wall time
    28 min
    Cost per run
    $15.78
  42. 42
    DeepSeekDossier

    low · Pi 1.3.0

    Caveat

    Bugs fixed

    DeepSeek V4.1 Flash 1017, Bugs fixed: 33.8 (scale 0.0 to 105.0)33.8
    Fix rate
    32.2%
    False-positive fixes
    0
    Wall time
    33.2 min
    Cost per run
    $0.46
  43. 43
    AnthropicDossier

    high · Claude Code 2.1.251

    1 seed. Based on one run, so no interval can be computed.

    Bugs fixed

    Fable 5.1, Bugs fixed: 33 (scale 0.0 to 105.0), single run, no repeated seeds33(1 seed)
    Fix rate
    31.4%
    False-positive fixes
    0
    Wall time
    36.3 min
    Cost per run
    $48.58
  44. 44
    DeepSeekDossier

    low · OpenCode 1.19.3

    Caveat

    Bugs fixed

    DeepSeek V4.1 Flash 1017, Bugs fixed: 32.8 (scale 0.0 to 105.0)32.8
    Fix rate
    31.2%
    False-positive fixes
    0
    Wall time
    41.2 min
    Cost per run
    $0.66
  45. 45
    GoogleDossier

    xhigh · Antigravity CLI 1.5.8

    Caveat

    Bugs fixed

    Gemini 4 Hydrogen, Bugs fixed: 32.7 (scale 0.0 to 105.0)32.7
    Fix rate
    31.1%
    False-positive fixes
    0
    Wall time
    35.8 min
    Cost per run
    $7.07
  46. 46

    max · Muse Code 1.1.1 / 1.2.1 / 1.3.0

    Caveat

    Bugs fixed

    Muse Spark 1.3, Bugs fixed: 32.2 (scale 0.0 to 105.0)32.2
    Fix rate
    30.7%
    False-positive fixes
    0
    Wall time
    85.6 min
    Cost per run
    $18.11
  47. 47
    OpenAIDossier

    max · Codex CLI

    1 seed. Based on one run, so no interval can be computed.

    Bugs fixed

    GPT-5.6 Terra, Bugs fixed: 32 (scale 0.0 to 105.0), single run, no repeated seeds32(1 seed)
    Fix rate
    30.5%
    False-positive fixes
    0
    Wall time
    159.7 min
    Cost per run
    $27.98
  48. 48

    max · Muse Code 1.4.1

    Caveat

    Bugs fixed

    Muse Wonder 1.0, Bugs fixed: 31.7 (scale 0.0 to 105.0)31.7
    Fix rate
    30.2%
    False-positive fixes
    0
    Wall time
    96.2 min
    Cost per run
    $27.87
  49. 48
    AnthropicDossier

    high · Claude Code 2.1.280

    Caveat

    Bugs fixed

    Opus 5.5, Bugs fixed: 31.7 (scale 0.0 to 105.0)31.7
    Fix rate
    30.2%
    False-positive fixes
    0
    Wall time
    23.9 min
    Cost per run
    $22.25
  50. 50
    OpenAIDossier

    max · Codex CLI

    Caveat

    Bugs fixed

    GPT-5.6 Luna, Bugs fixed: 31.3 (scale 0.0 to 105.0)31.3
    Fix rate
    29.8%
    False-positive fixes
    0
    Wall time
    187.1 min
    Cost per run
    $3.15
  51. 51
    UnbiasedDossier

    Claude Code 2.1.274

    Bugs fixed

    Pareto (ex-Union Alpha), Bugs fixed: 30.7 (scale 0.0 to 105.0)30.7
    Fix rate
    29.2%
    False-positive fixes
    0
    Wall time
    38.2 min
    Cost per run
    $4.81
  52. 52
    AnthropicDossier

    medium · Claude Code 2.1.280

    Caveat

    Bugs fixed

    Opus 5.5, Bugs fixed: 30.3 (scale 0.0 to 105.0)30.3
    Fix rate
    28.9%
    False-positive fixes
    0
    Wall time
    17 min
    Cost per run
    $15.68
  53. 53
    OpenAIDossier

    max · Codex CLI 0.155.1

    Caveat

    Bugs fixed

    GPT-6 Sol, Bugs fixed: 29.3 (scale 0.0 to 105.0)29.3
    Fix rate
    27.9%
    False-positive fixes
    0
    Wall time
    63.4 min
    Cost per run
    $9.33
  54. 54
    AnthropicDossier

    max · Claude Code 2.1.220

    1 seed. Based on one run, so no interval can be computed.

    Bugs fixed

    Fable 5, Bugs fixed: 29 (scale 0.0 to 105.0), single run, no repeated seeds29(1 seed)
    Fix rate
    27.6%
    False-positive fixes
    0
    Wall time
    57.3 min
    Cost per run
    $112.40
  55. 54
    AnthropicDossier

    low · Claude Code 2.1.251

    Caveat1 seed. Based on one run, so no interval can be computed.

    Bugs fixed

    Fable 5.1, Bugs fixed: 29 (scale 0.0 to 105.0), single run, no repeated seeds29(1 seed)
    Fix rate
    27.6%
    False-positive fixes
    0
    Wall time
    33.2 min
    Cost per run
    $33.00
  56. 54
    AnthropicDossier

    xhigh · Claude Code 2.1.267

    Caveat1 seed. Based on one run, so no interval can be computed.

    Bugs fixed

    Fable 5.1, Bugs fixed: 29 (scale 0.0 to 105.0), single run, no repeated seeds29(1 seed)
    Fix rate
    27.6%
    False-positive fixes
    0
    Wall time
    59.5 min
    Cost per run
    $53.61
  57. 54
    OpenAIDossier

    medium · Codex CLI

    1 seed. Based on one run, so no interval can be computed.

    Bugs fixed

    GPT-5.6 Sol, Bugs fixed: 29 (scale 0.0 to 105.0), single run, no repeated seeds29(1 seed)
    Fix rate
    27.6%
    False-positive fixes
    0
    Wall time
    47.6 min
    Cost per run
    $15.77
  58. 54
    OpenAIDossier

    medium · Codex CLI 0.159.0

    Caveat

    Bugs fixed

    GPT-6.1 Sol, Bugs fixed: 29 (scale 0.0 to 105.0)29
    Fix rate
    27.6%
    False-positive fixes
    0
    Wall time
    29.8 min
    Cost per run
    $1.76
  59. 54

    xhigh · Grok Build CLI (ACP) 1.0.30

    Caveat

    Bugs fixed

    Grok 4.7, Bugs fixed: 29 (scale 0.0 to 105.0)29
    Fix rate
    27.6%
    False-positive fixes
    0
    Wall time
    44.7 min
    Cost per run
    $22.96
  60. 60

    xhigh · Grok Build CLI (ACP) 1.0.5

    Caveat

    Bugs fixed

    Grok 4.6, Bugs fixed: 28.7 (scale 0.0 to 105.0)28.7
    Fix rate
    27.3%
    False-positive fixes
    0
    Wall time
    43.1 min
    Cost per run
    $18.60
  61. 61
    GoogleDossier

    high · Antigravity CLI 1.5.8

    Caveat

    Bugs fixed

    Gemini 4 Hydrogen, Bugs fixed: 28.3 (scale 0.0 to 105.0)28.3
    Fix rate
    27%
    False-positive fixes
    0
    Wall time
    22.4 min
    Cost per run
    $4.42
  62. 62
    OpenAIDossier

    low · Codex CLI

    1 seed. Based on one run, so no interval can be computed.

    Bugs fixed

    GPT-6 Astra, Bugs fixed: 27 (scale 0.0 to 105.0), single run, no repeated seeds27(1 seed)
    Fix rate
    25.7%
    False-positive fixes
    0
    Wall time
    32.1 min
    Cost per run
    $11.69
  63. 62
    AnthropicDossier

    max · Claude Code 2.1.220

    1 seed. Based on one run, so no interval can be computed.

    Bugs fixed

    Opus 5, Bugs fixed: 27 (scale 0.0 to 105.0), single run, no repeated seeds27(1 seed)
    Fix rate
    25.7%
    False-positive fixes
    0
    Wall time
    60 min
    Cost per run
    $54.94
  64. 64

    medium · Grok Build CLI (ACP)

    Caveat

    Bugs fixed

    Grok 4.7, Bugs fixed: 26.7 (scale 0.0 to 105.0)26.7
    Fix rate
    25.4%
    False-positive fixes
    0
    Wall time
    30.7 min
    Cost per run
    $16.76
  65. 65
    AnthropicDossier

    xhigh · Claude Code 2.1.248

    1 seed. Based on one run, so no interval can be computed.

    Bugs fixed

    Opus 5, Bugs fixed: 26 (scale 0.0 to 105.0), single run, no repeated seeds26(1 seed)
    Fix rate
    24.8%
    False-positive fixes
    0
    Wall time
    49.9 min
    Cost per run
    $63.25
  66. 65
    AlibabaDossier

    max · Claude Code 2.1.267

    Caveat1 seed. Based on one run, so no interval can be computed.

    Bugs fixed

    Qwen3.8-Flash, Bugs fixed: 26 (scale 0.0 to 105.0), single run, no repeated seeds26(1 seed)
    Fix rate
    24.8%
    False-positive fixes
    0
    Wall time
    97.3 min
    Cost per run
    $1.81
  67. 67
    AlibabaDossier

    max · Claude Code 2.1.267 / 2.1.272

    Caveat

    Bugs fixed

    Qwen3.8-Max, Bugs fixed: 25.7 (scale 0.0 to 105.0)25.7
    Fix rate
    24.5%
    False-positive fixes
    0
    Wall time
    117.1 min
    Cost per run
    $26.76
  68. 68
    OpenAIDossier

    xhigh · Codex CLI 0.155.1

    Caveat1 seed. Based on one run, so no interval can be computed.

    Bugs fixed

    GPT-6 Sol, Bugs fixed: 25 (scale 0.0 to 105.0), single run, no repeated seeds25(1 seed)
    Fix rate
    23.8%
    False-positive fixes
    0
    Wall time
    51.4 min
    Cost per run
    $7.67
  69. 69
    GoogleDossier

    medium · Antigravity CLI 1.5.8

    Caveat

    Bugs fixed

    Gemini 4 Hydrogen, Bugs fixed: 24.7 (scale 0.0 to 105.0)24.7
    Fix rate
    23.5%
    False-positive fixes
    0
    Wall time
    19.9 min
    Cost per run
    $2.10
  70. 70
    AnthropicDossier

    high · Claude Code 2.1.215 / 2.1.218

    1 seed. Based on one run, so no interval can be computed.

    Bugs fixed

    Fable 5, Bugs fixed: 24 (scale 0.0 to 105.0), single run, no repeated seeds24(1 seed)
    Fix rate
    22.9%
    False-positive fixes
    0
    Wall time
    31.4 min
    Cost per run
    $74.52
  71. 70
    AnthropicDossier

    medium · Claude Code 2.1.247

    1 seed. Based on one run, so no interval can be computed.

    Bugs fixed

    Opus 5, Bugs fixed: 24 (scale 0.0 to 105.0), single run, no repeated seeds24(1 seed)
    Fix rate
    22.9%
    False-positive fixes
    0
    Wall time
    30.5 min
    Cost per run
    $37.40
  72. 72
    XiaomiDossier

    Claude Code 2.1.278 / 2.1.280

    Caveat

    Bugs fixed

    MiMo-V2.6-Flash, Bugs fixed: 23.3 (scale 0.0 to 105.0)23.3
    Fix rate
    22.2%
    False-positive fixes
    0
    Wall time
    70.7 min
    Cost per run
    $0.49
  73. 72
    AnthropicDossier

    medium · Claude Code 2.1.283

    Caveat

    Bugs fixed

    Sonnet 5.5, Bugs fixed: 23.3 (scale 0.0 to 105.0)23.3
    Fix rate
    22.2%
    False-positive fixes
    0
    Wall time
    17.4 min
    Cost per run
    $11.11
  74. 74
    OpenAIDossier

    xhigh · Codex CLI

    1 seed. Based on one run, so no interval can be computed.

    Bugs fixed

    GPT-5.6 Luna, Bugs fixed: 23 (scale 0.0 to 105.0), single run, no repeated seeds23(1 seed)
    Fix rate
    21.9%
    False-positive fixes
    0
    Wall time
    135.6 min
    Cost per run
    $2.50
  75. 74

    high · Grok Build CLI (ACP)

    1 seed. Based on one run, so no interval can be computed.

    Bugs fixed

    Grok 4.6, Bugs fixed: 23 (scale 0.0 to 105.0), single run, no repeated seeds23(1 seed)
    Fix rate
    21.9%
    False-positive fixes
    0
    Wall time
    33 min
    Cost per run
    $15.73
  76. 74
    AlibabaDossier

    low · Claude Code 2.1.267

    Caveat1 seed. Based on one run, so no interval can be computed.

    Bugs fixed

    Qwen3.8-Flash, Bugs fixed: 23 (scale 0.0 to 105.0), single run, no repeated seeds23(1 seed)
    Fix rate
    21.9%
    False-positive fixes
    0
    Wall time
    98.1 min
    Cost per run
    $1.37
  77. 77
    XiaomiDossier

    Claude Code 2.1.278

    Caveat

    Bugs fixed

    MiMo-V2.6-Pro, Bugs fixed: 22.7 (scale 0.0 to 105.0)22.7
    Fix rate
    21.6%
    False-positive fixes
    0
    Wall time
    90.9 min
    Cost per run
    $0.86
  78. 78
    OpenAIDossier

    low · Codex CLI 0.159.0

    Caveat

    Bugs fixed

    GPT-6.1 Sol, Bugs fixed: 22.5 (scale 0.0 to 105.0)22.5
    Fix rate
    21.4%
    False-positive fixes
    0
    Wall time
    25.4 min
    Cost per run
    $1.34
  79. 79
    AnthropicDossier

    low · Claude Code 2.1.280

    Caveat

    Bugs fixed

    Opus 5.5, Bugs fixed: 22.3 (scale 0.0 to 105.0)22.3
    Fix rate
    21.2%
    False-positive fixes
    0
    Wall time
    11.5 min
    Cost per run
    $8.34
  80. 80
    GoogleDossier

    high · Gemini CLI (retired) + model-pinning gateway 0.56.0

    Caveat1 seed. Based on one run, so no interval can be computed.

    Bugs fixed

    Gemini 3.7 Flash, Bugs fixed: 22 (scale 0.0 to 105.0), single run, no repeated seeds22(1 seed)
    Fix rate
    21%
    False-positive fixes
    0
    Wall time
    96.8 min
    Cost per run
    $8.43
  81. 80

    medium · Grok Build CLI (ACP) 1.0.5

    1 seed. Based on one run, so no interval can be computed.

    Bugs fixed

    Grok 4.6, Bugs fixed: 22 (scale 0.0 to 105.0), single run, no repeated seeds22(1 seed)
    Fix rate
    21%
    False-positive fixes
    0
    Wall time
    26.5 min
    Cost per run
    $5.80
  82. 82
    DeepSeekDossier

    max · Claude Code 2.1.267 / 2.1.270

    Caveat

    Bugs fixed

    DeepSeek V4.1 Flash, Bugs fixed: 21.7 (scale 0.0 to 105.0)21.7
    Fix rate
    20.7%
    False-positive fixes
    0
    Wall time
    55.9 min
    Cost per run
    $0.78
  83. 83
    AnthropicDossier

    max · Claude Code 2.1.292

    Caveat

    Bugs fixed

    Haiku 5.5, Bugs fixed: 21.5 (scale 0.0 to 105.0)21.5
    Fix rate
    20.5%
    False-positive fixes
    0
    Wall time
    32.5 min
    Cost per run
    $5.83
  84. 84
    AnthropicDossier

    medium · Claude Code 2.1.267

    Caveat1 seed. Based on one run, so no interval can be computed.

    Bugs fixed

    Fable 5.1, Bugs fixed: 21 (scale 0.0 to 105.0), single run, no repeated seeds21(1 seed)
    Fix rate
    20%
    False-positive fixes
    0
    Wall time
    23 min
    Cost per run
    $23.73
  85. 84
    Moonshot AIDossier

    Claude Code 2.1.215 / 2.1.218

    1 seed. Based on one run, so no interval can be computed.

    Bugs fixed

    Kimi K3, Bugs fixed: 21 (scale 0.0 to 105.0), single run, no repeated seeds21(1 seed)
    Fix rate
    20%
    False-positive fixes
    0
    Wall time
    107.8 min
    Cost per run
    $25.27
  86. 84
    AnthropicDossier

    high · Claude Code 2.1.218

    1 seed. Based on one run, so no interval can be computed.

    Bugs fixed

    Opus 5, Bugs fixed: 21 (scale 0.0 to 105.0), single run, no repeated seeds21(1 seed)
    Fix rate
    20%
    False-positive fixes
    0
    Wall time
    37.4 min
    Cost per run
    $41.51
  87. 87

    xhigh · Muse Code 1.2.1

    Caveat

    Bugs fixed

    Muse Spark 1.3, Bugs fixed: 20.3 (scale 0.0 to 105.0)20.3
    Fix rate
    19.3%
    False-positive fixes
    0
    Wall time
    63.8 min
    Cost per run
    $13.71
  88. 88
    OpenAIDossier

    xhigh · Codex CLI

    1 seed. Based on one run, so no interval can be computed.

    Bugs fixed

    GPT-5.6 Terra, Bugs fixed: 20 (scale 0.0 to 105.0), single run, no repeated seeds20(1 seed)
    Fix rate
    19%
    False-positive fixes
    0
    Wall time
    56.6 min
    Cost per run
    $8.98
  89. 88
    OpenAIDossier

    high · Codex CLI 0.155.1

    Caveat1 seed. Based on one run, so no interval can be computed.

    Bugs fixed

    GPT-6 Sol, Bugs fixed: 20 (scale 0.0 to 105.0), single run, no repeated seeds20(1 seed)
    Fix rate
    19%
    False-positive fixes
    0
    Wall time
    33.9 min
    Cost per run
    $4.16
  90. 90

    high · Grok Build CLI (ACP)

    Caveat

    Bugs fixed

    Grok 4.7, Bugs fixed: 19.3 (scale 0.0 to 105.0)19.3
    Fix rate
    18.4%
    False-positive fixes
    0
    Wall time
    37.9 min
    Cost per run
    $12.29
  91. 91
    DeepSeekDossier

    high · Claude Code 2.1.267

    Caveat1 seed. Based on one run, so no interval can be computed.

    Bugs fixed

    DeepSeek V4.1 Flash, Bugs fixed: 19 (scale 0.0 to 105.0), single run, no repeated seeds19(1 seed)
    Fix rate
    18.1%
    False-positive fixes
    0
    Wall time
    26.1 min
    Cost per run
    $0.31
  92. 91

    max · Claude Code 2.1.267

    Caveat1 seed. Based on one run, so no interval can be computed.

    Bugs fixed

    GLM-5.3, Bugs fixed: 19 (scale 0.0 to 105.0), single run, no repeated seeds19(1 seed)
    Fix rate
    18.1%
    False-positive fixes
    0
    Wall time
    40.4 min
    Cost per run
    $15.93
  93. 91
    AnthropicDossier

    low · Claude Code 2.1.283

    Caveat

    Bugs fixed

    Sonnet 5.5, Bugs fixed: 19 (scale 0.0 to 105.0)19
    Fix rate
    18.1%
    False-positive fixes
    0
    Wall time
    8.5 min
    Cost per run
    $5.73
  94. 94

    high · Muse Code 1.2.1

    Caveat

    Bugs fixed

    Muse Spark 1.3, Bugs fixed: 18.7 (scale 0.0 to 105.0)18.7
    Fix rate
    17.8%
    False-positive fixes
    0
    Wall time
    75.6 min
    Cost per run
    $12.94
  95. 95
    OpenAIDossier

    max · Codex CLI 0.155.1

    Caveat

    Bugs fixed

    GPT-6 Luna, Bugs fixed: 18.3 (scale 0.0 to 105.0)18.3
    Fix rate
    17.4%
    False-positive fixes
    0
    Wall time
    99.2 min
    Cost per run
    $0.52
  96. 95
    AnthropicDossier

    xhigh · Claude Code 2.1.292

    Caveat

    Bugs fixed

    Haiku 5.5, Bugs fixed: 18.3 (scale 0.0 to 105.0)18.3
    Fix rate
    17.4%
    False-positive fixes
    0
    Wall time
    24.1 min
    Cost per run
    $4.80
  97. 97
    GoogleDossier

    high · Antigravity CLI 1.1.22

    Caveat

    Bugs fixed

    Gemini 3.8 Flash, Bugs fixed: 18 (scale 0.0 to 105.0)18
    Fix rate
    17.1%
    False-positive fixes
    0
    Wall time
    38.7 min
    Cost per run
    $11.03
  98. 97
    OpenAIDossier

    high · Codex CLI

    1 seed. Based on one run, so no interval can be computed.

    Bugs fixed

    GPT-5.6 Terra, Bugs fixed: 18 (scale 0.0 to 105.0), single run, no repeated seeds18(1 seed)
    Fix rate
    17.1%
    False-positive fixes
    0
    Wall time
    28.1 min
    Cost per run
    $5.89
  99. 97
    TencentDossier

    Claude Code 2.1.250

    1 seed. Based on one run, so no interval can be computed.

    Bugs fixed

    Hy4 Preview, Bugs fixed: 18 (scale 0.0 to 105.0), single run, no repeated seeds18(1 seed)
    Fix rate
    17.1%
    False-positive fixes
    0
    Wall time
    67.5 min
    Cost per run
    $3.13
  100. 97
    XiaomiDossier

    thinking · Claude Code 2.1.278

    Caveat1 seed. Based on one run, so no interval can be computed.

    Bugs fixed

    MiMo-V2.6-Pro, Bugs fixed: 18 (scale 0.0 to 105.0), single run, no repeated seeds18(1 seed)
    Fix rate
    17.1%
    False-positive fixes
    0
    Wall time
    74.9 min
    Cost per run
    $0.59
  101. 101

    max · Claude Code 2.1.269 / 2.1.270

    Caveat

    Bugs fixed

    GLM-5.3 Flash, Bugs fixed: 17.7 (scale 0.0 to 105.0)17.7
    Fix rate
    16.9%
    False-positive fixes
    0
    Wall time
    63.2 min
    Cost per run
    $1.03
  102. 102

    high · Grok Build CLI (ACP)

    1 seed. Based on one run, so no interval can be computed.

    Bugs fixed

    Grok 4.5, Bugs fixed: 17 (scale 0.0 to 105.0), single run, no repeated seeds17(1 seed)
    Fix rate
    16.2%
    False-positive fixes
    0
    Wall time
    27.9 min
    Cost per run
    $8.50
  103. 102

    xhigh · Claude Code 2.1.222

    Caveat1 seed. Based on one run, so no interval can be computed.

    Bugs fixed

    Muse Spark 1.2, Bugs fixed: 17 (scale 0.0 to 105.0), single run, no repeated seeds17(1 seed)
    Fix rate
    16.2%
    False-positive fixes
    0
    Wall time
    35.7 min
    Cost per run
    $13.99
  104. 104
    DeepSeekDossier

    max · Claude Code 2.1.267

    Caveat1 seed. Based on one run, so no interval can be computed.

    Bugs fixed

    DeepSeek V4-Pro, Bugs fixed: 16 (scale 0.0 to 105.0), single run, no repeated seeds16(1 seed)
    Fix rate
    15.2%
    False-positive fixes
    0
    Wall time
    37.2 min
    Cost per run
    $1.89
  105. 104
    GoogleDossier

    high · Antigravity CLI 1.1.20

    1 seed. Based on one run, so no interval can be computed.

    Bugs fixed

    Gemini 3.7 Flash, Bugs fixed: 16 (scale 0.0 to 105.0), single run, no repeated seeds16(1 seed)
    Fix rate
    15.2%
    False-positive fixes
    0
    Wall time
    22.7 min
    Cost per run
    $6.36
  106. 104

    high · Claude Code 2.1.269

    Caveat1 seed. Based on one run, so no interval can be computed.

    Bugs fixed

    GLM-5.3 Flash, Bugs fixed: 16 (scale 0.0 to 105.0), single run, no repeated seeds16(1 seed)
    Fix rate
    15.2%
    False-positive fixes
    0
    Wall time
    62.3 min
    Cost per run
    $0.94
  107. 104
    AnthropicDossier

    high · Claude Code 2.1.292

    Caveat

    Bugs fixed

    Haiku 5.5, Bugs fixed: 16 (scale 0.0 to 105.0)16
    Fix rate
    15.2%
    False-positive fixes
    0
    Wall time
    19.3 min
    Cost per run
    $3.56
  108. 104

    Claude Code 2.1.243

    1 seed. Based on one run, so no interval can be computed.

    Bugs fixed

    Ox Alpha (stealth), Bugs fixed: 16 (scale 0.0 to 105.0), single run, no repeated seeds16(1 seed)
    Fix rate
    15.2%
    False-positive fixes
    0
    Wall time
    59.4 min
    Cost per run
    $0.00
  109. 109

    low · Grok Build CLI (ACP)

    Caveat

    Bugs fixed

    Grok 4.7, Bugs fixed: 15.7 (scale 0.0 to 105.0)15.7
    Fix rate
    15%
    False-positive fixes
    0
    Wall time
    22.2 min
    Cost per run
    $9.36
  110. 110
    OpenAIDossier

    medium · Codex CLI

    1 seed. Based on one run, so no interval can be computed.

    Bugs fixed

    GPT-5.6 Terra, Bugs fixed: 15 (scale 0.0 to 105.0), single run, no repeated seeds15(1 seed)
    Fix rate
    14.3%
    False-positive fixes
    0
    Wall time
    20.2 min
    Cost per run
    $3.87
  111. 110

    low · Grok Build CLI (ACP) 1.0.5

    1 seed. Based on one run, so no interval can be computed.

    Bugs fixed

    Grok 4.6, Bugs fixed: 15 (scale 0.0 to 105.0), single run, no repeated seeds15(1 seed)
    Fix rate
    14.3%
    False-positive fixes
    0
    Wall time
    18 min
    Cost per run
    $4.14
  112. 110
    MistralDossier

    high · Claude Code 2.1.291

    Caveat

    Bugs fixed

    Mistral Large 4, Bugs fixed: 15 (scale 0.0 to 105.0)15
    Fix rate
    14.3%
    False-positive fixes
    0
    Wall time
    111.1 min
    Cost per run
    $11.95
  113. 110
    AnthropicDossier

    max · Claude Code 2.1.267

    Caveat1 seed. Based on one run, so no interval can be computed.

    Bugs fixed

    Opus 4.8, Bugs fixed: 15 (scale 0.0 to 105.0), single run, no repeated seeds15(1 seed)
    Fix rate
    14.3%
    False-positive fixes
    0
    Wall time
    108.8 min
    Cost per run
    $58.11
  114. 110
    AlibabaDossier

    xhigh · Claude Code 2.1.268 / 2.1.272

    Caveat

    Bugs fixed

    Qwen3.8-27B, Bugs fixed: 15 (scale 0.0 to 105.0)15
    Fix rate
    14.3%
    False-positive fixes
    0
    Wall time
    56.8 min
    Cost per run
    $5.55
  115. 115
    DeepSeekDossier

    Claude Code 2.1.220

    1 seed. Based on one run, so no interval can be computed.

    Bugs fixed

    DeepSeek V4-Flash 0731, Bugs fixed: 14 (scale 0.0 to 105.0), single run, no repeated seeds14(1 seed)
    Fix rate
    13.3%
    False-positive fixes
    0
    Wall time
    48.5 min
    Cost per run
    $1.52
  116. 115
    OpenAIDossier

    xhigh · Codex CLI 0.155.1

    Caveat1 seed. Based on one run, so no interval can be computed.

    Bugs fixed

    GPT-6 Luna, Bugs fixed: 14 (scale 0.0 to 105.0), single run, no repeated seeds14(1 seed)
    Fix rate
    13.3%
    False-positive fixes
    0
    Wall time
    57 min
    Cost per run
    $0.39
  117. 115
    OpenAIDossier

    medium · Codex CLI 0.155.1

    Caveat1 seed. Based on one run, so no interval can be computed.

    Bugs fixed

    GPT-6 Sol, Bugs fixed: 14 (scale 0.0 to 105.0), single run, no repeated seeds14(1 seed)
    Fix rate
    13.3%
    False-positive fixes
    0
    Wall time
    24.5 min
    Cost per run
    $2.39
  118. 118
    DeepSeekDossier

    high · Claude Code 2.1.267

    Caveat1 seed. Based on one run, so no interval can be computed.

    Bugs fixed

    DeepSeek V4-Pro, Bugs fixed: 13 (scale 0.0 to 105.0), single run, no repeated seeds13(1 seed)
    Fix rate
    12.4%
    False-positive fixes
    0
    Wall time
    26.6 min
    Cost per run
    $1.08
  119. 118

    Claude Code 2.1.247

    1 seed. Based on one run, so no interval can be computed.

    Bugs fixed

    GLM-5.3 Flash, Bugs fixed: 13 (scale 0.0 to 105.0), single run, no repeated seeds13(1 seed)
    Fix rate
    12.4%
    False-positive fixes
    0
    Wall time
    57.1 min
    Cost per run
    $0.79
  120. 118
    OpenAIDossier

    high · Codex CLI

    1 seed. Based on one run, so no interval can be computed.

    Bugs fixed

    GPT-5.6 Luna, Bugs fixed: 13 (scale 0.0 to 105.0), single run, no repeated seeds13(1 seed)
    Fix rate
    12.4%
    False-positive fixes
    0
    Wall time
    64.1 min
    Cost per run
    $0.57
  121. 118

    medium · Muse Code 1.2.1

    Caveat

    Bugs fixed

    Muse Spark 1.3, Bugs fixed: 13 (scale 0.0 to 105.0)13
    Fix rate
    12.4%
    False-positive fixes
    0
    Wall time
    58.3 min
    Cost per run
    $9.24
  122. 122
    AnthropicDossier

    medium · Claude Code 2.1.292

    Caveat1 seed. Based on one run, so no interval can be computed.

    Bugs fixed

    Haiku 5.5, Bugs fixed: 12 (scale 0.0 to 105.0), single run, no repeated seeds12(1 seed)
    Fix rate
    11.4%
    False-positive fixes
    0
    Wall time
    15.5 min
    Cost per run
    $2.66
  123. 123
    OpenAIDossier

    xhigh · Codex CLI 0.159.0

    Caveat

    Bugs fixed

    GPT-6.1 Terra, Bugs fixed: 11.3 (scale 0.0 to 105.0)11.3
    Fix rate
    10.8%
    False-positive fixes
    0
    Wall time
    15.9 min
    Cost per run
    $0.75
  124. 124
    AnthropicDossier

    low · Claude Code 2.1.292

    Caveat1 seed. Based on one run, so no interval can be computed.

    Bugs fixed

    Haiku 5.5, Bugs fixed: 11 (scale 0.0 to 105.0), single run, no repeated seeds11(1 seed)
    Fix rate
    10.5%
    False-positive fixes
    0
    Wall time
    8.7 min
    Cost per run
    $1.61
  125. 125
    AlibabaDossier

    8-bit · Claude Code 2.1.272

    Caveat

    Bugs fixed

    Qwen3.8-27B, Bugs fixed: 10.7 (scale 0.0 to 105.0)10.7
    Fix rate
    10.2%
    False-positive fixes
    0
    Wall time
    56.6 min
    Cost per run
    $2.97
  126. 126
    GoogleDossier

    low · Antigravity CLI 1.5.8

    Caveat

    Bugs fixed

    Gemini 4 Hydrogen, Bugs fixed: 10.3 (scale 0.0 to 105.0)10.3
    Fix rate
    9.8%
    False-positive fixes
    0
    Wall time
    7.2 min
    Cost per run
    $0.43
  127. 127
    OpenAIDossier

    high · Codex CLI 0.159.0

    Caveat

    Bugs fixed

    GPT-6.1 Terra, Bugs fixed: 9.7 (scale 0.0 to 105.0)9.7
    Fix rate
    9.2%
    False-positive fixes
    0
    Wall time
    12.6 min
    Cost per run
    $0.56
  128. 127

    low · Muse Code 1.2.1

    Caveat

    Bugs fixed

    Muse Spark 1.3, Bugs fixed: 9.7 (scale 0.0 to 105.0)9.7
    Fix rate
    9.2%
    False-positive fixes
    0
    Wall time
    36.4 min
    Cost per run
    $5.61
  129. 129

    low · Claude Code 2.1.269

    Caveat1 seed. Based on one run, so no interval can be computed.

    Bugs fixed

    GLM-5.3 Flash, Bugs fixed: 9 (scale 0.0 to 105.0), single run, no repeated seeds9(1 seed)
    Fix rate
    8.6%
    False-positive fixes
    0
    Wall time
    29 min
    Cost per run
    $0.42
  130. 129
    OpenAIDossier

    medium · Codex CLI 0.150.0-alpha.8

    1 seed. Based on one run, so no interval can be computed.

    Bugs fixed

    GPT-5.6 Luna, Bugs fixed: 9 (scale 0.0 to 105.0), single run, no repeated seeds9(1 seed)
    Fix rate
    8.6%
    False-positive fixes
    0
    Wall time
    15.4 min
    Cost per run
    $0.33
  131. 129
    OpenAIDossier

    high · Codex CLI 0.155.1

    Caveat1 seed. Based on one run, so no interval can be computed.

    Bugs fixed

    GPT-6 Luna, Bugs fixed: 9 (scale 0.0 to 105.0), single run, no repeated seeds9(1 seed)
    Fix rate
    8.6%
    False-positive fixes
    0
    Wall time
    24.4 min
    Cost per run
    $0.13
  132. 129
    AnthropicDossier

    high · Claude Code 2.1.215 / 2.1.218

    1 seed. Based on one run, so no interval can be computed.

    Bugs fixed

    Opus 4.8, Bugs fixed: 9 (scale 0.0 to 105.0), single run, no repeated seeds9(1 seed)
    Fix rate
    8.6%
    False-positive fixes
    0
    Wall time
    34.8 min
    Cost per run
    $21.30
  133. 129
    AnthropicDossier

    high · Claude Code 2.1.218

    1 seed. Based on one run, so no interval can be computed.

    Bugs fixed

    Sonnet 5, Bugs fixed: 9 (scale 0.0 to 105.0), single run, no repeated seeds9(1 seed)
    Fix rate
    8.6%
    False-positive fixes
    0
    Wall time
    32.8 min
    Cost per run
    $11.38
  134. 129
    AnthropicDossier

    max · Claude Code 2.1.267

    Caveat1 seed. Based on one run, so no interval can be computed.

    Bugs fixed

    Sonnet 5, Bugs fixed: 9 (scale 0.0 to 105.0), single run, no repeated seeds9(1 seed)
    Fix rate
    8.6%
    False-positive fixes
    0
    Wall time
    61.3 min
    Cost per run
    $17.96
  135. 135
    OpenAIDossier

    medium · Codex CLI 0.159.0

    Caveat

    Bugs fixed

    GPT-6.1 Terra, Bugs fixed: 7.3 (scale 0.0 to 105.0)7.3
    Fix rate
    7%
    False-positive fixes
    0
    Wall time
    9.6 min
    Cost per run
    $0.31
  136. 136
    OpenAIDossier

    low · Codex CLI 0.155.1

    Caveat1 seed. Based on one run, so no interval can be computed.

    Bugs fixed

    GPT-6 Sol, Bugs fixed: 6 (scale 0.0 to 105.0), single run, no repeated seeds6(1 seed)
    Fix rate
    5.7%
    False-positive fixes
    0
    Wall time
    11.1 min
    Cost per run
    $1.08
  137. 136
    OpenAIDossier

    low · Codex CLI 0.159.0

    Caveat

    Bugs fixed

    GPT-6.1 Terra, Bugs fixed: 6 (scale 0.0 to 105.0)6
    Fix rate
    5.7%
    False-positive fixes
    0
    Wall time
    10.1 min
    Cost per run
    $0.30
  138. 138
    GoogleDossier

    Claude Code 2.1.272

    Caveat1 seed. Based on one run, so no interval can be computed.

    Bugs fixed

    Gemma 4 31B, Bugs fixed: 4 (scale 0.0 to 105.0), single run, no repeated seeds4(1 seed)
    Fix rate
    3.8%
    False-positive fixes
    0
    Wall time
    221.8 min
    Cost per run
    $8.61
  139. 138
    OpenAIDossier

    low · Codex CLI 0.150.0-alpha.8

    1 seed. Based on one run, so no interval can be computed.

    Bugs fixed

    GPT-5.6 Luna, Bugs fixed: 4 (scale 0.0 to 105.0), single run, no repeated seeds4(1 seed)
    Fix rate
    3.8%
    False-positive fixes
    0
    Wall time
    5.8 min
    Cost per run
    $0.10
  140. 138
    OpenAIDossier

    medium · Codex CLI 0.155.1

    Caveat1 seed. Based on one run, so no interval can be computed.

    Bugs fixed

    GPT-6 Luna, Bugs fixed: 4 (scale 0.0 to 105.0), single run, no repeated seeds4(1 seed)
    Fix rate
    3.8%
    False-positive fixes
    0
    Wall time
    8.6 min
    Cost per run
    $0.060
  141. 138
    OpenAIDossier

    low · Codex CLI 0.155.1

    Caveat1 seed. Based on one run, so no interval can be computed.

    Bugs fixed

    GPT-6 Luna, Bugs fixed: 4 (scale 0.0 to 105.0), single run, no repeated seeds4(1 seed)
    Fix rate
    3.8%
    False-positive fixes
    0
    Wall time
    19.3 min
    Cost per run
    $0.20
  142. 142
    GoogleDossier

    4-bit · Claude Code 2.1.272

    Caveat1 seed. Based on one run, so no interval can be computed.

    Bugs fixed

    Gemma 4 31B, Bugs fixed: 3 (scale 0.0 to 105.0), single run, no repeated seeds3(1 seed)
    Fix rate
    2.9%
    False-positive fixes
    0
    Wall time
    24.9 min
    Cost per run
    $0.56
  143. 143
    AnthropicDossier

    Claude Code 2.1.292

    Caveat

    Bugs fixed

    Haiku 4.5, Bugs fixed: 1.5 (scale 0.0 to 105.0)1.5
    Fix rate
    1.4%
    False-positive fixes
    0
    Wall time
    16.9 min
    Cost per run
    $2.22
  144. 144
    OpenAIDossier

    Claude Code 2.1.269 / 2.1.272

    Caveat

    Bugs fixed

    gpt-oss-20b, Bugs fixed: 0 (scale 0.0 to 105.0)0
    Fix rate
    0%
    False-positive fixes
    0
    Wall time
    7.4 min
    Cost per run
    $0.11
  145. 144
    OpenAIDossier

    Claude Code 2.1.269 / 2.1.272

    Caveat

    Bugs fixed

    gpt-oss-120b, Bugs fixed: 0 (scale 0.0 to 105.0)0
    Fix rate
    0%
    False-positive fixes
    0
    Wall time
    3.7 min
    Cost per run
    $0.11

Per-bug fix matrix

Individual bug IDs resolved per run across 82 distinct bugs observed in this snapshot.

Least frequently solved bugs:
ModelVariantFixedFixed bug IDs
anthropic/claude-fable-5-2max · Claude Code58
#13#16#17#20#21#22#23#24#25#26#27#28#29#30#32#33#34#35#36#38#39#40#41#42+43 more
google/gemini-4-carbonhigh · Antigravity CLI57.3
#13#16#20#21#22#23#24#25#26#27#28#29#30#32#33#34#37#38#39#40#42#43#44#45+43 more
google/gemini-4-carbonmedium · Antigravity CLI56
#13#16#20#21#22#23#24#25#26#27#28#29#30#32#33#34#37#38#39#40#43#44#45#46+40 more
anthropic/claude-sonnet-5-5max · Claude Code51.3
#13#16#17#20#21#22#23#25#26#27#28#29#30#32#33#34#36#37#38#39#40#42#43#44+40 more
anthropic/claude-fable-5-2xhigh · Claude Code49
#13#16#17#20#21#22#24#25#26#27#28#29#30#32#33#34#35#36#38#39#40#41#42#43+36 more
google/gemini-4-carbonlow · Antigravity CLI46
#13#16#20#21#22#23#25#26#27#28#29#30#32#33#34#37#38#39#40#43#44#45#46#47+33 more
openai/gpt-6-astramax · Codex CLI45
#13#16#20#21#22#23#25#26#27#28#29#30#33#34#35#36#37#39#40#41#42#43#44#45+31 more
openai/gpt-6.1-solmax · Codex CLI44.3
#13#16#17#20#21#22#23#25#26#27#28#29#30#32#33#34#35#36#37#39#40#41#42#43+30 more
openai/gpt-5.6-solmax · Codex CLI43.5
#13#16#17#20#21#22#25#26#27#28#29#30#33#34#35#38#39#40#41#43#44#45#46#47+28 more
openai/gpt-6-astraxhigh · Codex CLI43
#13#16#20#21#22#23#25#26#27#28#29#30#33#34#35#36#37#39#40#41#43#44#45#47+19 more
anthropic/claude-fable-5-2high · Claude Code43
#13#16#20#22#25#26#27#28#29#30#33#34#35#36#38#39#40#41#42#43#44#45#46#47+24 more
anthropic/claude-fable-5-1max · Claude Code43
#13#16#20#22#23#25#26#27#29#30#32#33#34#36#39#40#41#43#45#47#49#50#59#60+19 more
deepseek/deepseek-v4.1-flash-1017max · DeepSeek Harness42.7
#13#16#17#20#21#22#23#25#26#27#28#29#30#32#33#34#37#38#39#40#41#42#43#44+36 more
openai/gpt-6.1-solxhigh · Codex CLI42.7
#13#16#17#20#21#22#23#25#26#27#28#29#30#32#33#34#35#36#37#39#40#41#43#44+28 more
deepseek/deepseek-v4.1-flash-1017max · Pi42.2
#13#16#17#20#21#22#23#25#26#27#28#29#30#33#34#36#37#39#40#41#42#43#44#45+33 more