Acropolis

SWE-Bench verified

SWE-bench Verified, run by Epoch AI.

Upstream source
CodingAgentic
Independent. Run by an operator independent of the model vendors.Epoch AI Benchmarking Hub by Epoch AIVersion 1Data as of 17 Nov 2026

No filters. Showing 35 of 35 rows.

  1. 1
    AnthropicDossier

    max

    Accuracy (best score across scorers)

    Opus 4.7, Accuracy (best score across scorers): 83.5% (scale 0.0 to 100.0), 95% confidence interval 80.2 to 86.883.5%(80.2–86.8)
    Mean score
    83.5%
  2. 2
    OpenAIDossier

    xhigh

    Tied #1

    Accuracy (best score across scorers)

    GPT-5.5 Pre-release, Accuracy (best score across scorers): 80.6% (scale 0.0 to 100.0), 95% confidence interval 77.1 to 84.180.6%(77.1–84.1)
    Mean score
    80.6%
  3. 3
    GoogleDossier

    high

    Tied #1

    Accuracy (best score across scorers)

    Gemini 3.5 Flash, Accuracy (best score across scorers): 79.3% (scale 0.0 to 100.0), 95% confidence interval 75.7 to 82.979.3%(75.7–82.9)
    Mean score
    79.3%
  4. 4
    AnthropicDossier
    Tied #1

    Accuracy (best score across scorers)

    Opus 4.6, Accuracy (best score across scorers): 78.7% (scale 0.0 to 100.0), 95% confidence interval 75.1 to 82.478.7%(75.1–82.4)
    Mean score
    78.7%
  5. 5

    max

    Tied #1

    Accuracy (best score across scorers)

    GLM-5.2, Accuracy (best score across scorers): 78.7% (scale 0.0 to 100.0), 95% confidence interval 75.0 to 82.478.7%(75.0–82.4)
    Mean score
    78.7%
  6. 6
    DeepSeekDossier

    max

    Accuracy (best score across scorers)

    DeepSeek V4-Pro, Accuracy (best score across scorers): 77.6% (scale 0.0 to 100.0), 95% confidence interval 73.9 to 81.477.6%(73.9–81.4)
    Mean score
    77.6%
  7. 7
    AlibabaDossier

    Accuracy (best score across scorers)

    Qwen3.7 Max, Accuracy (best score across scorers): 77.3% (scale 0.0 to 100.0), 95% confidence interval 73.5 to 81.077.3%(73.5–81.0)
    Mean score
    77.3%
  8. 8
    OpenAIDossier

    high

    Accuracy (best score across scorers)

    GPT-5.4, Accuracy (best score across scorers): 76.9% (scale 0.0 to 100.0), 95% confidence interval 73.1 to 80.676.9%(73.1–80.6)
    Mean score
    76.9%
  9. 9
    Moonshot AIDossier

    Accuracy (best score across scorers)

    Kimi K2.6, Accuracy (best score across scorers): 76.7% (scale 0.0 to 100.0), 95% confidence interval 72.9 to 80.476.7%(72.9–80.4)
    Mean score
    76.7%
  10. 9
    AnthropicDossier

    Accuracy (best score across scorers)

    Opus 4.5, Accuracy (best score across scorers): 76.7% (scale 0.0 to 100.0), 95% confidence interval 72.9 to 80.476.7%(72.9–80.4)
    Mean score
    76.7%
  11. 9
    AlibabaDossier

    Accuracy (best score across scorers)

    Qwen3.6 Max Preview, Accuracy (best score across scorers): 76.7% (scale 0.0 to 100.0), 95% confidence interval 72.9 to 80.476.7%(72.9–80.4)
    Mean score
    76.7%
  12. 12
    GoogleDossier

    Accuracy (best score across scorers)

    Gemini 3.1 Pro Preview (Custom Tools), Accuracy (best score across scorers): 75.6% (scale 0.0 to 100.0), 95% confidence interval 71.8 to 79.475.6%(71.8–79.4)
    Mean score
    75.6%
  13. 12
    AnthropicDossier

    Accuracy (best score across scorers)

    Opus 4.6, Accuracy (best score across scorers): 75.6% (scale 0.0 to 100.0), 95% confidence interval 71.8 to 79.475.6%(71.8–79.4)
    Mean score
    75.6%
  14. 14
    GoogleDossier

    Accuracy (best score across scorers)

    Gemini 3 Flash Preview, Accuracy (best score across scorers): 75.4% (scale 0.0 to 100.0), 95% confidence interval 71.6 to 79.375.4%(71.6–79.3)
    Mean score
    75.4%
  15. 15
    AnthropicDossier

    Accuracy (best score across scorers)

    Sonnet 4.6, Accuracy (best score across scorers): 75.2% (scale 0.0 to 100.0), 95% confidence interval 71.4 to 79.175.2%(71.4–79.1)
    Mean score
    75.2%
  16. 16
    OpenAIDossier

    high

    Accuracy (best score across scorers)

    GPT-5.3 Codex, Accuracy (best score across scorers): 74.8% (scale 0.0 to 100.0), 95% confidence interval 70.9 to 78.774.8%(70.9–78.7)
    Mean score
    74.8%
  17. 17

    Accuracy (best score across scorers)

    GLM-5.1, Accuracy (best score across scorers): 74.2% (scale 0.0 to 100.0), 95% confidence interval 70.3 to 78.174.2%(70.3–78.1)
    Mean score
    74.2%
  18. 18
    OpenAIDossier

    high

    Accuracy (best score across scorers)

    GPT-5.2, Accuracy (best score across scorers): 73.8% (scale 0.0 to 100.0), 95% confidence interval 69.8 to 77.773.8%(69.8–77.7)
    Mean score
    73.8%
  19. 18
    Moonshot AIDossier

    Accuracy (best score across scorers)

    Kimi K2.5, Accuracy (best score across scorers): 73.8% (scale 0.0 to 100.0), 95% confidence interval 69.8 to 77.773.8%(69.8–77.7)
    Mean score
    73.8%
  20. 20
    OpenAIDossier

    high

    Accuracy (best score across scorers)

    GPT-5, Accuracy (best score across scorers): 73.6% (scale 0.0 to 100.0), 95% confidence interval 69.6 to 77.573.6%(69.6–77.5)
    Mean score
    73.6%
  21. 21
    AnthropicDossier

    Accuracy (best score across scorers)

    Opus 4.1, Accuracy (best score across scorers): 73.3% (scale 0.0 to 100.0), 95% confidence interval 69.4 to 77.373.3%(69.4–77.3)
    Mean score
    73.3%
  22. 22
    GoogleDossier

    Accuracy (best score across scorers)

    Gemini 3 Pro Preview, Accuracy (best score across scorers): 72.9% (scale 0.0 to 100.0), 95% confidence interval 69.0 to 76.972.9%(69.0–76.9)
    Mean score
    72.9%
  23. 23

    Accuracy (best score across scorers)

    GLM-5, Accuracy (best score across scorers): 72.1% (scale 0.0 to 100.0), 95% confidence interval 68.0 to 76.272.1%(68.0–76.2)
    Mean score
    72.1%
  24. 24
    OpenAIDossier

    medium

    Accuracy (best score across scorers)

    GPT-5, Accuracy (best score across scorers): 71.5% (scale 0.0 to 100.0), 95% confidence interval 67.5 to 75.571.5%(67.5–75.5)
    Mean score
    71.5%
  25. 25
    AnthropicDossier

    Accuracy (best score across scorers)

    Sonnet 4.5, Accuracy (best score across scorers): 71.3% (scale 0.0 to 100.0), 95% confidence interval 67.2 to 75.371.3%(67.2–75.3)
    Mean score
    71.3%
  26. 26
    AnthropicDossier

    Accuracy (best score across scorers)

    Opus 4, Accuracy (best score across scorers): 70.7% (scale 0.0 to 100.0), 95% confidence interval 66.6 to 74.770.7%(66.6–74.7)
    Mean score
    70.7%
  27. 27
    OpenAIDossier

    high

    Accuracy (best score across scorers)

    GPT-5.1, Accuracy (best score across scorers): 68% (scale 0.0 to 100.0), 95% confidence interval 63.8 to 72.168%(63.8–72.1)
    Mean score
    68%
  28. 28
    OpenAIDossier

    high

    Accuracy (best score across scorers)

    GPT-5.1, Accuracy (best score across scorers): 65.9% (scale 0.0 to 100.0), 95% confidence interval 61.7 to 70.165.9%(61.7–70.1)
    Mean score
    65.9%
  29. 29
    OpenAIDossier

    medium

    Accuracy (best score across scorers)

    GPT-5 mini, Accuracy (best score across scorers): 64.7% (scale 0.0 to 100.0), 95% confidence interval 60.4 to 68.964.7%(60.4–68.9)
    Mean score
    64.7%
  30. 30
    OpenAIDossier

    medium

    Accuracy (best score across scorers)

    o3, Accuracy (best score across scorers): 62.3% (scale 0.0 to 100.0), 95% confidence interval 58.0 to 66.662.3%(58.0–66.6)
    Mean score
    62.3%
  31. 31
    AnthropicDossier

    Accuracy (best score across scorers)

    Claude 3.7 Sonnet, Accuracy (best score across scorers): 61% (scale 0.0 to 100.0), 95% confidence interval 56.6 to 65.361%(56.6–65.3)
    Mean score
    61%
  32. 32
    AlibabaDossier

    Accuracy (best score across scorers)

    Qwen3.6 Plus, Accuracy (best score across scorers): 57.9% (scale 0.0 to 100.0), 95% confidence interval 53.4 to 62.357.9%(53.4–62.3)
    Mean score
    57.9%
  33. 33
    GoogleDossier

    Accuracy (best score across scorers)

    Gemini 2.5 Pro, Accuracy (best score across scorers): 57.6% (scale 0.0 to 100.0), 95% confidence interval 53.1 to 62.057.6%(53.1–62.0)
    Mean score
    57.6%
  34. 34
    OpenAIDossier

    Accuracy (best score across scorers)

    GPT-4.1, Accuracy (best score across scorers): 48.5% (scale 0.0 to 100.0), 95% confidence interval 44.1 to 53.048.5%(44.1–53.0)
    Mean score
    48.5%
  35. 35
    OpenAIDossier

    Accuracy (best score across scorers)

    GPT-4o (Nov '24), Accuracy (best score across scorers): 31% (scale 0.0 to 100.0), 95% confidence interval 26.9 to 35.131%(26.9–35.1)
    Mean score
    31%