When will SWE-bench be solved? | Manifold

When will SWE-bench be solved?

12

Ṁ100Ṁ892

2030

0.9%

2024

1.8%

2025

36%

2026

58%

2027

1.4%

2028

1.4%

2029

0.7%

later

Resolves to the first year in which https://www.swebench.com posts a 90% resolved score in the “verified” category. The current score is 38.80% by Amazon Q Developer Agent.

Technical AI Timelines

Get

1,000

to start trading!

Sort by:

bought Ṁ5 YES

Relatively significant chance of never (which resolves at later). Frontier LLMs are just going to get more and more contaminated with this benchmark's test data - at some point authors might just stop tracking

People are also trading

Top SWE-Bench Pro score by Jan 1, 2027?

What will be the highest score on the SWE-bench pro private set before 2027?

Will DeepSWE be ≥90% solved at <$5/task before 2027?

AI resolves at least X% on SWE-bench without any assistance, by 2028?

AI resolves at least X% on SWE-bench WITH assistance, by 2028?

Will Claude Sonnet 5 exceed 85% on SWE-bench verified?

Will an autonomous agent resolve 90% of tasks on SWE-bench by 2027?

In what year will AI achieve a score of 95% or higher on the SWE-bench Verified benchmark?

Will DeepSWE be saturated before 2027 (≥90%)?

In what year will go be solved?

Related questions

Top SWE-Bench Pro score by Jan 1, 2027?

What will be the highest score on the SWE-bench pro private set before 2027?

Will DeepSWE be ≥90% solved at <$5/task before 2027?

AI resolves at least X% on SWE-bench without any assistance, by 2028?

AI resolves at least X% on SWE-bench WITH assistance, by 2028?

Will Claude Sonnet 5 exceed 85% on SWE-bench verified?

Will an autonomous agent resolve 90% of tasks on SWE-bench by 2027?

In what year will AI achieve a score of 95% or higher on the SWE-bench Verified benchmark?

Will DeepSWE be saturated before 2027 (≥90%)?

In what year will go be solved?