When will SWE-bench be solved? | Manifold

When will SWE-bench be solved?

11

Ṁ100Ṁ594

2030

0.6%

2024

0.6%

2025

88%

2026

3%

2027

3%

2028

3%

2029

1.3%

later

Resolves to the first year in which https://www.swebench.com posts a 90% resolved score in the “verified” category. The current score is 38.80% by Amazon Q Developer Agent.

Market context

Technical AI Timelines

Get

1,000

to start trading!

Sort by:

bought Ṁ5 YES

Relatively significant chance of never (which resolves at later). Frontier LLMs are just going to get more and more contaminated with this benchmark's test data - at some point authors might just stop tracking

People are also trading

Top SWE-Bench Pro score by Jan 1, 2027?

Will an autonomous agent resolve 90% of tasks on SWE-bench by 2027?

+13% 1d87% chance

What will be the highest score on the SWE-bench pro private set before 2027?

Best SWE-Bench Pro public score by June 30, 2026

Will Claude Sonnet 5 exceed 85% on SWE-bench verified?

+5% 1d52% chance

AI resolves at least X% on SWE-bench without any assistance, by 2028?

AI resolves at least X% on SWE-bench WITH assistance, by 2028?

In what year will AI achieve a score of 95% or higher on the SWE-bench Verified benchmark?

In what year will go be solved?

When Will 9x9 Go Be Solved?

Related questions

Top SWE-Bench Pro score by Jan 1, 2027?

Will an autonomous agent resolve 90% of tasks on SWE-bench by 2027?

What will be the highest score on the SWE-bench pro private set before 2027?

Best SWE-Bench Pro public score by June 30, 2026

Will Claude Sonnet 5 exceed 85% on SWE-bench verified?

AI resolves at least X% on SWE-bench without any assistance, by 2028?

AI resolves at least X% on SWE-bench WITH assistance, by 2028?

In what year will AI achieve a score of 95% or higher on the SWE-bench Verified benchmark?

In what year will go be solved?

When Will 9x9 Go Be Solved?