When will SWE-bench be solved? | Manifold

When will SWE-bench be solved?

8

100Ṁ420

2030

1%

2024

1%

2025

53%

2026

13%

2027

13%

2028

13%

2029

7%

later

Resolves to the first year in which https://www.swebench.com posts a 90% resolved score in the “verified” category. The current score is 38.80% by Amazon Q Developer Agent.

Market context

Technical AI Timelines

Get

1,000

to start trading!

Sort by:

bought Ṁ5 YES

Relatively significant chance of never (which resolves at later). Frontier LLMs are just going to get more and more contaminated with this benchmark's test data - at some point authors might just stop tracking

People are also trading

Will an autonomous agent resolve 90% of tasks on SWE-bench by 2027?

-9% 1d70% chance

Top SWE-Bench Pro score by Jan 1, 2027?

Best SWE-Bench Pro public score by June 30, 2026

AI resolves at least X% on SWE-bench without any assistance, by 2028?

AI resolves at least X% on SWE-bench WITH assistance, by 2028?

In what year will AI achieve a score of 95% or higher on the SWE-bench Verified benchmark?

In what year will go be solved?

When Will 9x9 Go Be Solved?

Related questions

Will an autonomous agent resolve 90% of tasks on SWE-bench by 2027?

Top SWE-Bench Pro score by Jan 1, 2027?

Best SWE-Bench Pro public score by June 30, 2026

AI resolves at least X% on SWE-bench without any assistance, by 2028?

AI resolves at least X% on SWE-bench WITH assistance, by 2028?

In what year will AI achieve a score of 95% or higher on the SWE-bench Verified benchmark?

In what year will go be solved?

When Will 9x9 Go Be Solved?

© Manifold Markets, Inc.•Terms•Privacy