When will SWE-bench be solved?
Basic
5
Ṁ282030
7%
2024
32%
2025
26%
2026
7%
2027
7%
2028
7%
2029
15%
later
Resolves to the first year in which https://www.swebench.com posts a 90% resolved score in the “verified” category. The current score is 38.80% by Amazon Q Developer Agent.
This question is managed and resolved by Manifold.
Get
1,000
and3.00
Related questions
Related questions
What will be the best performance on SWE-bench Verified by December 31st 2025?
What will be the best score on the SWE-Bench (unassisted) benchmark before 2025?
54% chance
80% on SWE-Bench Verified by Jan 1 2025
6% chance
Will an AI SWE model score higher than 50% on SWE-bench in 2024?
20% chance
AI resolves at least X% on SWE-bench WITH assistance, by 2028?
AI resolves at least X% on SWE-bench assistance, by 2025?
What will be the highest score achieved on SWE-Bench Verified in 2025?
AI resolves at least X% on SWE-bench without any assistance, by 2028?
Codebuff solves at least 40% of issues on SWE-Bench by March 31, 2025
16% chance
Will an autonomous agent resolve 90% of tasks on SWE-bench by 2025?
11% chance