When will SWE-bench be solved?
Basic
4
Ṁ202030
8%
2024
30%
2025
18%
2026
8%
2027
8%
2028
8%
2029
19%
later
Resolves to the first year in which https://www.swebench.com posts a 90% resolved score in the “verified” category. The current score is 38.80% by Amazon Q Developer Agent.
This question is managed and resolved by Manifold.
Get
1,000
and3.00
Related questions
Related questions
80% on SWE-Bench Verified by Jan 1 2025
39% chance
What will be the best score on the SWE-Bench (unassisted) benchmark before 2025?
39% chance
AI resolves at least X% on SWE-bench assistance, by 2025?
AI resolves at least X% on SWE-bench WITH assistance, by 2028?
Will an AI SWE model score higher than 50% on SWE-bench in 2024?
20% chance
AI resolves at least X% on SWE-bench without any assistance, by 2028?
Will an autonomous agent resolve 90% of tasks on SWE-bench by 2025?
18% chance
Will an autonomous agent resolve 90% of tasks on SWE-bench by 2026?
70% chance
Will an autonomous agent resolve 90% of tasks on SWE-bench by 2027?
72% chance