Will an AI SWE model score higher than 50% on SWE-bench in 2024?
14
33
Ṁ400Ṁ300
Dec 31
17%
chance
1D
1W
1M
ALL
Get Ṁ600 play money
Related questions
AI resolves at least X% on SWE-bench WITH assistance, by 2028?
AI resolves at least X% on SWE-bench without any assistance, by 2028?
Will an AI be capable of achieving a perfect score on the Putnam exam before 2030?
33% chance
Will an AI model outperform 95% of Manifold users on accuracy before 2026?
67% chance
AI resolves at least X% on SWE-bench assistance, by 2025?
Will xAI release an LLM with BIG-Bench score as good as GPT-4 before the end of 2024?
54% chance
Will an AI get a perfect SAT score before 2025?
81% chance
Will an AI be capable of achieving a perfect score on the Putnam exam before 2028?
20% chance
Will an AI achieve a perfect score on the Miklós Schweitzer Competition before 2035?
58% chance
Will OpenAI's next-gen math-focused model score at least 95% on the MATH benchmark?
71% chance