![](/_next/image?url=https%3A%2F%2Fstorage.googleapis.com%2Fmantic-markets.appspot.com%2Fcontract-images%2FJacobPfau%2Fnq3xhvjlop.jpg&w=3840&q=75)
By what factor will the cost for SotA SWE-agents drop from 2024 to 2025?
Basic
9
Ṁ8502025
1D
1W
1M
ALL
5%
<2x
7%
<10x
13%
<50x
20%
<250x
54%
>=250x
Algorithmic progress can be measured by reduction in cost to achieve equivalent performance. SWE-bench-lite is a popular benchmark for measuring scaffolded-LLM SWE capabilities.
By what factor will the cost of SWE-bench-lite SoTA drop between mid 2024-2025? Mid-2024 SotA is 43% costing $2,700 (per the devs), so this question will resolve Yes on the answer which most tightly bounds the reduction in cost to achieve 43% on July 1, 2025.
E.g. if in June 2025, 43% on SWE-lite costs $500 then that'd be a 5.4x reduction and the question would resolve (2) "<10x".
Get Ṁ1,000 play money
Related questions
Related questions
Will AI be Recursively Self Improving by mid 2026?
24% chance
Will self-improving AI agents crush SOTA in a complex environment (e.g. AAA game, tool use, science) in next 12 months?
35% chance
Will AI resolve P vs NP by 2050?
28% chance
Will we reach "weak AGI" by the end of 2025?
29% chance
SoAI 23 3/10: Will Self-improving Al agents crush SOTA in a complex environment (e.g. AAA game, tool use, science)?
42% chance
Will open-source AI win (through 2025)?
35% chance
Will some U.S. software engineers be negatively affected financially due to AI by end of 2025?
71% chance
AI resolves at least X% on SWE-bench assistance, by 2025?
AI resolves at least X% on SWE-bench WITH assistance, by 2028?
Will the "OpenAI hint at or claim to have AGI before 2025 end" market go below 10% before 2024 ends?
44% chance