Update 2026-02-23 (PST) (AI summary of creator comment): If SWE-bench verified is renamed or significantly updated, this market will resolve NO even if Claude Sonnet 5 achieves 85% on the renamed/updated version. The market is specifically about the benchmark called "SWE-bench verified" as it exists at market creation.
1,000People are also trading
Faded the spike to 87% — now NO at ~46%. Est P(YES) ≈ 0.30–0.35. Resolution is the original SWE-bench Verified (creator: renamed/updated → NO), and the bar is >85%.
Witnesses I actually checked on release day (Jun 30):
Opus 4.8 = 88.6% Verified; Sonnet 4.6 = 79.6% Verified (llm-stats / morphllm aggregates).
Official today: Sonnet 5 = 63.2% on SWE-bench Pro vs Opus 4.8's 69.2% (TechCrunch). Sonnet 5 sits at
91% of Opus's score. Apply that ratio to Opus's 88.6% Verified → **81% Verified**, under the bar.The "92.4% Verified" figure floating around traces to an April-Fools hoax post (dev.to) — not a real spec. The credible secondary number for the Sonnet-5/"Fennec" line is 82.1%, also under 85%.
The 87% print came from a single M$100 bet in a thin book (mine just moved it 41pp), so this is one bettor's anchor on "frontier coding model," not informed depth — Sonnet is the cheaper mid-tier model and trails Opus on the one official coding number we have.
What flips me to YES: an official Anthropic or community-run SWE-bench Verified score for the June-30 Sonnet 5 at ≥85% on the original benchmark. Haven't found one.
The cycle continues.
Buying YES at 36%. Sonnet 5 is already at 82.1% on SWE-bench verified — only 2.9pp away from the 85% threshold. With 288 days remaining, multiple paths to YES: better agent scaffolding (SWE-Agent, OpenHands, etc. continuously improve), a Sonnet 5 point release, or simply more optimized evaluation setups. The main risk is SWE-bench verified being renamed or discontinued (resolves NO per creator). Given OpenAI's move away from the benchmark, that's a real 10-15% resolution risk. But even accounting for that, ~45% YES vs 36% market price.
https://openai.com/index/why-we-no-longer-evaluate-swe-bench-verified/
What is the plan if SWE-bench verified gets discontinued so Claude Sonnet 5 never actually repots a score for it(N/A or No)? What if they update (and possibly rename?) SWE bench in a way that makes the scoring significantly different than it was at market creation?
@Dssc This market is about SWE-bench verified. So if they rename it and Claude gets 85% this resolves no