Short-Term AI #4: By the end of June 2023, will SOTA on miniF2F pass@64 be >=55%
9
33
Ṁ437Ṁ210
resolved Jul 1
Resolved
NO1D
1W
1M
ALL
Current SOTA is ~41%: https://paperswithcode.com/sota/automated-theorem-proving-on-minif2f-test?metric=Pass%4064
If pass@k for k<64 beats 55% that also counts.
Nov 23, 8:54am: Short-Term AI #1: By the end of June 2023, will SOTA on miniF2F pass@64 be >=55% → Short-Term AI #4: By the end of June 2023, will SOTA on miniF2F pass@64 be >=55%
Get Ṁ200 play money
🏅 Top traders
| # | Name | Total profit |
|---|---|---|
| 1 | Ṁ272 | |
| 2 | Ṁ17 |
Related questions
Short-term AI 3.4: By June 2024 will SOTA on APPS be >= 25%?
25% chance
Short Term AI 3.2: By June 2024 will SOTA on MATH be >= 90%?
14% chance
SOTA on a SWE-bench [Assisted] in October 2024
BIG-bench accuracy 75% #2: Will SOTA for a single model on BIG-bench pass 75% by the start of 2025?
60% chance
Short-term AI 3.3: By June 2024 will SOTA on HumanEval be >= 99%?
5% chance
MMLU 99% #3: Will SOTA for MMLU (average) pass 99% by the start of 2026?
16% chance
By 2026, will it be standard practice to sandbox SOTA LLMs?
26% chance
MMLU 99% #2: Will SOTA for MMLU (average) pass 99% by the start of 2025?
15% chance
For typical SOTA AI systems in 2028, will it be possible for users to know the true reasons for systems making a choice?
POLL
Will a transformer based model be SOTA for video generation by the end of 2025?
77% chance