Short-Term AI #4: By the end of June 2023, will SOTA on miniF2F pass@64 be >=55%
9
210Ṁ437resolved Jul 1
Resolved
NO1H
6H
1D
1W
1M
ALL
Current SOTA is ~41%: https://paperswithcode.com/sota/automated-theorem-proving-on-minif2f-test?metric=Pass%4064
If pass@k for k<64 beats 55% that also counts.
Nov 23, 8:54am: Short-Term AI #1: By the end of June 2023, will SOTA on miniF2F pass@64 be >=55% → Short-Term AI #4: By the end of June 2023, will SOTA on miniF2F pass@64 be >=55%
This question is managed and resolved by Manifold.
Get 
1,000 to start trading!
1,000🏅 Top traders
| # | Name | Total profit |
|---|---|---|
| 1 | Ṁ272 | |
| 2 | Ṁ17 |
People are also trading
Related questions
BIG-bench accuracy 75% #3: Will SOTA for a single model on BIG-bench pass 75% by the start of 2026?
86% chance
BIG-bench accuracy 75% #4: Will SOTA for a single model on BIG-bench pass 75% by the start of 2027?
86% chance
What will be true of the SOTA AI on the FrontierMath benchmark, before 2026?
BIG-bench accuracy 75% #5: Will SOTA for a single model on BIG-bench pass 75% by the start of 2028?
87% chance
MMLU 99% #3: Will SOTA for MMLU (average) pass 99% by the start of 2026?
6% chance
What will be true of the SOTA AI on the FrontierMath benchmark, before 2028?
What will be true of the SOTA AI on the FrontierMath benchmark, before 2027?
MMLU 99% #4: Will SOTA for MMLU (average) pass 99% by the start of 2027?
8% chance
MMLU 99% #5: Will SOTA for MMLU (average) pass 99% by the start of 2028?
44% chance
[Carlini questions] SOTA AI scores better than X% of other participants in competitive programming contest by 2027
91.5