MMLU 99% #5: Will SOTA for MMLU (average) pass 99% by the start of 2028? | Manifold

MMLU 99% #5: Will SOTA for MMLU (average) pass 99% by the start of 2028?

5

Ṁ150Ṁ129

2028

44%

chance

1H

6H

1D

1W

1M

ALL

Market context

Technical AI Timelines

Get

1,000

to start trading!

Sort by:

Aren't there errors in this dataset?

@MalachiteEagle Yeah this might actually be impossible, I'm not sure what the error rate is. (I didn't know this when I created the market.)

People are also trading

MMLU 99% #4: Will SOTA for MMLU (average) pass 99% by the start of 2027?

BIG-bench accuracy 75% #5: Will SOTA for a single model on BIG-bench pass 75% by the start of 2028?

BIG-bench accuracy 75% #4: Will SOTA for a single model on BIG-bench pass 75% by the start of 2027?

What will be true of the SOTA AI on the FrontierMath benchmark, before 2027?

Will a text model achieve 100% performance on the MMLU in five years?

Will the transformer architecture be replaced in SOTA LLMs by 2028?

Related questions

MMLU 99% #4: Will SOTA for MMLU (average) pass 99% by the start of 2027?

BIG-bench accuracy 75% #5: Will SOTA for a single model on BIG-bench pass 75% by the start of 2028?

BIG-bench accuracy 75% #4: Will SOTA for a single model on BIG-bench pass 75% by the start of 2027?

What will be true of the SOTA AI on the FrontierMath benchmark, before 2027?

Will a text model achieve 100% performance on the MMLU in five years?

Will the transformer architecture be replaced in SOTA LLMs by 2028?