Short-Term AI #2: By the end of June 2023, will SOTA on MMLU overall be >=85%?

Technical AI Timelines•Vincent Luczkow's Short-Term AI Predictions - Early 2023

13

81

Ṁ9.2K

Ṁ270

resolved Mar 14

Resolved

YES

1D

1W

1M

ALL

Current SOTA is ~75%: https://paperswithcode.com/sota/multi-task-language-understanding-on-mmlu

Get Ṁ200 play money

Related questions

Short-term AI 3.4: By June 2024 will SOTA on APPS be >= 25%?

BIG-bench accuracy 75% #2: Will SOTA for a single model on BIG-bench pass 75% by the start of 2025?

Short-term AI 3.3: By June 2024 will SOTA on HumanEval be >= 99%?

By 2026, will it be standard practice to sandbox SOTA LLMs?

MMLU 99% #4: Will SOTA for MMLU (average) pass 99% by the start of 2027?

Short Term AI 3.2: By June 2024 will SOTA on MATH be >= 90%?

MMLU 99% #3: Will SOTA for MMLU (average) pass 99% by the start of 2026?

MMLU 99% #2: Will SOTA for MMLU (average) pass 99% by the start of 2025?

Will an LLM Built on a State Space Model Architecture Have Been SOTA at any Point before EOY 2027? [READ DESCRIPTION]

For typical SOTA AI systems in 2028, will it be possible for users to know the true reasons for systems making a choice?

🏅 Top traders

#	Name	Total profit
1		Ṁ53
2		Ṁ37
3		Ṁ36
4		Ṁ20
5		Ṁ19

Sort by:

bought Ṁ50 of YES

Yep, it's over.

sold Ṁ68 of NO

https://openai.com/research/gpt-4 it is over 85

bought Ṁ70 of YES

Gpt4 will be out by then and whatever deepmind comes up with so no idea.

Short-Term AI #2: By the end of June 2023, will SOTA on MMLU overall be >=85%?, 8k, beautiful, illustration, trending on art station, picture of the day, epic composition

Related in Technical AI Timelines

Will an AI get gold on any International Math Olympiad by 2025?

Will a large language model beat a super grandmaster playing chess by 2028?

See more Technical AI Timelines questions

Related in Vincent Luczkow's Short-Term AI Predictions - Early 2023

Short-Term AI #5: By the end of June 2023, will there be a usable AI assistant that interfaces with keyboard and mouse?

Short-Term AI #3: By the end of June 2023, will SOTA on MATH be >= 70%?

See more Vincent Luczkow's Short-Term AI Predictions - Early 2023 questions

More related questions

Short-term AI 3.4: By June 2024 will SOTA on APPS be >= 25%?

BIG-bench accuracy 75% #2: Will SOTA for a single model on BIG-bench pass 75% by the start of 2025?

Short-term AI 3.3: By June 2024 will SOTA on HumanEval be >= 99%?

By 2026, will it be standard practice to sandbox SOTA LLMs?

MMLU 99% #4: Will SOTA for MMLU (average) pass 99% by the start of 2027?

Short Term AI 3.2: By June 2024 will SOTA on MATH be >= 90%?

MMLU 99% #3: Will SOTA for MMLU (average) pass 99% by the start of 2026?

MMLU 99% #2: Will SOTA for MMLU (average) pass 99% by the start of 2025?

Will an LLM Built on a State Space Model Architecture Have Been SOTA at any Point before EOY 2027? [READ DESCRIPTION]

For typical SOTA AI systems in 2028, will it be possible for users to know the true reasons for systems making a choice?

Technical AI Timelines questions

Will an AI get gold on any International Math Olympiad by 2025?

Will a large language model beat a super grandmaster playing chess by 2028?

Vincent Luczkow's Short-Term AI Predictions - Early 2023 questions

Short-Term AI #5: By the end of June 2023, will there be a usable AI assistant that interfaces with keyboard and mouse?

Short-Term AI #3: By the end of June 2023, will SOTA on MATH be >= 70%?

Related questions

Short-term AI 3.4: By June 2024 will SOTA on APPS be >= 25%?

Short Term AI 3.2: By June 2024 will SOTA on MATH be >= 90%?

BIG-bench accuracy 75% #2: Will SOTA for a single model on BIG-bench pass 75% by the start of 2025?

MMLU 99% #3: Will SOTA for MMLU (average) pass 99% by the start of 2026?

Short-term AI 3.3: By June 2024 will SOTA on HumanEval be >= 99%?

MMLU 99% #2: Will SOTA for MMLU (average) pass 99% by the start of 2025?

By 2026, will it be standard practice to sandbox SOTA LLMs?

Will an LLM Built on a State Space Model Architecture Have Been SOTA at any Point before EOY 2027? [READ DESCRIPTION]

MMLU 99% #4: Will SOTA for MMLU (average) pass 99% by the start of 2027?

For typical SOTA AI systems in 2028, will it be possible for users to know the true reasons for systems making a choice?