Will a LLM beat human experts on GPQA by Jan 1, 2025?
57
1kแน42kresolved Dec 20
Resolved
YES1H
6H
1D
1W
1M
ALL
GQPA dataset here: https://arxiv.org/abs/2311.12022
"Human expert" means 74%.
Currently, GPT-4 gets 39%.
The LLM is allowed to use external tools (e.g. Google, Wolfram Alpha).
This question is managed and resolved by Manifold.
Get 
1,000 to start trading!
1,000๐ Top traders
| # | Name | Total profit |
|---|---|---|
| 1 | แน1,198 | |
| 2 | แน880 | |
| 3 | แน731 | |
| 4 | แน621 | |
| 5 | แน605 |
People are also trading
Related questions
Will an LLM beat a Super GM Bot on chess.com by 2028?
51% chance
Will there be an LLM which scores above what a human can do in 2 hours on METR's eval suite before 2026?
67% chance
LLM Hallucination: Will an LLM score >90% on SimpleQA before 2026?
60% chance
What organization will top the LLM leaderboards on LMArena at end of 2025? ๐ค๐
Will an LLM (a GPT-like text AI) defeat the World Champion at Chess before 2035?
72% chance
Will the best public LLM at the end of 2025 solve more than 5 of the first 10 Project Euler problems published in 2026?
75% chance
Will there be an LLM (as good as GPT-4) that was trained with 1/100th the energy consumed to train GPT-4, by 2026?
83% chance
Will an LLM improve its own ability along some important metric well beyond the best trained LLMs before 2026?
50% chance
Will the most interesting AI in 2027 be a LLM?
70% chance
Will there be any simple text-based task that most humans can solve, but top LLMs can't? By the end of 2026
64% chance