Is GPT-4's human exam performance mainly due to memorisation?
17
370Ṁ1012resolved Jul 21
Resolved
NO1H
6H
1D
1W
1M
ALL
Resolves positive if convincing analysis comes out that GPT-4 is not much better than GPT-3.5 on human exams that are known to not appear in the training data, and negative if convincing analysis comes out showing the opposite.
This question is managed and resolved by Manifold.
Get 
1,000 to start trading!
1,000🏅 Top traders
| # | Name | Total profit |
|---|---|---|
| 1 | Ṁ26 | |
| 2 | Ṁ25 | |
| 3 | Ṁ20 | |
| 4 | Ṁ13 | |
| 5 | Ṁ9 |
People are also trading
Related questions
Will GPT-5 be capable of achieving superhuman performance in at least one exam that is typically taken by humans?
91% chance
Will GPT-5 ace exams?
79% chance
Will I be impressed by GPT-5?
75% chance
Will GPT-5 be capable of some form of online learning?
27% chance
Was GPT-4 trained in 4 months or less?
59% chance
GPT-4 #5: Will GPT-4 be a dense model?
1% chance
What is the main reason behind GPT-4o speed improvement relative to GPT-4 base model?
Will GPT-4.5 score at least 100 in an IQ test?
64% chance
Will GPT-5 have "the ability ... to autonomously replicate and acquire resources" per an ARC-like eval?
Will the performance jump from GPT4->GPT5 be less than the one from GPT3->GPT4?
70% chance