Did OpenAI use MUP for zero shot hyper-parameter transfer in GPT-4?
Basic
5
Ṁ1692025
81%
chance
1D
1W
1M
ALL
Maximal Update Parameterization is technique published last year by Yang et al. at Microsoft. https://arxiv.org/abs/2203.03466
This question is managed and resolved by Manifold.
Get
1,000and
3.00
1,0003.00Sort by:
@firstuserhere interesting that it is in the bibliography, although the reference in the first image is from a different section of the report with its own bibliography (that [16] actually refers to "DALL·E 2 Preview - Risks and Limitations.").
So the muP paper is in the bibliography, but not referenced anywhere.
@Stefan yep, and even then it's not actually used in gpt-4, the report only mentions the red team to have used the paper?
Related questions
Related questions
Will OpenAI release the weights of GPT-3? (2024)
3% chance
Will OpenAI release GTP-4.5 before GPT-5?
58% chance
Will OpenAI change their naming scheme (GPT-X) with the successor to GPT-4? (Ṁ200 subsidy!)
14% chance
Has openAI intentionally made chatGPT lazy to save inference costs?
21% chance
Will the next LLM released by OpenAI be worse than GPT-4 at MMLU?
16% chance
Did OpenAI intentionally handicap GPT4's image modality's ability to identify people?
83% chance
Will OpenAI's autonomous agent be based on GPT-4?
19% chance
Will OpenAI suggest GPT-4 is AGI?
4% chance
Will OpenAI provide access to GPT-4 weights to academic researchers not affiliated with OpenAI, by 2025?
14% chance
Will OpenAI open source the weights to one of the GPT family models in 2024?
3% chance