Did OpenAI use MUP for zero shot hyper-parameter transfer in GPT-4?
5
110Ṁ169Dec 31
81%
chance
1H
6H
1D
1W
1M
ALL
Maximal Update Parameterization is technique published last year by Yang et al. at Microsoft. https://arxiv.org/abs/2203.03466
This question is managed and resolved by Manifold.
Get 
1,000 to start trading!
1,000Sort by:
@firstuserhere interesting that it is in the bibliography, although the reference in the first image is from a different section of the report with its own bibliography (that [16] actually refers to "DALL·E 2 Preview - Risks and Limitations.").
So the muP paper is in the bibliography, but not referenced anywhere.
@Stefan yep, and even then it's not actually used in gpt-4, the report only mentions the red team to have used the paper?
People are also trading
Related questions
Will OpenAI's autonomous agent be based on GPT-4?
19% chance
Did OpenAI transcribe Youtube videos to train a GPT model as claimed by NYT?
89% chance
Will OpenAI change their naming scheme (GPT-X) with the successor to GPT-4? (Ṁ200 subsidy!)
4% chance
Will OpenAI release true multimodal image generation for GPT-4.5 before 2026?
16% chance
Will the next LLM released by OpenAI be worse than GPT-4 at MMLU?
16% chance
Will there be evidence in 2025 that in April 2023, OpenAI had a GPT-4.5 or higher model?
15% chance
Will OpenAI abandon discrete GPT releases in favor of continuous updates?
44% chance
Will OpenAI's next major LLM (after GPT-4) surpass 70% accuracy on the GPQA benchmark?
75% chance
Will OpenAI provide access to GPT-4 weights to academic researchers not affiliated with OpenAI, by 2025?
8% chance