Will second-order optimizers displace first-order optimizers for training LLMs by 2030?
24
1.7k
2029
38%
chance

Will something like Shampoo become the default instead of Adam?

Get Ṁ600 play money
Sort by:

Seems possible but <50%? Why do you think it's so high?