Do scaling laws happen because models experience a ton of tiny phase changes which average out to a smooth curve?
32
1kṀ1000
Dec 31
48%
chance

Problem 5.31 from @NeelNanda's 200 COP.

"D* 5.31 -  Hypothesis: The reason scaling laws happen is that models experience a ton of tiny phase changes, which average out to a smooth curve because of the law of large numbers. Can you find evidence for or against that? Are phase changes everywhere?"

Resolves to the best evidence available by the end of 2024.

AI
AI Safety
Mechanistic interpretability
Technical AI Safety
Get
Ṁ1,000
to start trading!
Sort by:

This seems ~vaguely plausible but I think proving it would require a level of understanding of the individual circuits that we won't have by 2025.

@vluzko End of decade?

I'm bullish but think the odds are less than 50%

Related questions

© Manifold Markets, Inc.•Terms + Mana-only Terms•Privacy•Rules