Proposition

Exited my NO position as I now believe that AI research in the next few years is going to mostly abandon architecture search in favor of adapting better RL-centric training ideas like GRPO.

https://www.liquid.ai/liquid-foundation-models

do people have takes on this?

In 2030, which AI paper will have the most citations?

What about hybrid models, like Jamba? They might be the best of both worlds.

Yes given that an architecture qualifies that levers a combination of transformer models and supporting infra components that wouldn’t be considered breakthrough technologies on their own (e.g. RAG).

So do mixtures of experts count? The linked page this not contain any actual details.

@EchoNolan I talked to Sasha, and his response is basically that as long as the E in the MoE is Transformer, its a transformer.

See: https://manifold.markets/HanchiSun/is-attention-all-you-need

@Tasty_Y and https://manifold.markets/ICRainbow/on-january-1-2027-a-transformerlike?r=SUNSYWluYm93

@_T_Grass

i have strong principled reasons this should stay at 50% for the next 24 hours

@jacksonpolack such as?

subsidy phasein

@jacksonpolack Hm, I will add in subsidy at a later point wherever the market stabilizes to maintain that