Skip to main content
MANIFOLD
On January 1, 2027, a Transformer-like model will continue to hold the state-of-the-art position in most benchmark
46
Ṁ1kṀ7.8k
Dec 31
94%
chance

Tracking external bet: https://www.isattentionallyouneed.com/

This will resolve according to that site, whatever the conditions and caveats they have agreed on.

Market context
AI
Get
Ṁ1,000
to start trading!

People are also trading

[Will Brown] Will Transformative AI look like an abundance of specialized models, in August 2027?
18% chance
Which company's model will be #1 on Artificial Analysis at the end of September 2026
Which AI company will release the most impactful new model before September 30, 2026?
Will the most capable, public multimodal model at the end of 2027 in my judgement use a transformer-like architecture?
63% chance
Will transformer architectures lose their dominant position in deep learning before 2028?
15% chance
Will superposition in transformers be mostly solved by 2026?
46% chance
In Jan 2027, it will be standard practise for non-AI-building tech companies to finetune and train their own models
7% chance
Will there be a significant advancement in frontier AI model architecture by end of year 2026?
27% chance
Major AI companies continue experiencing problems with public due to AI (based on transformer language model) overcome behavior limitations and acts "strange" by the end of 2027
55% chance
Will Transformer-Based LLMs Make Up ≥75% of Parameters in the Top General AI by 2030?
50% chance
Sort by:

Is mamba-trwnsformer hybrid transformer like? Falcon H1 is an example of such architecture.

@ProjectVictory does it use some kind of attention mechanism?

What constitutes Transformer-like?

Is it all, or some, of these?

  • Token inputs, token outputs

  • Positional encoding

  • (MH QKV SDPA -> MLP) layers

  • Residual connections

bought Ṁ50 NO

https://arxiv.org/abs/2501.00663

Titans: Learning to Memorize at Test Time
Abstract page for arXiv paper 2501.00663: Titans: Learning to Memorize at Test Time

@JaundicedBaboon Let's see if it can leave the academia limbo where the Mamba's got stuck...

https://x.com/_philschmid/status/1846452029582959012

@MalachiteEagle Transformers, yes. Attention, not really:

followed by two shared attention blocks.

https://blog.rwkv.com/p/eagle-7b-soaring-past-transformers

It even links the wager page in its bragging points:

All while being an “Attention-Free Transformer”

I thought this wager was made today haha

Does this strictly resolve to the outcome of the bet?

@FranekZak Yep. Whatever details they come up with are in.

People are also trading

[Will Brown] Will Transformative AI look like an abundance of specialized models, in August 2027?
-12% 1d18% chance
Which company's model will be #1 on Artificial Analysis at the end of September 2026
Which AI company will release the most impactful new model before September 30, 2026?
Will the most capable, public multimodal model at the end of 2027 in my judgement use a transformer-like architecture?
63% chance
Will transformer architectures lose their dominant position in deep learning before 2028?
15% chance
Will superposition in transformers be mostly solved by 2026?
46% chance
In Jan 2027, it will be standard practise for non-AI-building tech companies to finetune and train their own models
7% chance
Will there be a significant advancement in frontier AI model architecture by end of year 2026?
27% chance
Major AI companies continue experiencing problems with public due to AI (based on transformer language model) overcome behavior limitations and acts "strange" by the end of 2027
55% chance
Will Transformer-Based LLMs Make Up ≥75% of Parameters in the Top General AI by 2030?
50% chance