Will a Mamba-based LLM of GPT 3.5 quality or greater be open sourced in 2024?

1kṀ1874

Jan 1

73%

chance

ALL

Mamba is a next-generation architecture that seeks to improve on the shortcomings of transformers, mainly around context size and eliminating quadratic memory consumption during inference. https://arxiv.org/abs/2312.00752

YES resolution requires the Mamba LLM to match or beat GPT 3.5 on at least 5 popular benchmarks.

LLMs

Mamba

Get

1,000

to start trading!

People are also trading

Will an open-source fully functional Auto-GPT like LLM exist by the end of 2025?

90% chance

When will an open-source LLM be released with a better performance than GPT-4?

Size of smallest open-source LLM marching GPT 3.5's performance in 2025? (GB)

1.83

Which next-gen frontier LLMs will be released before GPT-5? (2025)

Will we have an open-source model that is equivalent GPT-4 by end of 2025?

82% chance

China will make a LLM approximately as good or better than GPT4 before 2025

89% chance

When will GPT-4 be open-sourced?

Will xAI develop a more capable LLM than GPT-5 by 2026

51% chance

Will it be possible to run an LLM of GPT-4 (or higher) capability on a portable device by 2027?

Sort by:

5mo

@Manifold can you resolve?
I believe the question creator is banned

5mo

@quantizor are you able to resolve?

5mo

time to resolve

8mo

does this count as Mamba-based? it's open and easily above GPT-3.5 quality

https://www.ai21.com/blog/announcing-jamba-model-family

https://huggingface.co/nvidia/mamba2-hybrid-8b-3t-4k

Here's a mamba-transformer-moe hybrid that's about as good as gpt-3.5. ai21.com/jamba.

5mo

@matt still using transformer, so I wouldn't count it : )

bought Ṁ10 NO

Looks like Gemini 1.5 used normal transformers and not Mamba, while also seeming to get around these shortcomings (1M context size). I expect this will cause interest in Mamba to wane, which decreases the chance someone will bother training and testing a Mamba LLM to GPT 3.5 level.

@adele I remain unconvinced that transformer architecture will be the long term winner due to its compute and memory-hungry nature. These are great improvements though.

People are also trading

Will an open-source fully functional Auto-GPT like LLM exist by the end of 2025?

90% chance

When will an open-source LLM be released with a better performance than GPT-4?

Size of smallest open-source LLM marching GPT 3.5's performance in 2025? (GB)

1.83

Which next-gen frontier LLMs will be released before GPT-5? (2025)

Will we have an open-source model that is equivalent GPT-4 by end of 2025?

82% chance

China will make a LLM approximately as good or better than GPT4 before 2025

89% chance

When will GPT-4 be open-sourced?

Will xAI develop a more capable LLM than GPT-5 by 2026

51% chance

Will it be possible to run an LLM of GPT-4 (or higher) capability on a portable device by 2027?

37% chance

People are also trading

People are also trading

Related questions