Will a Mamba 7b model trained on 2 trillion tokens outperform Llama2-13B | Manifold

Will a Mamba 7b model trained on 2 trillion tokens outperform Llama2-13B

Plus

21

Ṁ738

Jul 1

66%

chance

1D

1W

1M

ALL

Question will resolve positive if someone trains a Mamba (https://twitter.com/tri_dao/status/1731728602230890895) language model with <=7.5billion parameters on <=2 trillion tokens that outperforms Llama2-13B on the huggingface open llm leaderboard (https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard)

This question is managed and resolved by Manifold.

Get

1,000

and

3.00

Sort by:

https://huggingface.co/nvidia/mamba2-hybrid-8b-3t-4k

Related questions

Will a 15 billion parameter LLM match or outperform GPT4 in 2024?

Will a open source pure Mamaba LLM surpass 82 MMLU on MMLU (5-shot) before end of year 2024?

Will Llama 3-multimodal be natively mixed-multimodal? (VQ-VAE+next token prediction)

Will the next major LLM by OpenAI use a new tokenizer?

Will a single model running on a single consumer GPU (<1.5k 2020 USD) outperform GPT-3 175B on all benchmarks in the original paper by 2025?

Will the Jan 2024 version of the LLM detector "Binoculars" be effective against OpenAI's best model at end 2024?

When will OpenAI release a more capable LLM?

Will any open-source model achieve GPT-4 level performance on MMLU through 2024?

Will the next LLM released by OpenAI be worse than GPT-4 at MMLU?

How many active parameters will the largest Llama 3 have?

Related questions

Will a 15 billion parameter LLM match or outperform GPT4 in 2024?

Will the Jan 2024 version of the LLM detector "Binoculars" be effective against OpenAI's best model at end 2024?

Will a open source pure Mamaba LLM surpass 82 MMLU on MMLU (5-shot) before end of year 2024?

When will OpenAI release a more capable LLM?

Will Llama 3-multimodal be natively mixed-multimodal? (VQ-VAE+next token prediction)

Will any open-source model achieve GPT-4 level performance on MMLU through 2024?

Will the next major LLM by OpenAI use a new tokenizer?

Will the next LLM released by OpenAI be worse than GPT-4 at MMLU?

Will a single model running on a single consumer GPU (<1.5k 2020 USD) outperform GPT-3 175B on all benchmarks in the original paper by 2025?

How many active parameters will the largest Llama 3 have?

© Manifold Markets, Inc.•Terms + Mana-only Terms•Privacy•Rules