Will OpenAI release a tokenizer with vocab size > 150k by end of 2024?
9
Ṁ460Ṁ246Dec 31
42%
chance
1H
6H
1D
1W
1M
ALL
The GPT-2 model used r50k_base: vocab size = 50k
The GPT-3 model used r50k_base: vocab size = 50k
The GPT-3.5 model used cl100k_base: vocab size = 100k
The GPT-4 model used cl100k_base: vocab size = 100k
This question is managed and resolved by Manifold.
Market context
Get 
1,000 to start trading!
1,000People are also trading
Related questions
Will OpenAI announce a new full-size, frontier model >5.2 before March 1, 2026?
80% chance
Will AI (large language models) collapse by may 2026?
10% chance
Will OpenAI release another open source LLM before end of 2026?
70% chance
Will OpenAI release a social network in 2026?
20% chance
Before 2029, will OpenAI provide API access to a frontier LLM with 100,000,000+ context length?
49% chance