Will OpenAI release a tokenizer with vocab size > 150k in 2023?
20
620Ṁ6510resolved Jan 1
Resolved
NO1D
1W
1M
ALL
The GPT-2 model used r50k_base: vocab size = 50k
The GPT-3 model used r50k_base: vocab size = 50k
The GPT-3.5 model used cl100k_base: vocab size = 100k
The GPT-4 model used cl100k_base: vocab size = 100k
This question is managed and resolved by Manifold.
Get 
1,000 to start trading!
1,000🏅 Top traders
| # | Name | Total profit |
|---|---|---|
| 1 | Ṁ133 | |
| 2 | Ṁ52 | |
| 3 | Ṁ38 | |
| 4 | Ṁ36 | |
| 5 | Ṁ17 |
People are also trading
Related questions
Will OpenAI reveal thinking tokens by the end of June 2025?
13% chance
Will OpenAI release a tokenizer with vocab size > 150k by end of 2024?
42% chance
Will OpenAI release a tokenizer with more than 210000 tokens before 2026?
24% chance
Will the next major LLM by OpenAI use a new tokenizer?
77% chance
Will a flagship (>60T training bytes) open-weights LLM from Meta which doesn't use a tokenizer be released in 2025?
29% chance
Will OpenAI release next-generation models with varying capabilities and sizes?
64% chance
OpenAI to release model weights by EOY?
88% chance
Will OpenAI fold in 2025?
3% chance
Will OpenAI release a version of Voice Engine by the end of 2024?
81% chance
Will OpenAI have a new name by the end of 2025?
8% chance