Same as (https://manifold.markets/ZviMowshowitz/will-meta-ever-deploy-its-best-llm) except for Mistral instead of Meta.
Resolves YES if Mistral deploys an LLM, whatever name it might have, that is at least as strong as its best other LLM, either as a ChatBot, API access or as clearly used in another consumer product offering, and within 30 days of that deployment it is not possible to download the model weights.
Resolves to NO if Mistral releases the model weights to a model it self-describes as AGI, or that clearly constitutes AGI as per OpenAI's definition as of 1 Jan 2024.
Also resolves to NO if Mistral fails to release what they claim to be substantially improved LLM for 36 consecutive months, or for 24 months after the market closes.
Resolves to N/A on 1/1/35 if somehow none of the above criteria are met (to be safe).
Interestingly, mistral-medium might have recently leaked/been stolen, though it seems to have leaked late enough that the market should still resolve true AFAICT: https://manifold.markets/Vergissfunktor/is-miqu-a-leak-of-the-mistralmedium
@jacksonpolack best means strongest overall, regardless of size or cost, efficiency not relevant. So it would count if it was their strongest of any size.
Oh sorry, I mean because it was already released without open weights and has been for over a month, but before market creation, so this market wouldn't already resolve yes. Markets usually have a 'after market creation' implicitly but the 'ever' in the title is a bit ambiguous
@jacksonpolack Is it their best model? Would be interesting if they are indeed all talk after all, if so we can just resolve this YES right away and move on, and I'll have paid less than a dollar for useful info!
@ZviMowshowitz Yes, see here https://docs.mistral.ai/platform/endpoints/
Medium
This endpoint currently relies on an internal prototype model.
API name:mistral-medium
@RyanGreenblatt Looks like yes, this reddit post was made on december 17th and still not publically released https://www.reddit.com/r/LocalLLaMA/comments/18kib8y/evaluating_mistralmedium/