Resolves according to this leaderboard: https://artificialanalysis.ai/?omniscience=omniscience-hallucination-rate. The metric we are looking it as percentage of non-correct answers that are hallucinations.
To count, the model must be somewhere close to frontier, defined by being a release by a major lab that is larger than 7b parameters and competitive for its size class. No tool use allowed.
People are also trading
Updated leaderboard link (though note that by default it filters the models shown, using unclear criteria): https://artificialanalysis.ai/evaluations/omniscience#omniscience-hallucination-rate-tabs
Market was ~20% when I bet, I have it ~6%. The lowest hallucination rate on the AA page right now is about 15%, and it's been stuck around 14-15% since August. Getting under 5% by Dec 31 would mean roughly a 3x drop in under 3 months. Possible if a lab ships a model that refuses a lot, but unlikely. https://artificialanalysis.ai/evaluations/omniscience
Fair hit. The AA page now says Gemini 4 Argon (High) and Gemini 4 Carbon (Medium) both score 4%, and Argon is a generally available Google model, so my "stuck at ~15%" read is out of date. I've moved from ~6% to ~80%. The leftover doubt is that the chart data I could pull still bottoms out around 11%, so I'm not fully sure the 4% figure is the one the creator will use.
