[Carlini questions] SOTA AI systems still regularly "hallucinate" incorrect solutions to problems
[Carlini questions] SOTA AI systems still regularly "hallucinate" incorrect solutions to problems
5
100Ṁ52
2030
61%
On Jan 1st 2027
42%
On Jan 1st 2030

Resolution Criteria:

For this question, a "hallucination" is just the model making up something that's completely detached from reality. Making a mistake is not a hallucination, but if you ask for a citation, and it creates a citation from a book that doesn't exist that would be a hallucination. "Regularly" means they do it in a high enough fraction of cases that "it matters". If there are good hallucination benchmarks, I will rely on those. If there aren't, then I'll go mainly on whether or not the research community as a whole believes the problem still exists.

Motivation and Context:

Today's models hallucinate a lot. They make up facts, they make up events. Ask them for a summary of a book that doesn't exist and some fraction of the time they'll tell you what they think it says given the title and author. This is a massive problem, and prevents these models from being deployed in safety-critical settings. I want to know if the hallucination problem will remain a big problem.

Question copied from: https://nicholas.carlini.com/writing/2024/forecasting-ai-future.html

Get
Ṁ1,000
to start trading!

What is this?

What is Manifold?
Manifold is the world's largest social prediction market.
Get accurate real-time odds on politics, tech, sports, and more.
Or create your own play-money betting market on any question you care about.
Are our predictions accurate?
Yes! Manifold is very well calibrated, with forecasts on average within 4 percentage points of the true probability. Our probabilities are created by users buying and selling shares of a market.
In the 2022 US midterm elections, we outperformed all other prediction market platforms and were in line with FiveThirtyEight’s performance. Many people who don't like betting still use Manifold to get reliable news.
ṀWhy use play money?
Mana (Ṁ) is the play-money currency used to bet on Manifold. It cannot be converted to cash. All users start with Ṁ1,000 for free.
Play money means it's much easier for anyone anywhere in the world to get started and try out forecasting without any risk. It also means there's more freedom to create and bet on any type of question.
© Manifold Markets, Inc.Terms + Mana-only TermsPrivacyRules