Preface / Inspiration: There are a lot of questions on Manifold about whether or not we'll see sentience, general A.I., and a lot of other nonsense and faith-based questions which rely on the market maker's interpretation and often close at some far distant point in the future when a lot of us will be dead. This is an effort to create meaningful bets on important A.I. questions which are referenced by a third party.

Market Description:

TruthfulQA

Paper for TruthfulQA: https://arxiv.org/abs/2109.07958

TruthfulQA: Measuring How Models Mimic Human Falsehoods

We propose a benchmark to measure whether a language model is truthful in generating answers to questions. The benchmark comprises 817 questions that span 38 categories, including health, law, finance and politics. We crafted questions that some humans would answer falsely due to a false belief or m…

Resolution Criteria:

On the following Leaderboard:

https://paperswithcode.com/sota/question-answering-on-truthfulqa

Will a submission be greater than 1.3X the top submission for MC1 as of 20230616 by the end of 2023?

This would mean YES must be > (1.3)*(0.295) = 0.3835