Preface / Inspiration: There are a lot of questions on Manifold about whether or not we'll see sentience, general A.I., and a lot of other nonsense and faith-based questions which rely on the market maker's interpretation and often close at some far distant point in the future when a lot of us will be dead. This is an effort to create meaningful bets on important A.I. questions which are referenced by a third party.

Market Description:

Resolved by:

https://leaderboard.allenai.org/a-okvqa/submissions/public

A-OKVQA

A-OKVQA is a new knowledge-based visual question answering benchmark comprised of 25K questions that require a broad base of commonsense and world knowledge to answer. Questions in A-OKVQA are challenging, conceptually diverse, require knowledge outside the image, and in contrast to existing knowledge-based visual question answering datasets, they cannot be answered by simply querying a knowledge base. To ease working with unbounded knowledge sources, questions in the training set are paired with rationales that supply facts and snippets of reasoning needed to answer them. This benchmark encompasses both multiple-choice and direct-answer settings for questions.

Example A-OKVQA Question

Q: What type of parking is available here?

• (A) angle

• (B) parallel

• (C) valet

• (D) lot

Market Resolution Threshold:

• If Any A-OKVQA Question and Answer Challenge Submission Gets a % DA VQA Score of >0.85 by end of 2024, this resolves as YES, otherwise NO.

Mar 17, 12:10pm: Will Any A-OKVQA Rearrangement Challenge Submission Get a % DA VQA Score of >0.85 by end of 2024? → Will Any A-OKVQA Question and Answer Challenge Submission Get a % DA VQA Score of >0.85 by end of 2024?

Mar 22, 12:18pm: Will Any A-OKVQA Question and Answer Challenge Submission Get a % DA VQA Score of >0.85 by end of 2024? → Will A.I. Have Significantly More Common Sense by end of 2024?