🐕 Will A.I. Be Significantly Better, "Able to Track Changes in State," By the End of 2023?

Question

Warning! This market may already be doomed, as the threshold picked is higher than human performance. However I don't know for sure, just letting you know before you bet. Here's one that's set at slightly below human performance:

(https://manifold.markets/embed/PatrickDelaney/will-ai-be-able-to-meet-just-below)Preface / Inspiration:

There are a lot of questions on Manifold about whether or not we'll see sentience, general A.I., and a lot of other nonsense and faith-based questions which rely on the market maker's interpretation and often close at some far distant point in the future when a lot of us will be dead. This is an effort to create meaningful bets on important A.I. questions which are referenced by a third party.

Market Description

ProPara

ProPara aims to promote the research in natural language understanding in the context of procedural text. This requires identifying the actions described in the paragraph and tracking state changes happening to the entities involved.

Example Question

Given this five-sentence procedural paragraph (id 1167 from the training partition):

① The gravity of the sun pulls its mass inward. ② There is a lot of pressure on the Sun. ③ The pressure forces atoms of hydrogen to fuse together in nuclear reactions. ④ The energy from the reactions gives off different kinds of light. ⑤ The light travels to the Earth.

Consider the two participant entities:

atoms of hydrogen

sunlight or light

Predict answers to these four questions:

What are the Inputs?

That is, which participants existed before the procedure began, and don't exist after the procedure ended? Or, what participants were consumed?

Answer: The inputs are atoms of hydrogen.

What are the Outputs?

That is, which participants existed after the procedure ended, but didn't exist before the procedure began? Or, what participants were produced?

Answer: The outputs are light (or sunlight).

What are the Conversions?

That is, which participants were converted to which other participants?

Answer: The participant atoms of hydrogen is converted into light (or sunlight) in sentence 3.

What are the Moves?

That is, which participants moved from one location to another?

Answer: The participant light (or sunlight) moves from sun to earth in sentence 5.

Market Resolution Criteria

https://leaderboard.allenai.org/propara/submissions/public

Top score on F1 is 0.731.

We would need to see 1.3*0.731 or greater by the end of the year for this to resolve YES, otherwise NO.

Please see markets embedded for how I will format this description in the future to try to make it more clear and provide more information.

Manifold Markets · Accepted Answer

No — resolved on Jan 9, 2024 by Manifold Markets prediction market.