Preface:
Please read the preface for this type of market and other similar third-party validated AI markets here.
Third-Party Validated, Predictive Markets: AI Theme
Market Description
MMLU
Paper: https://arxiv.org/abs/2009.03300
Resolution Criteria:
https://paperswithcode.com/sota/multi-task-language-understanding-on-mmlu
GPT4 tops the list at 86.4 currently.
We seem to have been gaining around 10 points per year from Jan 2022 until early 2023.
Scaling out about 1/4th of the year hypothetically we should be at 93.9 by the end of 2023 assuming continued linear growth.
However, the test might not happen by then, so assuming we get a test by end of October, that would be (0.583 years)*(10 points/year) = 5.83 additional points, + 86.4 = 92.2
If > 92.2 by end of year, this resolves as YES.
New market for year posted at https://manifold.markets/PatrickDelaney/-will-ai-be-able-to-gain-a-much-bro ... still sitting at 86.4 for the year 2023 using GPT-4.