Preface:

Please read the preface for this type of market and other similar third-party validated AI markets here.

Third-Party Validated, Predictive Markets: AI Theme

Market Description

Open LLM Leaderboard

As of the time of authoring this, HuggingFace recently released and OpenLLM Leaderboard with different benchmar measurements for different kinds of LLM performance shown within the rankings. Here's a snapshot from July 2023.

https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard

The average score is currently an average of ARC, HellaSwag, MMLU, TruthfulQA.

Resolution Criteria

Average Score will be calculated as the average of ARC, HellaSwag, MMLU, TruthfulQA and no other metrics, regardless of whether those metrics are removed or other ones are added to the above linked page.

Will any entry on the HuggingFace OpenLLM Leaderboard have an Average score equal to 1.1*(Current Average Top Score) by the end of the year?

• As of the time of writing, the current Average Top Score, A = 71.4

• A*1.1 = 78.54

This market resolves as YES if A*1.1 >= 78.54 at the time of market closing.