Will model weights obtaining >90% on the Winogrande benchmark become publicly available in 2024? | Manifold

Will model weights obtaining >90% on the Winogrande benchmark become publicly available in 2024?

3

110Ṁ31

resolved Feb 25

Resolved

YES

1D

1W

1M

ALL

I will use the leaderboard at https://paperswithcode.com/sota/common-sense-reasoning-on-winogrande unless someone has a better suggestion. Model weights do not have to be licensed without restrictions but must be available for download.

Technical AI Timelines

Get

1,000

to start trading!

Sort by:

What model resolved this yes? I can see code on GitHub but I can't seem to find model weights for any of the top models.

@ProjectVictory https://github.com/allenai/rainbow?tab=readme-ov-file#downloading-the-weights Seems to fulfill the criteria, I would say, and comes in at 91.3 on the leaderboard. It's actually older than the question, so i guess this was just a matter of the leaderboard getting updated -_-

GitHub - allenai/rainbow: Neural models of common sense. 🤖

Neural models of common sense. 🤖. Contribute to allenai/rainbow development by creating an account on GitHub.

@Thomas42 thanks I missed the download instructions it seems.

Comment hidden

Related questions

Will AI image generating models score >= 90% on Winoground by June 1, 2025?

Will OpenAI offer a model that updates its weights while running during 2025?

Will OpenAI open source the weights to one of the GPT family models in 2024?

Will a flagship (>60T training bytes) open-weights LLM from Meta which doesn't use a tokenizer be released in 2025?

When will the weights of GPT-4 be publicly available (leaked or open sourced)?

OpenAI to release model weights by EOY?

Will any model get above human level on the Simple Bench benchmark before September 1st, 2025.

Will the gap between open-weights and frontier models on GPQA Diamond be at most 7%?

Will OpenAI allow near full access to the weights of their best-trained model to an external auditor by the end of 2030?

Will a state actor or other group succeed at stealing GPT-4 model weights by the end of 2026?

Related questions

Will AI image generating models score >= 90% on Winoground by June 1, 2025?

OpenAI to release model weights by EOY?

Will OpenAI offer a model that updates its weights while running during 2025?

Will any model get above human level on the Simple Bench benchmark before September 1st, 2025.

Will OpenAI open source the weights to one of the GPT family models in 2024?

Will the gap between open-weights and frontier models on GPQA Diamond be at most 7%?

Will a flagship (>60T training bytes) open-weights LLM from Meta which doesn't use a tokenizer be released in 2025?

Will OpenAI allow near full access to the weights of their best-trained model to an external auditor by the end of 2030?

When will the weights of GPT-4 be publicly available (leaked or open sourced)?

Will a state actor or other group succeed at stealing GPT-4 model weights by the end of 2026?

© Manifold Markets, Inc.•Terms + Mana-only Terms•Privacy•Rules