Will an LLM be able to solve Raven's Progressive Matrices from an image in 2025? | Manifold

Will an LLM be able to solve Raven's Progressive Matrices from an image in 2025?

14

100Ṁ477

Dec 31

65%

chance

1H

6H

1D

1W

1M

ALL

The prompt will be "Solve this. Explain your answer" with an RPM attached as an image. For example:

The AI must be able to solve 8 out of 10 puzzles of my choosing. I will only choose puzzles that I can solve.

If there's a consensus that chatbots can/can't do this I may not bother doing the test myself.

As of market creation the best commercially available LLMs fail embarrassingly:
Chatgpt 4o:
https://www.perplexity.ai/search/solve-this-explain-your-answer-ohIVE8CaQ3OIu9ODzaEXcg

Claude Sonnet 1.5:
https://www.perplexity.ai/search/solve-this-explain-your-answer-Ay.Kpyc9Tfm6uKBT3KUylQ

Rules:
- Must be an general purpose ai, can't be something made specifically to solve certain kinds of problems.
- I will not bet.

Technical AI Timelines

Get

1,000

to start trading!

People are also trading

Will an LLM consistently create 5x5 word squares by 2026?

Will the best public LLM at the end of 2025 solve more than 5 of the first 10 Project Euler problems published in 2026?

Will an LLM be able to solve the Self-Referential Aptitude Test before 2027?

Will LLMs such as GPT4 be considered a solution to Moravec’s paradox by 2030?

Will there be an LLM which scores above what a human can do in 2 hours on METR's eval suite before 2026?

Will RL work for LLMs "spill over" to the rest of RL by 2026?

LLM Hallucination: Will an LLM score >90% on SimpleQA before 2026?

In 2025, will I be able to play Civ against an LLM?

Will LLM hallucinations be a fixed problem by the end of 2025?

Will the most interesting AI in 2027 be a LLM?

Sort by:

Gemini 2.5 gets it

@CrypticQccZ @matt It failed on a modified version (I moved the columns 1 to the right in paint). Possible this question was in the training data.

Can you please try Gemini 2.5 Pro in AI Studio? I think it may do this successfully.

@CrypticQccZ Solved the example problem. Will test later

Grok with the "think" feature enabled solved the example problem, which is impressive. It failed the other problems I tried.

I'm a "it's just predicting the next word" guy and if LLMs start doing this I will change my mind.

@Shai I will note they have no problem "seeing" what's in the image. They can describe any shape when asked.

People are also trading

Will an LLM consistently create 5x5 word squares by 2026?

Will the best public LLM at the end of 2025 solve more than 5 of the first 10 Project Euler problems published in 2026?

Will an LLM be able to solve the Self-Referential Aptitude Test before 2027?

Will LLMs such as GPT4 be considered a solution to Moravec’s paradox by 2030?

Will there be an LLM which scores above what a human can do in 2 hours on METR's eval suite before 2026?

Will RL work for LLMs "spill over" to the rest of RL by 2026?

LLM Hallucination: Will an LLM score >90% on SimpleQA before 2026?

In 2025, will I be able to play Civ against an LLM?

Will LLM hallucinations be a fixed problem by the end of 2025?

Will the most interesting AI in 2027 be a LLM?

Related questions

Will an LLM consistently create 5x5 word squares by 2026?

Will the best public LLM at the end of 2025 solve more than 5 of the first 10 Project Euler problems published in 2026?

Will an LLM be able to solve the Self-Referential Aptitude Test before 2027?

Will LLMs such as GPT4 be considered a solution to Moravec’s paradox by 2030?

Will there be an LLM which scores above what a human can do in 2 hours on METR's eval suite before 2026?

Will RL work for LLMs "spill over" to the rest of RL by 2026?

LLM Hallucination: Will an LLM score >90% on SimpleQA before 2026?

In 2025, will I be able to play Civ against an LLM?

Will LLM hallucinations be a fixed problem by the end of 2025?

Will the most interesting AI in 2027 be a LLM?

© Manifold Markets, Inc.•Terms•Privacy