Will Victor Taelin’s new $10K bounty for AI reasoning be claimed by EOY 2025?
Will Victor Taelin’s new $10K bounty for AI reasoning be claimed by EOY 2025?
4
100Ṁ95
2026
75%
chance

Victor Taelin recently posted a challenge about the claimed reasoning abilities of AI/LLMs:

https://x.com/victortaelin/status/1844886809005687270?s=46

A short proof that LLMs (even o1) still can't reason:

Consider the problem of inverting a perfect binary tree. That's an old, entry-level interview question that humans and LLMs can solve easily. Now, let's add just 3 key nuances to make it new and unique:

1. It must invert keys ("bit-reversal permutation")

2. It must be a dependency-free, pure recursive function

3. It must have type Bit -> Tree -> Tree

These small changes are enough to move this problem out of the "memorized solution zone". It isn't on the internet. And, guess what? This is enough to make it completely intractable to modern AIs. All of them fail miserably at it, no matter how you prompt it.

This is very relevant, because the problem is still easy to a human researcher, and being capable of solving it is clear pre-requisite to contribute to CS research. Yet, all modern AIs fail miserably. As much as I love LLMs, truth is: they do NOT reason, and they will never do CS.

Some prompts for you to try:

gist.github.com/VictorTaelin/4…

I'm willing to give $10k to anyone who shows any AI capable of implementing this function correctly. It just won't work, no matter how long it thinks. (The solution is 7 lines of code!)

This market will resolve YES if anyone produces a demonstration that wins this $10k by the end of 2025. I will wait for confirmation from Victor before resolving YES. This market will resolve NO at the end of 2025 otherwise.

Get
Ṁ1,000
to start trading!

What is this?

What is Manifold?
Manifold is the world's largest social prediction market.
Get accurate real-time odds on politics, tech, sports, and more.
Or create your own play-money betting market on any question you care about.
Are our predictions accurate?
Yes! Manifold is very well calibrated, with forecasts on average within 4 percentage points of the true probability. Our probabilities are created by users buying and selling shares of a market.
In the 2022 US midterm elections, we outperformed all other prediction market platforms and were in line with FiveThirtyEight’s performance. Many people who don't like betting still use Manifold to get reliable news.
ṀWhy use play money?
Mana (Ṁ) is the play-money currency used to bet on Manifold. It cannot be converted to cash. All users start with Ṁ1,000 for free.
Play money means it's much easier for anyone anywhere in the world to get started and try out forecasting without any risk. It also means there's more freedom to create and bet on any type of question.
© Manifold Markets, Inc.Terms + Mana-only TermsPrivacyRules