Standard reinforcement learning is kinda dumb.

We humans reason about the mistakes we have done, and update in a more directed rational fashion.

Basically using understanding of the state space not just to solve problems but to also increase the speed of our learning as well.

Resolves YES iff before 2027 it will >50% likely that there exists such a model and has state of the art performance (in case of internal models). And for it to qualify it must explicitly have extra reasoning space to figure out how to update it's weights. Just like humans reason about what mistakes they made.

Karpathy mentioning something in this direction very recently:

• Update 2025-05-11 (PST) (AI summary of creator comment): The creator has further clarified the requirements for 'extra reasoning space' and 'human-like reasoning about mistakes' as outlined in the market description:

  • The 'rational reinforcement learning' must involve a process distinct from, and going beyond, learning achieved through mere token prediction.

  • The qualifying model must engage in a process of reasoning about why an event or error occurred. This reasoning is used to update its internal models or understanding of the relevant context (analogous to a human analyzing another individual's actions to update their mental model of that person).

  • Standard learning mechanisms such as backpropagation or simple reinforcement learning updates, by themselves, are not considered sufficient to constitute the required 'extra reasoning space'. The model must possess a more deliberative process for understanding and integrating lessons from its experiences or mistakes, rather than solely undergoing a standard algorithmic update.

• Update 2025-05-11 (PST) (AI summary of creator comment): The creator has further specified the process for determining if it is '>50% likely that such a model exists' by 2027, a key condition for market resolution:

  • This evaluation will be made by consulting AI models.

  • The AI models will be provided with the context of this market for their assessment.

  • The creator intends to use all state-of-the-art (SOTA) AI models available at the time of resolution (specific models to be determined in the future).

  • The determination will be based on an average across at least 10 answers obtained from these SOTA models.

• Update 2025-05-15 (PST) (AI summary of creator comment): Regarding the AI models that will be consulted for determining the market's resolution:

  • They will be given the ability to search the internet.

  • This search capability will include all sources mentioned in this market.