The breakthrough needs to make LLMs better at learning from context at test time. Some examples of results that could demonstrate this breakthrough would be a model that shows less degradation than contemporaries on long-context or one that can more effectively learn from mistakes during agentic tasks.
The breakthrough needs to be major and conceptual, it can't be something as simple as models getting better at in-context learning through more scaling.
In order to count this breakthrough must apply to general LLMs, not just using continual learning to solve a narrow problem or class of problems
A paper claiming to have a really good result isn't enough (like the Titans paper). The breakthrough must be widely accepted as legitimate and there should be publicly accessible models that use it
1,000People are also trading
Here is there most recent update: https://open.substack.com/pub/aifutures1/p/ai-futures-model-dec-2025-update?r=6lp84s&utm_medium=ios. Daniel and Eli think that the scenario as laid out will basically be the same except that it will happen a few years later. Daniel pointed out that things appear to be 1 year behind the original scenario. As of right now Daniel thinks things will start to unfold in 2029 while Eli thinks things will start to unfold in 2030.
@JoshYou I saw you voted no for a major breakthrough in LLM continual learning before 2027 but in another market you voted yes, that a breakthrough would occur. Do you think their be a breakthrough or not?
@JoshYou I apologize but I am still confused. It is probably just me but the the question criteria seem similar to me. How is the question meaningful different?
@MaxLennartson he didn’t provide a specific definition of continual learning, whereas my continual learning definition specifically requires models learning better from context at inference time.
His definition might include any model that can be continuously trained without catastrophic forgetting, which I don’t consider true continual learning
@JaundicedBaboon I am sorry but I am still a bit confused. In the “market context” it says “Continual learning in large language models (LLMs) refers to the ability of a model to acquire new information and skills over time without "catastrophic forgetting,". You don’t consider “any model that can be continuously trained without catastrophic forgetting” to be true continual learning. Should I not be that much attention to what the “market context” says?
@JaundicedBaboon Here is mr Mimi’s definition: "A significant advance in continual learning": The model should be able to remember facts or skills learned over a long period of time like a human. It should not make egregious errors related to memory that current bots in the AI village regularly commit.
@MaxLennartson if we basically solve the learning problem through much better in-context learning, I would guess mino resolves yes.
@MaxLennartson @JaundicedBaboon Do think this is a fair assessment that both definitions are different from each other?
A lightweight control mechanism of plastic weights that sit on top of the static weights. The LLM can be trained using RL to produce sequences of "continual learning tokens", upon which it optimizes some objective, backpropagating through the plastic weights. The control mechanism can be implemented as lightweight residual layers initialised at zero (no modification at initialisation).
This adds another scaling dimension on top of pretraining and reasoning. The key challenges include the formulation of the RL objective and the implementation of the control mechanism. Solving these problems enables agents to operate on much longer task horizons (7-30 human equivalent days) without getting stuck in typical LLM amnesia loops. The caveat is that the gradient steps to update the fast weights requires substantial GPU compute in order to perform the test-time training.