This seems relevant

Metaculus (@metaculus) on X

How about resolving this market accordingly?

Metaculus (@metaculus) on X

Last month GPT-6 Astra (@chatgpt) beat Montezuma's Revenge, and a lot of people declared that our "weakly general AI" question, originally launched in 2020, resolved. But…it still hasn’t resolved. We wanted to give everyone a quick update on where things actually stand and the steps we’re current…

I think if this market it going to be resolved in the original spirit of the question, then maybe we should actually read what the creator wrote about AGI on manifold 🤣 These arguments are going to go nowhere otherwise and it will just be people projecting their personal takes onto the definition

chatgpt gave me these:

/RemNi/will-we-get-agi-before-2033-34ec8e1d00fd

/RemNi/will-we-get-agi-before-2033-34ec8e1d00fd

(they write they don't think this timeline is realistic at the end of that comment)

/RemNi/will-earth-have-a-space-elevator-be-410e262e639f

/RemNi/will-we-get-agi-before-2031

/RemNi/will-we-get-agi-before-2025

/RemNi/attention-diffusion-search-agi

/RemNi/will-we-get-agi-before-july-2028

@SimoneRomeo

I think this one by itself it enough to answer your question down below why this market can't resolve yet:

/RemNi/will-we-get-agi-before-2033-34ec8e1d00fd

@0xseraphim

Dec 2023 was before LLMs started solving Millennium Prize Problems.

Ability to rapidly acquire new skills --> yes, they can download skills

Learn continuously --> yes, just see how personalized your ChatGPT has become and how better it is to support you for your workflow compared than incognito

Reason over long context --> yes, earning money in real life is an example

@0xseraphim LLMs can process video data. Besides Gemini that's natively multimodal, other LLMs will simply export it frame by frame and check each frame. Quite smart if you ask me

@SimoneRomeo but there's no "learning continuously from experience" is there? Especially not for video data. Humans can definitely do this. LLMs just can't get anywhere close to this capability in October 2026.

Think of like a youtube tutorial on how to use some complex GUI tool the human has never seen before (eg. like blender). Humans can watch that for 8 hours and then have a go and apply what they learned. Current AI models can't go from video data to learned and retained experience/skills in the same way. They can kind of approximate it by exhaustively captioning all the video frames, but it's not the same thing.

Let's recap what this market says:

"AGI can theoretically perform any intellectual task that a human being can. It involves the capability to reason, plan, solve problems, think abstractly, comprehend complex ideas, learn quickly, and learn from experience."

And the resolution criteria:

"Resolves as YES if such a system is created and publicly announced before January 1st 2027"

Current benchmarks show that in every single area that was mentioned, GPT Astra outperforms the human baseline:

Capability to reason — GPQA Diamond

GPT-6 Astra scores about 96%, substantially above published PhD-expert human baselines.

https://openai.com/index/gpt-6-astra/

Capability to plan — ARC-AGI-3

Astra has to infer the rules of unfamiliar interactive environments and formulate multi-step strategies. It used fewer actions than the median successful human on 96% of levels.

https://arcprize.org/blog/astra

Capability to solve problems — TroubleshootingBench

The task is to diagnose subtle failures in real biological lab procedures. Astra scored 48.44% versus 36.4% for the 80th-percentile PhD expert.

https://deploymentsafety.openai.com/gpt-6-astra

Capability to think abstractly — ARC-AGI-3

The benchmark uses novel abstract environments without explicit instructions. Astra must infer hidden mechanics and build useful internal representations.

https://arcprize.org/blog/astra

https://arcprize.org/arc-agi/3

Capability to comprehend complex ideas — GPQA / TroubleshootingBench

There is no universally accepted benchmark specifically called “complex idea comprehension,” but these test understanding and reasoning over PhD-level scientific material, where Astra exceeds expert-human comparisons.

https://openai.com/index/gpt-6-astra/

https://deploymentsafety.openai.com/gpt-6-astra

Capability to learn quickly — ARC-AGI-3

Humans and Astra encounter unfamiliar environments and must discover their mechanics through interaction. Astra beat median human action efficiency on 96% of levels.

https://arcprize.org/blog/astra

Capability to learn from experience — EBR-bench

Epoch designed Earthborne Rangers specifically to test improvement through repeated attempts and retained notes. Astra reached 21/21 after two playthroughs, while the best human reached 21/21 on the fifth.

https://epoch.ai/benchmarks/ebr-bench

As for AI leaders who have said AGI has been achieved:

Jensen Huang — NVIDIA CEO

Reportedly said: “I think we’ve achieved AGI,” and later “AGI has arrived.”

https://www.theverge.com/ai-artificial-intelligence/899086/jensen-huang-nvidia-agi

Ali Ghodsi — Databricks CEO

Argued that AGI had effectively arrived and that AI no longer had an intelligence problem.

https://en.sedaily.com/international/2026/06/18/has-ai-caught-up-to-humans-databricks-ceo-says-yes-openai

Greg Brockman — OpenAI President

Described the current period as the “AGI era” and reportedly said, “For me personally, I do think we’re there.”

https://agi-society.org/publications/evaluating-agi-claims

Can the market resolve YES already? And if not, what objective criteria are we still missing?

@RemNi @mods

forbes.com

Fair question. The longbets Turing test is a necessary but not sufficient condition. Maybe something like Aschenbrenner's drop-in remote workers.

@dreev could you update the market description with clear and objective criteria? We can't bet on a market that resolves on "maybe".

Also notice that my codex has been working autonomously online and earning real money fully unsupervised. So if your criteria is remote workers, I believe that was achieved months ago.

@SimoneRomeo it even completed the real Alan Turing test by solving unsolved Enigma messages. It used complex techniques to unpick a German message that to date remained encrypted. If this isn't the turning test and replicating all the actions for an undeniably clear result against the actual Alan Turing genius I am not sure what we can do to prove @RemNi 's maybe or most other humans in denial.

@JanChan AI is superhuman in a lot of ways, including, to my amazement, math problem-solving. And cyberhacking and code-breaking. It's also far beyond the average human (including me for sure) at art and music. An example of something AI can't do is learn a new skill from minimal training data. There's also something missing (though harder and harder to put one's finger on or demonstrate on command) with big-picture thinking / wisdom / decision-making.

I like this characterization from Judith Dada of where we're at on the road to AGI:

@dreev I think that works really well as an explanation for why we are not at ASI. I think it completely fails to address why this term, for their specific interpretation of AGI defined above, has not been met. Further, we have not begun to give this seemingly AGI access to limbs. Only one or two experimental robots have ever used LLMs to do things, Unitree seems to have had a good go at this autonomous house clean up. Others seem to fake it till you make it. We only give it a 1m context window, how can we criticise it for lacking human level long term memory, that's a choice not a restriction and by all accounts it has far better than human memory when given sufficient context like a folder full of files like transcripts or documents or other forms of digital long term memory. If you can't describe it either, we have to say this is not ASI or even SI as the orange one calls it, but it's plenty general enough to meet the definition above.

@dreev if you're nitpicking niche criteria you can still find realms where humans are better than AI. That's pretty much irrelevant as AI is already much more capable than an average human for the vast majority of tasks that don't require embodiment. It's not more capable than the sum of all humans together, but that's a pretty distorted definition of AGI and it was NOT in the original definition of the market. This market has not resolved only by changing resolution criteria as those that were mentioned were already clearly achieved. Pretty annoying if you ask me.

We're doing the best we can with an underspecified market and an absent creator. Another mod (see the addendum to the market description) made the call that this market will use my own AGI market, which is a work in progress, as its baseline. But also a hint that current AI is not AGI is the part of the original market description here that talks about learning quickly from experience. Current AI can leave itself endless notes and for many tasks that suffices. But if you try to replace a remote employee with AI, it soon becomes clear what's missing. I admit this is all changing fast.

If you're seeing contradictions in what we've committed to as the resolution criteria, do point those out.

PS: Other ways to approach the question of what counts as AGI:

• An adversarial Turing test is officially defeated (I realize this isn't satisfying if your view is that the conclusion is so far forgone that no one bothers to run the test)

• Eliezer Yudkowsky's market on AI hell breaking loose

• Scott Alexander's markets for various milestones or GDP/unemployment seeing a visible break

• Human-out-of-the-loop recursive self-improvement

@dreev there are various claims here, so let's break them down.

We're doing the best we can with an underspecified market and an absent creator. --> but there were indeed resolution criteria mentioned in the original descriptions. What's been done was to add other criteria besides what was mentioned.

that current AI is not AGI is the part of the original market description here that talks about learning quickly from experience. --> that's not clear at all! Practically any benchmark that aimed to assess this has fallen already.

But if you try to replace a remote employee with AI, it soon becomes clear what's missing. --> if you try to replace "all remote employees" then definitely but have you ever had leadership experience? Junior employees are pretty damn stupid, particularly without training. I challenge you to take 100 random college students, put them into 100 random positions regardless of their specialized experience, don't teach them anything and then tell me whether they or Fable will perform better. AI can earn money autonomously. This should be all it's required to know. I mean, damn, my codex delivered real work autonomously online across months, found prospects, pitched a service, bargained price, got paid, and even got a repeat business in one case. And you tell me this is not AGI. What?!

Current AI can leave itself endless notes and for many tasks that suffices --> so you just said it yourself: it does learn and it does improve

Other ways to approach the question of what counts as AGI --> there are infinite definitions of AGI, here's not about nitpicking them, but checking what the OP wrote and resolving accordingly

@SimoneRomeo The original market description is impossibly vague and Manifold norms are that it's incumbent on traders to ask clarifying questions before betting too much. That said, it's very much our goal to hew to the spirit of the original question.

Re: benchmarks: We've learned the last couple years that AI can ace benchmarks without that translating into skill at the thing the benchmark was trying to measure. For humans it works well enough to use exams or the SAT or playing chess as perfectly good proxies for competence and intelligence. AI is uncannily good at figuring out how to ace just the test without any generalization. Like an extreme version of all book smarts, no street smarts. I think this is getting less extreme and some benchmarks still do capture something meaningful beyond the benchmark itself. But this is the reason it's so hard to find a useful AGI benchmark. Many like ARC-AGI and GPQA are saturated. But many others aren't.

I realize this can look like goalpost-moving but see Dwarkesh Patel on when goal post shifting is justified.

@dreev man, I don't know how that essay may prove your point and it reads like someone who's trying to rationalize a bias to fit into his confused opinion. It states that "while we’re really close to (and in many ways have surpassed) what I would have defined as AGI in the past, the fact that model companies are not making trillions is revenue clearly reveals that my previous definition of AGI was too narrow". So what? The almost 100b profit of model companies is not enough? Even as 99% of the market share is unsaturated simply because the average user has never used the latest models or can't even imagine what they'd be able to do?

And YES AI is better at benchmarks than real world. But it is capable of generalization. There's overwhelming evidence of this. LLMs were driving a car without being taught how to drive. Do you know people that just sit into a car and start driving? And Dwarkesh says "but AI can't make videos as well as I do". Well no one can. He's top 1 billion podcast creators. Being better than him is not about having general intelligence. That's the realm of ASI, something that will be achieved soon but there's no reason to mix the two. If your definition of AGI is something that makes humans irrelevant, then why not call it ASI directly?

@SimoneRomeo I haven't investigated the Remote Labor Index enough to vouch for it yet but maybe that's a candidate benchmark for operationalizing the "is this AGI yet?" question.

More interestingly for Manifold, it sounds like our disparate intuitions on this question imply disagreement on reasonably short-term questions about AI replacing white-collar workers. It would be great to find or create markets on those questions and fight it out there.

@dreev also Open AI industrialised maths yesterday.

700 unsolved maths problems solved!

Sure not all interfaces have been built for AI, particularly in robotic crochet or robotic humans juggling balls, but this isn't about controlling arms it is about intelligence.

If today a giant spaceship landed and we were greated by a super intelligent giant space alien worm creature that could talk perfectly in all the languages of earth and rewrite all of our mathematics and showed us how well it understood all life in our planet and created beautiful music that we had never heard before, all with no arms would we say it lacks general intelligence?

Given the choice between life saving surgery from your average human, or from a human with access to chatgpt i would choose the chatgpt guy everytime and while you might win at jenga against an AI robot arm, it would probably beat all of us at chess and sudoku and you name it any obscure sub Sahara dice game that none of us have ever heard of.

Surely it's time to call this general.

@dreev Daniel, Remote Labor Index shows current models at 20% of white collar tasks. How many people do you know that could do the same? This is just the next of a series of benchmarks. And whenever AI will reach whatever threshold you like, people will look at it and say "but I can still do / feel / believe X, and therefore that is not AGI". This keeps happening every month. So you want to keep RLI as a resolution criteria for this market, do it. But I believe we long passed the threshold of when AI could be compared to individuals and it will soon be undeniably superhuman.

Daniel & Simone,

@dreev
@SimoneRomeo presumably we can agree that there would be a difference between the following 2 markets.

Will we get AGI before 2027? vs Will we get ASI before 2027?

Perhaps we can agree what the difference is.

As a minor addition the leader of the free world in his orange greatness with all of his access to the greatest AI experts, has renamed AI Super Intellgence.

While I grant you that could be used as compelling evidence against the market for ASI, surely it points towards AGI.

@JanChan I think you have a great point here. My definition of AGI is a system that is roughly equally capable to the average human.

I asked ChatGPT to give me a persona for this most average person in the world and ChatGPT called it Rahul.

Rahul Kumar is a 31-year-old married Indian man from a small town, with one child, 10 years of schooling, a modest income, a small retail business, and an Android smartphone.

After that sweet line of general intelligence we approach super intelligence even if there's going to be a long way between AGI and ASI.

@SimoneRomeo random fact, Rahul shares his name with an incredibly popular indian star who plays Manmohan in the classic 2009 film "3 Idiots".