This market resolves to the year in which an AI system exists which is capable of passing a high quality, adversarial Turing test. It is used for the Big Clock on the manifold.markets/ai page.
The Turing test, originally called the imitation game by Alan Turing in 1950, is a test of a machine's ability to exhibit intelligent behaviour equivalent to, or indistinguishable from, that of a human.
For proposed testing criteria, refer to this Metaculus Question by Matthew Barnett, or the Longbets wager between Ray Kurzweil and Mitch Kapor.
As of market creation, Metaculus predicts there is an ~88% chance that an AI will pass the Longbets Turing test before 2030, with a median community prediction of July 2028.
Manifold's current prediction of the specific Longbets Turing test can be found here:
/dreev/will-ai-pass-the-turing-test-by-202
This question is intended to determine the Manifold community's median prediction, not just of the Longbets wager specifically but of any similiarly high-quality test.
Additional Context From Longbets:
One or more human judges interview computers and human foils using terminals (so that the judges won't be prejudiced against the computers for lacking a human appearance). The nature of the dialogue between the human judges and the candidates (i.e., the computers and the human foils) is similar to an online chat using instant messaging.
The computers as well as the human foils try to convince the human judges of their humanness. If the human judges are unable to reliably unmask the computers (as imposter humans) then the computer is considered to have demonstrated human-level intelligence.
Additional Context From Metaculus:
This question refers to a high quality subset of possible Turing tests that will, in theory, be extremely difficult for any AI to pass if the AI does not possess extensive knowledge of the world, mastery of natural language, common sense, a high level of skill at deception, and the ability to reason at least as well as humans do.
A Turing test is said to be "adversarial" if the human judges make a good-faith attempt, in the best of their abilities, to successfully unmask the AI as an impostor among the participants, and the human confederates make a good-faith attempt, in the best of their abilities, to demonstrate that they are humans. In other words, all of the human participants should be trying to ensure that the AI does not pass the test.
Note: These criteria are still in draft form, and may be updated to better match the spirit of the question. Your feedback is welcome in the comments.
Related questions
People are amazed now how fast genai can come back with a seemingly thoughtful response to our queries. Let’s reserve our awe until the amount of time to prepare the query to transcribe is significantly longer than how fast the computer spits out some impressive but totally hallucinatied reply.
Here's my feedback...I'm not convinced that the original Turing paper, "Computing Machinery and Intelligence," was meant to outline a framework for, "thinking," as Turing states in the paper himself, page 8:
The original question, "Can machines think?" I believe to be too meaningless to deserve discussion.
I believe the paper was about trying to think through where to allocate research dollars, as that is brought up multiple times.
"Can machines think?" should be replaced by "Are there imaginable digital computers which would do well in the imitation game?"
It seems that AI researchers still wrestle with this problem and the consensus is that LLM's are a dead end, so high-end, hardcore AI researchers talk about world models, computing brains, mimicking nature, discovering new algorithms from brain structures, etc. The reason why Neural Nets and LLM's won out at the point of history we're in right now, is because of the massive amount of data produced by the internet, and NN's are a way for data scientists to process that data without as much human labor. Building brains from scratch (e.g. programming brains) or components of brains is highly labor intensive and much harder to do than just, "run a massive amount of data through a massively powerful NN."
The degree of misunderstanding that users on Manifold have in the field is understandably, extremely high. The degree of misunderstanding that people on social media, in the news, etc., who follow these topics, is even worse. I think that by talking about these topics without having at least a basic history of computing mathematics in the 20th Century is making us all stupider, not smarter.
This is a neat market, but nobody should be fooled into thinking that it is meaningful for any useful predictions of the future.
The software described in this market is far beyond what is necessary to cure diseases, automate almost all current labor, and do things that would improve human lives. It also is far beyond any reasonable definition of the word "AGI," as "AGI" has already been achieved when Claude 3 Opus scored an IQ of 101.
@SteveSokolowski IQ tests measure intelligence in humans. They make almost no sense outside of that.
Adversial Turing test is a pretty good test for AGI, IMO.
@jim Whether IQ tests are good or not, I have yet to find a useful task that Claude 3 Opus does worse at than the average human (excluding things it can't do, like lift up things, of course.)
The stupid tricks that people throw at it to count letters in sentences and solve logic problems don't matter to me.
@jim Humans cannot pass an adversarial Turing Test where they imitate an LLM. Does that mean that we aren't general intelligences?
@SteveSokolowski people say "a child could solve this logic problem" and I wonder how many children they've seen this year.
@MartinRandall AGI is generally defined in a anthropocentric way. AGI is an AI which can do the same intellectual work humans can do. That sort of thing. It's been this way from the moment the term was coined.
Anyway, I don't think it is necessary that an AI pass the adversial Turing test to be considered an AGI. I just think it's a good test, mostly in the sense that if it passes then it is hard not to be considered an AGI in the sense defined above. But I can imagine an AI that doesn't pass the test but which is nonetheless AGI.
@SteveSokolowski if you have ever tried to make AI agents, you will know how bad AI is at almost everything 😁 unless you put a massive amount of effort into writing countless pages of prompts and programming.
I don't really know what the average human is like. But even the dumbest of the people I know can follow a range of instructions vastly more wide than GPT-4. Their only weaknesses are that their knowledge is limited and they get bored.
@jim I'm pretty confident that the average human would not perform better than an AI agent would at this point. The agents have trouble because they are programmed to do things that only the best humans can do, like fix bugs in codebases. But AGI has already been achieved; these systems run circles around most humans now at most tasks. The problem is that all the supporting infrastructure around them, like traditional code to translate LLM output to speech and back for customer service, has been ridiculously slow to materialize for some reason.
My suspicion is that many people don't understand how dramatic the difference is between the best people and the average person. The bar to succeed in the world is so low that it's absurd. Just showing up to work on time every day is sufficient to lift you above the average human.
That's why I've maintained for some time that it's extremely easy to make lots of money and it's extremely difficult to keep it safe from scams, which is why I try to earn lots of money and not save for "retirement." Advice-givers don't recognize how simply working hard at something and showing up puts you so far above the average. People make it seem like that you have to be the best of the best to be successful, when the bar is so low that you don't actually have to be very good at all.
@SteveSokolowski I agree that AI agents can do some things well, and not others. But I don't think that's AGI! Definitely a lot of money to be made with pre-AGI agents though, especially as the next generation of LLMs are released.
Infinitely clonable, never lazy, can upload/download any skill they develop, cheap, fast, etc. -- AI agents are awesome.
here's a meta-market about how accurate this market will be:
https://manifold.markets/jim/will-manifolds-agi-clocks-1-yearave?r=amlt
feel free to make an improved version
This market has the same problems as all the others, and I think it discredits Manifold to put up a big countdown timer on this as though it means anything.
I expect AGI in less than 5 years, but conditional on us being alive a year afterwards I expect it in more than 10 years, so my incentive is to bet on "longer".
I love this market - but I’m chocked that so many people think AGI will happen in the next 5-6 years.
AGI will require further improvements in the algorithms. I’m sure we will get there - but not within. 3 years. Then we need compute and learning time…
Current technology “transformers” is amazing- it we need even more research.
@Magnus_ the definition of "AGI" here is just passing a turing test. It's pretty different from I think most peoples' intuitive sense of the word "AGI"