This market resolves to the year in which an AI system exists which is capable of passing a high quality, adversarial Turing test. It is used for the Big Clock on the manifold.markets/ai page.
The Turing test, originally called the imitation game by Alan Turing in 1950, is a test of a machine's ability to exhibit intelligent behaviour equivalent to, or indistinguishable from, that of a human.
For proposed testing criteria, refer to this Metaculus Question by Matthew Barnett, or the Longbets wager between Ray Kurzweil and Mitch Kapor.
As of market creation, Metaculus predicts there is an ~88% chance that an AI will pass the Longbets Turing test before 2030, with a median community prediction of July 2028.
Manifold's current prediction of the specific Longbets Turing test can be found here:
/dreev/will-ai-pass-the-turing-test-by-202
This question is intended to determine the Manifold community's median prediction, not just of the Longbets wager specifically but of any similiarly high-quality test.
Additional Context From Longbets:
One or more human judges interview computers and human foils using terminals (so that the judges won't be prejudiced against the computers for lacking a human appearance). The nature of the dialogue between the human judges and the candidates (i.e., the computers and the human foils) is similar to an online chat using instant messaging.
The computers as well as the human foils try to convince the human judges of their humanness. If the human judges are unable to reliably unmask the computers (as imposter humans) then the computer is considered to have demonstrated human-level intelligence.
Additional Context From Metaculus:
This question refers to a high quality subset of possible Turing tests that will, in theory, be extremely difficult for any AI to pass if the AI does not possess extensive knowledge of the world, mastery of natural language, common sense, a high level of skill at deception, and the ability to reason at least as well as humans do.
A Turing test is said to be "adversarial" if the human judges make a good-faith attempt, in the best of their abilities, to successfully unmask the AI as an impostor among the participants, and the human confederates make a good-faith attempt, in the best of their abilities, to demonstrate that they are humans. In other words, all of the human participants should be trying to ensure that the AI does not pass the test.
Note: These criteria are still in draft form, and may be updated to better match the spirit of the question. Your feedback is welcome in the comments.
Related questions
What desides if it AGI? And the genral query skills of current models are not great. And there is huge problem with the consept, how can it be AGI when it would just rebound and make artificial human information and there will also be syntetic data in the human genrated content it is trained on. It would just be a mess and have the fake do gooder vibe humans tend to put on for social clout. The very litle intelligence in form a dynamic self reflecting system like human brains. If people are honest this will never settle with just transformer models.
@ManifoldAI I created a related market here which will resolve based on this market.
@Soli Huh, if it could be expected that all the humans for the Longbets Turing Test would be in roughly the same location, couldn't you easily sniff out the AIs by asking about how the place smells on the day of the test?
@jim which is not the same thing at all 😅 check the question with my linked tweet and comment for some context
@GooGhoul very interesting thought, you are probably right but then i think this market is very wrong on the 2029 date since machines won’t be smelling anytime soon
@Soli it's obviously not the same thing, but isn't it like the same thing as a camera is for the visual modality?
@jim i am not familiar with spectrometers but if they can indeed be used to allow machines to smell the same way a camera allows them to see then yes would suffice
@Soli But where would it get the information to make the right lie?
It's not like anyone maps out the smell of places in some database that the AI could've been trained on or have access to.
@33cb The turing test is completed when you can't tell which is human between an AI and a human when speaking to them. Since it was thought up, we've learned that passing it can be done with no intelligence. So the turing test is just a first step.
Btw, Chat GPT4 passed a rigorous turing test last year.
@SteeeveTheSteve you're a describing an easy test. We can talk about a proper hard Turing test that can't be cheated without actually knowing how to conduct it.
Actually I don't know maybe the test you're talking about already meets my goalpost, just it's surprising news.
@SteeeveTheSteve Can’t I just buy the person I’m talking to an Amazon gift card and ask them to redeem it and then try to redeem it myself to confirm its been used?
Or give them your phone number & ask them to call you?
Or give them one of your social media accounts that you both have & ask them to send a friend request?
As far as I know no AI could currently do these kinds of arbitrary things & nearly any human could
From my read of the the long bets rules, that would be allowed
And that’s just some ideas off the top of my head, there’s probably stuff a cleverer questioner than I could ask who they’re talking to to do that doesn’t need outside verification like send them a YouTube link to an obscure version of a wagon-wheel illusion & ask them what it looked like to them or send them an illusion in ASCII art form.
A smart person with a 20 grand bet on the line could figure out if who they’re talking to isn’t human.
A Human is a biological human person as that term is understood in the year 2001 whose intelligence has not been enhanced through the use of machine (i.e., nonbiological) intelligence, whether used externally (e.g., the use of an external computer) or internally (e.g., neural implants).
Can you redeem a gift card without a computer? Impressive.
@robm Yeah, I just send the code to my mate to redeem GGEZ
More seriously, Using a computer to access Amazon & redeem a gift card isn’t what I’d call an intelligence enhancement but I suppose a judge might object
If the participants are not pushing the boundaries of the rules I don’t think it’s a good enough turing test
Do we reckon asking how many prongs this ASCII art fork has is within the rules?
(_) _
/ / (_) _
/ / / / (_)
/ / / / / /
/ / / / / /
/ / / / / /
/ / / / / /
/ / / / / /
/ / / / / /
/ / / / / /
/ / / / / /
/ / / / / /
/ /_/ / / /
/ \ \/ / /
/ \/ / /
\ / /
/ / /
/ /\ / /
/ / /\/_/
/ / /pjl
\/_/
Interviewer might have to go for a simpler design if they aren’t allowed to copy paste
@GooGhoul If it's allowed seems that asking your conversation partner to solve a Sudoku puzzle might root out an AI still in 2030:
https://manifold.markets/Mira_/will-a-prompt-that-enables-gpt4-to
There doesn't seem to be a market on whether GPT-5/6 will be able to solve Sudoku puzzles in one go
--