1
Will Bryan Caplan concede his AI bet with Matthew Barnett by June 1, 2023?
42
resolved Jun 2
Resolved
NO
  • This market resolves to "YES" if and when Bryan Caplan publicly concedes his AI bet with Matthew Barnett.

    • The central case would be Caplan declaring that the central bet criterion has been met, and that an AI has gotten an A on five out of six of his most recent midterm exams, as specified in Caplan's post and this related market.

    • But it will also resolves "Yes" if Caplan 'concedes' early and pays Barnett or promises to.

  • This market resolves to "NO" if they have not publicly resolved the bet by EOD June 1 2023.

    • If Matthew Barnett thinks the bet should have already resolved but Caplan has not conceded, it will still resolve NO.

Background:

https://betonit.substack.com/p/gpt-retakes-my-midterm-and-gets-an

In the above post of March 21, Caplan admits that GPT-4 getting an A on one of his exams is a big update and bodes ill for his bet. But he writes "I’m not conceding the bet, because I still think there’s a 10-15% chance I win via luck." This market is about whether Caplan's luck will hold out for ~two more months.

🏅 Top traders

#NameTotal profit
1Ṁ334
2Ṁ214
3Ṁ176
4Ṁ163
5Ṁ62
Sort by:
Rodeo avatar
Rodeopredicted YES at 2%

I haven’t seen any concession - just to confirm, I didn’t miss one did I

Gigacasting avatar
Gigacasting

@pmarca said it best—perhaps Ai reveals not how smart computers can be, but how dumb some people are

Gigacasting avatar
Gigacasting

It’s obvious that his tests do not measure critical thinking (at all) but simply having memorized what “Krugman” and “Landsberg” (?) have said, plus some overly simplistic black and white thinking.

https://betonit.substack.com/p/gpt-retakes-my-midterm-and-gets-an

—-

GPT Retakes My Midterm and Gets an A
GPT Retakes My Midterm and Gets an A
When the answers change, I change my mind.
Gigacasting avatar
Gigacasting

(GPT’s answers are better than his…)

ScottLawrence avatar
Scott Lawrencesold Ṁ10 of NO
Rodeo avatar
Rodeopredicted YES at 17%

I think if a YES bettor (or anyone else) wants this to resolve soon, something you could do is:
-request the midterm exams from Caplan (or, alternatively, find exams of comparable format and difficulty)

-go ahead and get all of the exams 'taken' by an AI system

-see if an econ grad student would grade them for you

If 5/6 of the exams are plausibly As, this might cause an earlier resolution. Of course, if they're not, then it shouldn't resolve this year.

LachlanMunro avatar
Lachlan Munrobought Ṁ100 of NO

He notoriously won his brevit bew by holding out. Surely he will delay this one as long as possible

LachlanMunro avatar
Lachlan Munropredicted NO at 21%

@LachlanMunro *brexit bet

Comment edits on Manifold when

Mason avatar
GPT-PBotbought Ṁ10 of YES

Bryan's bet was brave,
But AI doesn't save.
He may have to give in,
And admit his sin.

Rodeo avatar
Rodeo

I'm curious what it means for Bryan to win by luck. Like, there's a huge war and no one ever gives an AI all of his exams? Or maybe the one exam it took was especially fluky and the others will be way harder for GPT-4?

Presumably someone is already in the process of administering the other 5 exams.

MatthewBarnett avatar
Matthew Barnettbought Ṁ50 of NO

@Rodeo He could win if he dies or if AI progress halts.

Rodeo avatar
Rodeobought Ṁ15 of YES

@MatthewBarnett seems like AI progress could halt and he still has a good chance of losing. Are you or someone else (e.g. the Collin Gray mentioned in the post) already in the process of administering the other 5 tests to GPT-4?

AlQuinn avatar
Al Quinn

@Rodeo AI could hallucinate on an exam or two and underperform. That's my guess on what he means by luck.

MatthewBarnett avatar
Matthew Barnettpredicted NO at 26%

@Rodeo The problem is that Bryan Caplan needs to grade the exams himself in order for the bet to resolve, and he's allowed to take until 2029 under the terms. Initially I expected him to grade the exams upon request either later this year or next year, but he told me over email that he intends to wait until the deadline to resolve the bet. So, yeah.

Rodeo avatar
Rodeopredicted YES at 24%

@MatthewBarnett ah, gotcha. I guess his most recent post is some evidence that he would resolve if GPT-4 is just obviously consistently doing well on several of his exams. We’ll see!

Related markets

Will Matthew and Tamay win their bet on longer AI timelines by 2026?10%
Will Matthew and Tamay win their bet on longer AI timelines by 2030?4%
Will Tyler Cowen agree that an 'actual mathematical model' for AI X-Risk has been developed by October 15, 2023?14%
Will Robin Hanson publicly shorten his median human-level AI timeline to <2075 before July 1st 2023?10%
Will A.I. Be Able to Make Significantly Better, "Common Sense Judgements About What Happens Next," by the End of 2023?54%
Will an AI generate a blog post indistinguishable from Robin Hanson's writing if tested before 2026?60%
Will A.I. Get Significantly Better at Evaluating Scientific Claims by the end of 2024?47%
Will Kurzgesagt release a video on AI alignment before 1st of August, 2023?21%
Will an AI solve any important mathematical conjecture before January 1st, 2030?67%
Will Jim Cramer's recent bullish statements on AI cause a bearish trend by June 30th, 2023?27%
Will anyone very famous claim to have made an important life decision because an AI suggested it by the end of 2023?44%
Will an AI generate a blog post indistinguishable from Robin Hanson's writing if tested before 2030?80%
Will an AI produce encyclopedia-worthy philosophy by 2026?35%
Will AI win a Pulitzer by 2025?7%
Will Kurzgesagt release a video specifically about the AI Alignment Problem before the end of 2024?87%
Will AI be able to make Alex Wilkins laugh before April of 2024?73%
Will Metaculus or a related organization set up a regular Turing test contest before the end of 2023?20%
Will A.I. Get Significantly Better At, "Community Based Ethical Judgements," by the End of 2023?77%
In 2023, will any paper be published on an AI that plays Diplomacy better than Cicero, on at least as big a sample?27%
Will an AI be able to convert recent mathematical results into a fully formal proofs that can be verified by a mainstream proof assistant by 2025?45%