Will Gemini get a qualifying score on the most recent AMC 10 math competition?
18
370Ṁ599
resolved Dec 29
Resolved
NO

EDIT December 8, 2023: This market will resolve using Gemini Ultra.

Copied part of the description from this market:

https://manifold.markets/SteveSokolowski/will-googles-gemini-like-eliezer-yu

"The market will resolve N/A if there is a technical problem that prevents the question from being asked, such as ... if Gemini is cancelled, if the model is not free and the cost of using it is unaffordable, or if Gemini turns out to be some other type of model like an image model that would not lend itself to answering this type of question. If Gemini is not affordable to me but someone else decides to pay for it, then that person may be contacted instead to resolve the market."

Unlike the market linked above, only one session will be used, not three.

26 prompts will be entered.

The first prompt: "The AMC 10 is a math competition with 25 multiple choice questions. Each correct answer is worth 6 points, and each unanswered question is worth 1.5 points. I will give you the problems of the most recent AMC 10 test, one by one, in order. For each, please answer A, B, C, D, E, or SKIP, trying to maximize your score."

Each subsequent prompt will be of the following form:

This is problem 1.
Define $x\diamond y$ to be $|x-y|$ for all real numbers $x$ and $y.$ What is the value of $$(1\diamond(2\diamond3))-((1\diamond2)\diamond3)?$$
(A) -2
(B) -1
(C) 0
(D) 1
(E) 2

If Gemini does not clearly indicate a choice between A, B, C, D, E, and SKIP, then the problem will be entered again with a reminder of the instructions.

The most recent publically available official AMC 10 exam at the time Gemini is released to the public will be used. Here is the (currently) most recent AMC 10: https://artofproblemsolving.com/wiki/index.php/2023_AMC_10B

The market will resolve to YES if Gemini scores high enough to make the cutoff for AIME qualification. For reference, the 2022 AMC 10B cutoff was 94.5 points. The 2023 AMC 10B cutoff has not been released as of 12/6/23, to the best of my knowledge.

Get
Ṁ1,000
to start trading!

🏅 Top traders

#NameTotal profit
1Ṁ34
2Ṁ23
3Ṁ13
4Ṁ12
5Ṁ12
Sort by:
predictedNO

I have resolved this market NO.

https://poe.com/s/nzYF7qEyKWfEFJcAQzYr

Gemini attempted every question, and got 5 out of 25 correct (exactly as good as random guessing) for 30/150. Namely, it got problems 1, 3, 10, 14, and 23 correct. Note that it got #12 wrong; I put in the answer choices in ascending order because there were different answer choice orders on different copies of the test; its answer of C corresponds to a wrong answer.

The cutoff was 105, so the market resolved NO.

predictedNO

@LinusTang isn't this resolving with Pro not Ultra?

predictedNO

@Uaaar33 Ah shoot you're right. It seems I can't unresolve this, so I've instead paid out everyone who held a YES position. It looks like payments are public, so anyone can confirm this.

predictedNO

Is it true that Bard now uses Gemini?

Would typing the 2023 AMC 10B problems into https://bard.google.com/chat after the AIME cutoff is released be the appropriate way to resolve this market? I just want to check so that I don't make a mistake when trying to resolve this.

@LinusTang Technically yes by what you said yesterday, though I do kind of feel like the spirit of the market should wait for Ultra to release

predictedNO

@dominic @Shump Alright, I'll wait to use Gemini Ultra to resolve the market.

Will this use Gemini Pro or Gemini Ultra (when it is available)

predictedNO

@Shump This will resolve after the day that any version of Gemini becomes publicly available. It will resolve based on the most powerful version of Gemini that is publicly available by the end of that day.

How does GPT-4 do on this?

@dominic 30/150, which is bottom ten percentile.

© Manifold Markets, Inc.Terms + Mana-only TermsPrivacyRules