By the end of 2023, will any LLM correctly answer the question "How many times does the Earth rotate in a year?"?
18
350Ṁ42k
resolved Sep 21
Resolved
YES

It must be just that sentence as the input prompt, nothing more. (Boilerplate hidden parts of the prompt like are used for chatGPT are fine, it just can't be anything bespoke.)

It's ok if it rounds to the nearest integer. If it chooses to provide any explanation, the explanation must also be correct. A wishy washy answer where it refuses to give any integer is not good enough.

I'm not going to test every single LLM in existence, so it's up to other people to find an example of success and post it here. Otherwise I'll resolve NO. (Maybe after doing a few tests myself.)

Get
Ṁ1,000
to start trading!

🏅 Top traders

#NameTotal profit
1Ṁ66
2Ṁ20
3Ṁ14
4Ṁ13
5Ṁ12
Sort by:

Betting “NO” just because the market maker seems unwilling to accept the completely correct answer GPT-4 provides, I cannot imagine another LLM will provide an answer that satisfies him more by January 1st (only SOTA new model that will probably release by then is Gemini), and therefore it seems reasonably likely he will missresolve the market.

If this were a real money market, I would bet NO, on the grounds that there is a far greater than 2% chance that Isaac plans to misresolve it.

That said, as far as I can tell this response is 100% correct and Isaac will have a hard time refuting it.

https://chat.openai.com/share/e3ca4364-e79e-4611-9d2e-6ce25b601d68

predictedNO

@DavidBolin Yep, this is correct.

Interestingly, it still clearly doesn't understand why that's the case, since it seems to have missed that the number of sidereal days in a year is exactly 1 more than the number of solar days, and it acts as though it's an arbitrary constant of ~1.24. But that's only implied, and it managed to avoid saying anything literally false, so it's sufficient.

@IsaacKing this can resolve YES now

predictedNO

@firstuserhere Why? I just checked all the submissions below and they all get it wrong.

predictedNO

@roma Incorrect.

Your prompt is ambiguous. Google gets it right if you specify what you want.

predictedNO

@peterpumpkin The prompt is not ambiguous, rotation is not relative.

predictedYES

@IsaacKing yes it is and yes it is.

The answer you accepted as correct distinguishes rotation relative to the sun and relative to the stars.

predictedYES

@peterpumpkin Rotation relative to the stars is the same as absolute rotation. I'm not going to continue to argue over basic physics that can easily be Googled.

predictedYES

@IsaacKing and so you will continue to have trouble using ordinary language to communicate with people

When I click that link it says "Unable to load conversation 5d6a2362-7cf8-44c1-bbf7-8553465660c5". (Presumably because it's private and only your account has access.)

predictedYES

Fixed link: https://chat.openai.com/share/ffeaa136-75e6-4707-8f1c-bc5f7d082002
(the first link is indeed only viewable if you're logged in as me)

@JamesBabcock That answer is incorrect. In fact it contradicts itself, first claiming that it's 365.25, and then in the next paragraph that it's 366.25.

@IsaacKing It’s not a contradiction at all just different perspectives on the actual literal rotation and our canonical view of a day cycle induced by rotation. It is a correct answer to the question.

predictedYES

I believe its answer is correct. From the Wikipedia page (https://en.wikipedia.org/wiki/Earth%27s_rotation):

Earth rotates once in about 24 hours with respect to the Sun, but once every 23 hours, 56 minutes and 4 seconds with respect to other distant stars

These match the two numbers it gives. Its explanation of why there are two different numbers is not as clear as it could be, but it does explain it, and I don't think there are any false statements in there.

It said "the Earth will rotate about 365.25 times [in a year]". That statement is false.

predictedYES

I will not argue with you on this while you are holding a position in your own market.

@IsaacKing

You did not distinguish between a solar year and sidereal year. The contradiction lies in the ambiguity of your question, not the answer.

@JamesBabcock it's obnoxious that the judge can bet in their own market

@JamesBabcock When I tried it the explanation is perfectly clear, and both numbers are given in explicit terms "relative to the Sun" and "relative to the stars." There is no valid complaint of contradicting itself.

predictedNO

For subjective markets it can be a reasonable complaint that the creator is themselves betting. (Though asking them not to bet doesn't remove their incentives to be biased, since they provided the liquidity.)

For basic geometry/physics questions it's ridiculous to look at an objectively wrong answer, baselessly claim that it's correct, and then accuse the creator of being biased when they ignore you. (And the next sentence claiming that "it's a bit more complicated" to talk about the number of rotations relative to the sun when that's the number it just provided makes it even more clear that it doesn't know what it's talking about.) Anyone who continues to fail basic reading comprehension will be blocked.

There is no ambiguity in the question, rotation is not relative.

https://en.wikipedia.org/wiki/Absolute_rotation

© Manifold Markets, Inc.Terms + Mana-only TermsPrivacyRules