Which strategy will get the best Brier Score on the 2024 ACX prediction contest?
18
336
2.3K
2025
38%
The initial Jan 31st Metaculus user prediction, except Will Bitcoin go up over 2024? =99.999%
13%
Zvi's predictions https://twitter.com/TheZvi/status/1743282911699141066 (predictions entered into Metaculus)
11%
The initial Jan 31st Metaculus user prediction, except Will Bitcoin go up over 2024? =0.001%
10%
The average of the top three non-meta, non-other options in this market.
6%
The initial Jan 31st Metaculus user prediction, but adjusted for the overall average over/underconfidence from https://www.metaculus.com/questions/track-record/
4%
The Manifold probability averaged over the day of the 31st (Pacific Time)
3%
The Manifold probability averaged over the day of the 31st (Pacific Time), averaged with the initial Metaculus user prediction.
2%
The initial Jan 31st Metaculus user prediction
1.1%
Other

There is now a topic with a bunch of mirrored questions from the 2024 ACX prediction contest on Metaculus. This question asks what strategy for predicting these questions will get the best Brier score on all binary questions in this topic as of January 31st.

User submissions are valid, with a few caveats:

  • It must be somewhat reasonable to compute the prediction on January 31st.

  • Submissions must be described in 240 characters or less.

  • New submissions after the 31st will not be accepted (I will close the market on the 31st, disable new submissions, and then reopen).

  • Referencing other predictions is allowed, as long as there is provably no way that it could lead to a paradox.

If there is a tie for first I will resolve to % equally among the winners

I suggest an adversarial mindset.

Get Ṁ200 play money
Sort by:
The average of the top three non-meta, non-other options in this market.

Looks like these were Jack, Manifold, and Zvi at the end of January when this closed.

Ok I have reopened the market with new answers turned off, feel free to go wild now that there will be no new entrants. I'll try to make summaries of the main predictions that don't reference other predictions sometime soon.

The initial Jan 31st Metaculus user prediction, except Will Bitcoin go up over 2024? =0.001%

Very nice. I was wondering if someone would carry out this kind of attack.

Jack's predictions https://firstsigma.substack.com/p/predictions-2024
bought Ṁ10

Thanks to @Austin for the liquidity boost!

I'm writing my prediction post now. I want to clarify how it's going to work so you can set this market correctly (I'm going to link to it because I think it's fun).

  1. First I predict each question blind meaning I look at financial markets but not prediction markets.

  2. Second I look at Manifold and make trades (which are in, so if you want to know what side I'm on, you know, if it's M10 or less I'm just tracking mostly)

  3. Third is the number I'm settling on and entering into Metaculus.

The third number is the one you should use for Briar score here, although of course someone is welcome to also put in a market option for the blind ones. In which case, it's going to heavily depend on the few places I messed up and got a wildly off initial answer!

I won't trade on this market unless it gets a lot more liquidity, more fun to watch it play out, but I will say that if there was more liquidity I would definitely make some trades.

@ZviMowshowitz 😃 Ok, thanks for letting us know!

@Joshua I agree with Zvi that the Metaculus prediction is the one that makes the most sense for the answer you gave, so I'm editing it to clarify that. If you want to add the other ones you can do that too, or I might add them myself.

The initial Jan 31st Metaculus user prediction, but adjusted for the overall average over/underconfidence from https://www.metaculus.com/questions/track-record/

So right now the average prediction is 5% underconfident so just take all the initial forecasts and increase them to cancel that out.

bought Ṁ10 of Other YES

@Joshua the Manifold Calibration says that we're overconfident on average, but I don't think it says by how much?

Or am I missing that somewhere?

The Manifold probability averaged over the day of the 31st (Pacific Time), averaged with the initial Metaculus user prediction.

Lol if this and the individual manifold and metaculus ones are the top three then this is just the same as the top-three non-meta ones averaged.

bought Ṁ10 of Other YES

@BoltonBailey I'm hoping you get more submissions here haha

@BoltonBailey Ok quick rules update, if there is a tie for first I resolve to prob equally among the winners that seems most sensible.

The average of the top three non-meta, non-other options in this market.

This one seems more reasonable, but is still somewhat doubtful. What defines whether an option is meta or non-meta?

@BoltonBailey I guess you could say non-meta is anything that doesn't reference the market itself? But then I could make a side market that references this market and have the side market reference this one. Maybe "nothing with this market in the closure of things it recursively could reference"?

@BoltonBailey options which don't reference any other options in this market directly or indirectly are non-meta, and non-other is just ruling out "Other" from which answers are split.

@Joshua Ok seems good, if there isn't some unanticipated explosion here I think this is fine.

The highest probability option in this market other than this prediction and "other". (Resolves NO due to possible self-reference)

Nice try, but if if this is the second highest option, and the highest probability option says "the opposite of the second highest option" then you have a referential loop and it creates a paradox.

bought Ṁ10 of Other YES

@BoltonBailey Yeah true, feel free to edit to "resolves no".