Which mental operations will correlate (0,2) with successful forecasting?

6kṀ1888

Mar 8

40%

MENTAL SIMULATION: Vividly and precisely imagining what different scenarios might look like. This might include putting yourself into the shoes of key actors and seeing what motives, means and opportunities they have.

60%

WORKING BACKWARDS: Imagining a scenario has already happened or failed to happen, and working backwards to identify the key factors that led there. Or thinking about what a prediction implies about how the world is today.

44%

DIALECTIC: Exchanging back-and-forth arguments with yourself. Testing, whether predictions sound convincing when you try to express them in concrete words or even formal arguments.

62%

REFERENCE CLASSES: Thinking whether a scenario is a typical or an atypical member of some broader classes of scenarios.

79%

DECOMPOSITION: Breaking down a complex scenario into smaller, more easily estimatable components and checking if the conditional probabilities fit your intuitions - e.g. thinking about 3 steps necessary for an event to happen

50%

CONTRASTING: Asking yourself whether A or B sounds more likely (e.g. comparing two candidates with 5% chance). This can include adding up probabilities and asking "How likely is it that neither A or B happens?"

68%

REFLECTING ON BIAS: Thinking about the possible biases that influence your estimate or the estimates of others.

48%

PRIZE GUT CHECK: Imagining how you would feel if you lost or won a bet - i.e. if a given sum “offered” by the market seems worth it, similarly to how you would think about buying a product

49%

FREQUENCY GUT CHECK: Visualizing the meaning of a certain percentage. For instance, imagining 10 bubbles representing 10 worlds, or imagining how a chance compares to throwing a die (16,7%).

64%

SURPRISE CHECK: Checking how surprising you would find it if the likelihood turned out to be different. For instance, if a market predicts a 30% chance, would I find it weird if it predicted 50%?

67%

ROBUSTNESS CHECK: Thinking about how your estimate might change if some new evidence emerged, if something new happened, or if you changed your mind about some smaller piece of the puzzle.

49%

META-STRATEGY: Noticing how different techniques, thoughts or predictions contrast and interact in your mind.

⭐ CLICK HERE TO FILL OUT THE SURVEY

I'm running the survey above. To qualify, you need to be a Manifold user with an account that was registered before Feb 22nd, 2025 and made at least 10 bets.

In the survey, respondents are asked: "Imagine you are trying to predict a complex global event happening 5 years from now. Imagine you have 1 hour to come up with a prediction and you cannot use the internet.
For each technique, please indicate how useful you would find it in this exercise."

Then, they are asked to rank the techniques from 1 (not useful at all) to 5 (extremely useful). They are randomly ordered (except for the final questions 12-15). I will look at the users' Brier scores using the calibration app (thanks, sufftea!). Note that the Brier score doesn't measure calibration but the distance of negative bets to zero and positive bets to 1 - therefore low is good.

All techniques where the usefulness will negatively correlate with the user's Brier score so that the Pearson's coefficient is at least 0.2 will resolve YES. Correlations lower than 0.2 will resolve NO. For illustration, that's e.g. the correlation between male gender and enjoyment of danger.

The survey will stop gathering responses by the resolution date (March 8).

If there's > 100 respondents, I might randomly select 100 users that will form the sample.

I will not bet on this market. I reserve the right to remove suspicious responses.

Get

1,000

to start trading!

3 Comments

31 Holders

89 Trades

Sort by:

reposted

sick af market

bought Ṁ100 YES

Fascinating survey on forecasting techniques. Go take it!

This is a great question! I hope you get a lot of respondents. For Frequency Gut Check, I was about to say I don't do that at all, until I realized I have on rare occasion compared a low % to the odds of a rare drop in a video game, lol

Related questions

Related questions