ML's calibration
Grade: B+, Score: -1.11
Resolution probability
Probability after bet
Interpretation
- The green dot at (x%, y%) means when ML bet YES at x%, the question resolved YES y% of the time on average.
- Perfect calibration would result in all green points being above the line, all red points below, and a score of zero.
- The score is the mean squared error for yes and no bets times -100.
- Each point is a bucket of bets weighted by bet amount with a maximum range of 10% (sell trades are excluded).
YES bets
NO bets
3 largest bets for each bucket
10%
20%
- Will there have been a noticeable sector-wide economic effect from a new AI technology by the end of 2023?NOṀ50
- Will a Musk company release an LLM chatbot in 2023?YESṀ10
- Will language models or similar natural language processing technologies, such as ChatGPT, be integrated into dialogue trees for NPCs in triple-A games by the end of 2023?NOṀ10
30%
40%
50%
60%
70%
80%
- Will the average price of a dozen eggs drop below $3.50 in February 2023?NOṀ200
- Will "The Waluigi Effect" post on LessWrong receive >=800 karma by March 31st?NOṀ100
- Will GPT-4 be unreliable at reasoning about the physical, psychological, and mathematical world? (Gary Marcus GPT-4 prediction #2)YESṀ100
90%
95%
97%