Fermi Questions (fermiquestions.org) is a Wordle-like game where you try to guess the answer to Fermi estimation questions with 6 or less tries. After each guess, you'll see if your answer was too high or too low. You win if your guess is within ±20% of the correct answer.
Example questions:
- How many new cars were sold in the US in 2024?
- How many humans have ever lived (including those currently alive)?
- How many chickens are slaughtered for meat every year?
The skill of Fermi estimation is also extremely useful if you want to perform better here on Manifold. As Philip Tetlock shows in his book Superforecasting, many of the best forecasters break down complex questions into smaller, more manageable components which is exactly what you can practice when playing the game.
In the first 24 hours after going live it had some 1,100 unique visitors, which primarily came from Hacker News. Resolution source is the unique visitors count on the publicly available analytics dashboard: https://plausible.io/fermiquestions.org
Update 2025-08-04 (PST) (AI summary of creator comment): In addition to the default bot-filtering by the Plausible analytics tool, the creator reserves the right to manually exclude any suspicious activity from the final unique visitor count.
Update 2025-08-04 (PST) (AI summary of creator comment): A unique visitor is defined as a distinct visitor to the website within a 24-hour period, based on their IP address. A user who visits on different days will be counted as a new unique visitor for each day.
1,000People are also trading
I think this could be quite fun; I've also considered building an iOS/Android app previously. However, I’ve paused new development for the next month or so as I’m exploring another idea: a “Fermi Competition” for K–12 students.
The format would be around 25 Fermi questions tackled in teams or alone, with students providing both point estimates (that they believe are within ±50% of the correct answer) and confidence levels (for simplicity, one can choose from only 5 degrees of uncertainty). At halftime, a hint for each question would be revealed so teams can update their estimates and their stated confidence. The goal is to practice Fermi estimation, revising one's beliefs in light of new information, and probability calibration in a fun setting.
At the end, students can see how well-calibrated they were... bearing in mind that the sample size is small, so any conclusions come with error bars. There would of course be some prizes and medals for the best-performing students.
If anyone is interested in advising or helping run this, outside Switzerland/Austria which is where I'm located, I’d love to hear from you! (Email: daniel@fermi.org)
@AhronMaline I'll share an extensive update early next week! So far I think it's going very well, and I have a lot of features in the pipeline that will be added to the game soon! For the time being I wonder if you or someone else would mind giving me some feedback on the probability calibration mode: Should it by default only ask you for a probability estimate for the first guess you make? For all 6 guesses? Or ideally just for the first guess and the third guess (the one after the text hint)?
I'm currently leaning toward the last option, and I'm even thinking about removing the option to give a probability estimate for all guesses because it is a somewhat pointless exercise if the only new information you have are directional arrows. It only gets interesting when you have the text hint after the second incorrect guess, which in my opinion is uniquely valuable to practice updating your confidence in light of new information.
No one asked for it, but you can now enable a probability calibration mode where you can assign a probability to every estimate you make. Once you've made a number of estimates, you are able to see how well-calibrated you are on the chart in the stats modal.
@danftz How do you measure "actual accuracy"? Ratio guess / truth for guess < truth and truth / guess for truth < guess?
@4fa I assumed accuracy of X was how often your guess was within ±20% of the true answer when your confidence was X% (divided by how often your confidence was X%)
I wanted to give a quick update, which may move the market substantially. I have not traded on the market in the hours before posting this update and I'll refrain from trading in the next 12 hours.
- I've discovered that Plausible Analytics has been substantially undercounting our visitor numbers because many adblockers and browsers (like Brave) block the analytics script. Up to 60% of visitors may not have been counted. Tomorrow I'll fix this by using a proxy to serve the same script, which should ensure accurate traffic measurement going forward.
- I've attempted to run some Google Search ads, which has mostly been a failure, but Reddit advertising has exceeded expectations. I'm seeing high-quality traffic at surprisingly low cost-per-click rates (under $0.07) from targeted subreddits including r/slatestarcodex, r/PhysicsStudents, r/estimation, r/mathteachers, r/northernlion, and r/EffectiveAltruism. Given my previously stated advertising budget of $2-3k for this year, you can extrapolate what this means for our market. However, it's worth noting that the audience of the ads so far is somewhat small and very selected, so ad effectiveness will likely diminish as most of those people have been served ads.
- The site now includes statistics for individual questions, providing several useful insights:
Average number of attempts other users needed to solve each question
Median first guess for each question (surprisingly, these are often quite close to the correct answer!)
Percentile ranking of your initial guess compared to other users (log space)
Additional user statistics are coming soon, which will be quite nice and include whether you tend to guess too high or low, how your overall accuracy compares to other users, and much more.
@danftz Why r/northernlion? Just because he plays wordle-like games on stream? Very curious how effective that's been haha
@danftz I absolutely love the stats feature! Any chance of being able to export them some time in the future? 👀
Also lol
@Fion I like that! I'm currently working on improving the analytics for individual questions, and I find it fascinating to show what the median first guess of players was, along with some additional information. This is probably quite unique data we're gathering here because, to my knowledge, there is very little information on how well calibrated people are when answering estimation questions like the ones we have in our game.
After that is completed, I'll get to the revamp of the personal statistics and will certainly include the suggestion you've made about showing how often your initial guess was too high/low.