Nice post! Some thoughts that came to mind below.

First, small things:

• The text ends with a title "Making Good". Is this intentional?

• (I was confused by the X-link in "if you don't know what X is", but maybe I'm just missing some MTG context.)

Medium thing: I think the section about giving a probability range could be improved upon. In particular I think the part

"A statement of "I think this is 20% to 50% likely" means "I'm 95% confident that an ideal Bayesian reasoner with the same information available to me would have a credence somewhere from 20% to 50%, but I don't know exactly where it would be"."

is wrong and at least not very helpful (as you say). Four points about this:

• I think an ideal Bayesian reasoner would in the vast majority of questions output probabilities close to 0 or 1. Like, people have a lot of information, and I'd imagine doing something Solomonoff-induction-ish would give you very extreme probabilities for most questions. The issue is that humans are not quite that capable.

• I think William Erlhardt's comment below about the trading price interpretation is a useful perspective.

• My take n.o. 1: Probability ranges account for the fact that the conversion from natural-belief-strengths to numeric-probabilities is lossy, and the width of the range communicates how lossy it is.

• My take n.o. 2: Probability ranges communicate how much your given probability would change if you thought about it longer. By "longer" I mean something mundane (like 1 minute or maybe in more serious contexts 1 hour or more), not "long enough to do Solomonoff-induction type of computations" or anything.

Large thing: Here's an explanation I've come up with that I think is quite good at explaining what subjective, numeric probabilities are about. You can borrow some ideas from it if you like. (This is an excerpt from a longer text I wrote about the topic, quickly translated.)

---

"Humans of course don't naturally think of things in terms of probabilities. Humans don't think about volumes in terms of numerical decibels either. There has been a time when humans haven't thought about temperature in terms of numeric Celsius-degrees (because the scale hadn't been invented yet).

Nevertheless, some things are hotter than others, some sounds are louder than others and some beliefs are stronger than others. You notice this yourself, even though your body doesn't have a thermometer indicating numerical Celsius-degrees or a belief-strength-meter indicating numerical probabilities.

So where do we get the probabilities from? From the same place as the Celsius-degrees. You already have the information, even though not in numeric form. And it's not always easy to transform it to numerical form: maybe it's very cold outside and you think it's -30 degrees, even though it's only -20 degrees. Though you do improve in these transformations as you go: I bet you are better at estimating the lengths of events in minutes than the pitches of voices in hertzes.

Why would one want to convert their beliefs to probabilities? For the same reason that one converts distances to kilometers, mass to kilograms or time to hours (or quantity to numbers). And perhaps instead of "convert" one should say "measure".

Very often one only needs rough estimates. Sometimes "this will take 20 minutes" really communicates better than "this will take a moment" (of course assuming that everyone knows what 20 minutes means). Numerical values are not of course always used, but of course they are used sometimes. In addition one can do modeling with these concepts.

Similar considerations apply to probabilities. Sometimes "I'm 90% sure" really communicates better than "I'm quite sure" (of course assuming that everyone knows what 90 percentages means). And these concepts provide rather good tools for forming beliefs."

nit: "Laplace's aw of succession" - you accidentally the L in "law"

Typo: Tthere's no fundamental "50% ness" to a real coin. (interpretations of probability, second paragraph)

typo: "philsophically"

One interpretation of a probability range that you didn't mention is: trading prices.

If I say 5%-20%, I usually mean that I'd buy at 5% and sell at 20%, and the distance between those numbers tells you how much I'd change my mind if the listener was actually willing to bet me. If I say a flat 12% in this context, that communicates a very high level of certainty; a wider "spread" means I find it much more plausible that I don't have all the relevant information.

No one mentioned that I mixed up frequentist probability with propensity probability!

https://www.reddit.com/r/slatestarcodex/comments/18f1v0c/comment/kcs4em2/?context=3

Explain how the subjectiveness of probabilities is already there when considering a logical reasoner that has to function in the real world. The reasoner chooses its axioms, and observes new true statements from a specific point of view (i.e. its sensor values). A different reasoner may have different axioms, and observe different truths, thus judging a different set of statements as true; all while still being entirely logical. I think that failure to understand this point is a lot of where people's hang-ups with subjective probabilities stem from. It also is simpler to think about (seems pretty obvious once pointed out), helps priors seem like more of a reasonable thing, and shows that the subjective aspect is not inherent to the probability part.