Rival crowd prediction site Metaculus allows users to calculate a score quantifying their over- or underconfidence:

I was unable to find anything about this formula or what motivates it. Any hints or sources where to read more would be appreciated.

I'd also be happy to chat about whether we can adapt it for Manifold somehow. My intuition would be that if, say, a user here buys a share of YES at 80%, we would want 80% to be an underconfident estimate so that there is some room for profit left.