
After a comment from the creator of a market, we use OpenAI's o1-mini model to test whether the comment is a clarification to the resolution criteria, and if so, we add a note in the description, see /ian/will-we-judge-autoadding-clarificat
You might've seen a note on questions on the site that look like:
Possible clarification from creator (AI generated): Market will be resolved based on the results of a poll created by the market creator.
or:
Update 2024-31-12 (PST): - Resolution in case of shared title: The market will resolve as a 50/50 split between Magnus Carlsen and Ian. (AI summary of creator comment)
Those were auto-added by the feature in question. Has this been useful to you, or not?
It's exceptionally useful but just needs to be refined.
Ultimately this just comes down to tardiness of users: its super easy to just add these clarifications in ourselves, but most fail to do so hence why the comment section ends up becoming part of the resolution criteria (seasoned users will look at the comment section for more context).
I mention this because the main way this gets solved is when a clarification is made - imo - the author should be prompted if they a) accept the clarification is correct, and b) want it added to the description. If the author fails to accept the prompt, mods could also accept on their behalf. To make this a bit more stringent, you could have the feature actually evaluate whether the user has posted a clarification BEFORE the comment is posted and then prompt the user something along the lines of:
"This sounds like a useful clarification for other users. We suggest adding X to the resolution criteria {accept / decline / accept & modify}"
Here are some other suggestions (in order of importance):
Manifold needs to, in some way, commit to these clarifications being binding in terms of resolution, otherwise the feature is just going to become more redundant and cause further repeated issues.
Clarifications need to be better formatted to avoid mess.
Bullet points should be used.
The following text format should be used:
{disclaimer / warning} {list of clarifications}:
{timestamp} {clarification_1} {comment link}
{timestamp} {clarification_2} {comment link}
Disclaimer / warning needs to be more explicit and noticeable if the author is not prompted to accept / modify / review the clarification.
Clarifications should be retroactively removed and modified so that we don't have 101 different clarifications and to avoid contradictions.
P.S. where's the bounty?? ;)