🏅 Top traders

@traders here's the poll: https://manifold.markets/ian/is-autoadding-clarifications-from-t

Just got my first one, and it's incorrect: /NcyRocks/nc-youngs-crystallised-new-years-re

The market has an option for me doing something I wouldn't have predicted in 2024, not 2023; I was just giving examples of things that would have resolved YES in 2024. (Which means that these definitely won't resolve YES in 2025.)

I repeat that I don't appreciate this being done without my knowledge, let alone permission. It's not a huge deal, but it's really annoying and a violation of Manifold's usual principle of creator control.

@NcyRocks Thanks for the report! I will test this case on the model and tweak the prompt so it doesn't happen again. Oh - I think the problem there was that the answer text in the reply wasn't in the context.

@NcyRocks It was missing the close date in context, that appears to have fixed it!

On https://manifold.markets/Nightsquared/who-will-win-the-2024-world-blitz-c

I wrote a comment thread where my first comment included

(To be clear: this is not a clarification. This is just explaining the rules in the market. In case I made a mistake, the rules of the market stand.)

And on the third comment it auto-added a clarification :( Which didn't surprise me - I knew to check for it right away because I knew about this feature, but it's definitely not ideal.

@jack Thanks for the report! I added some notes in the prompt about creators' wishes and switched to o1-mini which seems to be better at following the prompt's orders. I will switch over to o1 once we get access as well.

Nice! I'll just note I do think this feature is pretty great overall - e.g. for a positive example the AI auto-added basically exactly what I wanted on https://manifold.markets/jack/will-netanyahu-remain-israels-pm-un-a635023577e5#kjuy4kxxais

Oh, here's an issue: the date is formatted as 2024-29-12, should be 2024-12-29 lol

Over 400 added! Probably will try hooking up o1 to see if it reduces the error rate

I think this is a pretty good feature, but it can make mistakes so there should be an opt-out for more experienced creators. Vote here if you agree or disagree

@bagelfan instead of a general opt out, I think it might be a good idea if every time the AI suggests an auto clarification, it shows it to the author and the author can accept, reject, or edit it. When I post an actual clarification I want to edit the description anyway and if the AI can help me do that that's fine - I just don't want it to make mistakes.

@jack That feels unnecessary, you can always edit the description after the AI has added its bit, and until you do the "AI clarification" is clearly marked as such and IMHO should be assumed to be non binding

Yeah but I think the biggest thing against that is that there's no notification to the author that it got added (as per a discussion thread below). Sure they are non binding, but that doesn't prevent them from being super confusing if they're inaccurate (users have to go read the comments, and the links don't work). Better to encourage the author to edit it than to force every reader to verify it (1 person who is already making a clarification vs N who are not as familiar)

I think this can be done in a way where it makes the authors job easier and doesn't get in their way unnecessarily!

I think they are pretty helpful on average, although there are some significant number of wrong/misleading AI clarifications, which hopefully can be improved!

Ah yes very helpful!

@ProjectVictory market: https://manifold.markets/strutheo/will-i-slay-all-20-top-creators-by

it helped you notice that the market creator gave contradictory resolution clarifications, ggs?

if it made a mistake based on the comments then L

@Bayesian The comment links are broken, so I'm not sure, but it seems AI invented this contradiction completely on itself. There was some discussion about those numbers being different, but I have no idea where it got three different criteria from.

@ProjectVictory lol I have seen that one, the AI trusted BTE’s overconfidence in reporting the new resolution criteria (that worked in his favor) and strutheo’s confusion as agreement. It didn’t make up the contradiction though afaict

@ian it made up a creator statement that was never made, because it was confusing who said what, so it was definitely incorrect. As long as readers click the link they will see what the author actually said, but it's super confusing otherwise.

@jack And the comment links seem to be broken (I tried chrome and firefox).

I love this feature - it's been a big help almost everywhere that I've seen it show up.

So far, it seems to be doing a very good job. However, the AI generated links in the market descriptions seem like they want to take me to the comments they're summarizing, and my app isn't taking me there when I click them. That's just a minor inconvenience, but the core function is very clean.