Ah yes very helpful!

@ProjectVictory market: https://manifold.markets/strutheo/will-i-slay-all-20-top-creators-by

it helped you notice that the market creator gave contradictory resolution clarifications, ggs?

if it made a mistake based on the comments then L

@Bayesian The comment links are broken, so I'm not sure, but it seems AI invented this contradiction completely on itself. There was some discussion about those numbers being different, but I have no idea where it got three different criteria from.

@ProjectVictory lol I have seen that one, the AI trusted BTE’s overconfidence in reporting the new resolution criteria (that worked in his favor) and strutheo’s confusion as agreement. It didn’t make up the contradiction though afaict

@ian it made up a creator statement that was never made, because it was confusing who said what, so it was definitely incorrect. As long as readers click the link they will see what the author actually said, but it's super confusing otherwise.

@jack And the comment links seem to be broken (I tried chrome and firefox).

I love this feature - it's been a big help almost everywhere that I've seen it show up.

So far, it seems to be doing a very good job. However, the AI generated links in the market descriptions seem like they want to take me to the comments they're summarizing, and my app isn't taking me there when I click them. That's just a minor inconvenience, but the core function is very clean.

As an user, I love this feature. It's great to see manifold going back to work on core usability instead of dumb moonshots.

Are creators notified when a possible clarification is added to the description?

@NcyRocks not yet, nice idea though! Could be worth adding, would you rather that happen?

@ian 😬 I’m a bit shocked that it doesn’t happen!! Even changing someone’s market description without their approval is iffy, but changing it without even telling them is pretty terrible! There’s no LLM that I consider reliable enough that I’d be okay with it adding to my market descriptions, even with the disclaimers it currently adds.

Another way of doing this that keeps the creator in the loop would be to prompt them when they post a comment, with something like “this seems like a resolution clarification; would you like to append this comment to the description?”. I’d feel a lot lot more comfortable with that than with it happening after the fact without my knowledge.

@ian I’d be okay with not being notified if the clarifications weren’t appended to the description - say, if they appeared above the comment section like one of Facebook’s AI comment summaries.

@NcyRocks Well, it says 'possible ai clarification' with a link to the comment, so I think traders will take the clarification with a grain of salt if it looks unreasonable. So far the clarifications are very reasonable, and with over 200 clarifications so far and very few reports of the clarification gone awry, I think it's safe to say it's not terrible. I can add a notification to the creator, though, that seems like a good idea.

@NcyRocks Actually I think creators will just get used to having AI read their comments and add clarifications, so the notification will end up being superfluous.

@ian Bug report (on my question about why dems lost election):

The AI didn't understand at all that the prize pool of 500% would be distributed, and was getting way too specific. I think I removed like 5 AI clarifications from my market.

@ian The AI is usually correct and is helpful, but it doesn't understand more complex resolutions. I am now seeing the value of the feature, but I think that an opt-out and notifying the the creator should be implemented

@bagelfan Thanks for the examples! I'll check em out. If you say that you will update the description yourself, the ai won't add a clarification

What’s the advantage to this vs. simply automating “the creator commented here…”? I think it misses some nuance.

@njmkw creators make way more comments than clarifications, that would end up cluttering the description way too much. It links to the comment, so you can check the AI’s work

@ian I think having the AI recognize a clarification and simply copy what the creator said may be more valuable than having it rephrase, but yes people should doublecheck what the AI says they said and make sure it’s what they said

@njmkw ah, that could work. Sometimes the comment they’re replying to should be included in the context, tho, which would lead to a bad experience if users have to navigate back and forth every time. Have any ideas in mind where it messed up?

Over 100 clarifications added so far!