Manifold recently added a feature where AI can edit my market description without my consent. Naturally these edits are terrible, frequently containing incorrect information and not understanding what the market is about at all.
This resolves YES if there's a way for me to turn this off, or it's removed entirely, or the AI gets smart enough that it stops making detrimental edits, or it otherwise stops annoying me.
Update 2025-02-19 (PST) (AI summary of creator comment): Arbitration Costs:
Paying for arbitration costs will not count as a valid solution to the issue.
1,000People are also trading
@LarsOsborne Probably nothing judging by this attempt: https://manifold.markets/IsaacKing/will-manifold-stop-using-ai-to-make#bw2gatvh529
@LarsOsborne Here's a classic chocolate chip cookie recipe:
Ingredients
2¼ cups all-purpose flour
1 tsp baking soda
1 tsp salt
1 cup (2 sticks) butter, softened
¾ cup granulated sugar
¾ cup packed brown sugar
2 large eggs
2 tsp vanilla extract
2 cups chocolate chips
Instructions
Preheat your oven to 375°F (190°C).
In a medium bowl, whisk together flour, baking soda, and salt. Set aside.
In a large bowl, cream together the softened butter and both sugars until light and fluffy (about 2-3 minutes with an electric mixer).
Beat in eggs one at a time, then add vanilla extract.
Gradually mix in the flour mixture until just combined. Don't overmix.
Stir in the chocolate chips.
Drop rounded tablespoons of dough onto ungreased baking sheets, spacing them about 2 inches apart.
Bake for 9-11 minutes, or until the edges are golden brown but centers still look slightly underbaked.
Let cool on the baking sheet for 5 minutes before transferring to a wire rack.
This recipe makes about 48 cookies. For chewier cookies, slightly underbake them. For crispier cookies, bake a minute or two longer until golden all over.
Note how @bens, a moderator and writer of Above the Fold for Manifold, does not trust their own AI summaries and puts this at the end of his markets:
@IsaacKing hahahahahahaha
Well, I think it’s detrimental for ppl like us that write (and update) clear criteria.
But it’s very useful for the majority of markets where creators often make critical clarifications that they don’t update in their descriptions.
Complete hallucination in /SaviorofPlant/what-will-happen-during-the-third-r, there is no such technical setup.
@IsaacKing If you can convince them to add you to this list, the AI will stop interfering in your markets:
https://github.com/manifoldmarkets/manifold/blob/4de01549b1d56ada1527f418fa47611ad71c9e45/backend/api/src/on-create-comment-on-contract.ts#L74-L76
@IsaacKing will you make sure to post all clarifications in your description if i add you to the list?
@ian I'll try, yeah. (Already do, but sometimes I forget.)
I wouldn't object to an AI that waits until an hour after any comment I place, checks if it thinks that comment is covered by the current description, and pings me to add a clarification if not. That seems like something that would actually be useful even with a noticeable error rate. (Just don't ping me multiple times for every comment I place in a drawn-out discussion about resolution criteria.)
Discussion on Discord: https://discord.com/channels/915138780216823849/1369743611197132881
Manifold has made it pretty clear they're refusing to let us disable the feature. Refused to provide a reason.
Manifold has now added a notification whenever the AI edits my questions, which is nice I suppose, in the same way it would be nice if a vandal who keeps destroying my property were to text me a heads up after every time they smash my mailbox.
An example of Manifold's AI making a blatantly wrong addition to my market description. (No, simply paying for arbitration costs would not count as a success.)
https://manifold.markets/IsaacKing/if-i-try-to-fight-paypal-for-my-350
@IsaacKing lol the ai in this market added that as a clarification. I definitely see the problem here.
I don't think we'll add a toggle to turn off the clarification, but I do think o1-o3, etc. will make them better. I also support adding a notification that the AI made an edit
@ian I was personally thrown off by this feature, and would prefer if it was a “suggested clarification” that the market creator can accept/reject. Even just a bot response like “@market_creator Do you want me to edit the description to add [edited text]? Thumbs up to accept.” would be great.
COI: I hold YES on this market.
@ian I suspect the inference costs on o3, even on low settings, could be prohibitive for this for the rest of 2025 - about an OOM more expensive than highest settings o1 on lowest settings o3 from what they've shown
@ian I work for OpenAI so I am obviously among these models’ biggest fans, but my two cents — I think it’s better to empower users, and it seems like this feature is maybe disempowering them at present.
Based on the example, I don’t think reasoning models will help here — the problem sounds like that the model both has incomplete information and has been shoehorned into believing its summary will be Relevant, when it was not.
And if it’s a summary of the market creator’s own comment, I think the market creator can be trusted to make the call. Empowering the human user to override the AI’s suggestion is almost always better for achieving good outcomes, even in non-existential situations like this one!