If someone solves Taelin's $10,000 LLM symbol manipulation challenge within a month, will they receive the money?
36
771
720
resolved Apr 16
Resolved
YES

https://twitter.com/VictorTaelin/status/1776677635491344744


See @IsaacKing's market:

The 1 month deadline refers to the date of the solution being posted publicly, not the date that Taelin confirms it or pays out. If Taelin tries to back out of the offer, that doesn't affect this market; it will still resolve YES if the solution is valid as per Taelin's stated rules.

If that market resolves YES, will the first team to have solved the challenge receive the full 10,000$?

EDITED FROM "If that market resolves YES, will the first person to have solved the challenge receive the full 10,000$?" to make it clearer / more precise

Get Ṁ600 play money

🏅 Top traders

#NameTotal profit
1Ṁ827
2Ṁ576
3Ṁ233
4Ṁ161
5Ṁ94
Sort by:

I encourage folks to check out Metaculus. They take rule construction for markets very seriously there. Which makes sense, as ambiguous markets can be easily manipulated when it comes to resolution.

@gpt_news_headlines Yeah, there's a reason Metaculus is taken a lot more seriously than manifold. I find this question resolution unfortunate.

@chatterchatty Metaculus actually does the exact same thing when unanticipated cases arise (and sometimes does it in much less justifiable cases, which has made me mad too). But as gpt_news_headlines says their rules are generally better written, so happens less often.

@jim Sure, I have no problem with some room for interpretation in very nuanced edge cases, but in this question's case clearly a lot of people interpreted the statement "X receives the full 10k" very differently from what the poll maker (apparently) intended, and the criteria's literal meaning is different from what they intended. In such cases I think - from the PoV of encouraging accurate forecasting - that it is more desirable for the question criteria themselves to guide the resolution, not the question author's hidden, unpredictable thoughts.

Otherwise what we're truly betting on is the subjective thoughts of the question's author, rather than the question itself.

@chatterchatty I'm not entirely unsympathetic but don't go to Metaculus for that - they take the opposite view.

it's like this: markets are meant to have utility: to inform decisions. Rules should be interpreted in service of that utility. This market's utility was obviously to let us know whether or not to bother trying to win the 10k. Any interpretation of the rules should be in line with that -- to resolve NO just because the person who won didn't want the full prize would be totally counterproductive. And, in the long run, such an approach would lead to less informative markets.

@jim I don’t think that was this market’s utility- and I don’t think we should be guessing at hidden states. I’ll leave it at that.

The market was made to predict whether Taelin would scam ppl that solved the challenge, and/or back out / find an excuse not to give out the prize money. Obviously, this didn't happen. being offered the $10,000 prize money, and accepting to only keep 75% of it, is in my mind equivalent to bob receiving the $10k, and creating a subchallenge of his own with $2.5k. but also, i had not foreseen this, and the "winner receives 10k" in the description is not literally met for most usages of the word receive. The comment I wrote here (https://manifold.markets/Bayesian/if-someone-solves-taelins-10000-llm#me5055cymv), that i would have been better to add to the description last week, also supports a YES resolution. I've decided to resolve the market YES, since I think the spirit of the market has been met. I'll managram NO holders the amount they spent on NO shares by market close, and strive to clarify the criteria in the future. Sorry about this.

@Bayesian

IMO rules lawyers suck and detract from the site. If a market is one that's useful and has a point, and an unanticipated case arises, the rules should always be interpreted in a way that follows from the point of the market, not according to the exact letter of the law. If anyone was betting NO because they anticipated something like this happening they deserve punishment not to be bailed out. But yeah I guess it's best to update description as soon as possible when an unanticipated case arises.

@Bayesian Your question's resolution criteria was clear and you did not offer feedback for a long time while the debates were ongoing. The person who received the money himself tweeted that he "received the 7.5k", not "the full 10k".

I appreciate the refund, but we would all benefit if prediction markets adhered to their resolution criteria as much as possible, and betting did not boil down to me predicting the question maker's hidden inner thoughts on what they -actually- meant. I think at best you should have resolved N/A, certainly not Yes.

For the record, I saw all this nonsense coming a million miles away which is why I said just "N/A" the market at the very beginning of all this. I encourage folks to do so now, take a step back, and just try to do better in the future. It's not worth it. Let's just create better markets.

@gpt_news_headlines Also, I encourage folks to be more mindful about overbetting. That is a form of manipulation in itself, especially when the EV is obviously not there.

@gpt_news_headlines it just resolves YES, lol.

Almost gave Bob an extra $2499 just to mess with you guys. What happened is that Bob solved it right after I announced the Wednesday deadline, so I felt it would be unfair w/ the other competitors, and asked if he would consider splitting it. He said we could do a 75% / 25% split. I agreed, and reinforced that it was his choice, as he was obviously entitled to the full amount. He agreed, so we did it that way. A few days later, Bob himself judged and announced @choltha as the winner of the remaining $2.5k, which were paid today.

Regarding my opinion - I think you should bet on famous people! I'm just a peasant. And you should be more clear on the specs. 10k is certainly not 7.5k, so, if someone has bet because they predicted this situation, they'd have the right to complain. They could also be advocating for more literal abiding to specs, which is fair. But as far as intentions go, it seems like the question was asking whether I'd refuse to pay, which I never did. I merely proposed a split, and was very clear that Bob was entitled to the full amount. It was ultimately his own choice how to handle his own prize.

@VictorTaelin

I think it's completely reasonable for you to defend yourself in this manner. I assume nobody pointed you to this market and you found it completely on your own (as otherwise, it would be a form of manipulation, a huge no no in prediction market land). For that reason, I will not take issue with your statement here. It's not your fault someone created this market.

However, as you said, 'bet on famous people'. It's a fair point, or another way of looking at is, do not allow NO bets or shorts on people. There is some great discussion on this problem here - https://discord.com/channels/915138780216823849/1229495298250772482

My feeeling here is N/A the market and let's call it a day. Lesson learned, be more careful about creating markets.

@gpt_news_headlines For the record I DMed him, I have no position here but I also think it would be fine for someone who did have a position to have DMed him too.

@gpt_news_headlines

(as otherwise, it would be a form of manipulation, a huge no no in prediction market land)

This is false. There is definitely no problem with getting information from the source, in fact, within Manifold's rules, it would likely even be allowed for a trader (not the creator though) to bribe Taelin to get the outcome they want.

@chrisjbillington And whilst Joshua DM'd him, in his reply (basically the same as the above comment) Taelin said he was already aware of this market. But yeah, if he wasn't, there wouldn't be any problem.

@Joshua Right, but what else is he going to say? "No, I chose not to pay him..mouhahah."

I mean... it's all good. N/A the market. Honestly, this just proves my point, so I'm good.

@Bayesian it sounds like however this resolves, you can resolve it now.

I can't find the post verifying the 2.5k was paid out though, @Gabrielle do you have a link to that?

@chrisjbillington See this gist comment: https://gist.github.com/VictorTaelin/8ec1d8a0a3c87af31c25224a1f7e31ec?permalink_comment_id=5021146#gistcomment-5021146

Cool! Congratulations @choltha. Will be sending you $2.5k for winning the second part of the challenge. Send me your addresses via DM on X.

choltha said thanks and has commented multiple times in the days since then without any complaint, so I think we can safely say that that has occurred.

I agree that the market can be resolved now and that no new evidence is likely to emerge at this point.

I'm gonna close this until Bayesian can get online, if there are no objections.

I'm online lol, good idea

@Bayesian Good luck with this one hahahaha

The whining NO bettors didn't read the winning prompt:

@Mira Sorry, is that a typo, the 'winning' no bettors? :)

The LLM didn’t do everything perfectly, could have led to more ambiguity with the winner only bringing home $5500