Will the "AI make a movie" market reach a new equilibrium by September 2024?
29
393
αΉ€550
resolved Feb 2
Resolved
YES

I've been a bit obsessed recently by Scott Alexander's market about whether an AI will be able to generate a high-quality film to a prompt by 2028. I find it surprising that it's been sitting almost exactly at 50% for almost 6 months. Right now there's some pretty heavy market-making going on, and I think we might have "Keynesian-beauty-contest'd" ourselves. (I'm as guilty of this as anybody.)

As well as the market makers, there are a few people who think the true probability is significantly higher and a few who think it is significantly lower. While it's possible both groups are wrong, and the true probability is 50%, it is also plausible that we are all missing something, and it should be significantly higher or lower.

If I'm right, then at some point, presumably the market will reach a new equilibrium?

To define "reaching a new equilibrium", I will require the market to settle either below 44% or above 58% for at least a month. I reserve the right to wait longer than a month if I think some whale is throwing mana at trying to make this market resolve positively. (Seems like a small risk since the main market is very liquid and this one probably won't be.)

Title says "by September", but specifically, it's a year today: 15th September 2024.

I may bet in this market. I promise not to abuse my (very minor) discretionary powers outlined above to try and turn a profit.

https://manifold.markets/ScottAlexander/in-2028-will-an-ai-be-able-to-gener

Get αΉ€200 play money

πŸ… Top traders

#NameTotal profit
1αΉ€186
2αΉ€105
3αΉ€64
4αΉ€49
5αΉ€33
Sort by:
predicted YES

This result has somewhat increased my credence that the main market was in an irrational state for ~9 months. It got up to around 50% in March 2023 and stayed there very stubbornly until December 2023, at which point it went down. I don't think there was any significant new information that came out in December - I think we all just decided to trade at a different price.

Was it mostly due to @firstuserhere and @Mira stopping buying YES? Possibly, but firstuserhere stopped buying in like September, didn't he? That didn't affect the market. Mira's dramatic exit sent the market briefly down to 0% then back up to 50%, before a steady decline to the 30% we see now. Causation or correlation?

I think I personally have a failure mode of updating too much on a prediction market. If you asked me my opinion on Scott's question without showing me the market, I think I would have said 10%. But when the market was at 50% for so long, I probably updated my estimate to like 45%. Markets know better than me, after all! At one point I was the second-biggest NO holder, but then I reduced my position and just did some market making. For a few months nobody else was doing much market making and I made quite a bit of profit from that. (Then others came in with heavier market making, most notably @DylanSlagh and I didn't make much more profit from it.) But I feel like I get virtually no prediction credit for my behaviour in the market. I might have had a well calibrated estimate at first, but the market convinced me I was wrong, and instead of gradually accumulating a big NO position (which would have given me big profits when the market finally moved to 30% in December) I just tinkered around the edges not really committing to anything. In fact, in my own small way, I contributed to keeping the market high - I had YES limit orders at 49%!

Lessons to learn? Hard to say. On the one hand, if you completely ignore the market, you see it's at 50%, you spend all your money on NO because your true estimate is 10% and then you have no more money. On the other hand, if you pay too much attention to the market you do what I did: contribute to keeping it high and make much less profit than you otherwise could. But there must be something to learn. We all messed up for 9 months. What were we playing at?

Grateful for others' input!

(Obviously it's possible the market price was rational and the information landscape changed in December. I don't think this is true, but it's possible.)

predicted YES

@Fion Thanks for the call out. I had fun market making. I think the fact mira destroyed all the YES limit orders had quite a bit of a psychological impact.

I thought 50% was too high but given that all the biggest companies in the world are frantically researching in the AI space there’s no telling what might come from that in the next couple years. I think 30% might still be too high but we will see.

predicted YES

@Fion My own estimate at the start was under 2% (Scott Alexander own prediction in his blog post). It the first day, I bought 700 NO from 40% to 30%. That was quite a lot for me at that time. Then I only bought NO, as the market was going up, I started wanting to buy only above 50% to get more profit. Maybe I should have buy more, getting more in loan, and still trade at a good price. So effectively I was also not driving the price down. I'm not sure I have enough weight for that anyway.


Of course the price can be driven by a few whales, but if there was really a consensus about it, there would be other whales on the opposite side, and also many other participants.

So I think the price was quite a fair representation of the view of Manifold population. There was a huge amount of hype over everything AI (there still is) and many have a simplistic view of it. They think progress is warranted to be more than exponential. They tend to ignore AI systematic failures and think they will be solved soon by just bigger models.

These things won't go away with only more of current technology:

  • lack of originality

  • no complex reasoning

  • for image, composition issues especially for unusual thing (see frog riding a bird market, DallE is not so bad)

  • for video, no physic model so weird thing happens.

  • hallucinations

I'm really impress by AI progress, but the bar for "AI make a movie" is just way too high.

predicted YES

@Fion my perspective is that it was irrational then and it's irrational now. It's still being propped up by entrenched positions and loan recycling. @mckiev in particular has sunk considerable cash money into big limit orders which stalls any market slides. Once the market stalls, it feels more like a futures/meta market where you try to predict the tug of war instead of the ground truth. We only got down to 30 because Mira nuked all the YES limit orders.

bought αΉ€5,000 of YES

I think this has happened now. I'll check properly when I'm at my computer

predicted YES

Last time it was at 44% or higher was 2nd January at 12.36 my time. It's now 19.44 my time. That's over a month. Resolves YES.

predicted YES

Just looking at the market graph the last time it was at 44% was Jan 2, although one could argue that the "equilibrium price" dropped below 44% percent around dec 31 and jan 2 was just a spike

This market seems too sensitive to manipulation. Do I understand correctly that one user moving the linked market inside the 44 - 58 range, even if for a second, would restart the month long β€œsettled” timer?

predicted NO

@alextes I'm deliberately allowing myself a bit of flexibility to reduce the risk of manipulation. If it jumps just within the 44-58 range for a second, I won't restart the "timer". If it stays there for a day I probably will. But I'll also take into account factors like:

  • how far outside the range the market was sitting before and after

  • how big the spike was

  • how many people are responsible for trades that drive the spike

  • whether the person/people responsible for driving the spike have a stake in this market.

I think manipulation is often obvious when you see it, and I think it's harder the vaguer the rules are. (Unfortunately honest trading is also harder with vague rules... My aim is to get the best of both worlds with being clear and transparent about what I plan to take into account, but not leave myself open to being forced to resolve on a technicality.)

If I suspect manipulation I'll discuss it in the comments to see what other people think, and ultimately act according to my best judgement.

Finally, I'll just note again that this market is still vastly smaller than the other market, and anybody making big trades in the other market to try and make profit in this market would probably lose more in that one than they'd gain in this one. Obviously I can't rely on that logic, but it's one more thing making us a little safer.

(I won't take it personally if you don't like the sound of this and would rather not trade!)

@Fion Thank you for the thoughtful reply. Indeed, in that case I’ll refrain from trading. If the core element we’re trying to predict was clear I’d be fine with vague rules + honest reputation but as β€œequilibrium” is a vague idea already having vague criteria on top is too much for me.

I’m sure others with higher risk profiles can have a good time and make mana still. Your intention at least is clear πŸ‘.

predicted YES

@alextes I don’t think manipulation is that big of a concern given that the original market has a huge amount of liquidity compared to this one. I would gladly pay 1000 mana for the opportunity to buy NO at ~50% again in the other market

bought αΉ€30 of YES

Kinda close at 45% rn

I'm thrown off by the word "equilibrium" and "settle" in the title and description. How will this market resolve if:

  • the linked market stays below 44% for a month but wildly fluctuates?

  • the linked market stays below 44% for a month but has a clear downwards trend?

bought αΉ€10 of NO

@FlorisvanDoorn fair point. I probably am making some assumptions with those word choices.

For the first one, it depends what you mean by "wild". If I judge that it is so erratic that it is likely to jump above 44% any time, then I might wait longer than a month to see if it settles. But if it seems to be oscillating around a particular level, and those oscillations don't take it above 44%, but they're a bit bigger than they are at the moment, that's fine. That's still "equilibrium", it's just in a potential well with less steep sides, to use a physics analogy.

For the second one... yeah, I think I'll have to count that as well. It's not settling to a value, but you could argue it's settling to a trend that is markedly different to the current one.

So yeah, maybe not the best choice of words, but basically the numbers are the main criteria, with the caveat that if in doubt, I reserve the right to wait and see.

predicted NO

@FlorisvanDoorn by the way, I should say that I wasn't expecting @firstuserhere to carry out this interesting experiment (removing limit orders) at the same time. If we get below 44% during their experiment, and if it seems to stay there, I'll probably invoke my wait-and-see in case they come back. I think this will be moot, because I don't think they'll let it stay below 44% for a month, but just getting it in now just in case.

predicted YES

@Fion P.S. I'm also not going to make trades on that market for the rest of September, to remove my influence as much as I can at this point.

boughtαΉ€10NO

@FionBull you don't think the market will move much in 1 year? (even towards NO side)

bought αΉ€10 of NO

@firstuserhere FionBull is my alt. I make bets with it instead of my main account when my gut tells me one thing but I don't feel like I can rationally justify the level of confidence. (It's Fion but more bullish)

So yeah, my instinct is that the market isn't going to move much. I don't understand why it's been so persistently flat, but I reckon it'll remain so.

To be honest, I think if it does reach a new equilibrium in the next year, it's more likely to be upwards than downwards. A big leap forward in AI progress could move the market up, but even in September 2024 there's a lot of time left until 2028, so I think a lack of AI progress will not necessarily move the market down.

bought αΉ€10 of YES

true probability is 50%

aka state of maximum uncertainty? I think with 1 year of extra time, we will accumulate enough bits of information to reach a new equilibrium

bought αΉ€10 of NO

Not directly related, but I've recently been reducing my position in the market. I think I over-Kelly'd myself initially, and I got spooked by @firstuserhere's prediction that it would reach ~65% by March.

Me leaving the market has had absolutely no impact, despite being the second-biggest NO holder at one point. If anything, it's got lower since I started selling NO.

bought αΉ€30 of YES

@Fion I will use this market's timelines as a checkpoint to re-evaluate the direction and amount of my bet

predicted YES

@Fion Doesn't look like it will get to 65% by March.