If you look at this, I think it looks good enough that you would reasonably have to consider for almost any video of this length that it might be AI generated.

https://x.com/kimmonismus/status/1871539069475238280

@JonTdb03 This video is pretty impressive, but I was able tell it was fake, since lego sharks are not real.

@Jacy This one from Sora shows a humming bird flapping its wings. What artifacts or issues do you see?

@RiskComplex can you link to the source?

@Jacy I'm personally the source. Is the gif not clear enough? I can make it an .MP4 if you prefer but the quality is the same.

@RiskComplex with what engine? It's just hard to tell what's going on with the pixelation and changing frame rate. Wing motion looks pretty good though!

Can anyone make or link ANY example with Sora, Veo, etc, that is anywhere near the quality of the video that Eliezer linked? In terms of detail and consistency

Like, all the generated videos so far are 1) aesthetically very interesting, 2) completely unbelievable if there is any kind of non-trivial movement

@PeterBuyukliev

https://x.com/minchoi/status/1866191030829084698
https://x.com/metastable_1/status/1758669580464263583

https://x.com/tsarnick/status/1868762696881234093?s=46

https://www.youtube.com/watch?v=G9RDHs9nx04

https://x.com/hua_weizhe/status/1868767410494619702?s=46

https://x.com/Plinz/status/1868885955597549624

https://x.com/shlomifruchter/status/1868974877904191917

https://x.com/i/status/1868900000463528054

@RiskComplex the videos you linked are all interesting but:

1. The monkey blinks and looks around. Eye movement like that is briefer and much simpler than the slow-motion moth flapping.

2. The bird saccades are similarly quick, though they do have a whole head moving (with a impressive set of feathers!) rather than just eyes.

3. The closest movements in the Twitter Veo 2 montage seem to be a long, consistent flower blooming and a jellyfish cap undulating. Both are much simpler than wing flapping, and the Veo 2 montage is of course made from many prompts and their selected outputs.

4. The YouTube Veo 2 montage is mostly human movement but otherwise similar to the above.

5. The cat running and jumping onto the couch is the most impressive IMO, though various limitations: distant shot, repetitions of a single type of movement with each step, etc.

6. Penguin waddling (and flying a bit at the end) with nice paraphernalia and background

7. Dropping blueberries in water (cool water movement! though lots of that in the training data from other things falling into water)

8. Dog running (about as impressive as the cat running, with a more complex and novel background but no jumping)

Does one of these in particularly strike you as comparable to the montage of moth wing flapping? I think the walls of links make it hard to judge the best-case examples.

I would also expect that if Sora or Veo could do this, you could find examples of wing flapping specifically instead of needing to compare examples from other animals, humans, plants, or inanimate objects.

@Jacy needing to carefully evaluate every example like this is the world the tweet suggests.

@Jacy there were things in there where my gut reaction was that it must be real footage that was being misrepresented as ai generated footage, I might just be more gullible than I thought I was, but I think that means it's there, even if knowing more about it's limitations would help me spot them better. You didn't give any reason why the bird in the second link would not qualify, I don't see why the cat (or the dog) footage being repetitive movements from far away would make it not qualify, the moth footage was equally repetitive movements imo, and I don't see how being far away much matters. I also don't think it matters that there would be lots training data from other things falling in water. The tweet wasn't about whether ai video generation will be good at everything, it's about whether you have to ask yourself whether it's real each time you see something like that, and I think at this point you do.

@LiamZ what do you mean? I was just going through the examples in detail to give a considerate response. I would have done that regardless of video quality because I wanted to explain my reasoning.

@bluerat the bird just isn't an attempt at motion or physiology as complicated as the moth montage. There was some more informative discussion of that video when it was first publicized that you could look at for analysis (e.g., comparison to footage of a real Victoria crowned pigeon). I don't have more to add on your other points, e.g., there still seems to be more repetition in the dog/cat videos, and accurate complexity seems to clearly be a lot harder with close-up video.

@Jacy The world you're describing sounds to me like "when I see cool nature videos on the internet, I think about the motion and physiology for a bit. If it seems sufficiently complicated, then I know the video is probably real. But if it's a short clip of a mammal or a non-flying bird I assume it could be AI-generated." Do you agree that's roughly what you're saying?

@Jacy your points generally seem like post hoc rationalizations and some of them are just saying that there’s probably close examples in the training data. That’s obvious because that’s the basis of the tech and there’s no reason moth footage wouldn’t also be included.

If you have to stop to wonder and evaluate based on subtle guesses about ill defined complexity of movement then we’re in that world.

Compare your reasons here to the reasons you’d have given in response to this video 7 years ago:

https://youtu.be/5DaVnriHhPc?si=AXTbygMTZ3OHolQq

@placebo_username no. I'm just explaining particular reasons why I think each of them isn't the the sort of video that would make it so, for Yudkowsky's typical follower, their "first thought will be to wonder whether it's real or if the AI's prompt was 'beautiful video..." as the tweet says.

@LiamZ it's your assumption that one has to stop and wonder that I disagree with, not necessarily the implications of that assumption.

@Jacy stopping to evaluate exactly how complex you think the motion is means you’d have a preceding moment to wonder if it could be generated.

@LiamZ you make it sound like the medusa, just looking at it once makes you question reality. We should all start looking at our screens through a mirror, just to be safe.

@Odoacre looking at photoshopped images also means we had to stop to wonder and check if surprising images are likely fake for the last 20 years. 😱

@LiamZ so you're arguing that the very existence of AI videos should make us question if random videos are real or not? If so then why should Elizier have predicted a date in the future ? AI video already existed when the OG tweet was made.

@Odoacre I’m arguing https://manifold.markets/GeorgeVii/will-this-yudkowsky-tweet-hold-up-2#b4v3ajzea9m

@Jacy But to be clear, your argument for videos 1-8 isn't that you can easily tell they're AI [comment edited], but rather that they're simpler than the moth video. So do you agree that for Yudkowsky's typical follower, when they see a viral short clip of a mammal, their first thought will indeed be "this might be AI generated"? In other words, if the original tweet had featured a cat video rather than a moth video, you would lean towards a YES resolution?

@LiamZ I'm not sure where you're getting that. I was evaluating the listed videos as a part of discussing this market's outcome, not because I was wondering if they were AI-generated. My reasons for evaluation are very different from the likely reasons that a typical Yudkowsky follower would stop to evaluate a random video they saw on Twitter.

@placebo_username I think their simplicity is one of the reasons that such videos would not make the typical Yudkowsky follower start doubting random Twitter videos. They are AI-generated, so I don't understand your first claim about them not being AI. I'm sorry if my statements are confusing here. I'm just trying to word them in very explicit ways for maximum clarity.

And no, I do not agree that AI can generate mammal videos at the level required by this market. The mammal thing isn't particularly important. It just came up because the cat and dog videos that I find most impressive out of these happen to be of mammals. Of course, the frequency of cat and dog videos is so high in the training data that I do expect AI to be better at generating those, all else equal.

@Jacy what I’m getting at is that, when doing your evaluation to differentiate you couldn’t say, “this cat has 6 legs. This looks like garbage, nobody would be fooled by this.” as you could a few years ago but instead “moth wings flapping from a still camera shot meet some undefined metric for being a more complex movement than a cat jumping in a moving camera shot.”

And some of your reasons are literally just saying, “this is in the training data” but that's obvious and moth videos are also included in the corpus so it reads as just needing to say something as the alternative is stating that it is convincing in those cases.

@LiamZ I wouldn't frame it that way, but I agree AI-generated cats usually now only have 4 legs.

AI is much better at things in things more frequent in the training data. Arguably, this is the defining limitation of modern AI architectures. I would guess that if we operationalized "things like blueberries dropping in a glass of water" and "things like the moth montage," then we'd identify more of the former in the training data, but I certainly agree that's really fuzzy and subjective.

Just to be clear, my list of 8 was not meant to be some knockdown proof that these videos would appear as clearly AI-generated if they showed up on social media. Indeed, I think many AI-generated videos are now indistinguishable! I just thought thinking through the upsides and downsides of each might be interesting as to why I continue to expect NO will be the correct resolution, given some YES holders seem to find that incomprehensible. (And as a case study in why Manifold systematically, time and time again, forecasts more progress in AI than what actually happens.)

@Jacy

I think many AI-generated videos are now indistinguishable!

Good luck convincing people this is true and also that they don’t need to stop to wonder about and evaluate the origin of nature videos.

I hold no YES shares.

@Jacy Sorry, there was a typo in my original comment. I have edited it, hope it makes sense now.

The "simplicity" is what I'm trying to ask about: Do you agree that when you see a "simple" video on twitter, it is now reasonable to wonder if it's AI-generated? i.e. if the original tweet had featured a cat rather than moths, you would have considered it a good take?

@placebo_username videos of cats with the complexity of the moth video? No. Videos of cats or moths doing the simpler things like walking from a distance with a simple background and a single clip instead of a montage? Maybe.

edit: removed