Maybe it should resolve NO. The vast majority of comments on this video appear totally unaware that it's AI, therefore NO, people's first thought is not "is this real or AI".

https://www.youtube.com/shorts/GmoQW683Ahk

I think this is actually proof AI video is already more than good enough to fool the average YouTube user. Just look at the comments.

Clearly the folks who commented on this video (and on the many other videos from that channel) should be asking themselves "is this real or is it AI".

*Edited for clarity

@SIMOROBO Ok, I'll bite.

AI is amazing at generating images. I failed to get more than 50% of the answers correct on the AI Art Turing Test on Astral Codex Ten.

https://www.astralcodexten.com/p/ai-art-turing-test

AI currently sucks at video generation. It can generate gorgeous images and then make a short clip with slight movements of the image, creating a video that feels very static. Longer videos generally have 0.5-5 second segments with jump-cuts between them. Sure, there are human-created videos like that, and those could be confused. And to be fair, the video Yudkowsky originally retweeted has this flavor, but contains still a lot more complex movements than any AI video I've ever seen.

Can you imagine a version of the AI Art Turing Test but for categorizing videos (not necessarily art)? I can already imagine it:

• Video 1: A complicated choreographed dance with complex movements of a group of dancers

• Video 2: A video of a beautiful bird that is mostly static

• Video 3: A clip of a football (soccer) match with an impressive goal, showing multiple players on the field

• Video 4: A video of a flower blooming with a blurred background

• Video 5: An impressive aerial arc shot of a beautiful mountain without cuts

• Video 6: A woman walking, and turning around. When turning, the back of her head transforms into her face.

I think I can pick the AI-generated videos from that list. And sure, maybe I'll miscategorize a human-created video with a lot of static shots as AI generated.

I think Yudkowsky's take is surprisingly good, and maybe his prediction will come true in a few years. We're just not there yet.

@FlorisvanDoorn OK, I'll bite back ;)

The obvious problem is that there is a lot of room for interpretation in this market. Here are the nine combinations that I think rational people could have expected was the true spirit of this market. Open to hear about other interpretations.

AI video generation is at a level of quality that makes it impossible for [anyone, most people, some people] to be able to tell what is real and what is AI for [every, most, some] subject[s].

Personally, I think we've hit the "most people, some subjects" mark based on that YouTube channel alone. I would certainly feel more confident about the resolution of this market if we had made it to "most people, most subjects". In all honesty though, I thought the current state was precisely the minimum required for a YES resolution. If most people are sometimes fooled, they should ask themselves if it's real or AI.

But I understand if some or even most participants here interpreted it at a higher standard of that definition. I just hope we're not moving the goalpost which appears to be overly common with AI and Turing-like tests.

Meanwhile, this makes me wonder how people here would think such a market should resolve if it was about image generation instead. And since you mention it, I would be curious how many here would have changed side after taking the ACX Art Turing Test. Maybe it's not new technology but a reality-check demonstration that is needed to correct one's biases.

I think with image generation we are simultaneously at "anyone, some" and "most, most", but perhaps even just "anyone, most".

@SIMOROBO

I think you're right that there is a (very) narrow slice of videos can fool a lot of people (maybe most, as you said). However, I think this is a very small slice of "beautiful videos". I'm pretty confident AI is not currently able to create a decent alternative for 95+% of the (top) videos on nature subreddits like r/natureisbeautiful or r/natureisfuckinglit

@SIMOROBO plot twist: most of the YouTube comments are bots (I beleive in dead internet theory)

@AlQuinn In that case, Twitter is also full of bots, therefore Yudkowsky was addressing bots with that tweet.

Why I think this is clearly a NO:

Go to /r/natureisfuckinglit and sample the comments of some of the recent posts that show any of the coherency in this moth video. Note how many posts have upvoted comments that mention AI or the possibility of the video being fake.

By my count, I've looked through a few dozen posts and found 0 mentions of AI or the possibilty of the gif being faked. AI video is simply not there yet for people to be extremely skeptical of all video content, which is what this tweet is predicting.

The question boils down to whether "Eliezer was importantly wrong about something, e.g., AI-generated video still sucks, or still couldn't be the cause for serious doubt about whether some random moth footage was made with a camera or not."

The answer is obvious that he wasn't importantly wrong about anything. There have been numerous instances of videos posted that are AI-generated but indistinguishable from reality and even more instances of videos posted and people arguing that they are/aren't AI.

Eliezer was correct with this tweet.

@RiskComplex The question isn't about AI videos in general, it's about every video which is as "beautiful" as the moth video being up for debate whether it's AI or not. What is "beautiful" can be very subjective, but for the question to resolve YES, then AI should at least be able to generate flying moths that look something like the video. But they can't, so this question should resolve NO.

Also, you saying that Eliezer was correct doesn't make sense, because resolving this YES now would mean that Eliezer was wrong, as he predicted AI videos would become that good 2025-2027, not before.

@RiskComplex Yudkowsky made a specific prediction about what my "first thought" would be "anytime" I saw a video "this beautiful". Not my second thought. Not for only some videos.

If he wanted to make a different prediction, that AI generated video would not suck, he could have used different words.

There are many people making vague or rhetorical prophecies and retrofitting them to reality. It's a silly game. We should hold Yudkowsky to higher standards than astrologers.

@MartinRandall Yudkowsky isn't the market-maker. And the market maker states that they will decide if this was a 'good take': YES if AI video is that good and NO if he missed something critical ("AI-generated video still sucks, or still couldn't be the cause for serious doubt about whether some random moth footage was made with a camera or not").

Those are the criteria for YES and NO, whether you like it or not.

@Pazzaz The market maker implies that it is indeed about AI videos in general. "AI-generated video really is that good" and "e.g., AI-generated video still sucks, or still couldn't be the cause for serious doubt about whether some random moth footage was made with a camera or not."

This is referring to the quality of AI video in general.

@RiskComplex "is that good" = as good as the moth video. It specifically calls out "moth footage". It's about videos which are similar to the moth video.

@RiskComplex I was mostly replying to your comment that "Eliezer was correct".

I think it's ambiguous now, but somewhat close to YES. There were cases of generated video being treated as real on reddit with thousands of upvotes.

e.g https://www.reddit.com/r/singularity/comments/1gh5p1y/ai_generated_video_gets_thousands_of_upvotes_on/

some cases are still ambiguous imo, like this one https://www.reddit.com/r/Damnthatsinteresting/comments/1fbmtiu/man_giving_water_to_a_snake/

weirdly static scene, bokeh balls not flashing in and out, but good furniture, but bottle that crumples. Like, i honestly don't know

and also some people suspect real videos now e.g.

https://www.reddit.com/r/ChatGPT/comments/1g3cjnr/is_this_video_ai/

I'd resolve this as YES, but if you count more uhh more common paranoia, then eh. Not yet, it's not really common

@Lavander https://viralhog.com/watch/file/395937723 same guy giving another snake water

ViralHog - Watch

@Lavander I reviewed the polar bear video and my first thought was "huh, is that what the Arctic looks like?". So not what Yudkowsky predicted.

Here's my pitch for why this ought to resolve YES:

The tweet does not say "AI video will be just as good as this moth video". It says "your first thought will be to wonder". Furthermore, the market uses even broader language: will the tweet "hold up"? Is it "a good take"? Could AI-generated video "be the cause for serious doubt"?

Every month or two a new AI video model is getting released. I keep seeing amazing new examples on my Twitter feed. When I see a random cool-looking video, my first thought is already to consider whether it's AI-generated or not. Sure, usually I can still tell one way or the other, after watching carefully. But this tweet was still a good take.

"hold up" is also super open to interpretation

@EMcNeill But the tweet also says that "anytime" you see a video like the flying moth video, you'll wonder if it's AI generated. But generative AI can't even generate flying birds, let alone flying moths. There is no reason to believe that such a video is AI generated.

This should resolve NO (if nothing changes before 2025).

@EMcNeill The AI Videos that have proliferated in the wild are just awful. There's a missing middle of videos that nearly pass, which makes me confident that lifelike AI videos pretty much only exist as promos. There has been nothing in the ballpark of that Moth video and I have zero skepticism when encountering videos with that kind of engrained detail and coherency.

This reddit post is evidence in favor of a YES resolution IMO https://www.reddit.com/r/ChatGPT/s/lSCArBlw5d

@DylanSlagh There is a world of difference between skepticism for a distant out of focus shot of a bear moving and a coherent high detail shot of multiple moth species taking flight organically. AI is not there yet.