When will I mistake an AI-generated song for a popular human-created song, in at least 1 of 5 tries?

Each year, sometime in December, I will ask a trusted person (called the "curator") to pick the 5 highest-ranked songs from the Billboard Hot 100 that I am unfamiliar with. For each song, they will generate a musically and lyrically similar full-length (>3 minute) song by giving an AI music generator a text prompt. I will listen to each pair of songs and guess which one is AI-generated. If I guess wrong at least once, the market will resolve to the current year.

When to run the experiment:

  • There must be a publicly available AI music generator that can generate 5 3-minute songs for less than $50, requiring less than two hours to set up and generate all 5 songs. It should be linked in the comments of this market.

  • I reserve the right to skip the experiment if I think it's >99% likely that I will guess right every time.

  • Between December 1-31 of the year in question, I will decide whether the experiment should be run that year, based on the requirements above.

    • If so, I should run the experiment between December 1-31.

    • If I have already run the experiment or decided not to run it, and then an improved AI music generator comes out during the month of December, I may re-run the experiment, but will not be obligated to.

  • If I am unable to run the experiment, I will find a trusted person without a stake in this market to do it instead. They will pick their own curator.

Process of the experiment:

  • The curator will look at the Billboard Hot 100. If the list no longer exists or we run out of eligible songs, we will choose a suitable alternative.

  • For each song, starting with #1, we will follow this process, until we have found 5 eligible songs.

    • The curator will ask me if I have heard of the artist(s) of the song (without mentioning the title). If I have heard of any artist, we will skip the song.

    • If the song doesn't have lyrics, is less than 3 minutes long, or was generated by an AI, we will skip the song.

    • When I listen to the song later in the experiment, I will stop if I realize I have heard it, and we will skip the song.

    • Otherwise, we will use the song.

  • For each song, the curator will generate a text description by giving an LLM with Internet access (Bard, Bing Chat, etc.) the following prompt:

    • Describe <SONG> by <ARTIST(S)> in 3-4 sentences. Mention the genre, musical style, instrumentals, a summary of the lyrics, and the gender and country of origin of the artist(s). Don't refer to the song or its artist(s) by name.

    • If necessary, the curator may retry, change the prompt, or cut parts of the response in order to get a satisfactory description.

    • The curator will append the text "The song was #<RANKING> on the Billboard Hot 100 in <YEAR>. It is <MINUTES>:<SECONDS> long." to the description. They will use the actual ranking and length of the human-made song and the current year.

    • I will not see the description until after the experiment.

  • The curator will generate a song by prompting the AI with the description.

    • If the song is less than 3 minutes long, they will regenerate it.

  • The curator will use a computer to randomly choose the order to play the two songs.

  • I will listen to the two songs in full.

  • Afterwards, I will guess which one was AI-generated.

  • If I am wrong about any of the 5 pairs of songs, the market resolves to the current year.

If an experiment occurred on November 19, the Billboard Hot 100 songs I would listen to, along with their associated AI prompts would be:

  • #4: "Snooze" by SZA

    • This song is a R&B ballad with a dreamy and soft sound. The instrumentals are minimalistic and subtle, with a sampled voice in the beat. The lyrics describe a relationship where the protagonist is willing to go to great lengths to make it work, even if it means sacrificing her own well-being. The artist is female and from the United States.

  • #5: "Standing Next To You" by Jung Kook

    • This is a retro-funk song with a modern touch. The song has a dramatic, jazz-funk, disco-pop instrumental that compliments the romantic retro aesthetic of the song. The lyrics express the artist’s commitment to stay by the side of his fans in every situation, no matter what. The artist is male and is from South Korea.

  • #6 "I Remember Everything" by Zach Bryan featuring Kacey Musgraves

    • This song is a sparse ballad that reflects a conversation between a man and a woman who used to love each other. The song talks about their past summer beach romance in an '88 Ford. The couple in question reaffirms the authenticity of their past feelings and experiences through their lyrics. The song is a collaboration between two country music artists from the United States.

  • #8 "Fast Car" by Luke Combs

    • This song is a cover of a folk-pop song by an artist from Cleveland, Ohio. The song is about the struggles of a couple trying to make ends meet and their hopes for a better life. The song features acoustic guitar and has a subtle folk-rock arrangement. The artist who covered the song is a man from the United States.

  • #9 "Thinkin' Bout Me" by Morgan Wallen

    • This is a country pop song with a dark, moody ambience. The song features rattling hi-hats and a deep, muted bass. The lyrics describe the singer’s ex-girlfriend who left him for someone better. The singer talks about how whatever it is she’s doing, if she’s thinking about him. The artist behind this song is male, from the United States.

I will not trade in this market, nor will any curator I choose.

Get Ṁ600 play money
Sort by:

This is an interesting market.

What will you do if an otherwise-qualifying human-generated song is less than 3 minutes long?

@BrunoParga As it says in the description, when selecting the song from the Billboard Hot 100, we'll skip any songs that are < 3 minutes

@CDBiddulph oh, okay, sorry. I thought only AI-generated ones below that length would be discarded. Thank you!

bought Ṁ20 2024 YES

@probajoelistic This is very impressive. I've seen Suno before, but this v3 version does seem to be even better than what I saw before. I can definitely tell there are some awkward rhythms (though sometimes that's because of awkward metre in the lyrics), and the tunes are just generally lower-quality than in popular human songs, but I could definitely see this getting good enough to fool me soon. Honestly the lyrics rather than the music are the biggest giveaway for me at this point, which is a little crazy

Remove 2023, It's 2024

@ShyamJayesh Unfortunately it's impossible to resolve individual options in a linked multiple-choice market to no :(

I asked about it, and the feature is hopefully coming soon

I believe there's still no public AI music generator that can generate a 3-minute song with lyrics, so if nothing else happens, 2023 will resolve to NO.

I think the biggest problem with this market is that the minute that AI generated music becomes better than human music, it will top the charts. The models that can do that first will be expensive, so record labels will be able to afford them.

Also, by definition, if AI music is better than human music, it will dominate the charts. That's what I expect to happen, so I won't bet on this market.

@SteveSokolowski I disagree, there's definitely a strong social aspect of music - going to concerts, connecting to the specific artist, and listening to songs with friends. Taylor Swift's fans are not going to instantly abandon her as soon as an AI starts churning out songs that are 10% catchier than hers. By analogy, chess fans didn't stop watching human grandmaster games when AIs became better at chess than humans.

Anyway, there may be a lot more AI generated music than human-made, possibly generated to suit a single listener's tastes. Even people who care 0% about the social aspect of music will likely just listen to their own bespoke songs, so few AI songs will get enough reach to top the charts.

If I can't find enough human-made songs on the Billboard Hot 100, I'll find another list that doesn't include AI songs.

I see that some people have YES bets on 2023. To be clear, if the state of AI music generation is the same this December as it is today, I don't think I'll run the experiment this year. The most advanced music generation AI I'm aware of is Lyria (https://deepmind.google/discover/blog/transforming-the-future-of-music-creation/) but this doesn't seem to be available to the public and can't create a >3 minute song as far as I can tell.

More related questions