See also: /SteveSokolowski/will-ai-achieve-superintelligence-i

With 2.25 years until the linked market resolves, and based on how close all types of models are to human level, I thought it might be interesting to hold a Turing Test variant, with mana for being right.

Song in question: https://shoemakervillage.org/temp/07-a_little_chaos.flac

Lyrics: https://shoemakervillage.org/temp/lyrics.txt

This market will resolve to YES if the following story is true:

I started out by using GPT-4o and Claude 3.5 Sonnet and asked them to create a song about music that is "ridiculously catchy," and would be as likely as possible to get stuck in a human's head. I obtained the Udio, Suno, and other audio models' documentation, as well as human-generated webpages where people posted prompting tips, and add them to GPT/Project knowledge bases.

I asked the models to generate the style, key, tempo, mood, and key for the song, and asked them to output both the lyrics and the bracketed singing, effects, and instrumentation instructions. I bounced their output back and forth until they largely agreed.

I used the audio models to generate the audio. I used the very first output as the arrangement. I then used "remix," "inpaint," and "extend" to get the rest of the song, selecting the best results out of the outputs.

I input the music into Gemini 1.5 Pro 002 and asked for an evaluation on a scale of 1 to 100. It returned a 62. I followed its instructions for improvement, and asked the other models if they agreed. I continued following Gemini's instructions and told it to continue making improvements until the song could be played on the radio. The final version was rated a 92/100 by Gemini.

The process required five hours from start to finish and cost $1.87.

I never did any of the following:

• I did not propose the idea or the song's storyline

• I never composed a single note

• I never recorded any instruments

• Nobody sung a single word

• I never manually wrote any lyrics; only objecting to inappropriate lyrics by telling the model to regenerate them

• I only used Audacity to add together the stems and losslessly compress the output

• My role was limited to copying data between models and selecting model outputs

• I never asked a human for an evaluation of the song

• Every single byte in the audio file is synthetically generated by models

I will wait one week for the market to close, and then reveal the evidence. If the story above is true, then the market will resolve to YES. Otherwise, it will resolve to NO. I won't bet on the market.

If significantly more people lose money than gain money, then clearly it is difficult to tell whether a human has produced music anymore, and the state of AI models is more advanced than many are aware of.

The resolution is NO and is explained in the first comment.