Nostradamnedus avatar
Nostradamnedus
closes Dec 31
Will a text prompt based AI music generator that most of the time cannot be told apart from human musicians be publicly available by the end of 2023?
40%
chance

Resolved yes if it beats human music at least 50% of the time in a general audience or expert poll across at least 10 samples of different prompts.

Sort by:
enemel avatar
enemel
is predicting NO at 38%

@Nostradamnedus Could you give your opinion on Mubert? You can enter a prompt and Mubert chooses pre-recorded samples based on that prompt and puts them together.

(It might seem weird for me to be the one to ask since I have one of the largest NO positions, but I'd rather ask this myself than have someone predicting YES bring it up, which probably would have happened at some point.)

cherrvak avatar
Konstantine Sadov
is predicting YES at 44%

https://audioldm.github.io/ Produces audio conditioned on text, not limited to just music. The fact that we can produce such wide variety of sound from a single model strengthens my belief that we're going to get some truly killer AI beatz soonish.

PatrickDelaney avatar
Patrick Delaney
bought Ṁ10 of NO

I'm starting out with an initial NO bet WAG not from having reviewed the literature or having read the comments yet, but from just my initial instinct that music has so many complicating variations across different genres, that it will have a sort of tinnyness to it which will take a while to resolve. However, if a research team comes up with a particular benchmark aimed at a single genre, then I could see this resolving within 2023.

So...will this market accept domain-specific or genre-specific music generators or does it have to include for example, 100 different genres? Does it need to be a generalized music maker? Obviously a generalized music maker is much more impressive, but harder to build.

Another question - does the AI generator constitute merely a large language model similar to how GPT text and image transformers operate? E.g. is it a zero-shot music generator? Or, since you mention, "AI" rather than something specific, e.g. a GPT, then can the AI music generator include some kind of, "listening filter," on the output side which eliminates all clearly horrible or nonsensical sounding music? What if that listening filter includes human intervention / mechanical turk style cherry picking...? Is it still an AI?

ChatGPT includes a lot of cherry picking and human intervention, in the form of constant fine tuning...yet it is still called AI, even though it's human-computer-hybrid intelligence.

dggL avatar
goddezz
is predicting NO at 46%

@PatrickDelaney Also considering DallE is still very distinguishable from human art, I assume training a good music AI is a bit more difficult to create. Not to mention the hurdle of vocals etc.

MathiasBonde avatar
Mathias Bonde
bought Ṁ200 of NO

These new models will fail on prompts such as "Jazz song with a complex drumbeat where the snare is only hit once throughout the chorus", similarly to how text-to-image models fail when asked to draw some specific number of objects.

Like text-to-image, scale might just be all you need to fix this, but doubt that will happen by the end of 2023.

cherrvak avatar
Konstantine Sadov
bought Ṁ100 of YES
PatrickDelaney avatar
Patrick Delaney
is predicting NO at 46%

@cherrvak they say right on the page, "Cherry-picked" so we don't know if this qualifies according to what the market maker has posted. We don't know if cherry picking is allowed.

dominic avatar
Dominic de Bettencourt
is predicting NO at 40%

@PatrickDelaney Also, it's not very good. The quality is pretty low and it can definitely be told apart from human music. Google's AI is much better IMO

Lauro avatar
Lauro Langosco
bought Ṁ50 of NO

Do the different prompts need to specify different genres? It seems like a prompt like "Repetitive house music" is much easier to do realistically than genres including singing or more varied compositions

dggL avatar
goddezz
bought Ṁ300 of NO

@Lauro I think a judge should be shown real and AI music, and if they are indistinguishable then AI wins.

JimHays avatar

@dggL got some big assertions baked into the word “real” there

dggL avatar
goddezz
is predicting NO at 56%

@JimHays by real I mean human made ofc

JimHays avatar

@dggL I know what you mean, but I disagree with you.

dggL avatar
goddezz
is predicting NO at 56%

@JimHays I don't really see how else it will be judged... especially due to how the question is framed

NickBaere avatar
Nick Baere
is predicting NO at 57%

@dggL From the prompt it would be judged against recorded music. Something like electronic instruments only would be interesting.

EMERPUSEMEM avatar

Depends if singing is included

bluerat avatar
blue rat
is predicting YES at 56%

@EMERPUSEMEM Benjamin cosman already asked that question and Nostradamnedus said there doesn’t have to be singing

HenriThunberg avatar
Henri Thunberg
bought Ṁ150 of YES

Published 7 hours ago: https://google-research.github.io/seanet/musiclm/examples/
I think the biggest issue now is sound quality, if it was just crisper many of these would have me fooled. Larger dataset + higher resolution on the audio and this resolves YES.

EricJang avatar
Eric Jang
is predicting NO at 47%

@HenriThunberg do samples count as "publicly available"?

HenriThunberg avatar
Henri Thunberg
is predicting YES at 50%

@EricJang I think the generator is what should be available, not the sound? So regardless if it's 30 sec or 3 min clips, what matters is if users can generate new ones?

What I meant was more an indication of advances and work being done in the field. I think the "Bella ciao" samples were especially impressive!

yaboi69 avatar
yaboi69
sold Ṁ69 of NO

@HenriThunberg This is impressive and promising, but questions remain. How cherry-picked are the examples? Can longer extracts follow a sensible form and structure? Can it stick closer to the descriptions than in the examples? When will a product like this be publicly available?

HenriThunberg avatar
Henri Thunberg
is predicting YES at 62%

@yaboi69 yeah sorry, a bit hasty in my "resolves YES" comment. I think @Nostradamnedus needs to set out what the test will look like.

Idk, something like "For a publicly available model: Given the best possible prompts on ten different instrumental genres, the hit rate is less than xx% in guessing whether it's AI output or not when paired with human-created samples, from a general audience (say Twitter or Manifold poll)."

Nostradamnedus avatar

@HenriThunberg Added criteria in description.

osmarks avatar

This seems like quite a high standard; I don't think we've achieved this for other nontrivial generative tasks.

bluerat avatar
blue rat
is predicting YES at 35%

@osmarks I mean we’ve had ai music generation capable of being indistinguishable from human generated music for years now haven’t we, I don’t think adding a requirement of text based prompts is that high of a standard, assuming someone actually tries I think music generation will be one of the easiest nontrivial tasks to reach that standard.

enemel avatar
enemel
bought Ṁ100 of NO

@bluerat

I mean we’ve had ai music generation capable of being indistinguishable from human generated music for years now haven’t we

could you give an example?

bluerat avatar
blue rat
is predicting YES at 31%

@enemel i'm was specifically thinking of Emily Howell which has been around since at least 2009, i'm seeing that actually theres some debate about whether it has passed the Turing test, and many of the original sources cited no longer exist, but it's good enough that I believe it could have passed a Turing test.

cherrvak avatar
Konstantine Sadov
bought Ṁ300 of YES
osmarks avatar

@cherrvak It doesn't seem very good from my limited testing. It's a bit jumpy and the sound quality seems bad somehow. Also, I couldn't make it do metal.

vluzko avatar
Vincent Luczkow
is predicting NO at 71%

My skepticism here is mostly just that I think it's reasonably likely that no major lab will try in the next year. On a technical level I think this is very doable.

Isaac228c avatar
Isaac
is predicting YES at 37%

@vluzko I think I read somewhere (no source right now) that stability ai are trying

BenjaminCosman avatar

Are we talking about instrumentals only here, or realistic sounding voice+lyrics (which sounds much harder)?

Nostradamnedus avatar

@BenjaminCosman Good point, I think instrumentals will suffice as long as not genre-limited.

BenjaminCosman avatar

@Nostradamnedus Also what does "most of the time cannot be told apart from human musicians" mean - at the low end we might have "most people listening to it out of context, especially a small snippet, would not immediately suspect it's AI", and at the high end we have something like "music experts given one obscure human piece of music and one AI one can't identify the AI one much better than 50% chance"

Nostradamnedus avatar

@BenjaminCosman A Turing test with non-experts.

Sophia avatar

@Nostradamnedus So something like Musenet?

?

Sophia avatar
Sophia
bought Ṁ10 of YES

@Sophia (That is not how I expected embed link to work, sorry.)

IsaacKing avatar
Isaac King
bought Ṁ0 of YES

@Nostradamnedus Who selects the human-created music that the judges listen to?

ManifoldDream avatar

Will a text prompt based AI music generator that most of the time cannot be told apart from human musicians be publicly available by the end of 2023?, 8k, beautiful, illustration, trending on art station, picture of the day, epic composition