So Riffusion exists:
https://www.riffusion.com/about

It uses Stable Diffusion to create visual images of spectograms, which encode audio information. This means that you can convert music recordings to spectogram images, fine-tune Stable Diffusion on a tagged set, and then use keywords to generate new spectograms, and decode those into new recordings.

Will this create a comparable freakout from musicians to what we saw from visual artists in reaction to Stable Diffusion?

This will be somewhat subjective to resolve, but let's say that if I observe a freakout from musicians, and it seems at least 25% as strong as the artist freakout to Stable Diffusion, then this market resolves YES.

You'll have to trust my good faith to resolve this as fairly as I can because a lot of it will be subjective temperature reading. I'll try to supplement it with objective and semi-objective metrics (keyword search counts of e.g. "ai art is bad/problematic", or observing the number of online art communities that explicitly ban generated works versus online music communities that do so, etc). The spirit of the market is the freakout doesn't have to be the same size, but it has to be within the same order of magnitude, and it needs to be clearly visible. If it reasonably feels like there's about 1/4 as many angry musicians upset about Riffusion as there are angry visual artists upset about Stable Diffusion, this resolves YES.

To be clear: this market is not just about Riffusion, but any upgraded versions of it, or any broadly similar generative musical AI that uses this technique (spectogram generation) as a significant part of its pipeline.