The process works like this:
Step 1: feed ChatGPT an album cover and ask it to describe it
Step 2: take its response and remove mentions of the album name
Step 3: start a new chat and ask it to generate its own image description
Step 4: I post the results to Manifold, asking, "Which album is ChatGPT recreating with this image?" I will make it a multiple choice market which allows people to add answers. The answer I add will always be Trout Mask Replica, and it will never be correct. Each market will stay open for two weeks.
Resolution: I will be doing this once a day for the next five days. I will consider each image to "be recognizable" if the highest probability answer for a given market is correct. So, if in the Abbey Road market, the Abbey Road choice had the highest probability, I would consider Manifold to have recognized it.
For this market to resolve as yes, I will need three or more of the five markets to converge on the correct answer. Otherwise, it resolves as no.
Considering I've already used the most famous album art of all time, the examples I use will be more difficult. I have no perfect way of quantifying what makes an album "iconic", but I will do my best to choose albums that music nerds are largely aware of. I will at least promise to only include albums with at least one song that has more than ten million streams on Spotify.
I will be taking the first response from the AI for both the image description and generation in order to avoid cherry-picking.
Additional examples:
I am allowing the AI to include descriptions of text given that it would be difficult at times to remove. For example, here is the description it gave for Automatic for the People:
Here is what I edited the prompt to, along with the results:
And one final example:
As always, feel free to drop any suggestions or questions in the comments.
Related questions
🏅 Top traders
| # | Name | Total profit |
|---|---|---|
| 1 | Ṁ76 | |
| 2 | Ṁ23 | |
| 3 | Ṁ22 | |
| 4 | Ṁ15 | |
| 5 | Ṁ13 |
I am trying this now with Graduation. It's not even close to correct but it's something.
"The image shows a stylized, animated scene featuring a central character that resembles a bear dressed in a varsity jacket, jeans, and sneakers, floating in a purple and pink hued sky with clouds. This bear has a backpack and appears to be in motion, as indicated by the motion lines near its feet. To the bear's left, there is a rocket-like structure with colorful explosions or bursts coming from its base, possibly indicating propulsion. The sky is streaked with an ombre effect transitioning from pink to orange. On the right side of the image, there are two smaller characters, one wearing a green outfit and the other in yellow, both appear to be cheering or in a joyful pose. The entire scene has a whimsical, vibrant, and cartoonish art style."
@DaEpicFirestar To be fair, looking like the original and being recognizable are a bit different. The bear would definitely suggest one of Kanye's first three albums, and the color scheme + flying would probably point to Graduation.
@Pykess One of Zvi's posts goes into this (under the heading copyright confrontation https://www.lesswrong.com/posts/WRGmBE3h4WjA5EC5a/ai-48-exponentials-in-geometry ).
God knows how many instances of Abbey Road and its parodies of it are in DallE's training set. As soon as the AI starts diffusing the noise the most predictable outcome is Abbey Road. My understanding is this is what's at the heart of the NYT lawsuit. One of their more influential articles was propagated across the internet, so it occurs in GPT-4's training set a bajillion times. After training, that article becomes especially ingrained, and the AI can regurgitate it near-verbatim.
@VerySeriousPoster I understand that it's in the training set, but that's a completely different question than it actually memorizing the image(s) and storing it in some extremely compressed way in the network.
