Based on this tweet from Spencer Schiff:
https://x.com/spencerkschiff/status/1910106368205336769?s=46&t=62uT9IruD1-YP-SHFkVEPg
Resolves YES if a mainstream AI model (so like, Google, OpenAI, Anthropic, xAI, DeepSeek) can repeatedly solve this benchmark, using this image and other similar ones I draw. I will be the judge of this (so please do not lobby me with your own attempts, except to alert me to the likelihood that a particular model can do this so I can try it for myself).
“Solving the benchmark” means being able to match the names to the colors of the stick figures, repeatedly and with simple prompts.
I’m not paying any money, so must be the free version of an AI model for me to verify! Again… FREE VERSION!
By the end of the year!
The image for reference in case the tweet is deleted:
…and here is what ChatGPT gives me now:
…and Gemini 2.5
I will not bet in this market to remain objective.
1,000People are also trading
I'll make a version where api calls for less than a few $ are allowed if that is ok. i think freeness affects the question a fair bit and makes a bigger % of the question dependent on stuff like which labs decide to offer frontier models for free which i'd rather isolate from the specific capabilities question
@Bayesian ya that's fair, I mainly just didn't want to litigate something where someone was claiming some paywalled AI could do it and was sending screenshots of their tests.