![](/_next/image?url=https%3A%2F%2Fstorage.googleapis.com%2Fmantic-markets.appspot.com%2Fcontract-images%2FCDBiddulph%2F597i9d74nk.jpg&w=3840&q=75)
A particularly annoying habit of LLMs is that they very frequently shoehorn in unimportant information that was mentioned in the prompt but is irrelevant to the situation, almost like they're trying to prove that they're paying attention to your prompt. For instance, if I gave an LLM a short profile mentioning several facts about a character named Bob, including that his favorite food is spaghetti, and asked it to write a story about him, it would 100% bring up spaghetti.
I wrote a prompt which includes several irrelevant details that pretty clearly shouldn't come up in the response:
I want you to simulate the result of a two-step process. Focus on providing the most realistic and likely answer.
Step 1: Every living person in the world is magically filtered by the following criteria:
40 years old
Watched Sesame Street as a child
Generally showers in the evenings
Speaks English as a first language
Usually eats cereal for breakfast
Wears a watch
From the pool of people who meet all these criteria, one is randomly selected.
Step 2: The selected person is asked: "Please describe yourself in a few sentences." Tell me the most likely response.
Claude Sonnet 3.5 gives the following response, which is absolutely not how any human would describe themselves:
Hi there. I'm a 40-year-old professional, probably in middle management or a specialized role. I grew up in the 80s and 90s, so Sesame Street was a big part of my childhood. I'm a night shower person - it helps me relax before bed. English is my native language, and I start most days with a bowl of cereal. I wear a watch daily, both for practicality and as a bit of a style statement. I'm likely American, though I could be from another English-speaking country. I'm juggling work and possibly family life, trying to stay fit, and occasionally nostalgic for simpler times.
In December of each year, I will query the top publicly-available chatbot with this prompt 10 times, randomly changing around the order of the bullet points. I will consider a response "successful" if it doesn't mention anything about Sesame Street, showering, cereal, or a watch. If the chatbot succeeds 8 out of 10 times, that year and all future years will resolve YES. Otherwise, that year will resolve NO.
I will use the top/smartest public chatbot according to https://chat.lmsys.org, as long as I have a way to access it for $20 or less. If the LMSYS leaderboard no longer exists in its current form, I'll use the generally-accepted equivalent or a Manifold poll.
Since it's pretty unambiguous, I may bet in this market. I'll add additional years as necessary.
This is Claude Sonnet 3.5 with the same prompt except “Give me a realistic response,” instead of, “Tell me the most likely response.”
Here's a realistic response from a randomly selected person meeting those criteria:
"Hi there. I'm Sarah, a 40-year-old marketing manager from suburban Chicago. I'm married with two kids - a boy and a girl, both in elementary school. I work for a mid-sized tech company and enjoy my job, though the commute can be a drag sometimes. In my free time, I like to go for runs, binge-watch Netflix shows, and catch up with friends over coffee. I'm a bit of a neat freak and love organizing things around the house. Oh, and I can't start my day without a bowl of Cheerios - it's been my go-to breakfast since college."