Each year, sometime in December, I will ask a trusted person (called the "curator") to pick 5 short stories from the New Yorker. For each story, they will generate a short story with the same opening using an AI text generator. I will read each pair of stories and guess which one is AI-generated. If I guess wrong at least once, the market will resolve to the current year.

When to run the experiment:

• Between December 1-31 of the year in question, I will run a test version of the experiment by myself with a few short stories.

• If I judge the stories generated by the AI to be very obviously AI-written, I may skip the experiment.

• If I have already run the experiment or decided not to run it, and then an improved AI text generator comes out during the month of December, I may re-run the experiment, but will not be obligated to.

• If I am unable to run the experiment, I will find a trusted person without a stake in this market to do it instead. They will pick their own curator.

Process of the experiment:

• I will pick what I judge to be the best available AI text generator that can generate 5 stories for less than $50, requiring less than two hours to set up and generate all 5 stories.

• The curator will look at the New Yorker's most recent fiction (https://newyorker.com/magazine/fiction/). If the fiction is no longer available or we run out of eligible stories, we will choose a suitable alternative.

• For each story, starting with the most recent, we will follow this process, until we have found 5 eligible stories.

  • If the story is less than 4,000 words or more than 8,000 words, was generated by an AI, has a visual component such as pictures that are crucial to the story, or anything else would make it easy to tell that this story comes from the New Yorker, we will skip the story. Otherwise, we will use it.

• For each story, the curator will do the following to generate another story:

  • The prompt will start as follows:

    • Continue the story based on the opening below. The story should be good enough to be published in the New Yorker and should be <WORD COUNT> words long. Repeat the opening first, then continue from there.

    • <WORD COUNT> is the number of words in the human-written story.

  • The curator will also append the first paragraph(s) of the original story, just enough to make 100 words or more.

  • If necessary, the curator may retry, change the prompt, or cut parts of the responses in order to get a satisfactory answer that is more than 4,000 words and less than 8,000 words. For instance, if the AI is accessed via a chat interface with limited text per response, they might need to add a prompt like "At the end of each response, tell me CONTINUE if you want to continue, or STOP if the story is done."

  • I will not see the prompt until after the experiment.

• The curator will paste both stories into a Google Doc, using a computer to randomly choose their order.

• I will read the two stories and guess which one was AI-generated.

• If I am wrong about any of the 5 pairs of stories, the market resolves to the current year.

I will not trade in this market, nor will any curator I choose.

This market is based on the format of https://manifold.markets/CalebBiddulph/when-will-i-mistake-an-aigenerated.