@Shump can you propose how you plan to adjudicate "comparable to a professional human written-novel"? Comparable to a mid-tier bestseller (say, top 100 best-sellers in a given year or something)? What is a "high-quality" novel? How do you know if it's "confirmed to exist"? I'm assuming this means you will have to read the novel it generates? Not just vaguetweets from an AI lab saying they have a model that can do this?

This is.... obviously going to resolve yes at this point, no?

@MalachiteEagle no

@bens https://x.com/sama/status/1899535387435086115

@MalachiteEagle

1) it's 1 page long

2) it's not very good, although that's a matter of taste I assume

3) similar capabilities existed 2 years ago, I don't think the amount of progress in the last 2 years, extrapolated over the next 3, would lead to this market resolving YES

But feel free to buy into my limit orders if you think otherwise :)

@bens fair response. I think OpenAI has cracked the inference time scaling problem for creative writing. This will lead to a different rate of progress over the next few years. Remains to be seen how this plays out for longer form text, but all things considered seems likely that this will be solvable with enough compute thrown at the problem.

@MalachiteEagle as an example, I've been using image models for a while (and I love them, FWIW). But Midjourney and DALL-E have essentially plateaued in quality for the last 2 years! To get from blurry images of cats to NEAR-photograph-level quality took a couple of years, but it might take far longer to get from there to generating photographs that are creative and beautiful enough to be in a National Geographic magazine, or on the cover of TIME magazine (and not as a novelty, that is).

Similarly, I think it's quite likely we can saturate the market with models that can generate short stories of the quality of a decent fanfiction writer but with better grammar. But to go from there to the level of ACTUALLY GOOD novels, the likes of which people choose to read because they are high quality and interesting to read... that might take far longer. Even with a relatively fast take-off, that's something that might not happen even with AGI! But in "ASI" territory, somewhere beyond that AGI horizon.

@bens I'd argue that writing a decent full length novel that people actually want to read is a good test for an AGI. If it's not able to do that then it's arguably not that general.

I empathize with what you're saying with regards to image generation. I believe that something may be about to happen in that domain with regards to inference-time scaling, where we move to a different rate of progress compared to the last two years. If there are broadly applicable methods in this space, then it's possible that there is an AI aesthetics overhang that we're about to go through.

Related: https://manifold.markets/CDBiddulph/when-will-i-mistake-an-aigenerated-f3faadb1a460?r=Q0RCaWRkdWxwaA

The main difference is that it's about short stories rather than novels, and has more rigorous resolution criteria

Does anyone know if there are samples or people who have tried to use GPT-4 max context length for writing a novel? Or similar projects with other models more trained for creative writing? I’d love to read them

The NaNoGenMo project might end up creating something that would arguably fit this criteria: https://nanogenmo.github.io/

From some cursory browsing, it seems like all the projects are one-off experiments that aren't actually trying to be "high-quality" novels. So probably not yet.

@ASomewhatRudeParakeet good catch! Ya, nothing there seems to be an attempt at long-form AI-gen text, but it seems reasonable that someone might try to post something on there in the next year or two

I confirm that it exists, can we resolve it to YES now?

@a2bb What's the novel?

@Shump Any new novel will do. Since a decent novel would be profitable to have under someone's name, and (even if it's copyrightable) much less so when branded as "AI-generated" (because today (and at 2028 even more so) everyone knows that AI-generated means boring crap), that person would swear that it was written, not generated, so the "author"'s word is not an evidence.

@a2bb AI as ghostwriter service

@a2bb That's just speculation. This market will not resolve based on speculation.

I will clarify that books written before this market was created ARE eligible here, as opposed to my usual rule, as long as they are revlealed to be so after the market was created.

@Shump What is not speculation?

@a2bb A reasonable level of evidence. An admission by the writer, a check by AI detection software (if those actually became good), a high quality journalistic investigation. Think something like the level of evidence required here

I feel like this comes down to the evaluation far more than the technology

I was trying to follow the criteria of the question about an AI movie. Please let me know if there are any improvements you are interesting in seeing. I think there's also space to creating a version with more objective criteria such as appearing in the NYT best sellers list.

@Shump It's possible I'm misunderstanding the movie market, but I don't think that market is going to come down to "are these movies as emotionally compelling as generally well-regarded movies" or whatever. I think if it's not, like, clearly clearly far worse than the worst movies in theaters, then it'd probably count?

Basically, I think the movie market is asking for "be comparable to a human movie-making team", whereas with books... there's a whole lot of garbage literature out there, and determining whether a given piece is too garbage to count is pretty subjective?

I dunno. This is a first impression more than a considered opinion, so maybe take it with a grain of salt.

@April The movie market says it needs to be comparable in quality to a high budget movie. There are bad big studio movies, but they all at least have a relatively high level of production quality and effort put into them. I think there might be controversy in that market if you get a movie on the level of say, emoji movie. Similarly, professional writers sometimes produce hot garbage, but they do have standards, unlike just any random book you can buy. This market simply asks if the AI can approach the level of a professional human writer.

@Shump I think this will be a fair market fwiw and that shump will be a good judge

How general-purpose does the AI need to be? Obviously an AI that only accepts one exact prompt wouldn't count, but should it be able to write any type of novel? Is hard coding, say, non dialogue acceptable or does everything need to be written by the AI? What about hard coding plot structures or characters?

@ASomewhatRudeParakeet It should be able to generate multiple different books, with a level of novelty comparable to that of a single human author. It doesn't need to be able to generate any kind of book, but it needs to be able to have enough variety to be more than a one-trick pony.