I am doing Inkhaven https://www.inkhaven.blog/ for the month of November. This means that I have to publish roughly 30 posts during November, mostly on AI safety and LLM evals in general. I have many drafts and rough ideas ready, but the actual writing will happen only during the month of November. I expect to publish at least 10 of them through my newsletter; where they are sent to about 1000 subscribers. My usual cadence for posting is less than once per month. The remaining posts will go to my website, my Random Features newsletter, the SPYLab blog, or my Twitter account.

Stats on the first post https://newsletter.danielpaleka.com/p/ab-testing-could-lead-llms-to-retain (Nov 1):

In the first 12 hours after publication, the post (or email) has been opened by 581 people. The stats tell me I have lost 3 subscribers and gained one new. I have not posted it anywhere public except Substack (and now to Manifold).

Resolves YES if I have more subscribers in the evening of November 30 than before I published the first post on November 1. Resolves NO otherwise.

A/B testing could lead LLMs to retain users instead of helping them

OpenAI’s updates of GPT-4o in April 2025 famously induced absurd levels of sycophancy: the model would agree with everything users would say, no matter how outrageous.