Nora Belrose says: I predict with 60% confidence that some DPO variant will more or less replace RLHF within 6 months. Outside of huge labs that can afford RLHF’s implementation complexity and instability it’s more like 80% chance.

Given the major labs probably won't talk, we will consider the non-huge lab scenario.

This resolves to YES if, excluding DM/OAI/Anthropic, DPO is a more popular technique in practice than RLHF at time of resolution.

I expect the answer to be obvious one way or another, if not I will attempt to settle via Twitter poll, if that isn't definitive I will ask experts and use best judgment.

https://twitter.com/norabelrose/status/1728456414535016536