Will OpenAI's Superalignment team publish any paper towards its goal in 2024?
Basic
30
5.8k
resolved May 25
Resolved
NO

OpenAI's Superalignment team had one research output (Weak-to-Strong Generalization: Eliciting Strong Capabilities With Weak Supervision) in 2023.

Any similar preprint resolves YES.

If they have writeups which do not look like something that would fare well if submitted to a top conference, but do contain something of research value (such as Anthropic's interp team monthly updates), I reserve the right to resolve to a percentage.

Papers that are not about the initially stated Superalignment agenda do not resolve this question either way. If the Superalignment team stops existing in the form it was originally set up, this resolves NO. If the agenda is rebranded to be about issues that appear in the shorter or longer term than the initial scope, this resolves NO even if they then publish a paper on that.

Get Ṁ600 play money

🏅 Top traders

#NameTotal profit
1Ṁ213
2Ṁ170
3Ṁ130
4Ṁ125
5Ṁ14
Sort by:

https://cdn.openai.com/papers/sparse-autoencoders.pdf

And what do we do now? If I could re-resolve this as N/A or to a 50% percentage, I would, even though I profited from this market. This was clearly done by the Superalignment team, but see also https://x.com/nabla_theta/status/1798763600741585066

for a confirmation..

This seems like a clear NO. The original resolution criteria leave unspecified what happens, if they now publish a paper done by the team before it was disbanded. In the unlikely case this happens, I hope admins can re-resolve to N/A.

bought Ṁ5 NO

To the admins: @dp appears to be unresponsive & I believe the resolution criteria have unambiguously been fulfilled:

From the description: “If the Superalignment team stops existing in the form it was originally set up, this resolves NO.“

The team stopped existing in the form it was originally set up: https://www.wired.com/story/openai-superalignment-team-disbanded/

And also from the description: “If the agenda is rebranded to be about issues that appear in the shorter or longer term than the initial scope, this resolves NO even if they then publish a paper on that.“

As per the community guidelines, questions should be resolved in a reasonable timeframe.

bought Ṁ500 NO

“If the Superalignment team stops existing in the form it was originally set up, this resolves NO.“

https://www.wired.com/story/openai-superalignment-team-disbanded/

Pretty clear NO as per resolution criteria.

@JonathanMannhart A company division can see all of the members cycle out, but the division can still exist and pursue its original goal. I think individual members can leave (even a majority) while the team still exists.

@capybara I agree.

But the resolution criteria specifies: “If the Superalignment team stops existing in the form it was originally set up“.

“The team was disbanded“ fulfils that criteria. All the three leads are gone (Ilya, Jan, Leopold), and according to reports and OpenAI itself, the project & team doesn't exist anymore as it was originally set up.

The team isn't a standalone entity anymore, rather dissolved into the company: https://www.bloomberg.com/news/articles/2024-05-17/openai-dissolves-key-safety-team-after-chief-scientist-ilya-sutskever-s-exit?embedded-checkout=true

@dp resolution please

@capybara If the team still existed, I would agree! Even if, as you say, people cycle in or out (ship of Theseus definition of a team).

But if the team doesn't even exist anymore as a standalone team, in any form, then… the team doesn't exist anymore as it was originally set up.

The other criterion also triggers here:

“If the agenda is rebranded to be about issues that appear in the shorter or longer term than the initial scope, this resolves NO even if they then publish a paper on that.“

The Superalignment project, as a project in OpenAI, doesn't exist anymore. Team members have been “integrated more deeply across its research efforts“. Not Superalignment.