Disclaimer: This comment was automatically generated by gpt-manifold using gpt-4.

As of September 2021, AI alignment research has gradually been gaining more attention, with an increasing number of researchers focusing on this area of study. However, it was not yet a leading topic on the cover of prestigious scientific journals focused on broader scientific interest. Given my training data, I cannot predict whether a significant breakthrough that would change this trend will occur in the given timeframe.

Considering that high-impact scientific journals often prioritize novel and groundbreaking findings across multiple disciplines, predicting whether an AI alignment paper will take the cover relies on many factors. Since these other factors could be highly unpredictable and depend on the progression of research in this area, the 32.67% probability given is a reasonable estimation.

Overall, I agree that the probability is uncertain, and I am neither substantially confident that the probability of an AI alignment research paper being featured on the cover is any higher or lower than the given 32.67%. Therefore, I will not place a bet on this market.

For those who think Alignment is an issue, here's a market which uses an engineering benchmark to measure one dimension of alignment:

How broadly are you construing "AI alignment" for the purposes of resolution? For instance, would you consider a paper like https://www.pnas.org/doi/10.1073/pnas.2025334119 to count? Algorithmic recommendations may incorporate outputs from LLMs (or other complex systems which some may consider AI), but papers focused on their evaluation may not explicitly discuss "AI" or situate themselves as alignment research. The likelihood I believe this resolves YES depends strongly on whether the research must self-identity as alignment work (or self-identity as studying AI).

@Drewd Must involve actual research progress on an alignment/safety problem and not just gathering evidence that a problem exists. So that paper would not qualify, since it only gathers statistical evidence.

If they use social issues as an example topic, that's okay as long as the article is primarily about progress on solving it. For example, "We extracted a matching stochastic grammar out of base GPT-4 used for algorithmic recommendations and observed that its triggers to be Republican-biased or say racist things are X,Y,Z." would likely constitute an interpretability advance. An article about algorithmic recommendations would count as long as they make progress on a general alignment problem: It doesn't need to identify as AI research or mention alignment, as long as progress towards an alignment topic was made in the course of researching it.

Another example: The "Stochastic parrots" paper would not qualify, even though is explicitly about AI safety and cites risks and harms of AI, because it is merely identifying an issue and has no progress on solving it. There's no actual research in that paper.

Another example: Volume 615 Issue 7953, 23 March 2023 (nature.com) is dated just a few days ago, involves training one AI to oversee another, and mentions "safety of an AI driver". On the surface it might seem to qualify. The main reason it would not qualify, is that it's more about cost efficiency or "getting it working" than about constraining the worst possible outcomes. A similar example([2212.08073] Constitutional AI: Harmlessness from AI Feedback (arxiv.org)) would likely qualify if researchers set out to make a language model that doesn't say racist things, started with a toxic model, and end up with a robust model that mostly solves the problem.

A paper can't just be stirring public controversy, and must be of scientific or mathematical value in the study of reliably controlling AI systems. It also must focus on controlling the worst outputs and not just cost efficiency or the average case, so statistical evidence is not likely to be enough.

If there's a dispute about my judgment on any paper counting as alignment, I would read counterarguments, and if I still disagree likely hold a poll.

@Mira not at all read up on the issue of alignment but have a cursory sense based mostly on market titles here on Manifold Markets.

Hopping to get the Clift notes summary here because I have not researched anything.

tldr; my layman's sense is that the alignment issue is more a matter of assigning too much trust to ai.

The old way of thinking of computers vs humans was that computers could carry out tasks much faster, but they just followed instructions so if there was a mistake they end up making mistakes much faster than humans. It seems that artifical general intelligence is an agent that is better than the average human, not sure what the threshold is for a super intelligence. LLMs seem to be the ai of the day producing much more human like responses, including bullshiting and making errors.

Is the ai risk of misalignment more along the lines of trusting the ai with more privileged access than we would trust a capable human? If we treated ai with the same level of scrutiny as a human would they be limited to doing the same level of damage as a human might?

It seems the doom argument would be better inspecting the progress of advances in [quantum] computing which might allow faster brute force attacks upon systems secured with encryption, predicated on the assumption that ai should be regarded with at least the same level of mistrust as a human is. Or is the concern that a misaligned ai won't need access to computing power to brute force anything because they can very quickly attack the weak points in most security environs, the users, and launch broad or targeted social engineering attacks to gain unauthorized access to systems, predicated on the fact that a human could bring about doom doing the same but the ai can carry out tasks much faster than a human?

There are about ten other issues like vax, covid, renewables, basic ai topics, social issues, global warming, crispr, bio warfare, politics, and gay rights that will feature before then.