Humanity is unable to create a superintelligence before 2060, despite generally trying to make smarter and smarter AI

'2060':9 'ai':18 'creat':5 'despit':10 'general':11 'human':1 'make':14 'smarter':15,17 'superintellig':7 'tri':12 'unabl':3

Humanity coordinates worldwide to significantly slow down the creation of superintelligence, buying enough time for other advancements that prevent AI Doom

'advanc':17 'ai':20 'buy':12 'coordin':2 'creation':9 'doom':21 'enough':13 'human':1 'prevent':19 'signific':5 'slow':6 'superintellig':11 'time':14 'worldwid':3

Humanity coordinates worldwide to entirely prevent the creation of superintelligence before 2060

'2060':12 'coordin':2 'creation':8 'entir':5 'human':1 'prevent':6 'superintellig':10 'worldwid':3

Humanity naturally takes so long to create a superintelligence that other advancements happen first, which prevent AI Doom when a superintelligence is created

'advanc':12 'ai':17 'creat':7,23 'doom':18 'first':14 'happen':13 'human':1 'long':5 'natur':2 'prevent':16 'superintellig':9,21 'take':3

The core problems of alignment are solved by a company's efforts, like OpenAI's Superalignment

'align':5 'compani':10 'core':2 'effort':12 'like':13 'openai':14 'problem':3 'solv':7 'superalign':16

The core problems of alignment are solved by a national effort, like the Manhattan Project

'align':5 'core':2 'effort':11 'like':12 'manhattan':14 'nation':10 'problem':3 'project':15 'solv':7

The core problems of alignment are solved by a global effort, like nothing ever seen before in humanity's history

'align':5 'core':2 'effort':11 'ever':14 'global':10 'histori':20 'human':18 'like':12 'noth':13 'problem':3 'seen':15 'solv':7

Human intelligence augmentation is developed, which makes everything else easier

'augment':3 'develop':5 'easier':10 'els':9 'everyth':8 'human':1 'intellig':2 'make':7

An "unaligned" superintelligence is created, but it does not want to cause AI Doom. It prevents the creation of other superintelligences.

'ai':13 'caus':12 'creat':5 'creation':18 'doom':14 'prevent':16 'superintellig':3,21 'unalign':2 'want':10

Multiple "unaligned" superintelligences are created, but none of them want to cause AI Doom.

'ai':13 'caus':12 'creat':5 'doom':14 'multipl':1 'none':7 'superintellig':3 'unalign':2 'want':10

Humanity first invents weaker AI that can solve the core problems of alignment, and then creates an aligned superintelligence

'ai':5 'align':13,18 'core':10 'creat':16 'first':2 'human':1 'invent':3 'problem':11 'solv':8 'superintellig':19 'weaker':4

The first people who create a superintelligence are able to control it, and order it globally prevent the creation of other superintelligences

'abl':9 'control':11 'creat':5 'creation':19 'first':2 'global':16 'order':14 'peopl':3 'prevent':17 'superintellig':7,22

Humanity invents a superweapon with which we can threaten any superintelligence with mutually assured destruction

'assur':14 'destruct':15 'human':1 'invent':2 'mutual':13 'superintellig':11 'superweapon':4 'threaten':9

It turns out that superweapons are very hard to create and so no superintelligence is able to pose a global threat through nanobots etc

'abl':16 'creat':10 'etc':24 'global':20 'hard':8 'nanobot':23 'pose':18 'superintellig':14 'superweapon':5 'threat':21 'turn':2

One or more smaller-scale disasters turn governments and public opinion against AI such that superintelligence is delayed enough for us to solve the alignment problem 

'ai':14 'align':26 'delay':19 'disast':7 'enough':20 'govern':9 'one':1 'opinion':12 'problem':27 'public':11 'scale':6 'smaller':5 'smaller-scal':4 'solv':24 'superintellig':17 'turn':8 'us':22

Large Language Models are naturally fairly well aligned

'align':8 'fair':6 'languag':2 'larg':1 'model':3 'natur':5 'well':7

Superintelligent AI is not developed by 2060

'2060':7 'ai':2 'develop':5 'superintellig':1

Superintelligent AI is never empowered enough to become a serious risk (ie it's just used for specific tasks and not given enough agency to make it risky)

'agenc':24 'ai':2 'becom':8 'empow':5 'enough':6,23 'given':22 'ie':12 'make':26 'never':4 'risk':11 'riski':28 'serious':10 'specif':18 'superintellig':1 'task':19 'use':16

Alignment is never "solved", but is mitigated well enough to avoid existential risk.

'align':1 'avoid':11 'enough':9 'existenti':12 'mitig':7 'never':3 'risk':13 'solv':4 'well':8

By means other than AI (Engineered pandemic? Nukes? Automated warfare? Etc.) we kill enough people to set humanity's tech level back a bit/a lot.

'ai':5 'autom':9 'back':22 'bit/a':24 'engin':6 'enough':14 'etc':11 'human':18 'kill':13 'level':21 'lot':25 'mean':2 'nuke':8 'pandem':7 'peopl':15 'set':17 'tech':20 'warfar':10

Weak AI causes enough social chaos and disruption that efforts to build better AI are set back such that that suprintelligence doesn't happen by 2060.

'2060':26 'ai':2,14 'back':17 'better':13 'build':12 'caus':3 'chao':6 'disrupt':8 'doesn':22 'effort':10 'enough':4 'happen':24 'set':16 'social':5 'suprintellig':21 'weak':1

Restrictions around access/use of compute/hardware

'access/use':3 'around':2 'compute/hardware':5 'restrict':1

Releasing the weights (or equivalent) of the most powerful AI models

'ai':10 'equival':5 'model':11 'power':9 'releas':1 'weight':3

Restricting access to the weights (or equivalent) of the most powerful AI models

'access':2 'ai':12 'equival':7 'model':13 'power':11 'restrict':1 'weight':5

OpenAI ceases to exist before AGI is made

'agi':6 'ceas':2 'exist':4 'made':8 'openai':1

Superintelligent AI will be much less motivated than humans, and in any case will automatically have humanlike goals due to training on human data.

'ai':2 'automat':15 'case':13 'data':24 'due':19 'goal':18 'human':9,23 'humanlik':17 'less':6 'motiv':7 'much':5 'superintellig':1 'train':21

Multiple "unaligned" superintelligences are created, and while some of them want to cause AI Doom (directly or indirectly), the ensuing handshake-hypowar results in a mostly-aligned Singleton.

'ai':14 'align':29 'caus':13 'creat':5 'direct':16 'doom':15 'ensu':20 'handshak':22 'handshake-hypowar':21 'hypowar':23 'indirect':18 'most':28 'mostly-align':27 'multipl':1 'result':24 'singleton':30 'superintellig':3 'unalign':2 'want':11

Humanity first invents weaker AI, and through hands-on experience with them, learns the methods and develops the tools to align a much stronger AI.

'ai':5,26 'align':22 'develop':18 'experi':11 'first':2 'hand':9 'hands-on':8 'human':1 'invent':3 'learn':14 'method':16 'much':24 'stronger':25 'tool':20 'weaker':4

Alignment ends up not a concern, AI turns out to naturally optimize for benign ends via benign means

'ai':7 'align':1 'benign':14,17 'concern':6 'end':2,15 'mean':18 'natur':11 'optim':12 'turn':8 'via':16

AIs that are better than humans at most cognitive tasks are developed and become widespread, and the world still appears vulnerable in 2060, but humans are still alive at that time.

'2060':23 'ai':1 'aliv':28 'appear':20 'becom':14 'better':4 'cognit':9 'develop':12 'human':6,25 'still':19,27 'task':10 'time':31 'vulner':21 'widespread':15 'world':18

Jesus returns and aligns AI (at least as likely as some of these options)

'ai':5 'align':4 'jesus':1 'least':7 'like':9 'option':14 'return':2

Many people are worried that in the next few decades humanity will create a superintelligence which brings extinction, enslavement, or some other terrible fate upon us in "AI Doom". This question asks how we avoided this by 2060, in the worlds that we did.

Please try to keep answers short and specific. Describe one thing, and describe it in a way that the average manifold user can understand. Don't assume your audience knows a bunch of very technical terms. Try not to present answers in a biased way like saying "We are lucky that [thing you think is unlikely] happens." Just say [thing happens].

If you have multiple things to say, say them in multiple submissions. Make as few assumptions as possible in each individual submission. You can elaborate in detail in the comments. It's better to submit something that doesn't overlap with existing answers too much, but submitting a much better version of an existing submission is also okay.

 in various formats. I think we mostly expect that humanity will not survive to resolve these questions, so they mainly represent the opinion of people willing to lock their mana up indefinitely. They also represent the opinion of people trading on other people's opinions in the short-term.

This questions tries a new way to incentivize long-term accuracy. In about a week, maybe a bit sooner if this question doesn't get much interest or a bit later if it gets more interest, this question will close. Then all answers will resolve N/A. All trades, profits, and losses will be reversed, and all mana returned.

If we all die and/or manifold ceases to exist, your reward will be the trader bonuses you got for submitting interesting answers. You'll also have bragging rights if your answer was voted high up before market close. I may also award bounties to very insightful answers.

If we survive until 2060, then I or another moderator will use the "unresolve" feature to undo the N/A resolution and put everyone's mana back into the market as it was at market close. All answers will then be graded for their general quality by the best experts I can find, and will resolve to % from 0 to 100. There will be a grading curve, so one can expect the answers to be graded relative to each other instead of being compared to a hypothetically perfect answer which was not submitted.

Hopefully, this format will function like a poll that is weighted by the amount of mana that people have and are willing to spend on it, and will produce more accurate results than an unweighted poll. 

Please do not submit answers that are too similar to existing answers, or which are just bad jokes. I will N/A answers that I think are not worth the space they take up in the market. Good jokes may be allowed to stay up longer than bad jokes. This market will be unranked, so as not to disrupt Manifold Leagues.

I am open to suggestions for improving this format, and may update these rules within the spirit of the question.