Could you clarify what you mean by "safe"?

How does this market determine if an AGI is "safe" without mathematical results? Would a single day of it existing without causing anyone to die be sufficient? Would it causing the death of a terrorist immediately disqualify it as "safe"? What about it giving advice leading someone to commit suicide?

I'm trying to imagine what the world would look like when this market resolved NO, if it were to do so.

If a (presumably safer than default RL training) AI is trained based off of logical inductor or infra bayes results does this resolve yes? In particular what about the case where suggestive mathematical results exist about this safe AI but those results aren’t anywhere near formalizing complete safety ( no x risk)

@JacobPfau I don't think it's meaningful to train an AI off of logical inductor or infra bayes stuff? Not sure what you mean, elaborate and I'll try to answer?
Second case it depends on exactly how suggestive they are. Analogy: if we get a safe AI and our understanding of AI is roughly at the level of our current understanding of P=NP (no formal proofs, lots of intermediate results, basically everyone is pretty confident on what the answer will eventually be), that would resolve YES.

@vluzko You can train an neural net to anticipate the results of N steps of logical induction, if I understand it correctly. This is a bit like expert iteration.

I don’t know much about Vanessa’s naturalized induction stuff but I assumed it generates some NN training targets as well.