The following is an argument for why AI safety organizations should consider my work. If Eliezer is not compelled by this argument, which proposition will he deny?
Will resolve if @EliezerYudkowsky claims to deny any proposition by number in the comment section of this prediction or agrees to review my work.
1. If AI develops the capability to control the environment better than humans, then humanity is doomed.
2. If we continue to scale AI capabilities, then it will eventually be able to control the environment better than humans.
3. 1 and 2 imply that if we continue to scale AI capabilities, then humanity is doomed.
4. We should not be doomed.
5. 3 and 4 imply that we should stop scaling AI.
6. If every person on the planet understood the alignment problem as well as Eliezer Yudkowsky, then we would not scale AI to the point where it can control the environment better than humans.
7. People only understand the things they have learned.
8. People learn the things that they have obvious incentives to learn.
9. 6, 7, and 8 imply that if people have sufficient and obvious incentives to understand the alignment problem, then we would not scale AI to the point where it can control the environment better than humans.
10. It is possible to build a machine that pays individuals for demonstrating they’ve understood something.
11. If individuals can see that they will earn a substantial cash reward for demonstrating they understand something, they will be incentivized to demonstrate they understand it.
12. 10 and 11 imply that it is possible to incentivize people to understand the alignment problem.
13. If a majority of people understood the actual risks posed by scaling AI, then they would vote for representatives that support legislature that prevents the scaling of AI.
14. 9 and 13 imply that if we sufficiently incentivize understanding of the alignment problem, then people would take action to prevent dangerous AI scaling.
15. If your goal is to prevent the scaling of dangerous AI, then you should be working on building mechanisms that incentivize awareness of the issue.
16. Krantz's work is aimed at building a mechanism that incentivizes the demonstration of knowledge.
17. 5, 12, 14, 15 and 16 imply that if your goal is to prevent the scaling of dangerous AI, then you should review the work of Krantz.
18. If AI safety orgs understood there was an effective function that converts capital into public awareness of existential risk from AI, then they would supply that function with capital.
19. 17 and 18 imply that Eliezer Yudkowsky and other safety organizations should review the Krantz system to help prevent doom.
This argument is one of many that should exist on a decentralized ledger like this:
https://manifold.markets/Krantz/krantz-mechanism-demonstration
If it did, we could be scrolling through the most important arguments in the world (on platforms like X) and earning livings by doing the analytic philosophy required to align AI/society.
This is how we build collective intelligence.
1,000On your concrete propositions:
1. If AI develops the capability to control the environment better than humans, then humanity is doomed.
This is wrong. If unaligned AI develops the capability to control the environment better than humans, then humanity is doomed.
10. It is possible to build a machine that pays individuals for demonstrating they’ve understood something.
I think this is unclear. How general is "something"? Does this refer to:
"It is possible to build a machine that, for all x, can decide whether an individual has demonstrated that they've understood x"?
"For all x, it is possible to build a machine that can decide whether an individual has demonstrated that they've understood x"?
"For some x, it is possible to build a machine that can decide whether an individual has demonstrated that they've understood x"? (This version seems uninteresting, so I assume not this, but I list it for completeness.)
For the sake of this particular argument, the weaker proposition "It is possible to build a machine that pays individuals for demonstrating they've understood the alignment problem" might be useful.
17. 5, 12, 14, 15 and 16 imply that if your goal is to prevent the scaling of dangerous AI, then you should review the work of Krantz.
This is wrong. 16 alone is insufficient; to believe you should review the work of Krantz, you must believe that the expected value of doing this is worthwhile. Anyone can produce "work" which is "aimed at" anything, but that does not guarantee that the work successfully achieves its aim, or even makes useful progress toward its aim. (Consider someone who says they have produced a proof that the earth is flat. Such a proof would be very important if true! But this does not mean that you should review the proof.)
18. If AI safety orgs understood there was an effective function that converts capital into public awareness of existential risk from AI, then they would supply that function with capital.
This is wrong. Simply being "effective" is insufficient; such a function must be sufficiently effective that the expected utility is higher than other possible uses of capital.
@jcb These are great questions and excellent critiques. Thanks!
1. Your right, that it would be stronger and remain valid if I added the property of being unaligned. I feel I could also argue that any self aligning AI is tautologically unaligned to humanity. What I'm trying to share is a mechanistically interpretable way of directly rewarding humans to align it.
I'm trying to pivot the world from machine learning to gofai.
10. What I'm saying is that it's possible to electronically consent to a given proposition being true. If we paid individuals to consent to propositions being true for use in legally binding contracts (think of "I've read and accepted xyz terms and conditions). That data would be valuable. People's explicit opinions are valuable. Especially on "how should a truly decentralized AI (that's supposed to be free and equally beneficial to all of humanity) behave and what beliefs should it have and why?".
It is possible to build a machine that, for all "propositions of truth" can print value according to how valuable society rates that "proposition of truth". It is possible to use blockchain technology to align constitutions.
17. It's a conjunction of all those propositions to imply 17. If someone claimed to have a proof that the world was flat in a hypothetical universe where the krantz mechanism was used as the primary mechanism of intellectual currency (human intelligence), they should stamp that proof on the alignment blockchain where it will be worth tremendous value if cooberated, but worthless if proven false. Other people print currency by proving it false. Blockchaining currency was a big deal. This is blockchaining public opinion..
18. Totally agree this is a more rigorous proposition. If these propositions existed on the sort of decentralized network that I hope exists one day I would have rewarded value to you by cooberating that your revision is in fact better.
The truth is, this prediction is just a subset of what I'd like to get on my demonstration mechanism or my "guess my beliefs question". For example, someone could easily take my liquidity by transferring these exact propositions over to "guess my beliefs" and guessing correctly. Imagine if every analytic philosopher in the world put their best arguments (for how to fairly align society and distribute all this new wealth) online in the public domain and printed IP crypto for doing it. If anyone thinks I'm wrong about something, ideally they should write and argument (with compelling propositions that I accept) that leads me to better consent to understanding why I'm wrong. I'd love to pay for that. I'd bet others would too.
At the end of the day, I want kids in the world to learn how to print their own currency by doing analytic philosophy about what we as humans ought do (in case AI takes all the physical jobs).
Thanks again for engaging positively and charitably!
Re 10: What you said here seems substantially different from the original proposition "It is possible to build a machine that pays individuals for demonstrating they’ve understood something."
What happened to "demonstrating understanding"? I can imagine how, given a set of someone's agreement/disagreement with certain propositions about a topic, a machine might decide whether they understand a topic. If this is what you're referring to, that's not very clear. The discussion of the value of people's opinions doesn't seem to connect to the original proposition 10.
Also, "a machine that pays individuals" has been replaced with "a machine that prints value". Perhaps this is what you meant originally by "pays"--I almost called that out as another point of unclarity in my previous comment, but I thought that this was answered by 18, where a funder, in this case AI safety orgs, would provide the capital to be paid out to individuals.
If indeed your proposition 10 is that the machine "prints value", I think that proposition is quite likely to be "denied" if Eliezer reviews this argument. I think that the possibility of building such a "machine" is not a simple proposition, as it seems to encompass an entire economic/blockchain system, the mechanics of which are not clear. (As far as I understand, this is the whole Krantz Mechanism for which you're advocating? Unfortunately, treating this as a preliminary proposition from which you derive "you should review the work of Krantz" is circular.)
Re 17: Sorry, I was unclear; I meant something like "the implication is not valid, though it might be with a stronger version of 16" (or, I suppose, a weaker version of 17).
Looking at 17 and nearby lines again, I think there are a couple of unneeded dependencies:
17 doesn't depend on 5 at all. 17 assumes "your goal is to prevent the scaling of dangerous AI". So the truth of "we should stop scaling AI" is irrelevant to 17.
I don't think 17 depends on 12 or 14 directly. Instead, I think 15 depends on both of these. (If 12 is false, then 15 does not hold, because you shouldn't focus on building the impossible. If 14 is false, then 15 does not necessarily hold, because building the mechanism won't necessarily help prevent dangerous AI scaling.)
So the simplified 17 is: 15 (If your goal is to prevent the scaling of dangerous AI, then you should be working on building mechanisms that incentivize awareness of the issue) and 16 (Krantz's work is aimed at building a mechanism that incentivizes the demonstration of knowledge) imply that if your goal is to prevent the scaling of dangerous AI, then you should review the work of Krantz.
But this still has the same problem. Clearly[*] 15 and 16 imply that if your goal is to prevent the scaling of dangerous AI, then Krantz's work is aimed at building a mechanism that you should be working on. But it doesn't follow that you should actually review the work of Krantz. What you "should" do depends on whether Krantz's work will actually help you accomplish your goal, which this argument has not demonstrated.
If 16 were stronger, like "16' Krantz's work successfully builds a mechanism that incentivizes the demonstration of knowledge" or "16' Krantz's work makes substantial progress toward a mechanism that incentivizes the demonstration of knowledge", then 15 and 16' would imply 17.
If 17 were weaker, like "17' if your goal is to prevent the scaling of dangerous AI, then the work of Krantz is relevant to your goal" or "17' if your goal is to prevent the scaling of dangerous AI, then the work of Krantz might be useful to you", then 15 and 16 would imply 17'.
But as written now, 15 and 16 do not imply 17.
[*] Technically I'm ignoring the distinction between "incentivize awareness" and "incentivize demonstration of knowledge", but I think this is basically addressable with your other propositions and not a fundamental flaw.
This market is much better than many of your others, in the sense that there's a nontrivial probability it might actually resolve: it's much more likely for Eliezer to evaluate 19 short, public statements than to "review your work", whatever that means.
(I think the probability is still not high, so YES bettors are likely heavily discounting for a low probability of actually getting a resolution. Therefore, if you're trying to identify weaknesses in your argument, you should consider YES bets to indicate a "probability of proposition denial" much higher than the market probability.)
But while the structure is more useful, it pins you down to committing to this particular version of your argument. If you discover a problem with one of your propositions, you would presumably want to modify your argument; you want Eliezer to review the best, strongest version of your argument, not an argument that you know is flawed. But then what can you do with this maket?
Edit this market: problematic as this would be unfair to traders who bet on the original statements
N/A this market: fails to reward traders who identified problem(s) with the argument in this market
Resolve this market: can't be done without Eliezer's review
Leave this market as is and create a second market with the updated argument: possible, but then it seems unreasonable to hope for Eliezer to review this market as well as the new one, so it doesn't really answer the question of what happens to this market.
Thinking out loud, this might work better as an unlinked multiple choice market (with Other) like this:
Which proposition(s) will be denied? A proposition resolves YES if either (a) I [Krantz] am convinced that it is flawed or (b) Eliezer Yudkowsky denies it by number in the comment section. If, before Eliezer's review, I am convinced that the argument is flawed, I may repair it by resolving the flawed propositions as YES and adding new propositions to replace them.
[edit: corrected NO -> YES to match this market]
Unlinked multiple choice also more naturally accounts for the possibility that more than one proposition might be denied.
@NivlacM I'm glad you liked the format. If Elon looked at my algorithm, we could transform X into a compiled feed of arguments of this sort. Then, we could allow everyone to earn a living by arguing with the world's most comprehensive collection of arguments. A machine that knows all your priors and which goal propositions you aim to prove, it could recommend the optimally accepted path to the things you don't know and were deemed beneficial by others. The verification of propositions could then also be used as a demonstrable social contract for the truth that rewards public consent. Individuals could effectively argue against any given person's set of priors (It would be super helpful if Eliezer earned a living by writing all of his arguments in a database I could access.) It's like building a machine analytic philosopher that pays you to evaluate its arguments.
What's your confidence in the following propositions?
It is possible to build a machine that demonstrates an individual understands something.
If 10 is false, then degrees and credentials are worthless.
@Krantz i suppose it's a matter of how certain you need to be that an individual truly understands the topic. And maybe I'm thinking of belief more than understanding. One can understand all of Yudkowsky's work but still disbelieve that AI is dangerous
@NivlacM I just want a lot of people to wager (with money sponsored from individuals that think AI is dangerous) on the issue by assigning a confidence to each individual proposition of Eliezer's argument aimed to predict which way consensus will resolve. Along with every other topic people want to gain support for.
13 (and, by extension, 6) seem to me to be obvious candidates for rejection. Aside from the fact that we don't know with certainty that scaling AI will result in existential catastrophe, even if we did know that with certainty, people often simply don't vote rationally (or, perhaps more importantly, altruistically). It would not be at all surprising to me if, even if everyone on Earth knew with certainty that scaling AI would wipe out humanity in 50 years, a majority of people nevertheless voted for a political party that was pro-AI-scaling but also promised to cut taxes/inflation in the interim, for example.
I don't think that's the only (or even the primary) reason. It seems to me that two much more challenging hurdles are hyperbolic discounting and general selfishness. Even if people know that a huge harm is definitely coming at some point in the future, they might nevertheless ignore it in order to focus on avoiding some smaller harm in the interim. And then perhaps more problematic, you have a big percentage of the voting population who simply aren't adequately motivated to avoid the huge future harm because they're old and won't be around to suffer from it.
A couple of gaps that I see right off:
Any incentive ≠ enough incentive
2 bitcoins has come up elsewhere. Sounds like a lot, until you compare to an average salary at OpenAI. No matter how much the incentive, you're up against a change bigger than the industrial revolution; it will have bigger incentives.
It is difficult to get a man to understand something, when his salary depends on his not understanding it.
The Krantz system aims ≠ the system achieves
You skip some steps: what the mechanism is, how it's funded, and evidence that it would be more effective than existing systems. I'm an advocate for off-label uses of prediction markets: insurance contracts, news discovery, etc. This sounds similar, but you don't discuss the difficulties already known in those systems, and you introduce more. You've mentioned elsewhere that your system needs proof-of-humanity, this is an unsolved problem despite many IQ-hours behind it, and it doesn't even make this list! A system that achieves what you're suggesting would require several major research breakthroughs. I'd love to see it happen, but I would not bet on this avenue working out.
1. In '9' I refer to 'sufficient and obvious incentive'. The 2 btc value was an example amount aimed at the general public (primarily individuals that are struggling financially). The actual amount that individuals would be rewarded would be determined by the market. In other words, the more important it is for someone to demonstrate consensus of a particular truth, the more the market will allocate to that particular verification. This value fluctuates as demand for particular truths become more or less controversial.
Let's look at an example. Assume I started a charity that was aimed at raising money to be directed at getting Yann Lecun to consent publicly to the proposition that 'It was dangerous to open source Llama 3.1 405.'. Assume we raised 1 million dollars and gave that to Yann, only to be used as a free wager on the prediction 'In 2030, a majority of experts will look back and agree that there was significant justification for believing with greater than 90% confidence that Llama 3.1 405 would not provide the foundation for nefarious actors to produce existentially risky artificially intelligent systems'.
What's important here, is not the amounts or the specific propositions I'm using, but the general principle that individuals can invest capital in a way that incentivizes any particular user to engage with and take a public stance on any particular crux issue that they would typically avoid asserting a public position on.
2. To determine whether or not the krantz system achieves what it aims to, requires review. That's what I'm hoping to get. Whether it would be more effective than existing systems, I would claim no systems currently exist (a mechanism that allows me to offer a sponsored wager only for the prediction of my choosing). Overall, there are several aspects of this project that will require further development. I don't think that suggests one should write it off. On the contrary, it seems to suggest it could use some funding and collaboration.
Thanks again for the feedback Rob!
In case anyone has a couple extra million dollars laying around and wants to send Yann a voucher to wager on this prediction..
https://manifold.markets/Krantz/in-2030-a-majority-of-experts-will?r=S3JhbnR6
a mechanism that allows me to offer a sponsored wager only for the prediction of my choosing
Would this be what you're after?
let people promote markets they like by offering free but unsellable trades.
@TheAllMemeingEye It's similar, but I was really just referring to the ability to send mana directly to another user that can only be used to wager on a particular prediction.
Like a gift card, but for designated predictions instead of department stores.
They can buy and sell as much of it as they like, but the funds don't get converted to mana that can be used anywhere unless they resolve.
If we had this, I'd send various individuals 100s of dollars to wager on predictions I think they should look at.
It's more in the spirit of surveying particular individuals than incentivizing new wagers from unknown users.