Robin Hanson has famously long AI timelines. For example in an interview from 2019, and more recently, a podcast, he seemed to argue that human-level AI is likely at least a century away. However, many are speculating that AI developments announced in early 2023 will be "wild" and will cause people to shorten their AI timelines.
This question resolves to YES if Robin Hanson publicly changes his mind before July 1st 2023, and clearly indicates that his median human-level AI timeline has moved to before 2075. Here is a non-exaustive list of example scenarios that would count towards positive resolution:
He releases a blog post indicating that he now expects robots to receive >50% of Gross World Product within 5 decades.
He says in a public talk published before July 1st that he now thinks ems will probably be invented in the 2060s.
He says in a podcast that he "now finds it very plausible" that the labor force participation rate will fall to <10% due to robots by 2050.
If it's ambiguous whether he's changed his mind, I'll tweet at him and ask. If he does not reply within roughly 2 weeks, the question will resolve to N/A. If he does not change his mind, this question resolves to NO.
Related questions
🏅 Top traders
| # | Name | Total profit |
|---|---|---|
| 1 | Ṁ1,114 | |
| 2 | Ṁ263 | |
| 3 | Ṁ209 | |
| 4 | Ṁ168 | |
| 5 | Ṁ129 |
@na_pewno On second thought I'll just resolve negatively. I think that would be the most fair outcome.
@MatthewBarnett Can you do both? I'd be interested to know his opinion even if it won't affect the resolution at this point 😅
Paper which goes into productivity increases from using ChatGPT.
"In an online experiment, we recruit 444 experienced, college-educated professionals and assign each to complete two occupation-specific, incentivized writing tasks. The occupations we draw on are marketers, grant writers, consultants, data analysts, human resource professionals, and managers. The tasks, which include writing press releases, short reports, analysis plans, and delicate emails, comprise 20-to 30-minute assignments designed to resemble real tasks performed in these occupations; indeed, most of our participants report completing similar tasks before and rate the assigned tasks as realistic representations of their everyday work."
ChatGPT basically doubled the speed of the participants, and increased their grade by around 1 point on a 7 point scale.
I wonder if stuff like this will change Hanson's mind soon? He seems to be pretty bullish on AI not being economically useful, but pretty soon it will be obviously useful.
It seems that Robin Hanson has not updated on GPT-4. I'm confused about why he still seems to have a 2150 median timeline, which was unchanged by developments in the last year, but I guess he's getting old, and so he might just be sticking with his priors. I have my sympathies.
@MatthewBarnett was GPT-4 an update towards short timelines for many close followers of AI? The vibe I've gotten this week is that it's pretty in line with expectations. As Hanson, Marcus, etc. suggest, it's still quite bad at the things LLMs are bad at.
@JacyAnthis Progress on various AI benchmarks and objective measures have been pretty smooth in the last few years, but many people (including me) updated their timelines anyway because (1) people don't always know how quantitative measures of performance translate into qualitative performance, and newer demos make this more clear, (2) the compute trend does not seem to have slowed down by much, which is crazy because it's been extremely rapid, (3) a lot of people don't track the benchmarks closely, and (4) the evidence for the scaling hypothesis continues to grow stronger with each new development that doesn't show steep diminishing returns.
@JacyAnthis I also think that Hanson's model of AI development here is just bad, and therefore he ought to update in light of (even very predictable) evidence.
To be clear, I think he could be right about long timelines by coincidence. I think it's plausible that we could soon get a dramatic slowdown in the rate of increase in price-performance for computing hardware, and we could also get a corresponding decrease in the rate of algorithmic progress and compute scaling. This would definitely push timelines far into the future. But I don't think Hanson has argued for that thesis at length anywhere recently.
I think his argument for long timelines is rather that current AI is simply "dumb" and we need qualitatively new advances to make it "smart", and I think that's just an incorrect argument. So, I expect him to at some point update his position once it's clear that the mechanism he proposes for long timelines is revealed to be nonexistent.
@MatthewBarnett Thanks for the clarification. My particular interest here is whether Hanson's lack of updating on GPT-4 is evidence of poor judgment assuming his belief prior to GPT-4 was right. For example, did he or would he have made predictions that the compute trend would have slowed down by more than he did?
Your points seem to mostly be about his prior being bad, which is interesting too but not as concerning as poor judgment even in the scope of one's own model. I think your points can be naturally translated to within-model critique with a substantial loss in force, and we can probably both reason to the limit about that instead of explicating those points—for now. Thanks!
@JacyAnthis I'm confused by what you're saying.
I think his judgement simply is poor, and I don't see why you'd disagree. Maybe you can help me understand what you're saying by replying to the following thought experiment.
Suppose the compute and price-performance trends continue indefinitely at the rate they've been going for the last 7 years and we get a new GPT-N every other year, with the modalities expanding to video, audio and robotics, and RLHF continuing to fine-tune the models with more and modestly better supervision. Under my model, this will likely yield AGI sufficient in capability to automate a high fraction of labor within 15 years, even if it still has some problems with hallucination, alignment, and robustness. I think Hanson would disagree with that statement.
In this scenario, if after every new GPT-N comes comes out Hanson tweeted that he wasn't surprised by it and the model wasn't introducing anything qualitatively new, then that would be really weird. Because at the end of the 15 year period, we would have AGI and then Hanson would say, "I wasn't surprised by any of this" and yet what happened completely inconsistent with almost all the public statements he's been making for the last 5 years.
@MatthewBarnett I'm not intending to make any statements about his judgment in general, so I think that clarifies your first point?
I agree Hanson would say that trajectory would not result in AGI within 15 years.
I don't know if those tweets would be "really weird," and that doesn't seem very relevant to me. Of course such tweets seem reasonable by his model and unreasonable by yours, and the occurrence of AGI at the end is decisive evidence against his. Other evaluations of the two models need to be made on bases other than "did they predict each GPT-N" because they both seem to do that completely. Maybe you mean that Hanson's model wouldn't have predicted the GPT-Ns even though he says it does, in which case, sure, then the GPT-Ns are evidence too.
Again, I'm not intending to have a discussion about Hanson's overall judgment. I was just wondering if his lack of updating on GPT-4 was within-model evidence of his poor judgment. This would be easier to explain with a whiteboard, but if you'll tolerate a brief toy example, say I believe a car is turning left because it tends to do 90-degree-leftward turns. It starts turning right, and I say, "This is in line with my expectations. I don't update because it is making a 270-degree-rightward turn, as I expected," then you should update towards (i) my model is wrong, (ii) I'm being unreasonable even if within my own model. => I'm wondering if Hanson's lack of update on GPT-4 is the latter sort of evidence.
@JacyAnthis To be clear, I'm also not making statements about his judgement in general; just in this case.
I think I understand what you're saying better now.
@StrayClimb disappointing argument almost entirely from history. I also notice that humans did not go extinct after inventing farming, but I don't really see the relevance to AI extinction risks.
In the scenario list I find the second one to be tricky. If I understood correctly he argues that genuine creation of human level ai is farther out than digitize and simulate capabilities aka Ems. I bet against him revising the timeline for GAI but think the 2nd scenario should not be considered to decide on this or the Market question should read "human level AI or EMs".
Btw his implications about societal change I believe will affect us much sooner. I do not think we need full capacity brain scan EMs to get those. Current LLM tech trained on the data output of individuals will be how scenarios he describes will be starting to manifest. One could argue this is happening already and I should maybe think about which indicator captures that and set up a prediction market about it.
Robin has recently reiterated his long timelines, e.g.,
https://twitter.com/robinhanson/status/1628155427841253376
https://twitter.com/returndotlife/status/1628086004342218757
@JacyAnthis
"This is why my median-estimate date for the creation of a human-level AI is roughly 2150. I have wide uncertainty around that, but I’m pretty sure it won’t be in the next thirty years. (And I’d be happy to bet on that.)"
The base rate of Robin Hanson changing his mind about AI is pretty low, but I tend to follow him pretty closely, and I get the sense that he's changing his mind right now. In early 2022, he was dismissing language models as mere "babblers". Now he's saying that GPT-line systems will be as big of a deal as airplanes. He also didn't want to take my bet with Bryan Caplan about AI progress recently, indicating that he now expects significant progress in the near future (he originally said he wanted to take the bet).