If there is a GPT-5 developed which is a similar amount better than GPT-4, as GPT-4 was to GPT-3, then will GPT-5 be capable of recursive self-improvement with a minimal amount of prompt engineering / scaffolding?
Note: recursive self-improvement, in the early stages, doesn't require novel scientific breakthroughs. It is sufficient to successfully integrate existing work that is not yet part of the model. Of course, to be 'recursive' this must be shown to repeat, and there must be evidence that the later generations are capable of advancements that the initial generation was not capable of.
Since this market is getting more interest, I thought I'd put some clarification here. I'm up for having a 3rd party arbiter of this question, details can be arranged closer to the close date.
If GPT-5 comes out before the stated close of the market, then the market will close as soon as the question can be evaluated. Subtle self-improvements which quickly plateau out, such as has been seen so far with GPT-4 using Reflexion, will not count. The process doesn't need to be entirely 'within' the model, as direct modifications of the model's weights. It could include external code wrappers and memory systems interfacing through an API. The system does need to show multiple steps of clear improvement, where the later steps are demonstrably better at making further improvements than the earlier steps.
As clarified in the comments, if the recursive self-improvement can't be clearly demonstrated using less than 3% of the FLOPs used in training GPT-5, then it doesn't count.
Related questions
Some interesting speculation about what might be on the horizon.... https://youtu.be/ARf0WyFau0A?si=X3DEqNkqEsp4W1OT
"CEO Sam Altman has privately suggested OpenAI may try to raise as much as $100 billion in the coming years to achieve its aim of developing artificial general intelligence that is advanced enough to improve its own capabilities, his associates said." - https://www.theinformation.com/articles/openais-losses-doubled-to-540-million-as-it-developed-chatgpt
@YoavTzfati Not until we are able to significantly alter our brains with genetic engineering and brain-computer interfaces. This is something stronger than just learning-as-usual.
I don't think the most recent note at the bottom of the description is a reasonable change to the market (it is specifically broadening the scope of the original question which was hyper-specific). Even worse that it does so at the favour of the market creator who has 98% of the total ~18 200 YES shares
@Gen Agree. If other possible AIs are crucial to the question, they should be included in a new, separate market.
@TylerColeman @Gen fair. I'll remove that from this question and make a separate market.
How much compute can the improvements require? Would you be open to giving a rough threshold as, say, a percentage of GPT-5's training compute?
@NoaNabeshima Nice questions. For this one, I'd say that it would make sense that the compute needed for a step which delivered a gain of x would need to be cheaper than a gain of capability roughly equivalent to x cost in FLOPs during training. Does that make sense?
@NoaNabeshima Hmm, I want really thinking of an x that large. I suppose my best answer is that the question needs to be answered with less compute than that, so if minimum viable step side was larger than the whole training cost then I'd resolve no even though that's an unclear edge case.
How much less would it have to be before it counts? Would any of 3%, 10%, 30%, 50% count?
@NoaNabeshima for the purposes of this definition, let's say a max of 3%. Not because that number constrains reality in some meaningful way, but because I think that it works be implausible to measure if it were more.
@NathanHelmBurger My expectation is that it will show up at very little extra compute, like less than a tenth of a percent. That there will be a series of small steps that can be taken in the direction of improvement, you can plot a straight or increasing line through them according to multiple benchmarks, and be able to say 'this trend could plausibly continue '.
Do the later steps need to be better in a large way? How large? Or do they just need to be demonstratably better to any degree, however small?
@NoaNabeshima Individual 'steps' can be small, since the idea of a 'step' is fairly arbitrary. What's important is the lack of plateauing after multiple steps. So, as you mentioned in your comment above, the steps must be cheaper than the ordinary training, and the trend of improvement has to seem at least linear (not slowing) for the range in which we are able to observe it. I believe these two requirements together describe the sort of accelerating process I am trying to pinpoint with the question.
Some thoughts on my current understanding of the AI development landscape (which may be wrong!): https://www.lesswrong.com/posts/GxzEnkSFL5DnQEAsZ/paulfchristiano-s-shortform?commentId=hEQL7rzDedGWhFQye
It’s pretty obvious this is already true.
It was true of google search (bouncebacks and click through as retune the algo)
And it’s true of the rlhf they are using.
The models change every day
@Gigacasting I agree it's kinda true in a weak sense, with small quickly-plateauing self-improvements, and with human-in-the-loop larger improvements. However, in the sense of "strong human-out-of-the-loop repeatable self-improvement"... not yet. That stronger case is what this question is about.
Related questions
