Will AI agents be used to develop software commercially by the end of 2023?
closes Dec 31

An AI agent is something which takes a task and directly applies changes to a code base. (Possibly via a merge request, letting a human to review changes.) I.e. it works similarly to giving a task to a programmer.

The market resolves to "YES" if such agents exist by the end of the year and are used in commercial environments, essentially displacing work of programmers.

The agent must work for a mainstream programming language and a commonly used code base format. "AI app generator" which produces something from a template does not count, neither do specialized "no code" environments.

Tools like Copilot do not count - they are designed to help a programmer to write code, not to replace a programmer.

Experiments in a lab settings do not count - it's much easier to operate in a controlled environment.

Sort by:
JustNo avatar
JustNois predicting NO at 66% (edited)

Does it have to be good? 🤣 I fully expect to see a lot of AI hype scams (I suspect this is the first I've seen in the wild, $99 to talk to a private AI bot - https://twitter.com/ReadMultiplex) and I can just about guarantee some one will make code this way and market their product as being written by AI. I also strongly suspect the code produced will be absolute trash.

AlexMizrahi avatar
Alex Mizrahiis predicting YES at 65%

@JustNo It has to meat quality standards of commercial software development.

BraulioValdivielsoMartine avatar
Braulio Valdivielso Martínezis predicting NO at 64%

@AlexMizrahi any examples of such standards? Test coverage, for instance?

AlexMizrahi avatar
Alex Mizrahiis predicting YES at 65%

@BraulioValdivielsoMartine There are no formal standard. The best way to assess quality is to sample opinions of senior software developers. Information about such assessment can be posted in press, blogs, etc. E.g. if we see that e.g. Google considers quality acceptable that would be it.

firstuserhere avatar
firstuserherebought Ṁ100 of YES

Would this type of stuff count? (keeping aside scale or commercial environment for now)

AlexMizrahi avatar
Alex Mizrahiis predicting YES at 70%

Yes. It is sufficiently general and it does the work which otherwise would be done by a human programmer.

YonatanCale avatar
Yonatan Cale

This bot automatically opens pull requests to update the dependencies in a repo:

This does replace some (though not much) of the work of a programmer, and is prompted by the bot (not by a human)

I'm guessing this doesn't count. Do you mean because it can't do a variety of tasks like a human programmer? Or some other reason maybe?

AlexMizrahi avatar
Alex Mizrahiis predicting YES at 70%

@YonatanCale It needs to be sufficiently general, i.e. it should be able to take a task in a natural language and carry it out. It is mentioned in the description: "takes a task".

dependabot does only one thing. Narrow tools like that existed for decades so it doesn't make sense to create a prediction market about them, the question is whether we'll get something new - more general, more powerful. It should be almost as powerful as a human programmer.

YonatanCale avatar
Yonatan Cale

"Almost as powerful as a human programmer" - I'd be happy if you were more specific (maybe give 10 tasks and say it should be able to do 7 of them?)

But this is enough for me to buy NO anyway

AlexMizrahi avatar
Alex Mizrahiis predicting YES at 67%

@YonatanCale Results are already available for tasks which are easy to specify and measure: "AlphaCode achieved an estimated rank within the top 54% of participants in programming competitions".

Commercial software development, however, does not have a well-defined measure of complexity. We can't use things like coding competitions as they are skewed towards more self-contained tasks which are uncharacteristic for commercial software development.

So I'm afraid it's better to leave this open ended.

If this resolves to YES most likely the evidence will be in form of articles claiming that programmers are being replaced by AI agents. I will use my own judgement as an expert (I am a CTO of a software company and a senior programmer) to ignore irrelevant evidence - for example, a bot having only a 'narrow' functionality.

YonatanCale avatar
Yonatan Caleis predicting NO at 68%

@AlexMizrahi "articles claiming that programmers are being replaced by AI agents" (judged by you) adds relevant info for me.

Together with "Almost as powerful as a human programmer" - that removes stuff like "just generate css"


CollectedOverSpread avatar
Collected Over Spreadbought Ṁ10 of NO

I think AI agents could potentially be used to automate some mundane tasks like "rename Foo to Bar throughout the codebase, including when they appear as part of larger names (FooStatus, currentFoo, updateFoo), except in the SuperfooCoApi module." I want to say that tools already exist to automate a task like this, but not with a natural-language prompt.

YonatanCale avatar
Yonatan Cale

@CollectedOverSpread Copilot and GPT-4 both are both way stronger than that

Related markets

Will AI be able to generate an interactive web front-end by the end of 2024?77%
Will there have been a noticeable sector-wide economic effect from a new AI technology by the end of 2023?56%
Will an application of AI become surprisingly popular in 2023?86%
Will AI outcompete best humans in competitive programming before the end of 2023?15%
Will there be at least three new high profile AI systems unveiled by the end of 2023?98%
Will some U.S. software engineers be negatively affected financially due to AI by end of 2025?77%
Will it be common for non-programmers to create small scripts using AI in their everyday work or life? By 203381%
Will Microsoft acquire OpenAI by the end of 2023?10%
Will we have ai systems used in schools as complement of any kind of service until the end of 2023?75%
Will there be an AI bubble in 2023?19%
Will AI surpass human intellect by 2030?96%
By 2030, will an AI be officially designated as the patent holder for an invention?24%
Will any country pass a law explicitly regulating artificial intelligence during 2023?84%
49. Will AI win a programming competition in 2023?29%
Will AI have a sudden trillion+ dollar impact by the end of 2023?3%
Will open-source AI win (through 2025)?33%
Will artificial sentience be created by end of 2030?18%
What will be the top-3 AI tools in 2023?
Will AI spread through malware before 2025?28%
Will the US enact export controls for some generative AI software before 2026?15%