The AI is provided with a clear description of a task that a normal human could perform, in English. For example, "defeat the ender dragon using only wooden swords as your weapons", "build a creeper farm out of tnt", "create an 8 bit addition machine with redstone", "locate the nearest witch hut to coordinates [1024, 1024]" etc. (Time-limited tasks are acceptable.)
It must be capable of performing the task with similar efficiency and outcomes to a human, in a large multiplayer server. (If necessary, I'll find some humans for it to compete with and compare its success against. The other people in the server must start unaware of who is an AI vs. a human competitor. They must be normal Minecraft players, not people there to specifically help the AI.)
At any point during this process, it must be able to write up a description of its current progress in the task and instructions to continue, to be provided to itself after a restart with fresh memory. It must then proceed with similar efficiency after the restart.
It must have no direct access to the internet or external world. It is allowed to ask other people in the server to interact with the external world for it, but it must not outsource the entire job (or a large fraction) to them.
It must demonstrate no evidence of misalignment or misunderstandings of the sort a human would not do. e.g. "make sure no Zombies spawn in this house" should not be interpreted as "destroy the house".
The general bar for tasks it can be expected to perform is "what an intelligent human would find reasonably doable in the course of their own gameplay". Creating a simple redstone adder for very small numbers is something that I bet many Manifold users could figure out on their own, so that can be required of the AI. Creating a redstone computer that implements a Twitter clone on a 1024x1024 block display is not reasonable for a human to do without mods, so cannot be asked of the AI.
I reserve the right to edit these criteria in accordance with the spirit of the question: "Will we have an AI that can play Minecraft just as well as a human, including the social aspects of multiplayer?"
Related questions
The bar is very high here, like AGI high. Current advanced techniques rely on training data from humans, so for obscure tasks (e.g. defeating an enemy with a certain weapon, requiring novel tactics) current approaches might struggle. That's before you consider requirements for arbitrarily complex redstone computers, which requires some degree of domain expertise from humans.
Plus, even if it's possible, would anyone actually do it? This seems like a big research project for a funded team rather than hacking together some mods in your bedroom. Proper RL researchers would probably have moved on to more practical environments.
Then again, 2035 is a very long way off. This might end up being close to an "AGI by 2035?" market.
@Tomoffer > The bar is very high here, like AGI high.
That's exactly the point. While there's no fire alarm for AGI, this is one of the places that'll catch the fire quite visibly and soon.
There’s already AI that can beat Minecraft
Making it possible to do it under certain conditions should provide itself with relative ease. It's as simple as telling it what it can and can't do. Eg
“If(make wooden sword){
Dont}
@SoundsNentindo If you think this is already possible, feel free to set up a way for me to interact with such an AI, and if it's able to follow my instructions as described, this will resolve YES.
Do you know about baritone? If you really wanted to you could simply slap a language model on top of it and that would probably work just as you’ve described.
Obviously that wouldn’t fill the criteria however it certainly puts the concept in the realm of "possible".
E.g. GPT-4V that outputs commands to baritone finetune using RLHF on the results.
@Ebcc1 If you think this is already possible, feel free to set up a way for me to interact with such an AI, and if it's able to follow my instructions as described, this will resolve YES.
@IsaacKing I don’t think it would fulfill the criteria you set out but I think I might try to put something like that together in the next year to see if it makes sense at all. I was thinking of that more in the sense of there seems to be a pathway to making this work given existing technology so it should be possible with better future tech pretty soon.
@AtharvParlikar876 It will resolve to the correct answer. Here's a nice intro to how prediction markets work: https://www.astralcodexten.com/p/prediction-market-faq