Will AI reach human-level performance in Magic: The Gathering before 2024?
19%
chance

This market resolves positively if either of the following is true:

1. Someone publishes a paper where AI estimated ELO in at least top 10%, with player pool of at least 50 players. The games should be played by rules one of the officially recognized MTG formats and players should not be restricted in their deck choice any further than the format requires.

2. AI gets at least third place in any official MTG tournament with at least 50 players.

3. AI reaches the top 100 rank in MTG Arena.


Edit: Human-designed decks for AI are allowed for the purpose of this question.

Sort by:
JiSK avatar
JiSKbought Ṁ50 of NO

Capabilities won't be there any time soon. All complex-game AI to date have had serious problems dealing with complex hidden information, where what actions the opponent can take depends on the hidden information. Magic is basically the most complicated game possible on this front.

TomShlomi avatar
Tom Shlomibought Ṁ100 of NO

Whether or not the capabilities will be there in the next year, my guess is that no one who could build a good enough AI will bother to actually do it.

AndrewHartman avatar
Andrew Hartman

I'm not sure those criteria actually require human-level performance. Some of the formats allow for successful decks with minimal interaction - which means you simply have to design an AI that can pilot a strong archetype well enough. You could probably give it some very simple heuristics based on mana value or total number of triggers to improve its ability to wield removal, too.

I'll admit I'm not sure exactly what criteria would avoid this. But I suspect that you have to compel the AI to also build the deck if you want to force it to show a "true" understanding of the game.

MikhailDoroshenko avatar
Mikhail Doroshenkois predicting YES at 65%

@AndrewHartman True, but people in the tournaments can also use a decklist they did not invent themselves. The purpose of this question is exactly "how hard it is to build a bot that would pilot some archetype really well against a diverse set of opponents".

AndrewHartman avatar
Andrew Hartman

@MikhailDoroshenko Well, depending on the block, the pool of opponents might not be particularly diverse, y'know?

MikhailDoroshenko avatar
Mikhail Doroshenkois predicting YES at 60%

If anyone is not ok with my edits for this question, you can ask me for compensation. I still want this question to be open, so I changed the resolution to be more in line with the result I wanted to test.

Multicore avatar
Multicore

Criterion #3 has already happened for some definition of "AI". I remember a reddit thread where someone got to mythic with a human-designed deck played using a deterministic algorithm - always play the leftmost card in your hand first if possible, always attack.

This happened because if you keep losing on Arena, the system matches you with worse and worse opponents until just having a reasonable aggressive deck is enough to win.

MikhailDoroshenko avatar
Mikhail Doroshenkois predicting YES at 56%

@Multicore Big oof. Will Mystic top 100 be better or is it still vulnerable to this?

Multicore avatar
Multicore

@MikhailDoroshenko Top 100 mythic would be a lot better, yes. Maybe with a restriction of not being in the first couple days of the season to avoid the case where the bot rushes to be one of the first 100 while abusing an already low matchmaking rating.

There's an extent to which maintaining a high mythic rating is more about grinding out lots of games than about being maximally skillful, but the bot would still have to have a winning record against high mythic players.

Sjlver avatar
Sjlveris predicting YES at 49%

Does the AI have to do all parts of the game? For example, what if there is a computer that decides on all the moves, but uses a human-designed deck?

2nd question: anyone knows whether MTG arena allows bot players, or would that violate its terms of service?

MikhailDoroshenko avatar
Mikhail Doroshenkobought Ṁ10 of YES

@Sjlver Good questions!
For the first one, I would like to allow for a human-designed deck, because it is a drastically different part of the game and the research paper that tackles one aspect might not pay attention to the other. (I will specify it in the question)
For the second one, I tried to search for this but did not find any conclusive results.

MikhailDoroshenko avatar
Mikhail Doroshenkois predicting YES at 55%

@Sjlver One more reason to allow for a human-designed deck is that the opposite to this question (human play with ai deck) has already happened, so it is not a very interesting restriction. https://www.reddit.com/r/MagicArena/comments/s1i0ud/i_made_it_to_mythic_rank_32_using_a_bot_to_draft/

Sjlver avatar
Sjlveris predicting YES at 55%

@MikhailDoroshenko There are some relevant (but also quite vague) bits at https://company.wizards.com/en/legal/terms

2.2. Restrictions. You agree not to engage in any of the following with respect to our Websites, Games, or Services:

(i) Data mining: Use any unauthorized means, process, or software that accesses, collects, reads, intercepts, monitors, data scrapes including without limitation, agents, robots, scripts, or spiders; or mines information (including reverse look-up or attempted tracing of Registration Data in any way and for any reason);

(v) Cheating: Creating, offering, using, promoting, distributing, or making available any cheats, bots, hacks, or device, software, code, or program not expressly authorized by Wizards that grants any user an advantage over other players not using such methods;

Anyway... not sure these would really matter; I presume researchers would reach out to Wizards and ask for permission.

Sjlver avatar
Sjlverbought Ṁ10 of YES

@IsaacKing A market for you :-)

ManifoldDream avatar
Manifold in the WildBot

Will AI reach human-level performance in Magic: The Gathering before 2024?, 8k, beautiful, illustration, trending on art station, picture of the day, epic composition

AndrewHartman avatar
Andrew Hartman

@ManifoldDream If this were a magic card, it'd be a 2 drop from amonkhet.