At the end of each year, I’ll play a game of chess against an LLM. Resolves YES if I lose, NO if I win, and 50% for a draw. On which years will I win?
Some more details: I’m rated around 1900 FIDE, 2100 USCF. The game will be played at a rapid time control. I’ll select the LLM from the top 3 of the leaderboard (https://huggingface.co/spaces/lmsys/chatbot-arena-leaderboard), or another benchmark if I believe it to be a better measure of general reasoning capabilities. On each move, I’ll provide the LLM with the game state in fen and pgn notation, and I’ll follow along on lichess. If the LLM makes 3 illegal moves, I’ll consider it a victory. Distinctions like Nd2 vs Nbd2 won’t count towards this, but I’ll ask the LLM for clarification. The LLM will play white.