Will there be an AI which is trained on a range of video games, and is then able to transfer to a new type of video game that it hasn't seen before? It needs to achieve human level performance after spending an hour in the new game. It does not need to achieve superhuman performance.

This question is from this AI Timelines dialogue, specifically this quote from Ajeya Cotra:
"...do you think you'd update if you saw a demonstration of sophisticated sample-efficient in-context learning and far-off-distribution transfer? 

E.g. suppose some AI system was trained to learn new video games: each RL episode was it being shown a video game it had never seen, and it's supposed to try to play it; its reward is the score it gets. Then after training this system, you show it a whole new type of video game it has never seen (maybe it was trained on platformers and point-and-click adventures and visual novels, and now you show it a first-person-shooter for the first time). Suppose it could get decent at the first-person-shooter after like a subjective hour of messing around with it. If you saw that demo in 2025, how would that update your timelines?"

I will attempt to resolve this market according to the spirit of the quote. If it is not clear if the AI is "human level", I will ask 3 people who are familiar with the new game if they think this qualifies, and take the majority.

The AI does not need to be trained via RL or even have played other video games. It does need to have not played the new type of video game before.

• Update 2025-03-10 (PST) (AI summary of creator comment): Update from creator:

  • The AI only needs to successfully transfer to a single new type of video game that it wasn’t trained on.

  • It does not need to demonstrate this ability across multiple new video game types.