Will a video game be released in which multiple NPCs' dialogue is based on live inference from an LLM, before 2026?
The player must have considerable freedom for conversation with the NPCs: picking from options is NOT sufficient. Input should be free text or following a flexible framework which allows fairly arbitrary conversation within the context. (I could imagine building sentences by picking words which are aligned with the universe etc to avoid bad publicity when someone asks something inappropriate.)
The LLM must have been originally trained with a parameter count of 7B+, just to establish a lower bound. The implemented parameter count after pruning etc may be lower, for cheaper inference.
The inference may be performed in the cloud (assuming subscription based), or locally, perhaps on GPU or AI accelerator (e.g. Ryzen 7840).
As for the game itself, it must achieve an average score of 80+ on metacritic with at least 60 reviews. This means there's a lower bound for quality and visibility.
Date based on game's release date (not early access). This might mean the market closes shortly after the end of 2025 as reviews come in, for example.
1,000
3.00@Tomoffer Good computers can already handle low-tier open source models, so local will be possible. Cloud compute seems very unlikely because it would cost the company money.
@ShadowyZephyr the video memory footprint for llama7b is on the order of 10GB, depending on quantisation, and that's before you do any actual graphics. I expect VRAM will scale rapidly in the coming years to better support local execution generative AI though.