On August 29, a groundbreaking research paper by Google researchers introduced a neural network capable of generating real-time graphics for the classic shooting game "DOOM" without relying on traditional game engines. The research, published on GitHub, also unveiled GameNGen, the first game engine entirely driven by neural models. Remarkably, GameNGen can simulate "DOOM" in real time at over 20 frames per second on a single TPU, Google’s custom AI accelerator. The visual quality produced is strikingly close to that of actual gameplay, eliminating the need for standard game engine components.
GameNGen operates by using diffusion models to predict each frame in real time. This allows the graphics to adapt dynamically based on player actions and interactions within complex environments. The training process for GameNGen occurs in two stages: first, an RL agent learns to play the game and logs its training sessions; second, a diffusion model is trained to generate the next frame based on the sequence of previous frames and player actions. This innovation suggests that AI could enable instant game creation and offer personalized experiences for every player.