New AI model can hallucinate a game of 1993's Doom in real time

Researchers from Google and Tel Aviv University have introduced GameNGen, an AI model capable of real-time simulation of the classic 1993 video game Doom. Utilizing image generation techniques from Stable Diffusion, GameNGen functions as a limited game engine, allowing for the generation of new gameplay frames at over 20 frames per second using a single tensor processing unit (TPU). In tests, human raters struggled to differentiate between actual gameplay footage and AI-generated clips, identifying the real footage only 58% to 60% of the time. The model builds on previous AI advancements in video game synthesis, including GameGAN and Google's Genie. GameNGen employs a two-phase training process, initially training a reinforcement learning agent to play Doom, followed by using recorded gameplay to train a custom Stable Diffusion model. However, the use of Stable Diffusion introduces graphical glitches and challenges in maintaining visual clarity over time. The researchers emphasize that interactive world simulation requires more than rapid video generation, as it must also respond to player inputs in real time, which can lead to instability in the generated content.

- GameNGen can simulate Doom in real time using AI image generation techniques.

- The model generates gameplay frames at over 20 frames per second.

- Human raters found it difficult to distinguish between real and AI-generated gameplay footage.

- The development involved a two-phase training process using reinforcement learning.

- Challenges include graphical glitches and maintaining visual consistency over time.