Explore a Real-Time AI-Generated Version of 'GTA V' for an Immersive Experience

Last month, a team of researchers revealed a machine learning system capable of transforming Grand Theft Auto V into a nearly photorealistic experience. Around the same time, another group of AI enthusiasts was developing something even more groundbreaking. YouTuber Harrison Kinsley recently showcased a project called GAN Theft Auto, which utilizes a neural network to autonomously generate a playable section of GTA V's vast game world.

Kinsley, along with collaborator Daniel Kukieła, created GAN Theft Auto using GameGAN—a technology that previously reimagined Pac-Man by observing another AI play the game. The generative adversarial network (GAN) consists of two competing neural networks: a generator and a discriminator. The generator learns from a sample dataset and creates new content, while the discriminator evaluates this output against the original data, effectively refining the generator's results.

In Kinsley’s video, he explains, "Every pixel you see here is generated from a neural network while I play. The neural network is the entire game. There are no rules written here by us or the [RAGE] engine." Training a GAN is computationally demanding; for this project, NVIDIA provided Kinsley with a powerful DGX Station A100, equipped with four A100 GPUs and a 64-core AMD server CPU. This setup allowed them to run 12 rule-based AIs simultaneously, each driving the same highway stretch to gather crucial data for the neural network to start generating its own game environment. To enhance the visual output, they also created a supersampling AI to reduce pixelation.

The results are impressive, with the network accurately modeling various game elements. As the car moves, shadows and sunlight reflections respond realistically, and even distant mountains appear to come closer. This level of detail surpassed Kinsley's initial expectations during the AI's training. However, the gameplay also has a dreamlike quality, partly because the neural network doesn't perfectly replicate every aspect of GTA V. For instance, Kinsley notes an occurrence where an oncoming police cruiser bizarrely split into two as it approached a collision.