NVIDIA Unveils AI Model for Rapid Generation of Objects and Characters in Virtual Worlds

NVIDIA is revolutionizing the creation of virtual 3D worlds with its new artificial intelligence model, GET3D. This innovative tool generates a variety of 3D objects, including characters, buildings, vehicles, and more, at an impressive rate of around 20 objects per second using a single GPU.

Researchers trained GET3D using synthetic 2D images sourced from multiple angles of 3D shapes. In just two days, they processed about 1 million images with A100 Tensor Core GPUs. According to NVIDIA’s Isha Salian, GET3D produces objects with "high-fidelity textures and complex geometric details." These objects take the form of triangle meshes similar to papier-mâché models, enhanced with textured materials.

GET3D's outputs are easily importable into game engines, 3D modeling software, and film rendering tools, allowing developers to efficiently populate virtual environments for games and the metaverse. NVIDIA also highlights applications in robotics and architecture.

The model demonstrates versatility; trained on a dataset of car images, GET3D can generate various vehicle types, including sedans, trucks, and race cars. Similarly, by learning from animal images, it can create lifelike renditions of foxes, rhinos, horses, and bears. The diversity and detail of the generated objects improve with a larger and more varied training dataset.

Furthermore, with the assistance of another NVIDIA tool, StyleGAN-NADA, users can apply unique styles to the generated objects using text prompts. This functionality allows for imaginative transformations, such as making a car appear burned-out, converting a house into a haunted version, or adding tiger stripes to an animal model.

The NVIDIA Research team behind GET3D envisions future enhancements where the model could be trained on real-world images instead of solely synthetic data. Additionally, the possibility of training GET3D on multiple 3D shape categories simultaneously could further expand its capabilities.