Chat with RTX: Create Custom Local Chatbots for Your Nvidia AI PC

Nvidia is launching Chat with RTX, a feature designed to create personalized local AI chatbots on Windows AI PCs. This innovation marks Nvidia's latest effort to make AI a mainstream tool utilizing its graphics processing units (GPUs).

Key Features of Chat with RTX

Local AI Interactions

Chat with RTX empowers users to harness the power of personalized generative AI directly on their devices. This technology utilizes retrieval-augmented generation (RAG) and TensorRT-LLM software, minimizing reliance on data center computing. Moreover, it enhances user privacy, eliminating concerns about data breaches during AI conversations.

Revolutionizing Chatbots

While traditional chatbots often depend on cloud servers powered by Nvidia GPUs, the new Chat with RTX enables users to leverage the processing capabilities of GeForce RTX 30 Series GPUs or higher, equipped with a minimum of 8GB of video RAM (VRAM). This significant shift allows users to enjoy generative AI locally.

A Personalized AI Experience

Nvidia emphasizes that Chat with RTX is more than just a chatbot; it serves as a customizable AI companion. By utilizing local GeForce-powered Windows PCs, users can experience generative AI with remarkable speed and privacy. The application employs RAG and TensorRT-LLM software for quick, contextually relevant responses based on local datasets. Users can connect the tool to their local files, creating a dataset for popular open-source large language models such as Mistral or Llama 2.

Instead of rummaging through various files, users can enter natural language queries—like asking for restaurant suggestions—and Chat with RTX will deliver swift, contextual answers. It supports multiple file formats, including .txt, .pdf, .doc/.docx, and .xml, providing a user-friendly experience.

Integration of Multimedia Content

Chat with RTX stands out with its ability to incorporate information from multimedia sources like YouTube. This feature allows users to ask contextual questions based on video content, such as seeking travel advice inspired by influencer videos or accessing tutorials from educational channels.

Thanks to its local processing capabilities, the application ensures fast results while keeping user data secure on the device—eliminating the need for cloud services and protecting sensitive information from third-party access.

System Requirements and Future Opportunities

To utilize Chat with RTX, users need at least a GeForce RTX 30 Series GPU with 8GB of VRAM, alongside Windows 10 or 11, and the latest Nvidia GPU drivers. Developers interested in optimizing large language models with RTX GPUs can access the TensorRT-LLM RAG developer reference project on GitHub. Nvidia invites developers to participate in the Generative AI on Nvidia RTX developer contest, running until February 23, offering chances to win a GeForce RTX 4090 GPU and a full conference pass to Nvidia GTC.

The Chat with RTX tech demo is now available for free download.