Ever since the groundbreaking launch of ChatGPT, the tech community has been eagerly anticipating a local AI chatbot that operates independently from the cloud. Nvidia has delivered with Chat with RTX, a local AI chatbot designed to help you sift through your offline data using an AI model.
This guide will walk you through the setup and operation of Chat with RTX. Keep in mind that since this is a demo version, you may encounter some bugs. However, this innovation promises to pave the way for more local AI chatbots and tools.
How to Download Chat with RTX
The initial step is to download and configure Chat with RTX, which involves more complexity than you might expect. While installation begins with running an installer, be prepared for potential issues and ensure you meet some minimum system requirements.
You’ll need an RTX 30-series or 40-series GPU with at least 8GB of VRAM, 16GB of system RAM, 100GB of disk space, and Windows 11.
Step 1: Download the Chat with RTX installer from Nvidia's website. Note that the compressed folder is 35GB, so the download may take some time.
Step 2: Once the download is complete, right-click the folder and select 'Extract All.'
Step 3: Inside the extracted folder, locate the setup.exe file and follow the installation prompts.
Step 4: Before installation starts, the installer will ask where to store Chat with RTX. Ensure you have at least 100GB of disk space in your chosen location, as the AI models will be downloaded there.
Step 5: Installation can take up to 45 minutes, so don't be alarmed if it seems to stall. It may also slow down your PC, especially during the AI model configuration. Feel free to step away while the installation completes.
Step 6: If the installation fails, simply rerun the installer and select the same data location to resume the process.
Step 7: After installation is complete, a shortcut to Chat with RTX will appear on your desktop, and the app will launch in your web browser.
How to Use Chat with RTX with Your Data
The primary feature of Chat with RTX is its ability to utilize your own data. It uses a method called retrieval-augmented generation (RAG) to sift through documents and provide answers based on the content. Instead of general inquiries, Chat with RTX excels at answering specific questions related to your dataset.
Nvidia provides some sample data to help you test the tool, but adding your own data will unlock its full capabilities.
Step 1: Create a folder to store your dataset and remember its location, as you’ll need to direct Chat with RTX to it. Currently, it supports .txt, .pdf, and .doc file formats.
Step 2: Open Chat with RTX and click the pen icon in the Dataset section.
Step 3: Navigate to your dataset folder and select it.
Step 4: In Chat with RTX, click the refresh icon in the Dataset section to regenerate the model based on your provided data. Remember to refresh the model each time you add new files or switch to a different dataset.
Step 5: After your data is added, choose the desired AI model from the AI model section. Chat with RTX offers Llama 2 and Mistral, with Mistral as the default. Although both models are worth exploring, Mistral is recommended for newcomers.
Step 6: You can begin asking questions now. Keep in mind that Chat with RTX does not retain previous context, meaning earlier responses do not inform later ones. Specific inquiries generally yield better results than vague ones. Be aware that sometimes Chat with RTX may reference the wrong data in its responses.
Step 7: If Chat with RTX stops functioning and restarting doesn’t help, you can resolve the issue by deleting the preferences.json file located at C:\Users\[YourUsername]\AppData\Local\NVIDIA\ChatWithRTX\RAG\trt-llm-rag-windows-main\config\preferences.json.
How to Use Chat with RTX with YouTube
In addition to personal data, Chat with RTX can analyze YouTube videos. The AI model utilizes the video’s transcript, but there are some inherent limitations.
It’s important to note that the AI model cannot interpret content not included in the transcript, so you won’t be able to inquire about visual elements, like what someone looks like in a video. Additionally, YouTube transcripts can occasionally be inaccurate, which may affect the responses you receive.
Step 1: Open Chat with RTX and, in the Dataset section, select the dropdown to choose YouTube.
Step 2: Paste the link to a YouTube video or playlist in the provided field. You will also see a number indicating the maximum number of transcripts you wish to download.
Step 3: Hit the download button next to this field and wait for the transcripts to finish downloading. Afterward, click the refresh button.
Step 4: Once the transcript is prepared, you can chat just as you would with your dataset. Remember that specific questions tend to work better than general ones, and if engaging with multiple videos, Chat with RTX may misunderstand references if the question is too broad.
Step 5: To discuss a new set of videos, you'll need to manually delete the previous transcripts. Use the button next to the refresh button to access the Explorer window, then delete old transcripts to make way for new ones.
Ranking
