NVIDIA ChatRTX's latest update brings support for additional features and models
The ChatRTX update adds new AI model support, including Google's Gemma and the bilingual ChatGLM3, enabling enhanced interaction with local documents. It also adds interaction with images via OpenAI's CLIP, and voice interaction through Whisper speech recognition.
Since its introduction, NVIDIA ChatRTX has empowered users to interact with their local data via NVIDIA RTX-powered Windows PCs and workstations. ChatRTX combines retrieval-augmented generation, NVIDIA TensorRT-LLM software, and NVIDIA RTX acceleration to deliver chatbot capabilities so users can query their notes and documents leveraging ChatRTX's capability to construct relevant responses on the user's device. The latest ChatRTX update introduces support for new models like Google's Gemma —which uses the same technology as the Gemini models— or the open, bilingual (English and Chinese) ChatGLM3.
This update also enables interaction with image data, due to OpenAI's Contrastive Language-Image Pre-training support. CLIP, a neural network that can learn visual concepts through natural language supervision, enables the model to recognize what it "sees" in image collections. CLIP support lets ChatRTX users interact with locally stored photos and images using text prompts without requiring metadata labeling. Voice interaction is enabled via Whisper speech recognition, letting users query ChatRTX with voice inputs that the model will process before producing text-based responses.
