NVIDIA ChatRTX Now Supports Google's Gemma & ChatGLM3
In February 2024, NVIDIA released ChatRTX, an experimental AI chatbot for users to interact with their local data with several LLMs to choose from. This release has been highly anticipated by AI enthusiasts due to its impressive features.
Highlights of ChatRTX:
NVIDIA’s ChatRTX, previously known as ‘Chat with RTX,’ is an AI chatbot designed to work locally on PCs with Nvidia GeForce RTX graphics cards. Users can provide the chatbot with documents, text files, PDFs, and YouTube videos to power its responses, enabling them to ask queries and receive answers. One key feature of ChatRTX is its support for several LLMs, including Mistral and Llama 2.
ChatRTX has now expanded its support to include Google’s Gemma, a lightweight Gemini derivative, and ChatGLM3, an English and Chinese LLM built on the general language model foundation.
The inclusion of Google’s Gemma model in ChatRTX is significant as it is optimized to run natively on powerful laptops or desktop computers. NVIDIA's app simplifies the process of running various models locally, allowing users to select the model that best suits their data for analysis or search.
Features of the Supported Models:
- Google’s Gemma: [Insert features here]
- ChatGLM3: [Insert features here]
ChatRTX has also added capabilities of OpenAI’s CLIP to enhance image recognition and incorporates Whisper for AI-based speech recognition support.
While ChatRTX is considered a “tech demo,” it showcases the possibilities of creating AI applications using the TensorRT-LLM RAG open-source reference project, making LLMs more accessible to users. The chatbot's locally enabled running features, including documentation and summarization, make it user-friendly and easy to train custom chatbots.
Recently, OpenAI launched a platform to assist developers in fine-tuning their AI models, hinting at a similar direction NVIDIA might be heading towards. With ChatRTX, users can benefit from an efficient ‘offline’ chatbot experience, ensuring data confidentiality and integrity.
Only Windows 11 PCs with an NVIDIA GeForce™ RTX 30 or 40 Series GPU with a minimum of 8GB of VRAM and 16GB of RAM can utilize ChatRTX. With the integration of new AI models, ChatRTX offers enhanced functionalities locally, with potential support for additional AI models in the future.
