NVIDIA Unveils New Open Models and Resources to Enhance AI Across Industries

NVIDIA has recently introduced a broad range of open models, datasets, and tools designed to enhance artificial intelligence (AI) innovation across all industries. This initiative expands the capabilities of what it refers to as the open model universe.

Among the new offerings are several families of models tailored for specific applications: the NVIDIA Nemotron for agentic AI, the NVIDIA Cosmos platform addressing physical AI, the NVIDIA Alpamayo series focused on autonomous vehicle technology, NVIDIA Isaac GR00T supporting robotics, and NVIDIA Clara aimed at biomedical applications. These resources are intended to empower organizations in developing AI solutions that address real-world challenges.

The scale of NVIDIA's initiative includes contributions such as open-source training frameworks and a massive collection of multimodal data. This encompasses approximately 10 trillion language training tokens, 500,000 trajectories related to robotics, 455,000 protein structural models, and 100 terabytes of vehicle sensor data. These extensive resources are positioned to significantly speed up innovation in language models, robot functionalities, scientific research, and the development of autonomous vehicles.

Several major tech companies, including Bosch, CodeRabbit, CrowdStrike, Cohesity, Fortinet, and others, are already adopting NVIDIA's open model technologies to enhance their offerings.

NVIDIA Nemotron: Enhanced Speech and Safety for AI Agents

The newly launched NVIDIA Nemotron 3 family has unveiled specific models focused on speech recognition, multimodal retrieval-augmented generation (RAG), and safety enhancements.

• Nemotron Speech comprises competitors on the leaderboard with a new ASR model designed to provide real-time, low-latency speech recognition capabilities, claiming to achieve 10x faster performance than similar models.
• Nemotron RAG features advanced vision-language models delivering accurate multilingual insights for improved document searching and data retrieval.
• Nemotron Safety introduces models such as the Llama Nemotron Content Safety model to bolster application safety and Nemotron PII for identifying sensitive data.

Collaborations are notably highlighted, with Bosch implementing Nemotron Speech technology for driver interactions, while Palantir is incorporating these models into its framework for specialized AI agents.

New Tools for Physical AI and Robotics

As the robotics segment continues its expansion, NVIDIA’s open models have become a leading offering on platforms like Hugging Face. The company is advancing physical AI development with the release of NVIDIA Cosmos foundation models. These models are designed to enhance machine capabilities in real-world applications.

• Cosmos Reason 2 is a leading reasoning VLM assisting robots and AI in interacting with physical environments more accurately.
• Cosmos Transfer 2.5 and Cosmos Predict 2.5 are significant models capable of generating large-scale synthetic video data for various conditions.

Many organizations, including Salesforce and Hitachi, are utilizing Cosmos Reason models to develop AI agents that improve both productivity and operational safety.

NVIDIA Alpamayo for Autonomous Vehicles

In efforts to enhance the development of autonomous driving technologies, NVIDIA introduces the NVIDIA Alpamayo family, which emphasizes reasoning capabilities as essential for navigating complex environments.

• Alpamayo 1 stands as the first large-scale reasoning VLA model for autonomous vehicles, requesting vehicles to understand and articulate their situational responses.
• AlpaSim serves as a simulation model allowing comprehensive training and evaluation of AVs.

This initiative is accompanied by over 1,700 hours of driving data across diverse and challenging conditions aimed at helping advance reasoning capabilities within AI systems.

NVIDIA Clara for Healthcare Innovations

The launch of the NVIDIA Clara models aims at bridging the gap between digital discovery and clinical applications in medical research.

• La-Proteina facilitates the precise design of large proteins, opening new avenues for drug candidate development.
• ReaSyn v2 aims to make AI-designed drugs feasible for synthesis.
• KERMT delivers predictions on drug interactions with the human body, enhancing safety assessments.
• RNAPro targets predicting RNA molecule structures, contributing to personalized medicine.

Accessibility of NVIDIA's Resources

NVIDIA has made its open models, datasets, and frameworks readily accessible through GitHub, Hugging Face, and various cloud platforms. These resources are designed to support developers in building and scaling AI applications effectively.

Many models are available as NVIDIA NIM microservices, optimized for security and scalability across multiple infrastructures ranging from edge devices to cloud solutions.

For more information on NVIDIA’s offerings, interested parties can tune in to NVIDIA Live at CES.