Nvidia's Nemotron 3.5 Super VL: A New Benchmark for Enterprise AI Agents

Nvidia has launched the Nemotron 3.5 Super VL, a multimodal AI model boasting 120 billion parameters, designed specifically for enterprise applications. This new model aims to facilitate the continuous operation of specialized AI agents, capable of handling diverse tasks including document analysis and image processing.

Nemotron 3.5 Super VL utilizes a Mixture-of-Experts architecture, whereby only a subset of 12 billion parameters is active for each token processed. This innovative design, intended to enhance efficiency, was developed in collaboration with the Nemotron Coalition, which encompasses organizations like Mistral AI, Thinking Machines Lab, Perplexity, and Black Forest Labs. The introduction of the Nemotron 3.5 marks a significant step in Nvidia's strategy to enhance AI capabilities in enterprise environments.

Alongside the Super VL model, two smaller versions dubbed "Bolt" are being released. The Bolt 74B-A9B 24GB model caters to local processing requirements, while the subsequent Bolt E4B 4GB version is aimed at voice assistants. According to Nvidia, both models provide what it describes as the "highest intelligence per gigabyte of memory." The 24GB version is particularly tailored for environments where local machines are used, expanding the applicability of AI to edge devices.

Maximizing Efficiency: Accuracy at a Reduced Parameter Count

The Super VL model was derived from the Nemotron 3 Ultra, which is Nvidia's flagship reasoning model. Remarkably, Super VL achieves similar accuracy on multi-step tasks with only around 20% of the parameters compared to Ultra. In assessments conducted using testing frameworks like Hermes and LangChain, Super VL reportedly outperforms Nemotron 3 Ultra in various agent tests. Additionally, in coding benchmarks such as SWE-bench and Terminal-Bench, it surpasses its predecessor, the Nemotron 3 Super.

Operating on a single GPU, Super VL reportedly achieves up to five times the throughput of comparable open models concerning agent tasks, as detailed on the Artificial Analysis leaderboard. In independent tests conducted by EXO Labs using the DGX Spark, Super VL emerged as the model demonstrating both superior accuracy and faster performance. This efficiency grants performance benefits for applications requiring sustained operations toward large volumes of tasks over extended periods.

Nvidia is marketing Super VL to enterprises seeking to customize the model with proprietary data and deploy it on in-house servers. This flexibility may appeal to industries such as insurance, where an agent could consistently analyze claims for potential fraud, or finance firms evaluating credit suitability based on their criteria. Nvidia supports this customization with tools like its NeMo framework and provides open datasets for fine-tuning.

Enhanced Compression Techniques for Improved Performance

Nvidia has employed its 4-bit NVFP4 format for the compression of Super VL, which allows the model to function entirely in 4-bit precision on Blackwell GPUs. For older graphics cards, the weights are maintained at 4 bits while processing computations at 16 bits, ensuring higher precision is retained for critical model components. To mitigate any potential quality loss from compression, Nvidia engaged in extensive training to align the outputs of the compressed model with those of the full model.

Moreover, the model includes a draft generation capability that allows for precursory token proposals. The main model then verifies these suggestions in a single computational pass, streamlining the generation process. When used on DGX Spark, Nvidia promotes its LiLiCorr methodology, which evaluates multiple draft tokens simultaneously to enrich output coherence, achieving speed increases of up to 2.3 times relative to conventional processes.

Processing Visual Data: Tables, Charts, and More

Nemotron 3.5 Super VL has been engineered to efficiently handle the various documentation forms commonly encountered in business: it can read and interpret OCR data, analyze tables and charts, and summarize detailed documents without relying on a separate vision model. This multimodal capability distinguishes it in a crowded AI market.

Nvidia asserts that Super VL achieves an 80.8% score on the CharXiv benchmark for chart interpretation, outperforming its predecessor, Nemotron 3 Nano Omni, which earned 63%. Further analysis shows an even higher score of 84% when the model further leverages Python code execution capabilities. Furthermore, Super VL ranks highly in the OCRBench v2 leaderboard among comparable open models.

Hardware Demands and Accessibility

The Bolt 74B-A9B 24GB version of Super VL operates as a hybrid Mixture-of-Experts model, activating only 9 of its 74 billion parameters per processing cycle. To fit within a 24 GB memory constraint, Nvidia has implemented strong 2-bit quantization strategies for weights.

This model is capable of performing local tasks that necessitate multi-step operations, tool access, or web searches. Nvidia's descriptions include applications for code writing, debugging, and optimizing PC component selections. Bolt 74B-A9B is particularly optimized for performance on Nvidia's RTX Spark chip, a new model tailored for AI functionalities, as well as compatible with various high-end Nvidia platforms.

However, the stringent memory requirement means 24 GB remains a considerable hurdle, typically confined to high-end systems. Many consumers may find it challenging to invest in such hardware, indicating the primary audience for this model consists of developers or businesses with access to advanced workstations.

Following up on the Bolt line, the anticipated Bolt E4B 4GB model is expected to cater to less powerful devices; it is designed for integration with GeForce RTX cards, Jetson modules, and other Nvidia hardware. This version targets voice-activated assistants and user interfaces that maintain natural interactions while managing multi-turn tasks efficiently.

Industry Adoption and Use Cases

Nvidia has observed that numerous organizations are actively customizing Super VL to meet specific requirements, particularly in sectors laden with heavy documentation and sensitive data. Companies like Scale AI are adapting the model for processing vendor credits, projecting substantial cost reductions and improved response times. Ramp employs the model to convert receipts into structured data and anticipates a tenfold increase in response speeds, while EY applies it for acquisition assessments.

A second wave of early adopters includes cybersecurity firms; for example, AI21 has tailored Super VL for Alice, a security company that focuses on analyzing risks and vulnerabilities, achieving notable improvements in accuracy. Other organizations, such as Dream and CrowdStrike, are utilizing Super VL for threat detection and defense implementations, respectively.

Open Access and Sustainability of Model Development

Nvidia is committed to transparency, making available the model weights in different formats: base version, a post-trained variation, and a 4-bit compressed edition. These releases are accompanied by comprehensive datasets and training recipes, featuring a focus on benchmarks for agent interactions, instruction adherence, long-text processing, and visual comprehension.

In a significant effort towards inclusivity, Nvidia collaborated with the Indian Institute of Technology Madras to create datasets that represent multiple languages across India, amounting to roughly 2.83 trillion tokens in 22 distinct languages. This initiative highlights a dedication to broadening the linguistic capabilities of AI models within multicultural contexts.

Nvidia's strategy involves utilizing its open models not just to engage practitioners in open AI development but to stimulate demand for its own hardware. Super VL is engineered to run extensively across various systems, from desktop configurations to cloud environments, and enterprises can deploy it as a microservice via Nvidia NIM, ensuring updated and validated configurations. As a result, while the models facilitate access, they simultaneously drive the need for enhanced computing infrastructure offered by Nvidia.

This positions Nvidia in direct competition with organizations like Meta, Google, Anthropic, and OpenAI, which are developing proprietary architectures optimized for their own chipsets. Additionally, Nvidia's recent financial investment in Hugging Face (approximately $12.9 billion) reinforces its commitment to fostering an ecosystem conducive to the propagation of open AI models, an approach that may well dictate market dynamics in the coming years.