CoreWeave and NVIDIA Team Up to Enhance AI Production Capabilities with New Infrastructure

Over the last decade, CoreWeave has been co-engineering advanced compute, networking, and software solutions with NVIDIA, yielding significant return on investment through multiple deployment generations. Recently, CoreWeave is advancing to the next stage with the introduction of cutting-edge NVIDIA infrastructure tailored for AI production.

Introduction of NVIDIA Vera Rubin NVL72 Systems

At the CoreWeave Fully Connected event in San Francisco, the company announced the availability of the NVIDIA Vera Rubin NVL72 systems, integrated with Spectrum-X 102.4T Ethernet networking capabilities. Cognition, an applied AI lab that developed the Devin AI software engineer, is the first client to employ Vera Rubin for production workloads.

In addition to Vera Rubin, CoreWeave has also unveiled NVIDIA Vera, specifically designed as the first CPU optimized for AI agents. The company rolled out CoreWeave Forge, a comprehensive platform for training, evaluating, and enhancing AI models and agents on NVIDIA accelerated computing.

Significant Performance Gains with Vera Rubin

Cognition’s implementation of training, reinforcement learning, and production inference for Devin has resulted in a rapid scaling to thousands of GPUs within nine months. Early performance benchmarks highlight that the Vera Rubin NVL72 outperforms the GB200 NVL72 baseline dramatically. Specifically, Cognition reported up to a 4.8x increase in token throughput during real-world software engineering tasks—improvements that enhance real-time code generation and multistep reasoning capabilities.

Silas Alberti from Cognition noted that agentic coding presents unique challenges, including complex workloads and high token volumes. The ability to integrate their operations on a single platform, complemented by NVIDIA’s and CoreWeave’s collaborative efforts, has proven instrumental in managing these challenges.

CoreWeave Cloud Offers NVIDIA Vera Rubin

CoreWeave made headlines by being one of the first cloud providers to deliver the NVIDIA Vera Rubin NVL72 directly to its customers. Early-access clients can quickly deploy NVIDIA’s full-stack AI factory capabilities on CoreWeave Cloud, with production clusters operational in mere days. This swift setup is attributed to the close collaboration between CoreWeave and NVIDIA teams, covering the entire technical spectrum from infrastructure to API access.

Customers can seamlessly manage capacity through services like CoreWeave Kubernetes Service and CoreWeave Mission Control, facilitating easier orchestration of resources.

NVIDIA Vera CPU: Enhancing Agentic AI Workloads

Agentic AI workloads demand both low-latency compute and extensive scalability, requiring thousands of isolated environments to run concurrently. The NVIDIA Vera CPU has been tailored specifically to cater to this need, providing impressive capabilities for simultaneous agent operations.

In a single rack deployment, CoreWeave has integrated 128 CPUs and 11,264 cores, enabling more than 11,000 concurrent environments for agent execution. Testing reveals that with the NVIDIA Vera CPU, CoreWeave experienced over 3x faster startup times for agent sandboxes, thus streamlining operational efficiency. Additionally, on Terminal-Bench, performance gains of 1.7x were recorded across various tasks.

Introducing CoreWeave Forge for Continuous AI Improvement

Historically, the feedback loop between model training and production has been fragmented across different vendor tools, leading to inefficiencies. CoreWeave Forge aims to bridge this gap by integrating Weights & Biases and other post-training tools into a connected environment designed for ongoing model enhancement.

The Forge platform offers several key features that enhance AI model development:

• CoreWeave ARIA: Now generally available, ARIA assists users in analyzing training runs, suggesting experiments, and managing code iterations, effectively feeding back production insights into the training loop.
• CoreWeave Agent Lens: This new service converts production performance data into actionable insights, enhancing failure detection by 20% while lowering costs to correct issues.
• CoreWeave Sandboxes: These allow for isolated CPU or GPU environments to run agents and evaluations, supporting various execution scenarios on existing infrastructure.
• Post-training enhancements: It leverages production signals to refine model quality while reducing latency and costs. Serverless options for supervised fine-tuning and reinforcement learning have also proven 1.4x faster at 40% lower costs than traditional setups.

Powered by NVIDIA Dynamo, an open-source inference framework, CoreWeave's managed inference service has also seen advancements. This includes capabilities for real-time updates of AI models without service disruptions, thus blending production and post-training tasks more effectively.

Case Studies and Proven Impact

The collaboration between CoreWeave and NVIDIA has attracted various sectors, from startups to large-scale enterprises. For instance, Ennoble Care, a home health service provider, chose CoreWeave for their clinical AI inference needs. Utilizing NVIDIA RTX PRO 6000 GPUs on CoreWeave Kubernetes Service, they aim to scale AI capabilities for documentation and decision support effectively.

CoreWeave's performance has also been validated through exceptional MLPerf results, maintaining a Platinum ranking across multiple rounds of measurement while serving the needs of nine of the top ten AI labs.

By offering a powerful platform that facilitates the transition from experimental concepts to real-world AI applications, CoreWeave and NVIDIA empower organizations to realize the full potential of agentic technology across various domains.