Modal Expands Reach with AWS and GCP Marketplaces, Enhanced User Experience, and Developer Collaboration Tools

Enterprise customers can now purchase and manage Modal through both Amazon Web Services (AWS) and Google Cloud Platform (GCP) Marketplaces. This new integration allows businesses to leverage their existing cloud spend commitments for Modal usage, streamlining the procurement process for companies that already use these cloud platforms.

This move represents a significant step for Modal, enhancing access for organizations that wish to implement Modal's capabilities within their cloud infrastructure. The integration is likely to appeal to a broader audience in the enterprise sector, making it easier for teams to scale their development efforts without the friction typically associated with transitioning between platforms.

Enhanced Sandbox Observability

Modal has recently updated its Sandbox feature, improving the user interface to provide clearer observability. Key enhancements include detailed resource information and a newly introduced execution timeline that outlines each Sandbox's lifecycle from creation to termination. This improvement aims to give users a more intuitive understanding of their Sandboxes and streamline debugging processes.

Such enhancements are crucial for developers working in fast-paced environments where quick iterations are necessary. Having detailed visibility into the lifecycle of their Sandboxes allows teams to monitor performance and address issues proactively, ensuring that resources are utilized effectively.

Client Updates

With the latest client updates, developers can run uv pip install --upgrade modal to access enhancements. The changelog highlights key features, including:

• Support for Python 3.14, alongside experimental support for free-threaded Python (3.14t) within Modal containers, allowing greater flexibility for developers seeking to optimize performance within their applications.
• New commands such as modal token info and timestamps now feature across key Command Line Interface (CLI) commands. This aims to make authentication and logs more straightforward for developers.
• Experimental async usage warnings help identify potentially blocking Modal APIs used in async contexts, assisting developers in preventing performance bottlenecks.

These updates underscore Modal’s commitment to continuous improvement and user-friendly development experiences, with added support for newer Python versions reflecting the evolving needs of the programming community.

Case Study: Ramp's Internal Coding Agent

Ramp's innovative internal coding agent, Inspect, operates within Modal's Sandbox, allowing developers to run sessions with a complete development environment. By utilizing Modal’s capabilities, Inspect writes approximately 30% of Ramp's production pull requests across both frontend and backend, contributing significantly to their development workflow.

This setup minimizes resource strain on developer laptops and enables seamless development processes. The integration of Inspect serves as a real-world demonstration of how Modal can enhance productivity without the traditional overhead associated with such intensive coding processes. For more insight, refer to the full breakdown from Ramp.

High-Performance LLM Inference

Modal is investing heavily in optimizing Large Language Model (LLM) inference capabilities. A recent deep dive examined three primary types of inference workloads: offline, online, and semi-online, analyzing how performance and cost trade-offs differ across these categories. Recommendations for utilizing engines such as vLLM and SGLang were also shared based on empirical evidence from teams operating at scale.

Architecting each workload efficiently on Modal can lead to decreased operational costs and enhanced model performance, critical as organizations increasingly adopt LLMs in their applications. For further depth on this topic, users are encouraged to read the full post or join our virtual event for direct insights from the Modal team.

Managing a Global GPU Fleet

Modal operates a globally distributed GPU fleet that manages over 20,000 concurrent GPUs, launching millions of instances monthly. The management of such a vast fleet requires rigorous health-check protocols, instance type testing, and monitoring for reliability. Modal shares valuable lessons from its operations, particularly concerning the challenges that arise when managing GPUs at scale and strategies used to detect and mitigate potential failure modes.

For a detailed look at these operations and insights, Modal invites readers to explore the full post, which outlines their best practices for maintaining GPU health and operational excellence.

Upcoming Events in February

This February, Modal is hosting numerous events in New York City and San Francisco, focusing on topics spanning reinforcement learning, AI development, and biotechnology. Here are a few key highlights:

• Cafe Compute: Feb 12 | New York
• FDEs in AI with Modal and Snowflake: Feb 18 | San Francisco
• Modal for Startups office hours: Feb 23 | San Francisco

For a complete list of events and activities, visit this page. Modal continues to engage with the developer community to foster collaboration and innovation within the tech sphere.