Updates on Modal: Accelerating GPU Workloads, Launching Notebooks, and Enhancing User Automation

📸 GPU Memory Snapshots (Alpha)

Modal has integrated GPU memory snapshots, which significantly enhance cold boot times for GPU workloads, achieving speeds up to 10 times faster. By leveraging these snapshots, users can bypass lengthy tasks such as Torch compilation, traditionally a bottleneck in model training. To activate this feature, existing users need only to include the enable_gpu_snapshot experimental option in their configurations.

For comprehensive guidance, visit our blog post.

📓 Introducing Modal Notebooks (Beta)

Modal's new Notebooks allow users to write and execute Python code directly in the cloud through a web interface. Built on the serverless Modal platform, users are charged only for the compute resources they actually utilize. This environment supports attaching custom images, integrating distributed storage, and enables real-time collaboration. Users can also share their work seamlessly on the web, enhancing collaborative programming.

To start using Notebooks, navigate to modal.com/notebooks or refer to our documentation.

🤖 Service Tokens

Workspace managers can now facilitate automation in deploying Modal applications through the introduction of Service Tokens, which users can generate for Service Users. This functionality allows for deployment automation without relying on individual member API tokens, enhancing security and control by decoupling deployments from personal credentials.

For detailed implementation, see our documentation here.

🏃 Run Modal Functions from the Browser (Beta)

Modal now simplifies the process of testing and debugging by generating a user interface for your Functions based on type annotations. This allows developers to easily trigger workflows and inspect outputs directly from the browser, improving the development cycle. To utilize this feature, ensure that your application is deployed using the modal client version 1.1 or later, and make sure all function arguments are properly annotated with types.

🧩 First-Class Support for uv

The integration of uv into the Modal ecosystem aims to enhance container build speeds. Users can employ uv_sync to synchronize their Modal image locally, while uv_pip_install allows for rapid package installations. This update is particularly advantageous for developers looking to maintain agile development workflows.

For more updates on this feature, refer to our changelog.

👩‍💻 Client Updates

Update your Modal client by running pip install --upgrade modal. Here are some notable enhancements from the recent changelog:

• Named Sandboxes now offer unique ownership over resources, promoting better resource management (1.1.1).
• New properties like .name and a method .info() for modal.Dict, modal.Queue, modal.Volume, and modal.Secret provide improved management of modal resources (1.1.1).
• Support for the 2025.06 Image Builder Version aims to minimize conflicts with user dependencies and enhance compatibility with existing Dockerfiles.

🏎️ Run OpenAI’s gpt-oss Model with vLLM

The newly available gpt-oss example allows users to deploy OpenAI’s open-weight reasoning model efficiently on the Modal platform using vLLM. This capability streamlines the deployment process to mere minutes, enabling quicker iterations in development. Access the functionalities by checking here.

🔉 Transcribe a Week of Audio in a Minute for a Dollar

The advancements in open speech-to-text models make them a cost-effective alternative to proprietary APIs. Users can now deploy an audio transcription service that is not only more than 100 times faster but also costs significantly less. For insights on setting this up, read our blog post here.

❤️ Customer Spotlights: Lovable, Marimo, Pipecat

• Lovable utilizes Modal Sandboxes at a large scale to run LLM-generated code, peaking over a million Sandboxes during a promotional event. Discover more in their case study here.
• Marimo, an open-source Python notebook platform, recently launched their cloud-hosted workspace utilizing Modal Sandboxes.
• Pipecat, a framework for real-time voice AI, implemented their new turn detection model on Modal.

📅 Recent Events

• A recent Voice AI event hosted over 100 developers, featuring insights from researchers at NVIDIA and Mistral AI, focusing on cutting-edge speech models. If you missed it, the livestream is available here.