Modal's Infrastructure: Boosting AI Research Through Enhanced Workflows and Collaboration

Between my PhD at Berkeley and my current role at Modal, I witnessed the rise of Weights & Biases as a key tool for experiment management in AI research. Widely adopted by teams working on transformative models like GPT-3 and GPT-4, this platform significantly streamlined the development process. It gave researchers standardized access to high-quality infrastructure, helping them manage their experimental data effectively. As I engage with the AI community today, seeing collaborations flourish through shared tools remains rewarding and inspiring.

At Modal, we are building on that foundation by offering a robust infrastructure designed to enhance the execution of AI experiments. Our platform’s speed, scale, and reliability ensure researchers can arrive at better answers more efficiently. The insights generated by our users feed back into our systems, creating a beneficial cycle that enhances AI research.

Despite being in relatively early stages, we've started to foster impactful collaborations with academic researchers from institutions such as Hazy Research, Scaling Intelligence, and IRIS. This blog post showcases their significant findings and illustrates how Modal exemplifies the acceleration of AI research.

Modal: Accelerating the Landscape of AI Research

Designed as a cloud infrastructure platform, Modal is tailored to foster rapid computational and data-intensive workloads, particularly those associated with artificial intelligence. The requirements of contemporary AI research demand substantial compute power: large GPUs and often multiple units operating in parallel. Traditional workstations can handle initial testing but quickly fall short when experiments require large-scale execution. High-Performance Computing (HPC) clusters present a solution with higher throughput yet are often hampered by downtime and queuing, especially near critical deadlines. In contrast, Modal maintains a high-quality GPU fleet that users can access by the second without the need for reservations.

Portability remains a challenge when transitioning code from workstations and clusters. Modal's infrastructure acts as an expansive, global computing platform, which eliminates the common issue of “it works on my machine” by facilitating seamless execution across varied environments.

Moreover, AI workloads necessitate robust isolation to prevent one malfunctioning process from adversely affecting the entire system. Many cloud providers have implemented sandbox environments, but they often lack the necessary resources or configurations. Modal, however, ensures strong isolation by default, maintaining performance integrity even with multiple concurrent workloads. This is particularly vital for experimental conditions where performance consistency is critical.

Leading AI Researchers Choose Modal for Infrastructure

Modal has attracted leading AI researchers who are engaged in pioneering projects that focus on self-improvement, particularly leveraging AI systems that enhance their performance. Two notable research directions include:

• Employing AI systems running on GPUs to optimize code-writing capabilities for GPUs themselves.
• Utilizing AI to autonomously produce new AI systems.

Within these domains, three standout projects leveraging Modal have emerged:

• KernelBench, led by Anne Ouyang, Simon Guo, and Will Hu from the Scaling Intelligence Lab.
• TTT-Discover, spearheaded by Mert Yuksekgonul from Stanford’s AI and AI for Science Labs.
• RL-4-MLE, directed by Anikait Singh of the IRIS Lab.

KernelBench: Setting the Standard for Automated Kernel Generation

KernelBench aims to address the automation of GPU kernel generation, posing a significant challenge due to the performance-sensitive nature of these elements within AI workloads. With increased focus on AI acceleration techniques, the KernelBench framework offers a benchmark dataset of 250 problems alongside a research environment conducive to generating performant kernels. This project notably drives efficiency, reducing redundant efforts within the field.

To effectively benchmark GPU performance, it must encompass both correctness and speed—although performance modeling remains constrained. Relying exclusively on human intervention to optimize kernels has proven inefficient. The KernelBench team chose Modal for its infrastructure, lauding its scalability and consistency in evaluation across various GPU types. Simon Guo, a researcher from the Scaling Intelligence Lab, remarked, “Benchmarking GPU kernel performance is quite tricky: results can vary across hardware configurations and software environments.”

TTT-Discover: Enhancing GPU Kernel Generation

Building on the KernelBench framework, TTT-Discover focuses on creating a system capable of generating GPU kernels that outperform human engineers. Their innovative approach leverages “test-time training,” allowing models to refine their weights based on real-time feedback. This project notably won first place in a competitive environment, showcasing its capability to establish a new state-of-the-art in triangular matrix multiplication—a critical operation in applications such as AlphaFold, a prominent molecular structure prediction model.

Mert Yuksekgonul emphasized Modal’s crucial role in providing a reliable and scalable infrastructure, stating that the platform's unique qualities allowed the team to conduct thorough experiments efficiently. Traditional attempts at timing kernels locally resulted in inconsistent performance due to differing hardware limitations, but Modal offered a scalable solution that provided consistent benchmarks essential for accurate evaluation.

RL-4-MLE: Automating the Machine Learning Lifecycle

The RL-4-MLE project aspires to deliver a comprehensive approach to automation within the workflow of machine learning research. This initiative seeks to create an automated research assistant that continuously learns and adapts, moving beyond conventional models that focus solely on finding optimal solutions. Alongside contributions from teams like IRIS and Scaling Intelligence, this project aims to understand and automate algorithm discovery across complex workflows.

Evaluating these systems presents unique challenges, pressing the limits of existing infrastructure. Anikait Singh emphasized the necessity of reliable, scalable environments for validating strategies effectively, explaining that local infrastructures lacked the needed capacity for parallel rollout during experiments. Modal's robust, isolated workspace addressed these issues, dramatically reducing study durations from weeks to mere hours.

Long-horizon learning in complex machine learning paradigms is a demanding task, requiring both extensive resources and sophisticated feedback systems. Modal has emerged as an invaluable partner, enabling researchers to develop and refine algorithms within a dependable setting, ultimately supporting groundbreaking advancements in AI technology.

Accelerate Your AI, Your Research, or Your AI Research with Modal

As artificial intelligence continues to evolve, the potential for these systems to enhance workloads and simplify processes becomes increasingly likely. This not only improves productivity but also promises to yield significant advancements in technological research and development. The cycle of AI research — where progress accelerates further advancements — opens exciting possibilities for the future.

Regardless of where you are on this journey, Modal stands ready to provide a scalable and reliable infrastructure that can help you move faster and think bigger. For those looking to leverage these advantages, I invite you to explore what Modal can do for you and your research initiatives.

Get started with Modal today!