Building a High-Performance Background Coding Agent with Modal at Ramp

Introduction: The Background Coding Agent Initiative

Ramp has developed an internal coding agent named Inspect, leveraging Modal to revolutionize its software development process. This agent has rapidly become responsible for over half of all merged pull requests within the company. By utilizing Modal Sandboxes, Ramp can provision full development environments in seconds, granting every team member—be it engineers, product managers, or designers—the capability to ship code seamlessly. This initiative effectively provides Ramp with extensive parallel engineering capacity, firmly rooted in its internal codebase and enterprise context.

The Vision: Fast Accessibility Across Teams

The AI team at Ramp recognized an opportunity to empower its entire workforce with coding capabilities, transcending traditional local agent limitations. Local setups typically require extensive configuration and integration with internal tools, making them less accessible to non-engineering staff. Therefore, a key objective was to create a background agent that matches local performance in speed and capability to drive widespread adoption. This meant the coding agent must integrate deeply with Ramp's existing technology stack, such as Sentry, Datadog, and LaunchDarkly, enabling end-to-end verification of code.

The Implementation: Ramp's Use of Inspect within Modal

Ramp's choice to utilize Modal for building Inspect stemmed from prior success with Modal in areas like fine-tuning and batch inference. Initial prototypes proved the concept quickly, demonstrating tangible value. Each session of Inspect operates within a Modal Sandbox, which provides a comprehensive development environment that includes essential tools like Postgres, Redis, and RabbitMQ. The agent, termed OpenCode, works in conjunction with a VS Code server for manual edits and a VNC stack to visually verify frontend changes.

In practice, this means that Inspect can execute actions akin to a human developer, including taking before-and-after screenshots and navigating web apps in a real browser to verify functionality. This setup eliminates the latency typically associated with remote operations, as everything runs within a single sandbox, ensuring low-latency access to services.

Architectural Features: Modal's Distributed Infrastructure

Inspect’s design leverages Modal’s distributed infrastructure, with several components working in harmony:

• Functions: These run on a cron job every 30 minutes, ensuring that Ramp’s repositories are cloned with the latest dependencies and current filesystem snapshots, resulting in minimal setup times.
• Dicts: They manage session locks and store metadata, allowing for seamless coordination across sessions.
• Queues: These routes prompts from various clients like Slack, web interfaces, and browser extensions into the right session, ensuring that multiple inputs can be processed without conflict.

This architectural framework not only supports rapid session scalability but also allows Inspect to manage hundreds of concurrent sessions with no interference between them. The complexity of infrastructure setup is abstracted away, letting Ramp’s developers focus on improving the agent, user experience, and integrations.

Efficiency: Fast Startup Times with Filesystem Snapshots

The Inspect agent is designed to provide near-instantaneous startup akin to running a local agent. Modal's filesystem snapshots are crucial in achieving this efficiency. Each session benefits from the latest snapshot that is no more than 30 minutes old, providing a quick way to sync with current repositories. This rapidly reduces the time to start working on tasks, enabling teams to hit the ground running.

Scaling with Sandboxes: Empowering Multi-User Collaboration

By encapsulating each session in its own sandbox, Inspect eradicates contention and load issues commonly seen with local setups. Builders can launch multiple instances for different tasks or variations, multiplying productivity without straining individual hardware resources. Inspect transforms the coding experience into one where users effectively have hundreds of concurrent development environments available to them.

Bringing Tools to Users: Integration into Existing Workflows

Inspect also broadens access for team members who prefer to minimize local installations, integrating directly into workflows where users already operate. This includes interfaces for Slack, web platforms with hosted VS Code editors, and a Chrome extension that facilitates UI modifications by non-engineers. All users interact with a unified session state, enabling real-time collaboration on projects.

Because the system requires no local setup, any employee can send a coding prompt that results in a fully functional sandbox environment being generated, akin to what an engineer would work with, thereby democratizing access to coding capabilities across the organization.

The Impact: Increased Productivity Across Teams

Without mandating its use, Ramp has observed Inspect rapidly become integral to their operations. Within months, Inspect initiated around half of all merged pull requests and has become largely self-sustaining, with over 80% of its own code being generated by the agent itself.

More than just improving code efficiency, Inspect's introduction has empowered product managers to directly implement feature changes and enabled designers to iterate on their designs more quickly, fostering a symbiotic relationship between intent and execution. This liberation of engineers allows them to focus their efforts on more complex and impactful projects, enhancing overall productivity.

Looking Ahead: The Future of Background Coding Agents

As AI models improve, background agents like Inspect are set to become even more vital in streamlining engineering processes. The current iteration of Inspect already generates review-ready pull requests, and as investment in AI infrastructure rises, the focus will shift from the ability to generate correct code to how many background agents can be effectively operated in parallel.

This paradigm shift presents a remarkable opportunity for teams to embrace and harness the power of advanced AI, facilitated by Modal's sandbox capabilities, marking a significant step forward in software development efficiency.