Addressing AI Security as an Engineering Challenge Across the Agent Stack

AI security fundamentally represents an engineering challenge demanding well-defined requirements, enforceable controls, and accountability within organizations. As AI systems grow in capability, the urgency to enhance security engineering practices escalates, while the necessity for broad access to defensive tools and expedited knowledge sharing becomes critical.

Long-Standing Security Principles Adapt to Emerging Technologies

The evolution of technology brought about by the internet and cloud computing has transformed software functionalities, yet it has not altered the foundational aspects of security: establishing identity, controlling access, minimizing exposure, and verifying the efficacy of security measures. AI agents exemplify a new frontier, allowing for capabilities such as reasoning and tool usage, which necessitate the application of these enduring security principles in innovative ways.

Organizations are eager to harness the productivity enhancements brought by AI, yet they confront the challenge of governing and securing these advanced systems effectively. This duality creates a pressing need for safety protocols that can keep pace with rapid technological growth.

Security is Integral to the Entire Agent Stack

The integrity of applications hinges on the interplay of various components, including code, data, identities, services, and the underlying infrastructure. As AI agents extend this framework, security becomes intertwined with how these elements collaborate. Each tier of the AI agent stack — models, harnesses, and runtime environments — possesses specific security responsibilities, necessitating comprehensive controls as data and actions traverse the system.

For example, consider a scenario where an agent is tasked with updating a customer record. If it encounters malicious instructions in an attached document, a rigorous security mechanism should prevent unauthorized data transfers, blocking the action and logging the incident for review by the security team. This incident logging should include details about the tool invoked, the authorization check performed, and the attempt's outcome, thus enabling a thorough investigation.

Integrating Security into Operational Frameworks of AI Agents

A resilient security perimeter must withstand potential missteps by agents. The operational environment must impose strict limits on file access, network destinations, and process interactions, independent of the agent's internal decision-making processes. While guiding instructions play a role in shaping behaviors, the overarching security model needs firm boundaries enforced at every level.

Each AI agent must possess a distinct, traceable identity with credentials tightly aligned to its specific tasks. Organizations must establish clear access policies detailing the type of information agents may interact with, permissible changes to systems, and the actions that necessitate human approval. Additionally, significant actions and alterations in permissions must always involve human oversight.

Moreover, teams must validate the sources and integrity of the tools and resources each agent employs. In the event of a failure, well-documented records should outline tool usage, authorization history, and outcomes, enabling investigators to understand how incidents unfolded. Clear protocols should exist for revoking permissions and containing security breaches, rendering this evidence usable in subsequent analysis.

To accommodate these protective measures, initiatives like NVIDIA OpenShell serve as secure runtimes that maintain policy enforcement beyond an agent's operational scope, creating a sandboxed execution environment that governs access to data, networks, and system resources.

Evidence of Security is Essential Pre-Deployment

Before AI systems go live, engineering teams are required to gather evidence demonstrating that security controls are effective at preventing unauthorized access to sensitive credentials or data. Testing must also cover vulnerabilities involving permissions and monitoring disruption, with evaluations repeated whenever significant modifications are made to models, tools, or workflows.

A designated owner must assess the outcomes of these tests, warranting system readiness for deployment and ensuring that any failures result in corrective measures. Should failures arise during testing or operational use, they must be replicated, scrutinized, and resolved, with each incident documented as a basis for repeatable tests throughout future system updates.

Notable examples include CrowdStrike's SafeMind, designed to fortify defenses through consistent attack simulations, and Palo Alto Networks' Prisma AIRS, focusing on continuous red team assessments as models and applications evolve.

Empowering Defenders with the Right Tools

When addressing security failures, teams need appropriate tools tailored to their distinct tasks, data, and operational landscapes. Both open and closed models fulfill crucial roles within this context. Closed models offer structured services, while open models empower security experts to inspect components, adjust strategies, and maintain oversight of their infrastructure.

This hands-on control during crises enables teams to replicate failures and validate fixes within their environment, while preserving sensitive evidence. Well-built AI systems can facilitate this investigative work, identifying vulnerabilities and confirming the effectiveness of fixes. Tools' contributions can be gauged through reproducible outcomes, verified rectifications, and quicker response times.

Examples like Capital One's VulnHunter for AI-assisted code security and ReversingLabs' Spectra Assure for AI-powered software package analysis to detect malware showcase the benefits of employing AI in security processes.

Enhancing Collective Defense by Sharing Knowledge

Sharing insights into security failures, effective controls, and verification processes enhances protective measures across teams. NVIDIA’s security research and the Open Secure AI Alliance foster this exchange by integrating research, applied tools, and expertise within the broader cybersecurity community.

Ultimately, addressing AI security as an engineering problem necessitates clear boundaries, assigned accountability, and demonstrable evidence of protective measures. By prioritizing open research and collaborative tool sharing, organizations move closer to achieving secure deployments while adapting to evolving capabilities.

Learn more about NVIDIA’s security research and join the Open Secure AI Alliance.