GAAP execution environment blocks AI agent data exfiltration
Researchers Robert Stanley, Avi Verma, and Lillian Tsai introduce GAAP (Guaranteed Accounting for Agent Privacy), an AI agent execution environment that deterministically enforces user-defined privacy permissions to prevent private data exfiltration — even against prompt injection attacks.
Score breakdown
Developers building agentic systems that handle sensitive user data can look to GAAP's Information Flow Control approach as a blueprint for enforcing privacy guarantees without relying on model trustworthiness or prompt sanitization.
- 01GAAP stands for Guaranteed Accounting for Agent Privacy, an execution environment for AI agents.
- 02It enforces user-defined permission specifications governing how private data — including personal and financial information — may be shared.
- 03Protections extend to disclosures made to the AI model and its provider, not just external third parties.
Robert Stanley, Avi Verma, and Lillian Tsai present GAAP (Guaranteed Accounting for Agent Privacy), a system that addresses a fundamental tension in AI personal assistants: these agents need access to sensitive user data to be useful, yet that access creates serious security and privacy risks. Adversaries can exploit prompt injection attacks to exfiltrate data, and users must also trust that the AI model provider itself will not misuse their information. GAAP resolves this by acting as an intermediary execution environment that enforces user-defined permission specifications, governing how private data may be shared — including with the underlying AI model and its provider.
The core technical mechanism extends Information Flow Control with novel persistent data stores and annotations.
The core technical mechanism extends Information Flow Control with novel persistent data stores and annotations. This allows GAAP to track the flow of private information both across multiple execution steps within a single task and across separate tasks that are temporally distant. Permission specifications are gathered through dynamic, directed user prompts rather than static configuration. Crucially, the guarantees are deterministic — they do not rely on the AI model behaving correctly or on prompts being attack-free. The paper's evaluation confirms that GAAP blocks all tested data disclosure attacks, including those that cause other state-of-the-art systems to leak private data to untrusted parties, while maintaining agent utility without significant degradation.
Key facts
- 01GAAP stands for Guaranteed Accounting for Agent Privacy, an execution environment for AI agents.
- 02It enforces user-defined permission specifications governing how private data — including personal and financial information — may be shared.
- 03Protections extend to disclosures made to the AI model and its provider, not just external third parties.
- 04GAAP's privacy guarantees are deterministic and do not require trusting the agent or assuming attack-free prompts.
- 05The system uses Information Flow Control augmented with novel persistent data stores and annotations.
- 06It tracks private data flow both within a single task across execution steps and across multiple tasks separated in time.
- 07Evaluation shows GAAP blocks all tested data disclosure attacks, including those that defeat other state-of-the-art systems, without significant impact on agent utility.