GAAP execution environment guarantees AI agent data privacy
Researchers introduce GAAP (Guaranteed Accounting for Agent Privacy), an AI agent execution environment that deterministically enforces user-defined privacy permissions to prevent private data exfiltration — even against prompt injection attacks.
Score breakdown
Developers building agentic systems that handle sensitive user data can look to GAAP's Information Flow Control approach as a model for enforcing privacy guarantees without relying on the trustworthiness of the underlying AI model or its provider.
- 01GAAP stands for Guaranteed Accounting for Agent Privacy, an execution environment for AI agents.
- 02It enforces user-defined permission specifications governing how private data may be shared, including with the AI model provider.
- 03GAAP provides privacy guarantees deterministically, without trusting the agent or requiring the model or prompts to be attack-free.
Robert Stanley, Avi Verma, and Lillian Tsai introduce GAAP (Guaranteed Accounting for Agent Privacy), a purpose-built execution environment for AI agents that provides deterministic confidentiality guarantees for private user data. The core threat model addresses two distinct risks: adversarial attacks such as prompt injection that attempt to exfiltrate user data through the agent, and the inherent trust problem of sharing sensitive personal or financial information with a potentially unscrupulous or compromised AI model provider. GAAP addresses both by enforcing user-defined permission specifications without ever requiring the agent or the underlying AI model to be trustworthy or attack-free.
The system works by collecting permission specifications from users through dynamic, directed prompts, then tracking how the agent accesses and uses private data throughout execution.
The system works by collecting permission specifications from users through dynamic, directed prompts, then tracking how the agent accesses and uses private data throughout execution. To accomplish this, GAAP extends Information Flow Control with novel persistent data stores and annotations capable of tracing private information flows across individual execution steps within a single task, as well as across multiple tasks separated in time. This cross-task tracking is a notable technical contribution, as most prior approaches focus only on single-session data flows.
The authors' evaluation confirms that GAAP successfully blocks all data disclosure attacks tested, including attacks that cause other state-of-the-art systems to leak private user data to untrusted parties, while imposing no significant penalty on agent utility. The result is a privacy enforcement layer that operates deterministically — a meaningful departure from probabilistic or model-level defenses that depend on the AI model's own behavior to protect user data.
Key facts
- 01GAAP stands for Guaranteed Accounting for Agent Privacy, an execution environment for AI agents.
- 02It enforces user-defined permission specifications governing how private data may be shared, including with the AI model provider.
- 03GAAP provides privacy guarantees deterministically, without trusting the agent or requiring the model or prompts to be attack-free.
- 04The system extends Information Flow Control with novel persistent data stores and annotations.
- 05GAAP tracks private data flows both within a single task and across multiple tasks separated in time.