Elasticsearch agent memory layer hits R@10 0.89 with zero tenant leaks
Elastic's Noam Schwartz details a persistent, multi-tenant agent memory layer built on Elasticsearch using three indices, hybrid retrieval with RRF and a cross-encoder reranker, supersession, decay, and per-user DLS isolation — achieving R@10 0.89 across 168 questions with zero cross-tenant leaks.
Score breakdown
The architecture consolidates vector storage, keyword search, audit history, and per-user access control into a single Elasticsearch deployment, replacing the fragile multi-service approach and the context-stuffing workaround that degrades with scale.
- 01R@10 recall averages 0.89 across 168 QA-style evaluation questions
- 02Zero cross-tenant leaks were recorded in the evaluation
- 03Three Elasticsearch indices map to episodic, semantic, and procedural memory categories from cognitive science
Noam Schwartz at Elastic argues that a 1M-token context window is a scratchpad, not a memory system, and that stuffing prior session context into the context window breaks down on cost, latency, and the "lost in the middle" effect — where models ignore facts placed far from the prompt's edges. The proposed alternative is a persistent long-term memory layer built entirely on Elasticsearch, structured around three indices corresponding to the episodic, semantic, and procedural memory categories from cognitive science (surfaced for LLM agents in the COALA framing).
Recall is handled by a single hybrid query combining RRF with a cross-encoder reranker.
Episodic memory stores time-stamped user-turn events before any extraction or interpretation; semantic memory holds distilled, stable assertions about the user that survive across sessions; and procedural memory stores multi-step playbooks carrying `success_count` and `failure_count` counters incremented by a consolidation LLM when a user confirms whether a fix worked. Recall is handled by a single hybrid query combining RRF with a cross-encoder reranker. When a user contradicts a stored fact, the old version is superseded rather than deleted, preserving an audit trail. Per-user DLS isolation ensures each user's memory remains invisible to every other user, and the architecture is designed to be reachable by any MCP-speaking client rather than tied to a single agent runtime.
On a QA-style evaluation over 168 questions, the system achieves R@10 0.89 with zero cross-tenant leaks. The full open-source implementation is available on GitHub, and Elastic's Agent Builder — referenced in the post — is now generally available.
Key facts
- 01R@10 recall averages 0.89 across 168 QA-style evaluation questions
- 02Zero cross-tenant leaks were recorded in the evaluation
- 03Three Elasticsearch indices map to episodic, semantic, and procedural memory categories from cognitive science
- 04Hybrid retrieval uses RRF combined with a cross-encoder reranker
- 05Supersession preserves old fact versions rather than deleting them, maintaining an audit trail
- 06Procedural memory playbooks carry `success_count` and `failure_count` counters updated by a consolidation LLM
- 07The architecture is designed to be reachable by any MCP-speaking client, not tied to one agent runtime
Topics
Summary and scoring are generated automatically from the original article. We always link back to the publisher and never republish images or paywalled content. Last processed Jun 19, 2026 · 10:25 UTC. How this works →