More on the topic...
Generating detailed summary...
Failed to generate summary. Please try again.
Elastic built an agent memory system on Elasticsearch with three dedicated indices—episodic, semantic and procedural—plus a shared catalog index. Episodic entries capture every user turn with timestamps before the AI responds. Semantic entries store stable facts about the user (“Sarah owns a Lumio Hub v2,” “Hub was reset in March”). Procedural entries hold multi-step playbooks (“troubleshoot Zigbee disconnects”) tagged with success and failure counts that update whenever the user confirms an outcome. Separating these types lets each index follow its own write rate, decay rules and update logic without bogging down the others.
To fetch relevant memory, they run a two-stage hybrid retrieval: BM25 inverted-index search and Jina v5 dense embeddings fused via Reciprocal Rank Fusion (RRF), pulling 80 candidates from each leg with a rank_constant of 30. Those 160 results go to a Jina v2 cross-encoder reranker, which scores query–document pairs jointly. That mix catches literal matches—version numbers, error codes—and semantic paraphrases, avoiding the “lost in the middle” problem of large context windows. A single recall_memory call fans across all three indices plus the shared catalog; the retriever’s scoring and per-index decay automatically surface the right hits.
Writes happen automatically: every user message becomes one episodic document (ID, raw text, timestamp). Elasticsearch assigns IDs and enforces per-user isolation via document-level security so no cross-tenant leaks occur. Consolidation into semantic or procedural memory runs separately: LLMs extract stable facts or refine playbooks based on success_count and failure_count. Supersession, not deletion, handles contradictions—older facts stay in the audit trail while fresh ones take priority under a time-decay ranking function.
On a QA-style test of 168 queries, recall at rank 10 hit 0.89 with zero leaks between tenants. All memory types and the catalog sit in one Elasticsearch cluster, avoiding extra round-trips or multiple services. The code’s on GitHub and integrates as an MCP-compatible tool, so any LLM-based agent runtime can tap into the same memory layer without custom integrations.
Questions about this article
No questions yet.