AgentMemBench: A Systematic Benchmark for Evaluating Long-Term Memory Management Strategies in Conversational AI Agents

Short summary

AgentMemBench benchmarks five memory management strategies for conversational AI agents across three datasets and 491 annotated question turns. External key-value stores (EKV) dominate on every quality axis, achieving macro Recall@5 of 0.792, while recency windows, summaries, and entity graphs collapse at long horizons. EKV's recall advantage carries a footprint cost of ~5,100 tokens vs ~300 for in-context methods, revealing an explicit accuracy-efficiency trade-off. All code and artifacts are released for reproducibility.

• •External key-value stores dominate all other memory strategies on quality
• •Recency windows, summaries, and entity graphs fail at long-range recall
• •EKV costs ~17x more memory footprint than in-context methods but is the only strategy that scales

Generated with AI, which can make mistakes.