Results on GVD and Efficiency

backbonemethodAcc.Corr.Cohe.
GPT-4o-miniTiM84.578.890.8
MemoryBank78.473.391.2
MemGPT87.983.289.6
A-Mem90.486.591.4
MemoryOS93.391.292.3
Qwen2.5-7BTiM82.273.285.5
MemoryBank76.370.382.7
MemGPT85.180.286.9
A-Mem87.279.587.8
MemoryOS91.882.390.5
GVD · retrieval accuracy / correctness / coherence, scored by DeepSeek-R1 · MemoryOS over A-Mem with GPT-4o-mini: +3.2% / +5.4% / +1.0%
0 10 20 30 40 500 1K 2K 5K 10K 20K recalled tokens per response (log scale) average F1 (LoCoMo) MemoryBank 432 tok · 3.0 calls · F1 6.84 TiM 1,274 tok · 2.6 calls · F1 18.01 A-Mem* 2,712 tok · 13.0 calls · F1 26.55 MemoryOS 3,874 tok · 4.9 calls · F1 36.23 MemGPT 16,977 tok · 4.3 calls · F1 29.13 circle area ∝ LLM calls per response fewer calls than A-Mem* (4.9 vs 13.0) · far fewer tokens than MemGPT (3,874 vs 16,977)
23 / 25