Results on the LoCoMo Benchmark
0
10
20
30
40
50
F1
26.65
27.02
35.27
single-hop
+32.35% vs best baseline
33.23
45.85
41.15
multi-hop
+23.83% vs best baseline
9.15
12.14
20.02
temporal
+118.80% vs best baseline
41.04
44.65
48.62
open-domain
+18.47% vs best baseline
TiM
MemoryBank
MemGPT
A-Mem* (re-implemented)
A-Mem (as reported)
MemoryOS
LoCoMo · GPT-4o-mini · F1 by question type
average over the four types vs the best baseline: +49.11% F1 · +46.18% BLEU-1 · average rank 1.0 on both metrics
* on multi-hop, the A-Mem value its authors report (45.85) exceeds MemoryOS (41.15); the re-implementation under the same environment gives 33.23, which is the basis of the +23.83%
22 / 25