A memory benchmark is most useful when the system cannot simply pass the whole history into the model.
That is why benchmark scale matters. LoCoMo is useful for conversational recall, but the public conversations average around 26k tokens. BEAM is more demanding for memory architecture because it scales coherent histories up to 10M tokens.
But benchmark scores are still directional.
For production systems, the final question is whether memory improves the real task: completion rate, fewer turns, better personalization, fewer repeated mistakes, lower support burden, or better conversion.