Why we're looking at this
Long-running agents need memory that persists across sessions — a support agent should remember a customer's history, a clinical agent should remember a patient's prior visits. Naively dumping everything into a vector store and retrieving top-k results is the default, and it's usually wrong for production use cases.
What we're seeing
- Semantic retrieval alone loses ordering and recency signals that matter a lot in practice — the most relevant memory is often the most recent one, not the most semantically similar one.
- Structured memory (explicit fields the agent updates, not just free-text embeddings) is more reliable for anything with compliance requirements. In Physiolaxy, patient history is stored as structured clinical fields precisely so every AI recommendation stays auditable back to a specific data point.
- Retrieval quality degrades faster than teams expect as a knowledge base grows past a few thousand documents, without a deliberate re-ranking or chunking strategy.
Open questions we're still chasing
We're still evaluating where long-context models start to make retrieval-augmented memory unnecessary versus where retrieval remains cheaper and more controllable — see our related notes on RAG versus long-context tradeoffs.