Legal and compliance work is rarely a matter of retrieving one rule and applying it mechanically. Interpretation depends on precedent, internal risk appetite, regulator expectations, and how similar cases were resolved. A useful LLM system needs more than a document index; it needs institutional memory.
Memory should not mean an uncontrolled store of prior conversations. It should mean curated, reviewable experience: examples of previous interpretations, escalation outcomes, exception handling, and policy changes. Each memory item should have provenance, scope, and expiry conditions.
The design challenge is governance. A compliance assistant that remembers too little repeats the same analysis. One that remembers carelessly imports stale or unauthorized reasoning. The right architecture treats memory as a controlled knowledge asset, not as a hidden prompt extension.