Designing Institutional Memory for AI Assistants

Most enterprise AI assistants retrieve documents well, but retrieval isn’t memory. When one assistant serves many people across an organization, it has to retain what teams learn, control who can see it, reconcile conflicting facts, and retire what’s stale.

What makes this different from traditional system design is that the core decisions aren’t deterministic. A large language model decides what’s worth remembering, what kind of knowledge it is, whether a new statement updates or contradicts an existing one, and how important it is. Embeddings decide which memories are semantically related. These components are powerful but probabilistic, so the architecture has to be built around the fact that they will sometimes be wrong.

This session designs an AI-driven institutional memory system from the ground up:

LLM-driven write path: an extraction stage where the model turns raw interactions into explicit operations (add, update, invalidate, or no-op), grounded in the existing memories in the same scope rather than judged in isolation.

Semantic conflict detection: using embeddings to find related memories across teams, and letting the model classify whether they agree, refine, or contradict each other.

Access model: visibility enforced at the storage layer, outside the model, so a hallucination or prompt injection can’t leak one team’s memory to another.

Confidence-aware human review: routing low-confidence judgments, cross-team conflicts, and promotions to shared knowledge through a human approval step instead of trusting a threshold.

Read path: retrieval ranked by semantic relevance, model-assigned importance, and recency decay, with every access logged.

Evaluating the AI itself: why unit tests can’t verify model judgment, and how scenario-based evaluation catches regressions when prompts or models change.

For each component, we’ll cover the trade-offs, including which decisions to delegate to the model and which must stay deterministic. Examples come from Noscium, an open-source framework I’m building. It’s early-stage, so the focus is the architecture and design reasoning, not a finished product.