Remember with scope and provenance
AI Agent Memory Architecture
Separate task state, preferences and authoritative knowledge, then design memory writes, retrieval, correction, expiry and deletion with a worked ledger.
AI agent memory architecture defines what information survives a turn, a task or a conversation, and how that information can influence later work. The difficult decision is not whether to add a database. It is which information should be remembered, under whose authority, for which purpose and until when.
This guide is for architects and product owners designing continuity without turning every conversation into a permanent source of truth. The illustrative case is a maintenance service assistant that works with recurring service agreements. It needs to remember an open case, retrieve the current agreement and handle a customer's temporary scheduling preference without confusing those three things.
Separate working context, task state and reusable knowledge
Working context is what the model receives for its current decision. Task state records the progress and facts needed to continue a particular case. Reusable knowledge or preferences may apply across cases. These layers can share infrastructure, but their purpose, authority and retention rules differ.
A conversation transcript is evidence of what was said, not automatic proof that every statement is true. A generated summary is a derived interpretation that may omit qualifications. A business system record can be authoritative for some fields while still requiring an applicable version and access check.
LangChain's memory overview distinguishes thread-scoped and cross-thread memory. That is a useful implementation distinction, while the business design must further define what each stored item means. See the memory documentation.
Memory storage is also different from model training. Saving a preference in an application database does not inherently retrain model weights. Clearing a conversation, deleting a stored memory and changing a model are different operations; the product should explain which one a user control actually performs.
Choose the authority of each remembered item
In the maintenance example, an approved agreement defines the service window. A customer says that the next visit should happen later in the day. A technician records that access was difficult on a previous visit. These statements have different scope and should not be merged into one unqualified customer profile.
The agreement remains a source-controlled business record. The next-visit request is a case-specific preference pending the relevant scheduling confirmation. The technician's note is an observation with its own date and author, not a permanent rule that the site is inaccessible.
Classify stored content as verified fact, user-stated preference, temporary instruction, observation or derived summary where those distinctions matter. Preserve provenance and uncertainty. A high confidence score generated by a model does not convert an inference into an authorised fact.
A worked memory ledger
Assume the agreement permits a routine window of 09:00–12:00, while the customer requests 14:00–16:00 for the next visit. These times are illustrative. The assistant should remember the request without rewriting the agreement or treating the later window as approved.
| Information | Memory category | Authority and lifecycle |
|---|---|---|
| Routine window 09:00–12:00 | Retrieved agreement fact | Read from the applicable approved agreement; refresh when its version changes |
| Requested next visit 14:00–16:00 | Case-specific user preference | Applies only to the identified visit; does not itself approve the change |
| Access difficulty on a previous visit | Dated observation | Keep source and relevance; do not generalise indefinitely |
| Pending scheduling review | Durable task state | Persist until the case is resolved under the agreed retention policy |
| Generated conversation summary | Derived context aid | Retain source references and avoid overriding authoritative records |
This ledger prevents a common design error: allowing a convenient summary to become more authoritative than the underlying agreement. It also makes correction possible because each item has a clear source and scope.
Define a minimum record contract
A reusable memory item should carry enough metadata to decide whether it belongs in a future task. Useful fields can include subject, tenant or access scope, purpose, source reference, source version, validity period, verification status and a link to any item it supersedes. Choose the minimum needed for the use case rather than collecting every available attribute.
Keep identity stable. Two people with similar names or two sites with similar labels must not share memory by semantic resemblance. Resolve the authorised business identifier before retrieval. A memory namespace is an organisational boundary only if the application and storage access actually enforce it.
Separate validity from storage retention. A temporary preference may stop being applicable after the visit even if a limited case record is retained for an agreed operational purpose. Conversely, a retained agreement can remain historically relevant without being the current rule for a new case.
Control the write path
Do not let every plausible model statement become persistent memory. Define which events can create or update each category. A user can express a preference, a source owner can publish an agreement change and a completed tool operation can update task status. These authorities should not be interchangeable.
A practical pattern is to create a memory candidate with provenance, then apply the appropriate validation or review before promoting it to reusable state. The required review depends on consequence. A harmless display preference may need a simple confirmation; a contractual or eligibility fact should come from its authoritative system.
Record corrections as deliberate changes. If a customer says the previous address was wrong, the assistant should identify the field and permitted update path rather than silently modifying several remembered references. The guardrails guide explains how write authority remains outside persuasive text.
Retrieve by relevance and validity
Before placing an item in model context, check its access scope, purpose and applicability. Then retrieve the information relevant to the current task. Similarity search can help locate candidate memories, but it should not decide whether an expired preference or another tenant's record is authorised.
Use current source lookup for facts that must be current. Caching a service agreement indefinitely because it was once correct creates a freshness problem. Define the revalidation trigger: a new case, a source version change, an explicit correction or a time-based rule chosen by the business owner.
If remembered content conflicts with an authoritative source, preserve the conflict and follow the agreed precedence rule. Do not blend them into a compromise fact. In the example, a later requested visit window remains a request until the scheduling system or authorised reviewer confirms it.
Manage context without losing qualifications
The model does not need every historical message for every decision. Select the relevant evidence, current task state and unresolved questions. Keep references to longer material so the system can retrieve details when needed.
Anthropic's context engineering discussion describes dynamically selecting relevant information rather than carrying all available material into every turn. The application must still preserve the meaning and authority of what it selects. See the context engineering discussion.
When summarising, retain negation, dates, scope and unresolved status. “Customer requested a later visit” must not become “customer's service window is later”. A compact summary should be a navigation aid with provenance, not a lossy replacement for the record needed to authorise an action.
Do not store hidden reasoning as a substitute for operational evidence. Store the concise decision, relevant sources, validated action and result. Reviewers need an inspectable account of what happened, not an unverified narrative of the model's internal process.
Design correction, expiry and deletion together
A correction may affect the primary record, derived summaries, search indexes and cached context. Map those copies so the system can stop using superseded information. Deleting only the visible chat message may leave a separate memory item available to future tasks.
Define deletion behaviour for the actual storage architecture, including replicas, backups and operational records under the applicable policy. Do not promise immediate removal from every location unless the implementation and retention arrangements support that promise. Legal obligations should be assessed for the organisation and data involved.
The product should make the user's control understandable: forget a preference, correct a fact, close a case or request deletion are different actions. Confirm what changed and what remains for a legitimate defined purpose. The Saudi PDPL guide addresses the legal assessment separately.
Test memory through time and across boundaries
A useful evaluation includes a new session after a preference is stored, a case after that preference expires, a source revision and an explicit correction. Check that the system retrieves what remains applicable and stops using what has been superseded. A single conversation cannot prove cross-session behaviour.
Test identity separation with distinct authorised users and tenants. Test access removal while a case remains open. Also test a malicious or mistaken instruction attempting to store a false business rule. The evaluation guide explains how to separate critical failures from ordinary usefulness.
Measure whether memory improves the task rather than merely whether retrieval occurs. Useful continuity can reduce repeated clarification, but irrelevant memory can distract the system or introduce stale assumptions. Compare an appropriate no-memory baseline for the same cases.
Integrate memory with execution without confusing them
Task checkpoints support recovery, while reusable memory supports later context. Neither proves that an external action completed. Store the operation receipt and destination state according to the action contract. A remembered statement that a visit was booked is not a substitute for the booking system's confirmation.
The orchestration guide covers resumption and side effects, and agent architecture shows where memory fits among identity, evidence and tools. Open for AI can scope agent development with the appropriate continuity. To review a memory design for your workflow, bring examples of a lasting fact, a temporary preference and a correction that must reach future conversations.