AI Agent Architecture

An AI agent architecture is the arrangement of software, data, identity and decision controls around a model. Its most important boundary is where an interpretation becomes an authorised action. A model may propose what to do next; the application still needs to establish who requested it, which evidence is valid, whether the action is allowed and whether it actually completed.

For enterprise architects and engineering leads, a useful architecture explains both the successful path and the ambiguous one. This guide develops an illustrative warranty assistance design: the agent gathers product and service evidence, explains the applicable warranty terms and prepares a service request. It does not invent coverage, approve an exception or silently alter a customer's entitlement.

Start with responsibilities, then choose components

The first diagram should describe responsibilities rather than vendor logos. Identify the entry point, identity boundary, task controller, evidence services, permitted tools, durable state, review interface and operational signals. A single application may initially implement several responsibilities. Separating them conceptually still makes failures and ownership easier to reason about.

Write down what must remain true regardless of model output. In the warranty example, the customer can access only their permitted product records; the relevant terms must correspond to the product and purchase context; and creating a service request does not itself confirm warranty coverage. These invariants belong in application controls and acceptance tests.

Do not begin with multiple agents, a vector database or long-term memory simply because they appear in other designs. A structured product registry may answer the core query directly. A model can interpret the customer's description while ordinary code performs exact identifier lookup and date comparison. The automation comparison helps identify where model choice is actually useful.

Keep identity outside conversational persuasion

The entry layer authenticates the caller and establishes the tenant, user role and permitted case. That context should be supplied by trusted application code, not inferred from a message claiming to be an administrator. Each evidence service and action endpoint must enforce its own access boundary.

A tool called get_product_record should receive a validated case or product identifier within the caller's scope. It should not accept an arbitrary tenant identifier chosen by the model. Where service credentials are required, the application must still enforce the end user's delegated authority rather than exposing the full privileges of the service account.

Record authority separately from content. A PDF can contain warranty facts, but its text cannot grant permission to issue a replacement. An internal note can inform the review, but its presence in retrieval does not make it an approved policy. The dedicated security architecture guide develops these trust boundaries further.

Build an evidence path with provenance

Retrieval should return usable evidence, not an unlabelled text block. Include a stable source identifier, version or update time, relevant excerpt and access context where needed. The response builder should be able to distinguish an authoritative warranty term from a customer's account of what happened.

Use exact lookup for exact questions. A serial number should resolve through a system that preserves its identity; semantic similarity alone is a poor basis for deciding that two product records refer to the same item. Search can help locate explanatory passages, but the application should preserve the link between the passage and the correct product terms.

Define freshness rules per source. Product ownership may require a current lookup, while a historical warranty document may need the version effective at purchase. “Newest document” is not always “correct document”. When evidence conflicts, return the conflict and a review path instead of letting the model blend incompatible facts into a smooth answer.

Give the task controller a bounded state model

The controller tracks what the system is trying to accomplish and what has already happened. A useful state for this example includes the case identifier, verified product, evidence references, unresolved questions, proposed outcome, approval record and command status. Keep raw evidence references separate from the prose shown to the user.

Separate transient conversation from durable case state. A customer can close a browser and return later, but that should not erase a pending approval or duplicate a request. Conversely, a conversational remark should not automatically become a permanent customer fact. Memory architecture covers retention, correction and reuse across interactions.

Framework persistence can support this separation. LangGraph documents thread state in checkpointers and application data in stores. Those mechanisms provide building blocks; your application still defines ownership, retention and the meaning of each record. See LangGraph persistence documentation.

Separate proposing a command from executing it

Use a narrow command such as “create service request for this verified case” rather than a general database write tool. Validate required fields, caller authority, current case status and any approval before execution. Return an explicit receipt or an explicit unresolved outcome. A model-generated statement that the request was created is not a receipt.

Microsoft's CQRS pattern separates commands that change data from queries that retrieve it. An agent application can use that separation of responsibilities without automatically adopting a fully distributed CQRS infrastructure. See the Azure architecture guidance.

For the warranty design, evidence lookup is a query; saving a draft is a limited write; creating the service request is a command. Approval of an exceptional replacement is a different command with different authority. Do not hide these distinctions behind one flexible “manage warranty” tool.

Proposed contracts in a warranty assistance architecture

Component	Contract	Failure behaviour
Product lookup	Return the authorised product record and its version	Ask for clarification if identity is unresolved
Terms retrieval	Return applicable terms with source and effective context	Escalate contradictory or missing evidence
Model task	Explain evidence and propose the next permitted step	Leave unsupported eligibility unknown
Review interface	Bind approval to the specific proposal and evidence	Invalidate approval when material inputs change
Request command	Validate authority and return a durable request receipt	Reconcile uncertain outcomes before retrying

Design the uncertain path before calling the architecture complete

Suppose the service request endpoint times out after receiving the command. The agent cannot know from the timeout alone whether the request was created. Blindly retrying may create a duplicate. The application needs an operation identifier, a status lookup or another documented reconciliation method supported by the receiving system.

If the underlying system offers no reliable duplicate protection, the architecture must reflect that limitation. The safe operating mode may require a reviewer to check the destination before another attempt. A prompt instruction to “avoid duplicates” does not create a transaction guarantee that the API lacks.

Also specify cancellation and expiration. A customer may withdraw while a case awaits review. A previously available service slot may disappear. Revalidate material conditions immediately before a consequential command, and make the user-facing status match the durable state. Detailed retry and resumption design belongs in agent orchestration.

Treat the model as a replaceable decision component

Define the input and output contract independently of one model's phrasing. Structured proposals should have validated fields and explicit unknown values. Preserve evidence references through transformations. Model choice can then be assessed on the task rather than becoming entangled with every integration.

Version instructions, tool descriptions, retrieval configuration and output schemas together with the relevant release record. A model change can alter which tool is selected or whether clarification is requested even when the interface looks identical. Evaluation should compare the complete behaviour, including tool calls and resulting state, not only the final paragraph.

Fallback behaviour needs the same discipline. A second model may have different capabilities or data processing arrangements. Falling back to a generic answer when evidence services fail may be inappropriate for a warranty decision. A controlled unavailable state can preserve trust better than a plausible response without support.

Instrument decisions without copying every document

Record the run identifier, relevant versions, source references, tool outcomes, approval transitions and final state. Use restricted evidence access when investigation requires document content. Uncontrolled full-prompt logging can create another store of customer information that no one has designed or governed.

Distinguish technical failure from business non-completion. A run that successfully asks for a missing serial number may be behaving correctly. A run that returns polished text but loses the service request receipt has failed operationally. The architecture must let evaluators inspect both categories; the testing guide describes suitable evidence.

Review the architecture through a case walkthrough

Ask the team to walk one normal case, one conflicting-evidence case, one permission failure and one interrupted command through every component. At each transition ask: what is trusted, what is stored, who may act, what is visible to the user and how does the system recover? Missing answers identify design work more effectively than adding another box to a diagram.

The resulting architecture document should include these walkthroughs, component contracts, source ownership, access boundaries, versioning and operating limitations. It should also identify which capabilities are proposed, which have been proven in the intended environment and which are deferred. That distinction makes the design reviewable without presenting an architecture sketch as a production guarantee.

Check what the review can actually prove

A design walkthrough establishes coherence, not implementation correctness. Ask for separate evidence that access checks execute in the receiving services, that the receipt can be retrieved after a timeout and that a withdrawn source is no longer used. Keep these items in the delivery acceptance plan. This prevents an attractive architecture from being mistaken for a completed control assessment and gives engineers concrete work to verify before release.

Open for AI can scope AI agent development around the required business boundary. To discuss an agent architecture for your systems, bring a redacted workflow, the authoritative data sources and an example of the most consequential permitted action. Those inputs determine the design more usefully than a preferred framework alone.