AI Agent Orchestration

AI agent orchestration manages the execution of a task across model calls, tools, state, approvals and external systems. Planning decides what to attempt next. Orchestration determines whether that attempt is allowed, records what happened and makes recovery possible when execution stops partway through.

A useful orchestration design must answer a difficult question: if the process crashes after another system accepts a request, what happens when it starts again? This guide follows an illustrative equipment-loan workflow in which an agent prepares a reservation, a supervisor approves it and a controlled service records the loan. The focus is reliable execution, not a claim that any particular platform makes every operation safe automatically.

Model business state before drawing the agent loop

Define the states that matter to the receiving team. For the equipment loan, a request can be received, awaiting information, ready for review, approved, being committed, confirmed, cancelled or awaiting reconciliation. These states describe business progress, not simply which prompt ran most recently.

Define permitted transitions and their evidence. “Approved” requires an authorised approval for a particular proposal. “Confirmed” requires a destination receipt. “Awaiting reconciliation” means an operation may have completed but its outcome is not yet known. Making that state explicit prevents a model from filling the uncertainty with reassuring text.

Keep the state machine understandable. Not every model thought needs a durable state. Persist the information needed to resume correctly, audit consequential actions and present a truthful status to the user. The architecture guide explains the broader component boundaries.

Give each task and operation a stable identity

A task identifier connects the user's request, evidence, proposal, approval and final outcome. Individual operations need their own identities when they can be retried or reconciled. Do not use a freshly generated identifier for every retry if the receiving service must recognise the same intended operation.

Separate a repeated delivery of the same event from a genuinely new request. Two identical-looking equipment requests may be intentional, while a duplicated event may represent only one request. The business intent and event contract should determine identity, not a guess based solely on similar text.

AWS discusses caller-provided request identifiers as a way to support safe retries in idempotent API designs. Whether that protection exists depends on the receiving service's contract. See the AWS Builders' Library discussion.

Separate model decisions from durable commands

The model can propose the equipment, dates and reason for a loan. Application code validates the fields, verifies the requester's access and checks the business rules. The command layer accepts only the validated operation within its permitted scope.

Store the proposal version before approval. Record who approved it, what they saw and any conditions attached. If the equipment, dates or material terms change, the orchestrator should apply the agreed reapproval rule rather than attaching an old approval to a new command.

Immediately before commitment, recheck conditions that can change: equipment availability, caller authority and approval validity. A reservation proposal that was correct earlier can become invalid while awaiting review. The human oversight guide explains the design of meaningful approval.

Use checkpoints for continuity, not transaction magic

A checkpoint records enough execution state to continue after interruption. It can include the current business state, evidence references, proposal version and operation status. It does not necessarily prove what happened in a remote system between two checkpoints.

LangGraph's interrupt mechanism can pause execution and preserve state for external input. Its documentation describes the persistence and resumption requirements. Application designers must still handle external side effects and revalidation. See LangGraph interrupts.

Choose checkpoint boundaries around meaningful transitions. If one large step reads sources, requests approval and performs a write, recovery may repeat too much work. Separating preparation, approval and commitment makes the recovery contract easier to inspect, provided the state transitions themselves are implemented consistently.

Classify failures before deciding to retry

A temporary network error, an invalid field, a permission denial and an ambiguous write outcome are not the same failure. Retrying all of them with the same policy can waste resources, bypass intended decisions or duplicate effects.

Proposed failure policy for an equipment-loan orchestrator

Observed condition	Correct classification	Proposed response
Read service temporarily unavailable	Potentially transient dependency failure	Bounded retry, then a visible unavailable state
Requested dates are invalid	Input or business validation failure	Ask for correction; do not repeat unchanged input
Requester lacks access	Authorisation failure	Stop the action and use the approved escalation route
Commit request times out	Outcome may be unknown	Reconcile using the operation identity and destination state
Approval no longer matches the proposal	Stale authority	Return for the required review
Cancellation follows a confirmed reservation	New business operation	Apply the destination's cancellation rules and verify the result

This table is a proposed policy to adapt to the actual receiving systems. The team must verify which outcomes can be queried and which operations support safe repetition.

Walk the uncertain-commit scenario

The supervisor approves the equipment loan. The orchestrator sends the command with its operation identifier. The destination accepts it, but the response is lost before the orchestrator records confirmation. On restart, the durable task shows an attempted command without a verified result.

The next step is reconciliation, not a new model decision about whether to book again. If the destination can return the existing result for the operation, record that receipt and continue. If it confirms no operation occurred, follow the documented retry path. If the result remains ambiguous, stop for an authorised review.

Keep the user informed with a precise status. “We are checking whether the reservation was recorded” reflects the state. Claiming either a confirmed loan or a clean failure without evidence can prompt the user or another employee to create conflicting work.

Test this scenario by introducing a failure at the relevant boundary in an authorised test environment. A successful normal run does not demonstrate recovery after a lost response. The evaluation guide describes how to inspect destination outcomes as well as messages.

Use compensation where the business process requires it

A workflow may reserve equipment and then fail to arrange an associated approved service. Reversing the first step is a business operation with its own conditions. It may be better to offer a permitted alternative, release a temporary hold or ask the responsible person what to do.

Microsoft's compensating transaction guidance explains that recovery across services is application-specific and may not restore the exact original state. Compensation can itself fail and require tracking. See the Azure pattern guidance.

Record the information needed for the chosen recovery. Do not assume that restarting the agent undoes an external reservation. Likewise, a cancellation request should not be marked complete until the destination confirms the relevant state or the case is explicitly escalated.

Bound concurrency and duplicated work

Decide whether two runs may modify the same case simultaneously. If not, enforce an appropriate claim, lock or version check in the application. The model cannot coordinate concurrent workers reliably through a prompt asking them to avoid conflict.

When parallel work is useful, give it separate outputs and a defined merge point. Read-only evidence gathering may run concurrently while one owner controls the final write. The multi-agent guide covers delegation and conflict-preserving consolidation.

Set limits for total execution time, tool calls, worker creation and retry attempts. Use backoff or scheduling appropriate to the dependency rather than repeatedly calling a service that is already unavailable. When limits are reached, preserve partial evidence and expose the unresolved state to an owner.

Version the workflow that must resume

A long-running case may pause under one configuration and resume after a release. Decide whether it stays on its original compatible workflow, migrates through an explicit procedure or requires review. A new tool schema or approval rule can make blindly resuming an old checkpoint invalid.

Record the relevant model, instruction, tool and workflow versions with the case. Use compatible contracts at stable boundaries and define migration for material changes. Do not let a deployment silently reinterpret a previously approved proposal.

Distinguish task state from reusable memory. The former records what this workflow has done; the latter may inform another interaction. The memory architecture guide addresses that separate lifecycle.

Make operations able to understand and intervene

Provide views of queued, running, waiting, failed and uncertain cases, with the reason and responsible owner. An operator should be able to identify whether the next step is a retry, a source correction, a new approval or destination reconciliation.

Define stop controls precisely. Stopping new tasks is different from pausing in-flight work or disabling a tool. Operations also needs a safe resumption procedure and a record of manual interventions. Monitor stuck approvals and growing exception queues, not only server uptime.

Open for AI can scope agent development with these execution contracts. To review orchestration for a proposed agent, bring the external actions, their receipts and the receiving systems' retry and cancellation behaviour. Those details determine whether a workflow can recover as reliably as it can demonstrate success.