Task and execution evidence
AI Agent Task Tracing
Design task, attempt and trace relationships that preserve parallel work, human handoffs, retries and unresolved outcomes.
AI agent task tracing connects the work a user requested with the executions, tool calls, handoffs and decisions that followed. Its purpose is to reconstruct causality and outcome. A conversation transcript or a list of model calls can be useful evidence, but neither is automatically a complete task history.
The central design choice is to keep task identity separate from execution identity. One task can have several attempts and traces. One conversation can contain several tasks. A background worker can process work long after the initiating request has ended. A useful tracing design preserves those distinctions instead of forcing everything into a session-shaped tree.
Choose identifiers for the questions they answer
A task identifier answers “Which piece of work is this?” An attempt identifier answers “Which execution tried to advance it?” Trace and span identifiers describe technical execution relationships. An external action reference identifies a request or result in another system. A reviewer decision reference identifies an authorised human disposition. None should be assumed to substitute for all the others.
Use opaque identifiers that do not embed names, email addresses or confidential case details. Keep any mapping to sensitive business records in an appropriately controlled system. The identifier should remain meaningful for correlation after a task is retried, a worker is replaced or a trace is sampled out.
Document identifier creation and ownership. If two entry services can admit the same work, task deduplication is an application concern; generating two different trace IDs will not resolve it. The idempotency and retry guide covers the separate control needed to avoid repeated business effects.
Illustrative graph: research brief BR-64
Consider a hypothetical internal planning agent preparing a venue-research brief. Task BR-64 asks for a comparison using synthetic venue records. The first execution checks capacity and accessibility in parallel. One lookup is delayed, so the agent pauses for a planner to clarify a requirement. A later execution resumes and assembles the brief. It does not make a booking or contact a venue.
The capacity lookup finishing first does not make it the parent of the accessibility lookup. Both were initiated by the same execution. The resumed execution belongs to BR-64 but is not the same attempt. The planner's clarification affects the later brief even if it arrives through a separate application with its own trace.
| Recorded object | Relationship | Meaning |
|---|---|---|
| Task BR-64 | Durable work reference | The requested planning brief |
| Attempt BR-64-A | Advances BR-64 | Initial execution before clarification |
| Capacity lookup | Child operation of BR-64-A | One independently observed source request |
| Accessibility lookup | Parallel child of BR-64-A | A separate source request, not caused by the capacity result |
| Clarification decision | Resolves a review request for BR-64 | Authorised input to subsequent work |
| Attempt BR-64-B | Same task, linked to the prior handoff and clarification | A new execution using the updated requirement |
| Brief acceptance | Judges the resulting brief version | A business disposition distinct from execution completion |
This table is a conceptual application model, not an OpenTelemetry wire format. Map it to the SDK and backend you use, then verify that the operator can recover the relationships. The OpenTelemetry guide covers the current standard and the versioning decisions involved.
Use parent-child relationships and links deliberately
OpenTelemetry describes spans, events and span links for representing execution and causal associations. A child relationship is useful for work performed within an operation. A link can express a relevant causal connection when the work is not naturally represented as a simple child in the same execution tree.
For fan-out, capture each branch independently and record the join condition. Does the brief need both sources, either source or an explicit reviewer override? A join that proceeds after one result is different from a join that waits for all required evidence. Preserve which inputs were actually consumed, not merely which calls were started.
For fan-in, a later operation may depend on several prior results. Keep the relationship to each relevant source operation and its version. Do not invent a sequence by sorting completion timestamps. A timeline is a display of events; causality comes from the recorded relationships and application contract.
Carry context across queues and process boundaries
OpenTelemetry's context propagation guidance explains sending context through a carrier and extracting it at the receiver. In a queue-based application, define the permitted message metadata and verify that producer and consumer instrumentation use it consistently. Test a real asynchronous boundary, not just two functions in one process.
Preserve the durable task reference separately from any transient execution context. A delayed message may be processed after the original worker has disappeared. A resumed workflow must still identify the task, the applicable input revision and the reason for resumption. Store only approved correlation data, not an entire execution object containing credentials or unrestricted content.
Treat incoming trace context from outside your trust boundary as untrusted metadata. It must not grant access or authorise a tool call. Apply the application's authentication and authorisation independently. A caller who knows a trace identifier should not gain the ability to read the corresponding task.
Represent retries without rewriting history
Distinguish a provider client's internal retry from a new application-level attempt to advance the business task. The former may appear within one logical operation according to the chosen instrumentation; the latter needs its own attempt identity and reason. Record what was known before retrying, especially when a prior external action was uncertain.
Do not overwrite the first failure with the final successful result. Operators need to see the initial attempt, the retry decision and the result of each relevant action. Conversely, avoid counting internal retries as separate accepted business tasks. The task-and-attempt relationship makes both interpretations possible without conflating them.
For BR-64, a repeated read of a synthetic source can be associated with the lookup operation and its attempts. If the agent instead wrote to a business system, tracing would record the action references, while separate application controls would determine whether another write is permitted. Trace visibility is evidence, not duplicate-prevention enforcement.
Make waits, cancellation and late results explicit
A task waiting for a planner has a valid non-terminal state. Record the review request and waiting interval in the durable lifecycle. The executing span can end when its actual work ends; the task can remain open. On resumption, create the appropriate new execution and connect it to the prior state.
If the planner cancels BR-64 while a lookup is still running, record the cancellation decision and whether cancellation reached each active component. A late result should not automatically reopen the task or trigger a final brief. The application needs a state check; the trace should show how the late event was handled.
Distinguish requested cancellation from confirmed stopping. External services may complete work already accepted. Keep that uncertainty visible where it matters. The incident recovery guide addresses reconciliation and containment when a trace reveals unfinished or ambiguous external work.
Record evidence references, not imagined internal reasoning
A trace can capture observable inputs, outputs, tool requests, selected sources, validation results and application decisions. It does not reveal all of a model's internal computation. Do not label a generated explanation as a verified account of why the model acted. If an explanation is collected, identify its origin and treat it as another output to assess.
Record the versions of the instructions, source records, tool contract and runtime configuration needed to interpret the task. A reference is useful only if an authorised investigator can resolve it later. Where full content is unnecessary or too sensitive, retain an approved excerpt or content reference with a documented access and retention policy.
The current GenAI agent semantic conventions distinguish agent invocation and tool execution concepts. Align technical attributes with the applicable convention version while keeping business-specific meanings explicit. The telemetry field contract helps prevent custom fields from becoming an undocumented collection of arbitrary strings.
Validate the graph from an investigator's perspective
Prepare expected graphs for a clean task, parallel work, a retry, a human wait, a cancellation and missing telemetry. Compare captured relationships with those expectations. Ask an engineer unfamiliar with the implementation to explain which evidence produced the accepted brief and which earlier results were superseded.
Check event time and ingestion time separately. Delayed export and clock differences can distort a displayed timeline. Preserve causal references and record timing uncertainty where relevant. A negative-looking duration in a cross-system view should trigger clock or mapping investigation rather than an invented story about execution order.
Reconcile the graph with admitted tasks. Sampling or exporter loss may leave partial traces, so show completeness limits explicitly. Preserve a minimal task-state record where the operating contract requires it. Audit evidence may need stronger completeness and access controls than a sampled diagnostic trace.
A useful tracing deliverable
Deliver an identifier dictionary, event-and-relationship map, context propagation rules, content policy and a set of known trace fixtures. Include the operator queries that reconstruct each fixture. These artifacts make an SDK upgrade or backend migration testable without depending on the original developer.
Discuss a task-tracing design for your agents with a masked workflow containing a queue, retry or human handoff. AgentOps services can be scoped to the evidence gaps and operating decisions involved. The objective is a task history that helps an authorised person understand what happened and decide what should happen next.
FAQ
Your questions, answered.
What is the difference between a task ID, trace ID and session ID?
A task ID identifies a piece of business work, a trace ID identifies a technical execution graph, and a session ID groups interactions according to an application-defined boundary. Their relationships are not necessarily one-to-one. A long task can span several traces, while one conversation session can contain several tasks. Document the mapping explicitly.
Should a long-running agent task use one span for its entire lifetime?
Not automatically. Model actual execution durations with spans and preserve long waits in the durable task lifecycle. A resumed worker can create a new execution linked to the task and prior handoff. The design should support reconstruction without requiring a process or span to remain active throughout an extended human approval wait.
How should parallel agent tool calls appear in a trace?
Represent each observed operation separately under its actual initiating context and preserve the join rule. Calls started in parallel should not become a false sequential chain just because one finishes first. Record which results were used, which were superseded and whether the next operation required all results or only a permitted subset.
Can tracing reveal the actual internal reasoning of an LLM?
Tracing records observable execution: messages, tool interactions, selected sources, application decisions and outcomes, subject to your content policy. It does not expose all internal model computation. A generated explanation is another model output, not a verified causal account. Use concrete source and action evidence when investigating why a result was accepted.
How do we trace a cancelled task that receives a late tool result?
Record the cancellation request, the state decision and any confirmation that active work stopped. Preserve the late result as an event connected to the task, then show whether the application ignored, retained or reconciled it. Do not let arrival order alone reopen the task. External actions may need additional reconciliation.
Are complete agent traces sufficient as audit records?
Not necessarily. Diagnostic traces can be sampled, delayed, edited in presentation or missing events after export failures. An audit requirement may need stronger completeness, access, retention and integrity controls. Link the relevant evidence, but define the audit contract separately. A visually complete execution graph is not proof that all required accountability records exist.