OpenTelemetry for AI Agents

OpenTelemetry can provide a common way to instrument and transport evidence from AI agents, but successful export is only the first acceptance test. Your backend must interpret the operations correctly, preserve task relationships and expose missing data. Your application must still decide what counts as an authorised action or an acceptable business outcome.

For engineering teams, the safest starting point is a versioned telemetry contract. Pin the SDK, instrumentation packages, semantic conventions and collector configuration used by the service. Then test a representative agent fixture through the complete pipeline. This makes interoperability an observed result rather than an assumption attached to the OpenTelemetry name.

Separate the protocol from the GenAI conventions

The OTLP specification defines telemetry encoding and transport, including HTTP and gRPC, and marks trace, metric and log signals stable. The GenAI semantic conventions currently have Development status in their dedicated repository. The former GenAI documentation location states that the material has moved. Those are different maturity questions.

OTLP compatibility therefore does not prove that two tools use identical agent attributes or interpret them in the same way. A backend may accept an event while ignoring a field that another backend uses to construct an agent view. Recheck the applicable convention status and implementation support when selecting a release; do not treat an old tutorial as a permanent schema contract.

Create a compatibility record

Version record for an OpenTelemetry agent implementation

Layer	Record	Why it matters
Language SDK	Language, package versions and runtime	Context, export and shutdown behaviour are implementation-specific
Instrumentation	Framework and provider instrumentation versions	Coverage and emitted fields can change independently
Semantic conventions	Release or commit and stability of used definitions	Attribute names and meanings must be interpretable later
Collector	Distribution, components and configuration revision	A processor must exist and be enabled in the deployed pipeline
Transport	OTLP protocol, endpoint, authentication and size policy	Client and receiver must agree on delivery
Backend mapping	Accepted schema and any transformation rules	Successful ingestion does not guarantee the intended view
Custom business fields	Names, types, meanings and owners	Task outcomes are not defined automatically by technical conventions

Keep this record with the service release, but allow a telemetry-only configuration change to have its own revision. That lets an investigator distinguish a real behaviour change from a field rename, changed sampling rule or missing exporter. Include a rollback procedure for the telemetry configuration itself.

Identify the service and the source of instrumentation

OpenTelemetry resources describe the entity producing telemetry, while instrumentation scope identifies the software unit emitting it. For an agent worker, set a meaningful service identity and deployment information. Preserve the instrumentation library name and version so an unexpected span shape can be traced to its producer.

Do not put per-task identifiers into resource identity. A resource describes the producing entity; a task reference belongs on the relevant observations or in the connected application record. Similarly, do not confuse the model provider's version with your service release or instruction revision. Each answers a different diagnostic question.

When several libraries instrument the same call, inspect their scopes for duplicate coverage. Choose a deliberate arrangement rather than enabling every available integration. A duplicate span can distort call counts and latency views even when each individual integration appears to work.

Represent agent, model and tool operations separately

The current agent conventions distinguish agent invocation and tool execution, including remote and in-process agent concepts. The model-operation conventions describe logical model operations. Use the applicable definitions for the pinned version; do not rename every HTTP request as an agent invocation.

For example, an agent can make one model request that proposes two tools, execute those tools, then make another model request to prepare an answer. Capture the observable operations and their relationships. The tool proposal is not evidence that execution happened, and a completed model operation is not evidence that the business task was accepted.

Use a clearly owned custom namespace for business-specific attributes where no applicable convention exists. An illustrative field such as example.task.id is a custom design choice, not a standard OpenTelemetry attribute. Document its type, permitted values and content classification. Task tracing explains how it connects multiple attempts without replacing trace context.

Propagate context across real execution boundaries

Test context through the framework, asynchronous tasks, queues and custom tools actually used by the service. Automatic propagation inside one runtime does not prove propagation into another process. Define which metadata is carried and where extraction occurs; confirm that the receiving operation joins or links to the intended context.

Keep the durable business-task reference available when execution context is absent or a later attempt starts a new trace. Never use a trace identifier or propagated field as proof of user identity or action authority. Authentication and authorisation belong to the service's security controls.

OpenTelemetry baggage propagates key-value context and can cross service boundaries. Avoid credentials and sensitive personal content there. Restrict propagation to intended destinations and review untrusted incoming context. Correlation convenience should not create a new data-disclosure channel.

Choose the content boundary before export

Decide which instructions, messages, retrieved passages and tool values may be recorded. Make metadata-only capture the starting design when it answers the operational question, then justify additional content. Use version references, categories and approved excerpts where they provide sufficient evidence.

OpenTelemetry's sensitive-data guidance makes clear that instrumentation requires context-specific review and supports minimisation or processing of sensitive attributes. Decide whether a field must be removed inside the application or may reach a trusted collector. Removing it later in the pipeline does not undo an earlier transfer.

Use synthetic markers to test root messages, nested tool arguments, exception text and metadata. Check all export destinations. A field can be absent from the primary trace view while surviving in logs, debug exports or another processor branch. Agent telemetry design provides the more detailed field, retention and cardinality contract.

Configure the collector as a controlled processing path

The Collector configuration documentation defines receivers, processors and exporters that are enabled through service pipelines. Review the deployed component set and processing order. A redaction processor listed in a configuration file but absent from the active path provides no protection.

Specify the endpoint protocol, authentication, transport protection and network exposure for your environment. Avoid copying an unrestricted development listener into production. Bound queues and memory, decide how retryable export errors are handled and monitor discarded or delayed records. Flush behaviour at process shutdown should be tested with the actual SDK and runtime.

OTLP acknowledgements apply between adjacent sender and receiver nodes; the specification does not establish an end-to-end guarantee across every intermediary. Treat application-to-collector acceptance and final backend availability as different observations. Keep telemetry delivery failures visible instead of assuming a successful first hop proves complete retention.

Use a conformance fixture, not a connectivity screenshot

Create a synthetic fixture called OT-17 for this illustrative test. It starts a task, invokes a model, runs two harmless lookup tools, crosses a queue boundary and returns a deliberately rejected result. Repeat it with a worker interruption and an unavailable exporter. The fixture makes no external business changes.

For each run, inspect expected operation types, parent or link relationships, service and instrumentation identities, custom task reference, error information and content masking. Verify that a rejection is represented as the defined business outcome rather than inferred solely from a technical span status.

Then query the backend as an operator would: find the task, locate the failed branch, identify the application and instrumentation versions, and determine which evidence is missing. Export the resulting record and check that its essential meaning survives. This is a stronger interoperability test than merely seeing a span appear.

Manage schema changes as operational changes

When a convention or instrumentation library changes, compare the fixture before and after the upgrade. Check field names, value types, operation boundaries, token units where used and score mappings. A dashboard may continue rendering while its underlying interpretation changes.

If an adapter maps old and new attributes, preserve enough version information to explain the transformation. Do not silently combine differently defined metrics into one historical trend. Mark the transition or recompute a comparable series where the retained evidence supports it.

Use release control for changes that affect required operating evidence. The acceptance gate should ask whether operators can still make the necessary decisions. Production monitoring should also watch the telemetry pipeline, not only the agent it describes.

What OpenTelemetry does not decide for you

It does not select the correct business outcome, authorise external actions, establish Saudi data-handling compliance or guarantee that every framework emits complete evidence. Those remain explicit application and operating responsibilities. Its value is a common technical foundation whose actual behaviour can be tested and maintained.

Discuss an OpenTelemetry contract for your AI agents with your language runtime, frameworks, intended backend and a masked execution flow. AgentOps services can be scoped to instrumentation, verification and operating handover requirements. Keep the contract and fixture alongside the service so later changes remain reviewable.