How to Trace AI Agents with OpenTelemetry in Python

OpenTelemetry can show why an AI agent was slow or expensive—not merely that its HTTP request took three seconds. A useful trace separates the agent run, model inference, retrieval, memory, and tool execution while preserving their parent-child relationships. That makes “the agent is flaky” a debuggable claim: perhaps the model retried, a tool timed out, or one prompt version doubled token use.

This guide targets OpenTelemetry AI agent tracing in Python. It uses the current OpenTelemetry GenAI attribute names, exports over OTLP, and keeps prompt content out of telemetry by default. The pattern is framework-neutral, so it can sit under a Pydantic AI agent, an in-house orchestrator, or a system that exposes tools through MCP.

What changed in OpenTelemetry GenAI observability

As of July 26, 2026, the GenAI conventions live in a dedicated OpenTelemetry GenAI semantic-conventions repository. The older GenAI pages under the core semantic-conventions 1.43.0 documentation now point developers to that repository. This matters because copied examples using older attribute names can fragment dashboards.

The conventions are also explicitly marked Development, not Stable. Treat them as a shared, fast-moving vocabulary rather than a frozen API. A July 23 change, for example, added a previous-response ID attribute for APIs that continue context from a prior response or interaction.

For the Python plumbing, the current OpenTelemetry SDK release is 1.44.0, published July 16, 2026, and requires Python 3.10 or newer. OTLP itself is at specification 1.11.0 and is Stable for traces, metrics, and logs. Pinning the Python packages while the GenAI conventions evolve gives you a reproducible starting point.

Design the trace before adding code

Use one trace for one logical agent invocation. A practical hierarchy is:

Do not create a plan span around every reasoning step. The official agent conventions reserve it for a planning or task-decomposition phase that instrumentation can actually distinguish. Likewise, an inference span should cover the complete logical operation, including automatic retries, and end only when the response is fully received or the call fails.

Step 1: Install and configure OTLP export

Create a Python 3.10+ environment and pin the current matching packages:

Required Python packages

Use Python 3.10 or newer and install matching 1.44.0 releases of the OpenTelemetry API, OpenTelemetry SDK, and OTLP HTTP exporter. Pin all three packages to the same version so the tracing API, SDK, and exporter remain compatible.

Configure a tracer provider once at process startup. The HTTP exporter expects the trace endpoint, including /v1/traces:

Tracer-provider setup

Configure the tracer provider once when the process starts:

1. Create a resource with the service name, service version, and deployment environment.
2. Create the tracer provider using that resource.
3. Attach a batch span processor and the OTLP HTTP exporter.
4. Read the collector trace endpoint from an environment variable, with a local Collector endpoint as the development fallback.
5. Register the provider globally and create a named tracer for the agent service.

OpenTelemetry recommends batching and treats the Collector as a production best practice. Send telemetry to the Collector and let it manage authentication, filtering, retries, sampling, and routing. Do not hard-code backend credentials in application code.

Step 2: Instrument the agent, model, and tools

The following wrapper accepts your existing model and order-lookup functions. It records operational metadata but deliberately excludes prompts, model outputs, and tool arguments:

Instrumentation pattern

Instrument the existing model and tool functions without recording prompts, model outputs, or tool arguments:

1. Start one internal root span for the complete agent invocation.
2. Create an internal child span around each tool call and record bounded error types when a tool fails.
3. Create a client child span around the complete logical model operation, including automatic retries.
4. Record provider, requested model, response model, streaming status, finish reason, and token usage.
5. Mark both the failing child span and root agent span when an exception escapes.

Set attributes needed for head-sampling decisions when each span is created, not after the operation returns. Check whether a provider SDK already emits conforming spans before adding wrappers, because double instrumentation creates duplicate model calls and inflated metrics.

Use the exact requested model identifier in the request-model attribute and record the model that actually answered in the response-model attribute. If a provider reports both consumed and billable tokens, the conventions say to report the billable count. Reasoning output tokens, when available, should also remain included in total output tokens.

Step 3: Build dashboards that answer operational questions

Start with a small set of service-level views rather than indexing every attribute:

1. Agent success rate and p95 duration by agent name and version.
2. Model p50/p95 latency by provider, request model, and response model.
3. Input and output tokens per successful run, plus cache-read tokens when the provider exposes them.
4. Tool failure rate and duration by tool name and a low-cardinality error type.
5. Streaming time to first chunk using the standard response timing attribute.
6. Trace outliers where tool-call count, inference-call count, or total duration exceeds a known-good baseline.

The current metric vocabulary covers client operation duration, token usage, agent duration, inference-call count, tool-call count, and tool duration. These metrics are also in Development, so verify their exact names against the repository before upgrading instrumentation.

Privacy, cardinality, and cost guardrails

Prompt and response capture is opt-in for good reason. GenAI input and output message attributes, prompt variables, system instructions, tool arguments, and tool results may contain PII, secrets, customer records, or proprietary prompts. Begin with metadata-only spans. If content is indispensable for a short debugging window, require explicit authorization, redact before export, restrict access, set a retention limit, and disable capture automatically.

Do not invent a conversation ID when none exists. The conventions specifically advise against substituting a new UUID, trace ID, or content hash. When a genuine conversation identifier is available, keep it on traces or logs; avoid turning it into a metric dimension, where unbounded cardinality can make the backend expensive or unusable.

Also sample intentionally. Keep all errors and unusually slow traces, then apply a lower rate to routine successes. Because the conventions are not stable, pin dependencies, record the convention revision in your engineering documentation, and test dashboard queries before each upgrade.

Validate the trace end to end

Do not stop when the application runs without an exception. Send one deterministic test request and inspect the exported trace. The root agent span should contain both the tool and model spans as children; their durations should fit inside the root’s time range. Confirm the model request and response identifiers match the values returned by your SDK, not display names copied from configuration.

Next, force a tool timeout and a model error separately. Each failing child should carry an exception, an error status, and a bounded error type; the root should also fail. Restart or terminate the process gracefully and verify the batch span processor flushes its queue. Finally, disconnect the Collector during a load test. Telemetry loss may be acceptable, but an unavailable observability backend must not become an outage in the user-facing agent.

Async agents need one extra check: launch concurrent requests and verify tools remain attached to the correct parent trace. OpenTelemetry context normally propagates through Python async tasks, but custom thread pools, queues, and process boundaries require explicit context propagation.

Production rollout checklist

• Confirm exactly one span exists for each agent, inference, retrieval, and tool operation.
• Verify parent-child context survives async tasks, queues, and service boundaries.
• Send telemetry through a Collector and test exporter failure without blocking requests.
• Compare recorded token counts with provider billing for a controlled sample.
• Trigger one model error and one tool timeout; confirm the error-type value stays low-cardinality.
• Confirm prompts, outputs, tool arguments, and secrets are absent from exported spans.
• Load-test telemetry overhead and choose a sampling policy before full rollout.

Limitations and conclusion

OpenTelemetry tells you what happened across an agent run; it does not judge answer correctness by itself. Quality evaluations, human feedback, and business outcomes still need their own data model. The GenAI repository defines an opt-in evaluation-result event, but events and the surrounding conventions remain in Development and language support varies.

The safest 2026 implementation is therefore incremental: trace the agent boundary, inference calls, and tools; export through OTLP; retain model, token, latency, and error metadata; and keep content disabled. That foundation is already enough to locate slow tools, runaway loops, model-routing surprises, and cost regressions without binding your observability stack to one AI provider.

Primary references

• OpenTelemetry GenAI semantic conventions
• GenAI spans and attribute requirements
• GenAI agent span conventions
• OpenTelemetry Python exporters
• OTLP 1.11.0 specification
• opentelemetry-sdk on PyPI