How to trace Pydantic AI agents with OpenTelemetry

When you finish, each agent run is one trace in mcpd with the prompt and answer of every model request, the arguments and result of every tool call, and the tokens and cost of the run. You need a Pydantic AI agent and a running mcpd daemon on port 4317 (see the Quickstart).

1. Install the packages

Besides Pydantic AI, install the OpenTelemetry SDK, its OTLP over HTTP exporter, and the OpenInference span processor for Pydantic AI.

install

pip install "pydantic-ai-slim[openai]" \
  openinference-instrumentation-pydantic-ai \
  opentelemetry-sdk opentelemetry-exporter-otlp-proto-http

The span processor is what makes prompts readable and searchable in mcpd. Pydantic AI writes the messages of a model request in the attributes gen_ai.input.messages and gen_ai.output.messages. mcpd keeps them under Attributes but does not show them as the step's input and output, and full-text search does not read them. The processor copies them into input.value and output.value, which mcpd stores as the input and the output.

2. Point the exporter at mcpd

mcpd takes OTLP on port 4317, over HTTP and gRPC. The HTTP exporter defaults to port 4318, so name the endpoint.

environment

# The base URL. The exporter adds /v1/traces itself.
export OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317
# Only when mcpd runs with --ingest-key.
export OTEL_EXPORTER_OTLP_HEADERS="Authorization=Bearer YOUR_INGEST_KEY"

3. Set up tracing and instrument every agent

Run this once when the process starts. Agent.instrument_all() turns on instrumentation for every agent, using the tracer provider you set.

tracing setup

from opentelemetry import trace
from opentelemetry.sdk.resources import Resource
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
from openinference.instrumentation.pydantic_ai import OpenInferenceSpanProcessor
from pydantic_ai import Agent

provider = TracerProvider(resource=Resource.create({"service.name": "support-agent"}))
# Order matters: rewrite each span first, then export it.
provider.add_span_processor(OpenInferenceSpanProcessor())
provider.add_span_processor(BatchSpanProcessor(OTLPSpanExporter()))
trace.set_tracer_provider(provider)

# Every agent in the process emits spans from here on.
Agent.instrument_all()

4. Run an agent

The agent code does not change. The agent's name becomes the name of the run in mcpd.

agent.py

agent = Agent(
    "openai:gpt-4.1-mini",
    name="support_agent",
    instructions="You answer questions about orders.",
)


@agent.tool_plain
def lookup_order(order_id: str) -> str:
    """Look up where an order is."""
    return f"Order {order_id} is at the Leeds depot."


result = agent.run_sync("Where is order 4417?")
print(result.output)

# A script exits before the batch is sent unless you flush. A server does not need this.
provider.shutdown()

5. Read the trace

Open localhost:4317 and select the newest run in Runs, or read it from the API:

read the run

TRACE=$(curl -s "http://localhost:4317/v1/api/traces?limit=1" | jq -r '.traces[0].trace_id')
curl -s "http://localhost:4317/v1/api/traces/$TRACE" \
  | jq -r '.spans[] | [.name, .kind, .model, .usage.input_tokens, .cost.total_cost] | @tsv'

one agent run with a tool call

$ curl -s "http://localhost:4317/v1/api/traces/$TRACE" | jq -r '.spans[] | [.name, .kind, .model, .usage.input_tokens, .cost.total_cost] | @tsv'invoke_agent support_agent	agent_run			chat gpt-4.1-mini	model_generation	gpt-4.1-mini	812	0.000384execute_tool lookup_order	tool_call			chat gpt-4.1-mini	model_generation	gpt-4.1-mini	812	0.000384

Pydantic AI span	mcpd kind	What it holds
`invoke_agent <name>`	`agent_run`	The run: the user prompt and the final output
`chat <model>`	`model_generation`	One model request, with tokens and cost
`execute_tool <tool>`	`tool_call`	One tool call, with its arguments and result

Pydantic AI also sets gen_ai.conversation.id on its spans, and mcpd records it as the run's session id. The conversations guide shows how to read a session back.

Troubleshooting

What you see	Why, and what to do
Steps have no input or output, and search finds nothing	The `OpenInferenceSpanProcessor` is missing, or it was added after the exporter's processor.
A prompt served from the provider's cache costs as much as an uncached one	Pydantic AI reports cached tokens as `gen_ai.usage.cache_read.input_tokens`, a name mcpd does not read yet. mcpd prices those tokens at the full input rate, so the figure is an upper bound when caching is on.
No trace arrives	A script exited before the batch was sent: call `provider.shutdown()`. Or `Agent.instrument_all()` was never called.
The exporter logs 401 or 403	The daemon has ingest keys. Set `OTEL_EXPORTER_OTLP_HEADERS` as in step 2.

mcpd is free and open source. Start free at mcpd.sh , or run the daemon yourself.