Setting Up OpenTelemetry in FastAPI

A FastAPI service instrumented the wrong way fails in one of two ways: it loses async context so handler spans detach from their request, or it exports synchronously and stalls the event loop under load. This page solves that precise problem — wiring OpenTelemetry into FastAPI so every request produces a correctly parented span that exports off the request path. It is written for backend engineers and SREs already running an ASGI service, it is part of the OpenTelemetry SDK setup guide within Distributed Tracing and OpenTelemetry in Python, and it applies that provider lifecycle to FastAPI's event-loop architecture. The broader treatment of instrumenting Python web frameworks applies the same ASGI/WSGI pattern to Django, Flask, and Starlette directly.

FastAPI is built on Starlette and ASGI, so instrumentation hooks the ASGI application rather than individual routes. That is why a single instrument_app call covers every endpoint, including ones added later, and why the wrapping must respect the async call chain — a synchronous middleware injected in the wrong place breaks the await that drives the whole stack.

Prerequisites

Async context loss usually starts as a version mismatch between the instrumentation packages and Starlette's routing layer, so pin the versions that ship together. Mismatched opentelemetry-api and opentelemetry-sdk versions raise an ImportError during provider initialization instead of failing quietly, which is the good case; the bad case is an instrumentation package built against a different Starlette middleware contract that installs cleanly and then drops context at runtime.

pip install \
  "opentelemetry-sdk>=1.30.0,<2.0.0" \
  "opentelemetry-instrumentation-fastapi>=0.51b0,<1.0.0" \
  "opentelemetry-exporter-otlp-proto-grpc>=1.30.0,<2.0.0"

export OTEL_SERVICE_NAME="fastapi-backend"
export OTEL_RESOURCE_ATTRIBUTES="deployment.environment=production,team=platform"
export OTEL_EXPORTER_OTLP_ENDPOINT="otel-collector:4317"

The dependency order is deliberate. opentelemetry-instrumentation-fastapi pulls in a compatible opentelemetry-instrumentation-asgi, and that ASGI package — not the FastAPI one — is the layer that actually wraps Starlette's application callable. Upgrading the SDK without moving the instrumentation package to a matching beta is exactly how the wrapper falls out of step with Starlette and loses context, which is what the bounded ranges prevent. Set programmatic resource defaults as well as the environment variables, so the service never falls back to a generic unknown_service label when a variable is missing from one deployment target.

Implementation

1. Bootstrap the provider before the app exists. Build the resource and provider, attach a BatchSpanProcessor (never SimpleSpanProcessor, which exports synchronously and blocks the loop), and set the global provider. This is the same deterministic lifecycle the SDK setup guide describes, applied one module earlier than you might expect: the provider must exist before any instrumentation resolves its tracer, or the middleware binds to a no-op tracer that silently records nothing.

2. Instrument after constructing the app. Call FastAPIInstrumentor.instrument_app(app) once the FastAPI() instance exists so the ASGI middleware wraps every route natively. The middleware extracts traceparent and tracestate from inbound requests — the server-side half of context propagation and baggage — and opens a SERVER span, so a call arriving from an instrumented upstream joins the existing trace with no manual header parsing.

3. Nest manual spans for business logic. Auto-instrumentation captures only the request boundary. Open a child span with tracer.start_as_current_span() inside the handler or a dependency to record the work that defines your latency. Because FastAPI runs on a single event loop, the active span lives in a contextvar and stays correct across every await within one request, so the manual span simply nests under the server span the middleware already created. The only places you must intervene are thread-pool offloads and fire-and-forget tasks, where the context does not follow automatically — the boundary cases covered in the async tracing patterns guide.

4. Flush on shutdown. Force-flush and shut down the provider in the lifespan handler so spans buffered at process exit are not dropped during a graceful restart. The yield in the lifespan context separates startup from shutdown; everything after it runs once when the server begins draining, which is the right moment to call force_flush before shutdown closes the exporter connection.

Two details make this robust under real traffic. First, instrument_app must receive the same tracer_provider you registered globally; passing it explicitly removes any ambiguity about which provider the middleware uses and avoids a subtle bug where the middleware binds to a stale default installed by another import. Second, excluded_urls keeps health checks, metrics scrapes, and the docs UI out of your traces — these fire constantly, carry no diagnostic value, and would otherwise dominate span volume and cost. Exclude them by path fragment so a load balancer's liveness probe never creates a span.

import os
import asyncio
from contextlib import asynccontextmanager
from fastapi import FastAPI, Request
from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.resources import Resource
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import OTLPSpanExporter
from opentelemetry.instrumentation.fastapi import FastAPIInstrumentor
from opentelemetry.trace import SpanKind

# 1. Resource + provider, bootstrapped before the app is built.
resource = Resource.create({
    "service.name": os.getenv("OTEL_SERVICE_NAME", "fastapi-backend"),
    "deployment.environment": os.getenv("DEPLOYMENT_ENV", "production"),
})
provider = TracerProvider(resource=resource)
provider.add_span_processor(BatchSpanProcessor(            # async export, never SimpleSpanProcessor
    OTLPSpanExporter(
        endpoint=os.getenv("OTEL_EXPORTER_OTLP_ENDPOINT", "otel-collector:4317"),
        insecure=os.getenv("OTEL_EXPORTER_OTLP_INSECURE", "false").lower() == "true",
    ),
    max_export_batch_size=512,
    max_queue_size=2048,
    schedule_delay_millis=5000,
))
trace.set_tracer_provider(provider)


# 4. Flush buffered spans on graceful shutdown.
@asynccontextmanager
async def lifespan(app: FastAPI):
    yield
    provider.force_flush(timeout_millis=5000)
    provider.shutdown()


app = FastAPI(lifespan=lifespan)
FastAPIInstrumentor.instrument_app(                        # 2. instrument after app construction
    app,
    tracer_provider=provider,
    excluded_urls="healthz,metrics,docs",
)

tracer = trace.get_tracer(__name__)


@app.get("/process/{item_id}")
async def process_item(item_id: str, request: Request):
    # 3. Manual child span for business logic, nested under the server span.
    with tracer.start_as_current_span(
        "process_item_logic",
        kind=SpanKind.INTERNAL,
        attributes={"item.id": item_id},
    ) as span:
        await asyncio.sleep(0.05)
        span.set_attribute("processing.status", "completed")
        return {"item_id": item_id, "status": "processed"}

Spans inside yield-based dependencies

When a route depends on a yield-based dependency — a database session, a unit of work, an authenticated principal — the dependency's setup runs before the handler body and its teardown runs after the response is produced. Both halves sit inside the server span but outside any span the handler opens, so connection-acquisition time and commit time vanish into the server span's unattributed remainder. If that time matters, open a span inside the dependency itself and it nests correctly under the request.

from fastapi import Depends
from sqlalchemy.ext.asyncio import AsyncSession

async def get_session() -> AsyncSession:
    # Setup and teardown each get their own recorded span.
    with tracer.start_as_current_span("db.session.acquire"):
        session = SessionLocal()
    try:
        yield session
    finally:
        with tracer.start_as_current_span("db.session.commit"):
            await session.commit()
            await session.close()


@app.get("/orders/{order_id}")
async def read_order(order_id: str, session: AsyncSession = Depends(get_session)):
    with tracer.start_as_current_span("load_order", attributes={"order.id": order_id}):
        return await load_order(session, order_id)

Expected Output: four spans in one trace, with the dependency's work attributed rather than absorbed.

GET /orders/{order_id}   SPAN_KIND_SERVER    duration=41ms
  db.session.acquire     SPAN_KIND_INTERNAL  duration=3ms
  load_order             SPAN_KIND_INTERNAL  duration=29ms
  db.session.commit      SPAN_KIND_INTERNAL  duration=8ms

If your dependency hands back an async SQLAlchemy session, the query spans themselves come from the database instrumentation rather than from hand-written spans; see tracing SQLAlchemy async queries for wiring that layer underneath these.

Background tasks close after the server span

BackgroundTasks callables run after the response is sent, which means after the ASGI middleware has already ended the server span. A span opened inside a background task therefore has no active parent and starts a fresh trace unless you capture the request context first and re-enter it in the task. Capture the context with contextvars.copy_context() or, more explicitly, hold a reference to the request span and attach it as a link so the two traces remain queryable together.

from opentelemetry import context as otel_context
from fastapi import BackgroundTasks

def emit_receipt(order_id: str, parent_ctx) -> None:
    token = otel_context.attach(parent_ctx)        # re-enter the request's context
    try:
        with tracer.start_as_current_span("emit_receipt"):
            send_receipt(order_id)
    finally:
        otel_context.detach(token)


@app.post("/orders/{order_id}/confirm")
async def confirm(order_id: str, tasks: BackgroundTasks):
    tasks.add_task(emit_receipt, order_id, otel_context.get_current())
    return {"status": "queued"}

The same capture-and-attach pattern is what carries context over a broker when the work leaves the process entirely, as in propagating trace context across Celery tasks.

Sampling a high-traffic endpoint

On a busy service you should not record every request. Set a sampler when you construct the TracerProvider, in the same place the resource is fixed: ParentBased(TraceIdRatioBased(0.1)) keeps 10% of the traces this service roots while honouring any decision already carried in the inbound traceparent, so a request an upstream gateway chose to sample is recorded here too and the trace does not come back with holes in it. Leave error-and-slow-request retention to tail sampling in the collector rather than encoding that logic in the application, where the decision has to be made before you know how the request ends. The trade-offs between head and tail approaches are worked through in sampling strategies for distributed tracing.

Configuration Options

Size max_queue_size against concurrency rather than throughput: the queue only has to hold what accumulates between flushes, so twice your peak in-flight request count leaves headroom for a slow collector without letting a backlog grow unbounded. When the queue is full the processor drops spans instead of applying backpressure, which is the correct trade for a web service — losing a sample of traces is always better than adding latency to live requests. server_request_hook runs with the server span already open, so anything it sets lands on the request's own span; keep it synchronous and cheap, because it executes on the event loop.

Verification

Send a request and confirm the collector receives a server span and the nested process_item_logic child sharing one trace_id.

curl -s localhost:8000/process/12345

Expected Output (collector side):

{
  "resourceSpans": [{
    "resource": {"attributes": [
      {"key": "service.name", "value": {"stringValue": "fastapi-backend"}},
      {"key": "deployment.environment", "value": {"stringValue": "production"}}
    ]},
    "scopeSpans": [{"spans": [{
      "name": "process_item_logic",
      "kind": "SPAN_KIND_INTERNAL",
      "attributes": [
        {"key": "item.id", "value": {"stringValue": "12345"}},
        {"key": "processing.status", "value": {"stringValue": "completed"}}
      ]
    }]}]
  }]
}

A correctly wired service shows two spans per request: an auto-generated GET /process/{item_id} server span and the manual child nested beneath it. The server span also carries the standard HTTP attributes — method, route, and status code — applied by the ASGI instrumentation, so you can filter and aggregate by route in the backend without adding them yourself. Note that the route appears as the templated path, not the concrete 12345: that is what keeps span names low-cardinality, and the actual value lives in the item.id attribute where it can be queried without exploding the number of distinct names. The same low-cardinality discipline is covered in depth under span lifecycle and attributes.

To verify without a collector, attach a ConsoleSpanExporter through a SimpleSpanProcessor in development and watch the two spans print to stdout in parent-child order on each request. Confirm the child's parent_span_id matches the server span's span_id; if it is empty or points elsewhere, async context was lost — usually because the CLI launcher was used instead of instrument_app, the mistake covered below. For a test-suite assertion rather than an eyeball check, swap in an InMemorySpanExporter, drive the app with httpx.AsyncClient, and assert both that two spans were exported and that they share a trace_id; that single assertion catches a detached child and a missing server span at once.

Correlating those traces with your application logs is a separate step: the trace id has to reach the log record before a backend can join them. If you are choosing a logging stack for this service at the same time, choosing a logging library for FastAPI covers the options, and adding trace ids to log records covers the wiring.

Common Mistakes

• Error signature: RuntimeWarning: coroutine 'Starlette.__call__' was never awaited, or child spans with no parent. Root cause: the opentelemetry-instrument CLI wrapper injected middleware around an app it discovered late, bypassing FastAPI's async ASGI stack. Remediation: drop the CLI wrapper and call FastAPIInstrumentor.instrument_app() programmatically after the app object is constructed, so the wrapping happens at a point where the ASGI chain is fully assembled.

• Error signature: asyncio.exceptions.TimeoutError at peak load, followed by SpanExportError: Export timed out. Root cause: SimpleSpanProcessor runs a synchronous gRPC export at the end of every request, blocking the event loop for the duration of the round trip. Remediation: use BatchSpanProcessor, set max_queue_size to twice expected concurrency, keep schedule_delay_millis between 2000 and 5000, and bound OTEL_EXPORTER_OTLP_TIMEOUT so a stalled collector cannot hold the exporter thread indefinitely.

• Error signature: the final requests before a deploy are missing from the backend. Root cause: the provider is never flushed, so spans buffered in the batch queue die with the process. Remediation: call provider.force_flush() and provider.shutdown() in the lifespan handler as shown above, and make sure the process actually receives a graceful signal — a hard SIGKILL skips lifespan teardown entirely, so configure a real termination grace period in your orchestrator.

• Error signature: traces appear when running a single uvicorn process but stop entirely under gunicorn --preload -k UvicornWorker. Root cause: the TracerProvider and its exporter thread were created in the parent before fork(), and threads do not survive forking, so each worker holds a provider whose background exporter never runs. Remediation: build the provider inside the worker — in a gunicorn post_fork hook, or by dropping --preload so each worker imports the app module itself.

Related

• OpenTelemetry SDK setup — the parent guide covering provider lifecycle, resources, and processor tuning in depth.
• Instrumenting Python web frameworks — the same ASGI and WSGI pattern applied across frameworks.
• Instrumenting Django with OpenTelemetry — the WSGI counterpart to this walkthrough.
• Instrumenting aiohttp client requests — propagating the context this page creates to the next service.
• Sampling strategies for distributed tracing — choosing the sampler for a high-traffic endpoint.

Frequently Asked Questions