Using contextvars for Request Tracing in Python

You need every log line a request produces to carry the same correlation ID, across await points and offloaded work, without passing that ID through every function call. This guide implements that with contextvars, the only standard-library primitive that stays correct when one OS thread multiplexes many coroutines. It is a focused task within Context Variables and Thread Safety and part of the Python Logging Fundamentals and Structured Data reference.

Prerequisites

contextvars is built in from Python 3.7; the examples target Python 3.11 or newer. Install an ASGI framework for the middleware example:

pip install "starlette>=0.37.0,<1.0.0"

Optionally set a service name so downstream tooling labels the records:

export OTEL_SERVICE_NAME="orders-api"

Implementation

The flow has four steps: declare the slot, seed it at the request boundary, read it in a filter, and reset it on teardown.

1. Declare one module-level ContextVar. Define it once with a None default so every module that imports it shares the same slot and the filter can detect a missing value. Declaring it per request would reset the slot and defeat propagation.

2. Seed the value in ASGI middleware. Middleware owns the whole request lifecycle, so it is the right place to extract the inbound identifier and guarantee the matching reset. Prefer an explicit X-Trace-ID header, then the trace_id segment of a W3C traceparent, and finally a generated UUID so a request is never untraceable.

3. Read the value in a logging filter. The filter runs once per record, reads the active value, and attaches it as a LogRecord attribute. The formatter string then consumes that attribute with a %(trace_id)s placeholder. Reading inside the filter rather than caching it on the handler ensures the value reflects the coroutine actually emitting the record. These patterns build on structured logging with the standard library.

4. Reset on teardown. Reset the variable in a finally block so its value never survives into the next request the worker handles. This matters most under connection reuse: a keep-alive worker that skips the reset starts the next request with the previous request's trace ID still active, and the leak is invisible until two requests share an ID in your dashboards.

The header precedence in step two is deliberate. An explicit X-Trace-ID lets an upstream proxy or test harness pin a known value; the W3C traceparent carries the trace ID propagated by an instrumented caller, sitting in the second hyphen-delimited segment of a 00-<trace-id>-<span-id>-<flags> string; and the UUID fallback guarantees that even an uninstrumented client produces a correlatable request. Preserving an inbound trace ID rather than minting a fresh one is what keeps a single logical operation stitched together as it crosses service boundaries.

The middleware seeds and resets the context:

import contextvars
import uuid
from starlette.middleware.base import BaseHTTPMiddleware
from starlette.requests import Request
from starlette.responses import Response

trace_id_ctx: contextvars.ContextVar[str | None] = contextvars.ContextVar(
    "trace_id", default=None
)


class TraceMiddleware(BaseHTTPMiddleware):
    async def dispatch(self, request: Request, call_next) -> Response:
        # Prefer explicit header, then traceparent's trace-id, then generate.
        traceparent = request.headers.get("traceparent", "")
        parts = traceparent.split("-")
        inbound = request.headers.get("X-Trace-ID") or (
            parts[1] if len(parts) > 1 else None
        )
        token = trace_id_ctx.set(inbound or uuid.uuid4().hex)
        try:
            return await call_next(request)
        finally:
            trace_id_ctx.reset(token)  # runs on success or exception

Expected behaviour: every request resolves trace_id_ctx to a 32-character W3C trace ID or a generated UUID hex, and teardown unconditionally restores the previous value so no identifier bleeds into the next request.

The filter stamps that value onto every record, with a fallback for code that runs outside a request:

import logging

# trace_id_ctx is imported from the module above.


class TraceFilter(logging.Filter):
    def filter(self, record: logging.LogRecord) -> bool:
        record.trace_id = trace_id_ctx.get() or "no-trace"
        return True


logger = logging.getLogger("app")
logger.setLevel(logging.INFO)
_handler = logging.StreamHandler()
_handler.setFormatter(logging.Formatter("[%(trace_id)s] %(message)s"))
_handler.addFilter(TraceFilter())
logger.addHandler(_handler)

logger.info("processing step")

Expected Output:

[no-trace] processing step

Inside a request the same line would read [4bf92f3577b34da6a3ce929d0e0e4736] processing step. The no-trace fallback fires only for code emitting outside any request scope, which makes orphaned log lines easy to spot.

Coroutines spawned during the request inherit the value with no extra work, because asyncio copies the context when it creates a task:

import asyncio
import logging

# trace_id_ctx and the configured logger from above.


async def worker() -> None:
    logging.getLogger("app").info("inside spawned task")


async def main() -> None:
    trace_id_ctx.set("req-abc-123")
    await asyncio.create_task(worker())  # task inherits the context snapshot


if __name__ == "__main__":
    asyncio.run(main())

Expected Output:

[req-abc-123] inside spawned task

For synchronous WSGI servers the same principle holds, but the finally lives in different machinery. Wrap the application callable so the set happens before the request is dispatched and the reset happens in a finally around the response iterator, or register a teardown callback that the framework guarantees to run before the worker returns to the pool. Background workers - schedulers, queue consumers, retry loops - have no request to inherit from, so they should set their own trace ID at the start of each unit of work and reset it when that unit finishes; relying on the no-trace default there produces a stream of uncorrelatable log lines exactly when an incident makes correlation most valuable.

Configuration Options

Verification

Confirm propagation with a test that asserts a spawned task observes the parent's trace ID and that teardown clears it:

import asyncio
import contextvars

trace_id_ctx: contextvars.ContextVar[str | None] = contextvars.ContextVar(
    "trace_id", default=None
)


async def child() -> str | None:
    return trace_id_ctx.get()


async def main() -> None:
    token = trace_id_ctx.set("req-abc-123")
    try:
        seen = await asyncio.create_task(child())
        assert seen == "req-abc-123", seen
    finally:
        trace_id_ctx.reset(token)
    assert trace_id_ctx.get() is None
    print("propagation OK")


if __name__ == "__main__":
    asyncio.run(main())

Expected Output:

propagation OK

For offloaded work, verify that a ThreadPoolExecutor task reads the caller's value only after an explicit copy_context(); without it the worker observes the None default. This is the same boundary discussed in the parent guide on context variables and thread safety.

A cheap continuous check is to assert in CI that no log line escapes with the no-trace sentinel during a request-path test. Pipe the structured output through a parser and fail the test if any record produced inside a simulated request carries the fallback value. Because the fallback only appears outside request scope, its presence inside one is a precise signal that a boundary was crossed without propagation - a missing middleware, a thread pool without copy_context(), or a generator that captured context too early. Treating the sentinel as a test assertion rather than just a log convenience turns propagation bugs into build failures instead of incident-time surprises.

Common Mistakes

Resetting with the wrong token, or no token at all. Calling set() without keeping its return value, or calling reset() with a token created in another context, raises RuntimeError: cannot reset context: token created in a different Context. Capture the token in the same scope as the set and reset it in that scope's finally block; never reset a token across an await into a different task.

Using threading.local() in an async framework. Thread-local storage shares one slot per OS thread, and asyncio runs many requests on one thread, so identifiers interleave and downstream dashboards show mismatched trace IDs. Replace every threading.local() with a ContextVar.

Forgetting to reset on teardown. A request that sets the variable but skips the reset leaves its trace ID active when the worker picks up the next request, so the following request logs under a stale ID. Wrap the mutation in try/finally; for synchronous WSGI servers, register a teardown callback that resets before the worker returns to the pool.

Frequently Asked Questions