Binding Context Variables in structlog

Request-scoped fields such as request_id, user_id, and trace_id must appear on every log line a request emits, even from deep helper functions that never received a logger argument. This page is for backend engineers and SREs running structlog in threaded or asyncio Python services who want to bind those fields once at the edge and have them merged automatically everywhere else. It is a focused task within the structlog architecture and setup reference, part of the Modern Python Logging Libraries Deep Dive guide; the same async-safety concerns are covered from the standard library angle in context variables and thread safety.

Prerequisites

Install structlog with a pinned range. Nothing else is required for the core API: the store is the standard library's contextvars module, and the web example below assumes you already run an ASGI framework.

pip install "structlog>=24.1.0,<26.0.0"

Set no special environment variables. Context values live in process memory and never persist across processes, which is why a task queue boundary needs explicit propagation rather than inheritance — the pattern shown in propagating trace context across Celery tasks. If you are arriving from logging.LoggerAdapter and still porting a configuration across, start with migrating from standard logging to structlog.

Implementation

The mechanism has three moving parts: the merge_contextvars processor in your pipeline, the binding calls (bind_contextvars / unbind_contextvars), and a per-request reset with clear_contextvars. It is worth distinguishing this from logger.bind(). A bound logger threads context through a logger object you must keep passing down the call stack; context variables instead live in a context-local store that any call site reads, so a helper three frames deep that calls structlog.get_logger() fresh still gets the fields. The two compose: bind stable per-logger fields with bind(), and request-scoped fields with bind_contextvars().

1. Add merge_contextvars to the processor chain. This processor copies the current context-local values into the event dictionary before rendering. It must run before the renderer and before any processor that filters on those keys, otherwise the values are not yet in the event dict when that processor runs. Place it early, immediately after the log-level processor. A common subtle bug is putting it after a filtering processor that drops records by request_id — that filter would never see the field.

import structlog

structlog.configure(
    processors=[
        structlog.contextvars.merge_contextvars,   # pull request-scoped vars in first
        structlog.processors.add_log_level,
        structlog.processors.TimeStamper(fmt="iso"),
        structlog.processors.JSONRenderer(),
    ],
    # cache_logger_on_first_use speeds up repeated get_logger() calls
    cache_logger_on_first_use=True,
)

Precedence follows directly from how the processor works: it takes a copy of the context-local dictionary and then updates it with the incoming event dict. Anything passed at the call site — or already bound onto the logger with bind() — therefore wins over a context variable of the same name. That is the behaviour you want: a generic tenant_id bound at the request edge can be overridden for one specific line without disturbing the rest of the request.

2. Bind values once at the request boundary. Call clear_contextvars() first so a reused worker cannot inherit a previous request's fields, then bind_contextvars() with the request-scoped keys. Every logger obtained anywhere in the call stack now emits these fields.

import uuid
import structlog
from structlog.contextvars import bind_contextvars, clear_contextvars

log = structlog.get_logger()


def handle_request(user_id: str) -> None:
    clear_contextvars()                       # drop any stale context first
    bind_contextvars(
        request_id=str(uuid.uuid4()),
        user_id=user_id,
    )
    log.info("request received")
    charge_account()                          # helper logs with no logger passed in


def charge_account() -> None:
    # This function never saw request_id, yet it appears on the line.
    structlog.get_logger().info("account charged", amount=42)

Expected Output:

{"event": "request received", "request_id": "5f1c...e9", "user_id": "u-7781", "level": "info", "timestamp": "2026-06-19T10:15:30.123456Z"}
{"event": "account charged", "amount": 42, "request_id": "5f1c...e9", "user_id": "u-7781", "level": "info", "timestamp": "2026-06-19T10:15:30.124001Z"}

3. Remove individual keys, scope, or reset entirely. Use unbind_contextvars("key") to drop a single field mid-request, clear_contextvars() to wipe everything, and bound_contextvars() as a context manager for nested scopes that must restore the prior state on exit. When you need precise nested undo without a with block, capture the tokens bind_contextvars returns and pass them to reset_contextvars — this restores exactly the keys you changed, leaving sibling fields untouched.

from structlog.contextvars import (
    bind_contextvars,
    unbind_contextvars,
    bound_contextvars,
    reset_contextvars,
)

bind_contextvars(tenant_id="acme")
log.info("tenant work started")
unbind_contextvars("tenant_id")               # subsequent lines drop tenant_id

with bound_contextvars(step="reconcile"):     # auto-resets on block exit
    log.info("inside scoped block")           # carries step=reconcile
log.info("outside scoped block")              # step is gone again

tokens = bind_contextvars(attempt=1)          # keep tokens for precise undo
log.info("retrying")
reset_contextvars(**tokens)                    # restore prior value of attempt only

Under the hood structlog keeps one ContextVar per key, created lazily the first time that key is bound and never destroyed. "Clearing" therefore does not delete the variable, it sets a sentinel value that merge_contextvars skips. That is invisible in normal use, but it explains two things: unbinding a key that was never bound is silent rather than an error, and the number of distinct keys you ever bind is a small, permanent per-process cost — so keep key names bounded, exactly as you would for label cardinality in Prometheus.

4. Wire it into a request lifecycle. In an ASGI app, bind in middleware and clear in a finally block so a crashed handler still leaves a clean context for the next request on that worker. Binding the same fields once at the edge is the structlog equivalent of using contextvars for request tracing with the standard library.

import uuid
import structlog
from structlog.contextvars import bind_contextvars, clear_contextvars

log = structlog.get_logger()


async def context_middleware(request, call_next):
    clear_contextvars()
    bind_contextvars(
        request_id=request.headers.get("x-request-id", str(uuid.uuid4())),
        path=request.url.path,
    )
    try:
        return await call_next(request)
    finally:
        clear_contextvars()                   # guarantee isolation per request

5. Keep the merge in the chain when records route through logging. Many services render through structlog.stdlib.ProcessorFormatter so that third-party libraries writing to the standard library end up in the same JSON stream. In that layout the processors are split in two: the ones structlog runs before handing off, and the foreign_pre_chain applied to records that originated in logging. merge_contextvars belongs in both, or records from libraries such as an ORM or an HTTP client arrive without request_id while your own lines carry it — a mismatch that is easy to miss until you filter on the field in your log backend. The wider handoff is covered in structlog JSON logging in Django.

import logging
import structlog

pre_chain = [
    structlog.contextvars.merge_contextvars,  # foreign records need it too
    structlog.stdlib.add_log_level,
]

formatter = structlog.stdlib.ProcessorFormatter(
    foreign_pre_chain=pre_chain,              # applied to logging.* records
    processor=structlog.processors.JSONRenderer(),
)
handler = logging.StreamHandler()
handler.setFormatter(formatter)
logging.getLogger().addHandler(handler)

Context boundaries: tasks, threads, and processes

The store is context-local, so what matters in production is which boundaries copy a context and which do not. asyncio.create_task and asyncio.gather copy the current context at task creation, so fields bound before the fan-out appear on all child tasks automatically — a request that launches three concurrent sub-tasks sees its request_id on all three — while anything a child binds afterwards stays local to that child. asyncio.to_thread copies the context too, because it runs the callable through contextvars.copy_context().run. A raw loop.run_in_executor call does not, and neither does a bare threading.Thread or a ThreadPoolExecutor.submit: the worker starts in whatever context it happens to hold, which is usually empty. Process boundaries never inherit anything, so a Celery task or a multiprocessing worker must re-bind from data carried in the message itself.

The copy is one-way, and that asymmetry is the single most surprising behaviour in practice. A child task inherits a snapshot; it never writes back. So a field bound inside a handler is invisible to the middleware that called it if that middleware ran the handler in a separate task — which is exactly what Starlette's BaseHTTPMiddleware does. Bind everything the middleware's own response log line needs before awaiting the handler, and treat values discovered inside the handler as data to return, not as context to propagate upwards.

import asyncio
import contextvars
from structlog.contextvars import bind_contextvars


async def fan_out() -> None:
    bind_contextvars(request_id="req-A")      # bound BEFORE the tasks are created
    await asyncio.gather(step("load"), step("verify"))   # both inherit request_id
    await asyncio.to_thread(cpu_work)          # inherits: runs in a copied context


def submit_to_pool(pool) -> None:
    ctx = contextvars.copy_context()           # executors do NOT inherit
    pool.submit(ctx.run, cpu_work)             # so carry the context explicitly

threadlocal vs contextvars

structlog ships two context backends, and the choice is the single most important correctness decision here. The structlog.contextvars API is built on contextvars.ContextVar, whose value is per context rather than per thread. The asyncio event loop copies the current context when it schedules each task, so two coroutines that bind different request_id values never see each other's data even though they run on the same thread and interleave across await points. That property is exactly what makes the contextvars API the only one safe under asyncio.

The legacy structlog.threadlocal API stores state in thread-local storage. Under threads alone it is fine — each thread has its own dictionary — but under asyncio it breaks, because one event-loop thread runs many coroutines, and they all share that single thread-local dictionary. A value bound by request A leaks into request B the moment A awaits and B resumes. Prefer contextvars in all new code; reach for threadlocal only to keep an old synchronous codebase running unchanged, and never mix the two (see the mistakes below).

Configuration options

Verification

Confirm isolation with a short asyncio test: two concurrent tasks bind different request_id values and neither leaks into the other. Each task runs in its own copied context, so the assertions hold. The await asyncio.sleep between bind and read is the crucial part — it forces the loop to interleave the two tasks, which is precisely where a threadlocal backend would corrupt the result.

import asyncio
import structlog
from structlog.contextvars import bind_contextvars, merge_contextvars, clear_contextvars

structlog.configure(processors=[merge_contextvars, structlog.processors.KeyValueRenderer()])


async def worker(rid: str) -> dict:
    clear_contextvars()
    bind_contextvars(request_id=rid)
    await asyncio.sleep(0.01)                  # yield to the other task
    # capture the merged event dict structlog would render
    return merge_contextvars(None, "info", {"event": "done"})


async def main() -> None:
    a, b = await asyncio.gather(worker("req-A"), worker("req-B"))
    assert a["request_id"] == "req-A", a
    assert b["request_id"] == "req-B", b
    print("isolation verified:", a["request_id"], b["request_id"])


asyncio.run(main())

Expected Output:

isolation verified: req-A req-B

If the two values had been swapped or shared, the binding leaked across tasks, indicating you are on the threadlocal backend or forgot the per-request clear_contextvars().

Calling the processor directly, as above, is deliberate: it asserts on the merged event dict without depending on a renderer. In a pytest suite the equivalent one-liner is assert structlog.contextvars.get_contextvars() == {"request_id": "req-A"}, which reads the store itself and stays valid whatever your production processor chain looks like. For an end-to-end check, assert against the emitted line instead — send output to a JSONRenderer and parse it, so that a misplaced merge_contextvars in the real chain is caught rather than bypassed.

Common mistakes

• Error signature: log lines render correctly but never contain request_id, even though the bind call clearly ran. Root cause: merge_contextvars sits after the renderer (or after a filtering processor), so the event dict was already serialized — or filtered on a key that had not been merged yet — by the time the context values were added. Remediation: move merge_contextvars to the top of the processors list, ahead of JSONRenderer and any filter_by_level-style processor, and add it to foreign_pre_chain as well when routing through ProcessorFormatter.

• Error signature: a request's log lines carry the user_id of a previous request, intermittently and only under load. Root cause: ASGI and threaded servers reuse workers, so a context bound in one request survives into the next unless it is reset — and an exception that skipped the cleanup path leaves the stale values in place. Remediation: clear_contextvars() at the start of the request and again in a finally block, mirroring the discipline described in context variables and thread safety.

• Error signature: binds appear to do nothing at all — the store is always empty at render time — in a codebase that also imports structlog.threadlocal. Root cause: the two backends use different storage, so binding with bind_threadlocal while merging with merge_contextvars silently drops every value. Remediation: pick one backend and use its matching bind, merge, and clear helpers consistently; for anything touching asyncio that choice must be contextvars.

• Error signature: after a short nested scope ends, the rest of the request logs without request_id, user_id, or any other correlation field. Root cause: clear_contextvars() was used to undo one temporary key, and it wipes the whole store rather than that key. Remediation: undo a single nested change with reset_contextvars and its saved tokens, or wrap the scope in bound_contextvars, and reserve clear_contextvars for the request boundary only.

• Error signature: background jobs and thread-pool work log without correlation while the request that submitted them logs correctly. Root cause: loop.run_in_executor, ThreadPoolExecutor.submit, and every process boundary start the worker outside the submitting context, which is never inherited. Remediation: submit contextvars.copy_context().run instead of the bare callable, prefer asyncio.to_thread where it fits, and across process or queue boundaries carry the identifiers in the message and re-bind them on arrival, as with adding trace IDs to log records.

Related

• structlog architecture and setup — the parent reference covering the processor chain, bound loggers and configuration order that this page builds on.
• Migrating from standard logging to structlog — how to keep existing logging call sites working while the context store becomes the source of correlation.
• Using contextvars for request tracing — the same request-scoped pattern implemented with the standard library alone.
• structlog JSON logging in Django — where the bound fields end up once records route through ProcessorFormatter and a JSON renderer.
• Loguru vs structlog for microservices — how the two libraries compare once per-request context is a hard requirement.

Frequently Asked Questions