structlog Architecture and Setup

structlog separates the act of capturing a log event from the act of rendering it: a log call produces a dictionary, not a string, and that dictionary flows through an ordered chain of processor functions until a terminal renderer turns it into text or JSON. That inversion is what makes structured logging practical in Python without hand-formatting every message. This guide, part of the Modern Python Logging Libraries Deep Dive, explains the pipeline, gives a production-ready bootstrap configuration, and covers the async-safety and performance details that decide whether structlog helps or hurts under load. It links to the architectural comparison in the standard library versus third-party libraries, to the incremental path in migrating from standard logging to structlog, and to its focused companion on binding context variables in structlog.

The mental model that makes structlog click is that filtering, context injection, redaction, and serialization are all just functions in a list, which is what gives the pipeline its determinism and its testability. A processor takes three arguments and returns a dictionary, so you can call one directly in a unit test and assert on the keys it adds or removes — there is no logging framework to mock and no global state to reset. That testability is not incidental: it is the reason a platform team can package a processor chain as a shared library, write tests that lock its output schema, and roll it out across services knowing every one of them emits the same fields in the same shape. The sections below set that pipeline up correctly and then guard the two places teams get it wrong: async context and per-call cost.

Prerequisites

Pin structlog so the processor API and JSON shape stay stable across deployments, and pin OpenTelemetry if you intend to inject trace context. orjson is optional but worth pinning if you serialize large event dicts on a hot path.

pip install \
  "structlog>=24.1.0,<26.0.0" \
  "opentelemetry-sdk>=1.30.0,<2.0.0" \
  "orjson>=3.9.0,<4.0.0"

The two environment variables the bootstrap below reads are the only knobs the deployment needs; everything else is code:

export LOG_LEVEL=INFO       # INFO in production, DEBUG for an investigation
export LOG_FORMAT=json      # json in containers, console on a developer laptop

The examples assume Python 3.11 or newer so that contextvars propagates correctly across asyncio tasks.

Concept & architecture

structlog replaces the handler-and-formatter graph of the standard library with a flat, ordered list of processors. A log call constructs an event_dict whose first key is the event name, then hands it to each processor in turn. A processor receives the wrapped logger, the called method name ("info", "warning", and so on), and the current dict, and returns a possibly modified dict. Early processors enrich and filter; the final processor is a terminal renderer that returns a string — JSONRenderer for production, ConsoleRenderer for a local TTY. Because every processor is an ordinary function, the chain is deterministic and trivially unit-testable: feed in a dict, assert on what comes out.

Two configuration choices govern behavior more than any other. wrapper_class=make_filtering_bound_logger(logging.INFO) builds a bound logger that discards sub-INFO calls before any processor runs, so a dropped DEBUG line costs almost nothing — the same threshold decision described in log levels and severity mapping, but enforced one layer earlier. logger_factory decides where the final string goes: PrintLoggerFactory() writes straight to stdout, while stdlib.LoggerFactory() routes through the standard library so structlog coexists with libraries that log via logging. That coexistence is the foundation of the incremental cutover covered in migrating from standard logging to structlog.

The choice of factory is more consequential than it first appears. PrintLoggerFactory() is the simplest path and is perfect for a greenfield service that logs only through structlog, but it bypasses the standard library entirely, which means a third-party dependency logging through logging never passes through your structlog chain — its records land in whatever format the root logger happens to have. stdlib.LoggerFactory() is the right default for any real service, because it lets you attach a ProcessorFormatter to the root logger and capture both structlog records and foreign stdlib records through one chain, giving a single schema across your code and everything it imports. When in doubt, route through the standard library: you can always simplify later, but retrofitting dependency capture into a PrintLoggerFactory setup means reconfiguring the whole stack. (If you serialize with orjson, note that it returns bytes, so pair JSONRenderer(serializer=orjson.dumps) with structlog.BytesLoggerFactory() rather than the text factories.)

Immutability is the property that makes the pipeline safe under concurrency. When you call log.bind(key=value), structlog returns a new bound logger carrying the added key rather than mutating the one you called, so two coroutines that bind different values to the same base logger never interfere. The event dict each call constructs is likewise local to that call. This copy-on-bind discipline is why one configured logger can be shared freely across an entire async application without locking, and why structlog avoids the class of bugs that comes from mutating shared logging state — the same hazard examined in context variables and thread safety.

structlog actually offers two layers of context, and distinguishing them prevents a lot of confusion. The first is per-logger context, attached with log = log.bind(component="payments"), which travels with that specific bound logger instance and suits static fields scoped to a module or object. The second is the contextvars-backed layer, set with bind_contextvars, which is global to the current execution context and is the right home for request-scoped identifiers that must reach code holding a different logger instance. The merge_contextvars processor folds the second layer into every event dict; the bound logger carries the first. A common mistake is to reach for bind() with a request_id and then wonder why a helper function's own logger does not see it — the answer is that request_id belonged in the contextvars layer all along. The full lifecycle of that layer, including nested scopes and reset tokens, is the subject of binding context variables in structlog.

Processor ordering is itself a design decision with real consequences. Put merge_contextvars first so every downstream processor and the renderer can see the merged fields. Put any redaction or field-dropping processor before the renderer so secrets never reach serialization. Put format_exc_info before the renderer if you log exceptions, so the traceback becomes a structured field rather than an unhandled object. Put EventRenamer("message") last but one if your log store expects the human-readable text under a message key. The renderer is always last, because once the dict has become a string no further processor can inspect or transform its fields.

Step-by-step implementation

Step 1 — Define the processor chain in order. Build the list in a function rather than inline, so the same chain can be imported by tests and by the ProcessorFormatter used for foreign records. merge_contextvars goes first so request context reaches everything downstream; level and timestamp follow; exception formatting sits immediately before the renderer.

# Tested with structlog>=24.1.0,<26.0.0
import logging
import os
import sys
import structlog


def build_processors() -> list:
    """Shared chain: everything except the terminal renderer."""
    return [
        structlog.contextvars.merge_contextvars,      # request context first
        structlog.processors.add_log_level,           # "level": "info"
        structlog.processors.TimeStamper(fmt="iso", utc=True),
        structlog.processors.StackInfoRenderer(),     # honours stack_info=True
        structlog.processors.format_exc_info,         # traceback -> "exception"
    ]

Step 2 — Choose the terminal renderer from the environment. Keep the chain identical everywhere and swap only the final processor, so a developer sees readable colored output and the very same code in a container emits machine JSON with no divergence in the fields produced.

def terminal_renderer():
    fmt = os.getenv("LOG_FORMAT")
    use_console = fmt == "console" or (fmt is None and sys.stderr.isatty())
    if use_console:
        return structlog.dev.ConsoleRenderer(colors=True)
    return structlog.processors.JSONRenderer()

Step 3 — Configure exactly once at bootstrap. Call structlog.configure() a single time before any worker, thread, or event loop starts, and enable cache_logger_on_first_use=True so the bound logger is built once instead of being rebuilt on every call. make_filtering_bound_logger reads the level from the environment so an investigation only needs a redeploy of the same image with LOG_LEVEL=DEBUG.

def configure_logging() -> None:
    level = getattr(logging, os.getenv("LOG_LEVEL", "INFO").upper(), logging.INFO)
    structlog.configure(
        processors=[*build_processors(), terminal_renderer()],
        wrapper_class=structlog.make_filtering_bound_logger(level),
        cache_logger_on_first_use=True,          # build the bound logger once
        logger_factory=structlog.stdlib.LoggerFactory(),
        context_class=dict,
    )

Step 4 — Bind request-scoped context in middleware. Call clear_contextvars() at the start of each request so nothing survives from the previous one handled by the same task, bind the identifiers you want on every line, and let merge_contextvars do the rest. The following ASGI middleware is framework-agnostic and works under any ASGI server.

import asyncio
import uuid


class LoggingContextMiddleware:
    def __init__(self, app):
        self.app = app

    async def __call__(self, scope, receive, send):
        if scope["type"] != "http":
            return await self.app(scope, receive, send)

        headers = dict(scope.get("headers") or [])
        request_id = headers.get(b"x-request-id", b"").decode() or uuid.uuid4().hex[:12]

        structlog.contextvars.clear_contextvars()      # never inherit the last request
        structlog.contextvars.bind_contextvars(
            request_id=request_id,
            path=scope.get("path"),
            method=scope.get("method"),
        )
        log = structlog.get_logger()
        log.info("request_started")
        try:
            await self.app(scope, receive, send)
        finally:
            log.info("request_finished")
            structlog.contextvars.clear_contextvars()

Expected Output:

{"request_id": "req-8f3a9c", "path": "/api/v1/orders", "method": "GET", "event": "request_started", "level": "info", "timestamp": "2026-06-19T14:22:01.123456Z"}

Step 5 — Log events, not sentences. Inside handlers, pass an event name plus key-value pairs. The renderer builds the final line; you never format a string yourself.

async def process_order(order_id: str, amount: float) -> None:
    log = structlog.get_logger().bind(component="orders")   # per-logger context
    log.info("order_validated", order_id=order_id, amount=amount)
    try:
        await charge(order_id, amount)
    except PaymentError:
        # format_exc_info turns the live exception into a structured field
        log.exception("order_charge_failed", order_id=order_id)

Expected Output:

{"request_id": "req-8f3a9c", "path": "/api/v1/orders", "method": "GET", "component": "orders", "event": "order_charge_failed", "order_id": "ORD-992", "level": "error", "timestamp": "2026-06-19T14:22:01.481902Z", "exception": "Traceback (most recent call last):\n  File \"app/orders.py\", line 41, in process_order\n    await charge(order_id, amount)\nPaymentError: card_declined"}

Note what the last line demonstrates: the contextvars layer (request_id, path, method), the bound-logger layer (component), the call-site keywords (order_id), and the structured traceback all arrive in one flat object that a log store can index. Swap format_exc_info for structlog.processors.dict_tracebacks and the exception value becomes a nested list of frames instead of one escaped string, which is friendlier to backends that can index nested JSON.

Configuration reference

Two entries deserve a note. context_class is where per-logger bound values live; dict is correct on modern Python because it preserves insertion order and is the fastest option, and the historical reason to choose OrderedDict no longer applies. And the renderer row is the only environment-specific line in the whole configuration — everything above it is identical on a laptop and in production, which is precisely what makes local output trustworthy as a preview of what the log store will receive.

Async & concurrency considerations

The reason structlog is safe in asyncio is merge_contextvars. It reads from contextvars, whose values asyncio copies per task, so each coroutine sees its own bound context and never another request's. A thread-local approach would silently leak across coroutines sharing one OS thread, which is the single most common source of mislabeled logs in async services. Bind in middleware, clear at request end, and the context stays correct across every await point without passing identifiers through every function signature.

Thread boundaries are where that guarantee gets subtle. asyncio.to_thread() copies the current context into the worker thread, so logs emitted there still carry the request's fields. loop.run_in_executor() does not — it calls the function with whatever context the pool thread happens to hold, which is usually empty. If you must use an executor, capture contextvars.copy_context() in the caller and invoke the function through ctx.run(...) inside the worker. Across a fork() boundary the situation is different again: child processes inherit the parent's configuration but not its in-flight context, which is why prefork servers should call configure_logging() in a post-fork hook rather than relying on import-time setup alone.

The performance lever is cache_logger_on_first_use. With it disabled, get_logger() rebuilds the bound logger and re-resolves the processor chain on every call, which roughly doubles per-call latency under high QPS. With it enabled, the bound logger is constructed once and reused for the life of the process. Pair that with early level filtering so sub-threshold records are dropped before any processor — including JSON serialization — runs.

The filtering bound logger is where structlog earns its keep under load. make_filtering_bound_logger(logging.INFO) returns a logger whose debug() method is a no-op that returns immediately, before the event dict is even constructed. A hot loop peppered with log.debug(...) calls therefore costs essentially nothing in production while remaining fully active when you drop the level to DEBUG. Contrast that with building the event dict and only checking the level inside a processor: that pays dict construction and argument evaluation for every dropped line. The same wrapper classes also expose awaitable variants — await log.ainfo(...) — which run the chain in a thread executor so a slow renderer cannot stall the event loop; use them only where you have measured a problem, because the executor hop costs more than the chain does for a short chain.

The number of processors matters linearly. Each one is a function call on every surviving record, so a chain of fifteen costs noticeably more than a chain of five. Keep the production chain lean: context merge, level, timestamp, exception formatting, any redaction, and the renderer is usually enough. CallsiteParameterAdder, which attaches filename, line number, and function name, is the classic example of a processor worth having in development and worth removing in production — it inspects the stack on every call. And remember that the most expensive processor is almost always the renderer itself, which is exactly why nothing should reach it that the level filter could have dropped.

Finally, structlog does not own the write. It renders a string and hands it to the factory; if that write blocks — a slow file system, a full pipe, a network sink — the calling coroutine blocks with it. For a non-blocking path, route through stdlib.LoggerFactory() and put a bounded QueueHandler and QueueListener pair behind it so the actual I/O happens on a background thread.

Microservice deployment considerations

In a containerized service the deployment rule is simple: render JSON to stdout and let the platform's log agent ship it. The application stays stateless, the agent owns buffering and retry, and a collector outage never becomes an application latency spike. Keep stdout for normal logs and reserve stderr for the process's own fatal startup errors, so the orchestrator can distinguish a crash-on-boot from routine output. Because the renderer is the only environment-specific stage, the same image runs identically across staging and production with LOG_FORMAT choosing the final step. Writing to files inside a container instead pulls you into log rotation concerns that the platform already solves.

Across a fleet, the highest-leverage move is to package the configuration. Ship a small internal module whose one public function calls structlog.configure() with the agreed chain, and have every service import and call it at startup. That turns schema consistency into a dependency-version problem rather than a copy-paste discipline problem: bump the package, and every service picks up the new fields on its next deploy. It also gives you one place to add a redaction processor or a new semantic-convention field for the whole organization at once. Version the schema explicitly — emit a log_schema field, or at minimum treat a field rename as a breaking change with a deprecation window — because dashboards and alert rules are consumers of that schema just as surely as any API client.

Under prefork servers such as gunicorn, call the shared configure function from the post_fork hook (or an ASGI lifespan startup) so every worker configures itself after the fork, rather than inheriting a partially initialized state. When you are choosing between libraries for a specific service shape rather than configuring one you have already chosen, the head-to-head comparison of structlog, Loguru, and the standard library lays out the trade-offs, and Loguru vs structlog for microservices narrows it to the distributed-tracing case.

Production code examples

Correlating logs with OpenTelemetry spans

This example injects trace context into every structlog record. A small processor reads the active span on each call and attaches its trace_id and span_id, giving log-and-trace correlation without custom middleware and without remembering to bind at every span boundary.

# Tested with structlog>=24.1.0,<26.0.0 and opentelemetry-sdk>=1.30.0,<2.0.0
import asyncio
import logging
import structlog
from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import ConsoleSpanExporter, SimpleSpanProcessor

trace.set_tracer_provider(TracerProvider())
trace.get_tracer_provider().add_span_processor(
    SimpleSpanProcessor(ConsoleSpanExporter())
)
tracer = trace.get_tracer(__name__)


def inject_trace_context(logger, method_name, event_dict):
    """Attach W3C-format ids for the innermost active, sampled span."""
    span = trace.get_current_span()
    ctx = span.get_span_context()
    if ctx.is_valid and ctx.trace_flags.sampled:
        event_dict["trace_id"] = format(ctx.trace_id, "032x")
        event_dict["span_id"] = format(ctx.span_id, "016x")
    return event_dict


structlog.configure(
    processors=[
        structlog.contextvars.merge_contextvars,
        inject_trace_context,                    # runs on every surviving record
        structlog.processors.add_log_level,
        structlog.processors.TimeStamper(fmt="iso", utc=True),
        structlog.processors.JSONRenderer(),
    ],
    wrapper_class=structlog.make_filtering_bound_logger(logging.INFO),
    cache_logger_on_first_use=True,
)


async def handle_order() -> None:
    with tracer.start_as_current_span("process_order"):
        log = structlog.get_logger()
        log.info("order_validated", order_id="ORD-992", amount=45.99)


if __name__ == "__main__":
    asyncio.run(handle_order())

Expected Output:

{"trace_id": "0000000000000000a1b2c3d4e5f60718", "span_id": "1a2b3c4d5e6f0718", "event": "order_validated", "order_id": "ORD-992", "amount": 45.99, "level": "info", "timestamp": "2026-06-19T14:22:03.987654Z"}

There are two ways to wire this correlation, and the choice has consequences. A processor, as above, is harder to forget because it runs on every line regardless of where the call sits, and it always reflects the innermost active span even as spans nest. Explicit bind_contextvars(trace_id=...) at span entry is more visible in the code and puts the ids in the bound context where non-logging code can also read them, but it must be repeated at every span boundary and goes stale when a child span starts. For most services the processor is the better default. Either way, only attach ids for sampled spans — correlating to a span that was never exported leaves a dangling reference the backend cannot resolve.

structlog does not parse the traceparent header itself. The OpenTelemetry instrumentation extracts it and establishes the span context through the same mechanism described in context propagation and baggage, and structlog simply reads whichever span is current when each line is logged. That clean separation — OTel owns context extraction, structlog owns rendering — is why the integration needs no custom middleware and why it survives refactoring that moves where spans start. The equivalent pattern for the standard library is covered in adding trace IDs to log records.

Redacting sensitive fields before serialization

The second example is the processor every production chain eventually grows: a redactor that runs before the renderer, so a secret can never reach the serialized line even if a caller passes it by accident. It also shows the non-blocking sink wired behind the stdlib factory.

# Tested with structlog>=24.1.0,<26.0.0 (standard library only otherwise)
import logging
import logging.handlers
import queue
import sys
import structlog

SENSITIVE = {"password", "authorization", "api_key", "card_number", "set_cookie"}


def redact(logger, method_name, event_dict):
    for key in list(event_dict):
        if key.lower() in SENSITIVE:
            event_dict[key] = "[redacted]"
    return event_dict


def configure() -> None:
    # Bounded queue: back-pressure instead of unbounded memory growth.
    log_queue: queue.Queue = queue.Queue(maxsize=10_000)
    handler = logging.StreamHandler(sys.stdout)
    listener = logging.handlers.QueueListener(log_queue, handler)
    root = logging.getLogger()
    root.handlers = [logging.handlers.QueueHandler(log_queue)]
    root.setLevel(logging.INFO)
    listener.start()                       # writes happen on a background thread

    structlog.configure(
        processors=[
            structlog.contextvars.merge_contextvars,
            structlog.processors.add_log_level,
            structlog.processors.TimeStamper(fmt="iso", utc=True),
            redact,                        # must precede the renderer
            structlog.processors.JSONRenderer(),
        ],
        wrapper_class=structlog.make_filtering_bound_logger(logging.INFO),
        cache_logger_on_first_use=True,
        logger_factory=structlog.stdlib.LoggerFactory(),
    )


if __name__ == "__main__":
    configure()
    structlog.get_logger().info(
        "user_authenticated", user_id="u-8812", password="hunter2"
    )

Expected Output:

{"event": "user_authenticated", "user_id": "u-8812", "password": "[redacted]", "level": "info", "timestamp": "2026-06-19T14:23:11.004512Z"}

Because the redactor is an ordinary function, its test is three lines — call it with a dict containing password and assert the value is [redacted] — and that test is the enforcement mechanism for a compliance rule across every service that imports the shared chain.

Common mistakes

• Error signature: fields appear on some lines and vanish on others, or a ValueError surfaces from inside the chain under load. Root cause: structlog.configure() was called more than once — often once at import and again in a worker startup hook — invalidating cached loggers while records were in flight. Remediation: configure exactly once during bootstrap; use structlog.configure_once() or guard with structlog.is_configured() in code paths that may run twice, and configure inside post_fork rather than at both import and fork time.
• Error signature: log fields cannot be filtered or aggregated in the log store; the whole message sits in one opaque event string. Root cause: the caller preformatted the message, as in log.info(f"user {uid} paid {amount}"), so nothing was ever a separate key. Remediation: pass an event name plus key-value pairs — log.info("user_paid", user_id=uid, amount=amount) — and let the renderer build the final text.
• Error signature: logging CPU scales with request rate and profiles show time inside structlog._config. Root cause: cache_logger_on_first_use is left at its default of False, so the bound logger and processor chain are rebuilt on every get_logger() call. Remediation: set cache_logger_on_first_use=True and call get_logger() once per module or per request rather than per log line.
• Error signature: a request's request_id shows up on log lines belonging to a different request. Root cause: context was bound with bind_contextvars but never cleared, so the next request handled by the same task inherited it. Remediation: call clear_contextvars() at the start of every request in middleware and again in a finally block, or use the bound_contextvars context manager so the scope unwinds automatically.
• Error signature: p99 latency spikes correlate with log volume even though the processor chain is short. Root cause: structlog renders the string but does not own the write, so a slow sink blocks the caller. Remediation: route through stdlib.LoggerFactory() with a bounded QueueHandler/QueueListener pair so the write moves off the request path, and never log to a network sink synchronously.

Related reading

• Modern Python Logging Libraries Deep Dive — the parent guide covering library architecture, configuration, and cost control.
• Binding Context Variables in structlog — the lifecycle of the contextvars layer in detail.
• Migrating from Standard Logging to structlog — an incremental cutover with ProcessorFormatter.
• Loguru vs structlog for Microservices — exact JSON configuration for distributed tracing.
• structlog vs Loguru vs Standard Library Logging — the decision framework before you commit to a chain.
• structlog JSON Logging in Django — applying this configuration inside Django's LOGGING dict.

Frequently Asked Questions