Python Observability

Loguru vs structlog for Microservices: Exact JSON Config for Distributed Tracing

You are choosing between Loguru and structlog for a microservice that must emit deterministic JSON to a log aggregator while carrying W3C trace context across async boundaries, and you need the exact configuration for each rather than a feature list. This page is for backend engineers and SREs wiring a service that already emits traces and now needs its logs to join them. It is a focused task within Structlog Architecture and Setup, part of the Modern Python Logging Libraries Deep Dive; for the full three-way picture including the standard library, see structlog vs Loguru vs standard library logging.

structlog processor chain vs Loguru sink pipeline structlog moves a record through merge_contextvars, an OpenTelemetry extraction processor and JSONRenderer. Loguru moves it through logger.bind, level filtering and sink routing, then serialization inside a callable sink. Both converge on a single stdout JSON stream carrying a 32-hex trace_id and a 16-hex span_id for the aggregator. structlog transformation in the processor chain Loguru transformation inside the sink merge_contextvars extract_otel_context JSONRenderer() logger.bind(request_id) level filter + sink routing json.dumps in the sink one stdout JSON stream consumed by the aggregator {"level": "info", "trace_id": "…32 hex", "span_id": "…16 hex"}
Both pipelines converge on one stdout JSON stream; they differ in how context is attached and where serialization happens.

Prerequisites

Pin both libraries and an OpenTelemetry API for span-context extraction. The API package is enough — the extraction code below only reads the ambient span, so the SDK and exporter live wherever your service already configures tracing.

pip install \
  "structlog>=24.1.0,<26.0.0" \
  "loguru>=0.7.0,<0.8.0" \
  "opentelemetry-api>=1.30.0,<2.0.0"

No environment variables are required for the snippets below; both write JSON to stdout so a platform collector can ingest them directly. If your service does not yet start spans, every example still runs and emits the all-zero trace identifiers that the W3C specification reserves for "no active trace", which is the correct fallback rather than an omitted field.

Implementation

For a microservice the comparison reduces to three questions a benchmark alone will not answer: how does request context attach and survive across await, how much CPU does each library spend per record under sustained load, and how cleanly does the JSON match your aggregator's expected shape. The walkthrough below answers the first by configuring both libraries to emit an identical flat schema; the overhead and shape differences fall out of how each gets there.

The decisive difference is context propagation. structlog uses Python contextvars natively, so a value bound inside a request handler stays isolated per coroutine and survives await points without being passed explicitly — the mechanics are covered in binding context variables in structlog and, from the standard library angle, in context variables and thread safety. Loguru's logger.bind returns an immutable bound logger you must thread through, or you extract the active span context synchronously at the point of logging. Both must read trace identifiers from opentelemetry.context and format them per the W3C specification: a 32-character hex trace id and a 16-character hex span id. Getting those identifiers onto the record is what lets your aggregator pivot from a log line to the trace it belongs to, the same correlation described in adding trace IDs to log records and produced upstream by context propagation and baggage.

Context across an await: structlog contextvars vs a Loguru bound logger Top timeline: a structlog request binds request_id and trace_id with bind_contextvars, awaits a database call, and a helper coroutine that received no logger still emits both fields. Bottom timeline: a Loguru handler binds request_id onto a returned logger object, awaits the same call, and the helper logs through the module-level logger, so the emitted line carries the trace_id but has lost request_id. structlog — bound context survives the await bind_contextvars() request_id · trace_id await db.fetch() helper() logs no logger passed in time {"event": "handled", "request_id": "req_a", "trace_id": "4bf9…"} context intact Loguru — the bound logger must be passed along log = logger.bind() returns a new logger await db.fetch() helper() logs module-level logger time {"event": "handled", "trace_id": "4bf9…"} request_id lost
The trace id comes from the ambient span either way; only the request-scoped binding differs, and it is the field that silently disappears when a helper coroutine logs through the module-level Loguru logger.

Step 1 — Configure structlog with an OTel-extraction processor. Add a processor that reads the current span and writes 32 and 16 hex digit identifiers into the event dict, then renders JSON. Order matters: context merge first, level and timestamp next, the renderer last.

import structlog
from opentelemetry import trace
from opentelemetry.trace import INVALID_SPAN_CONTEXT


def extract_otel_context(logger, method_name, event_dict):
    """Inject W3C-compliant trace IDs from the active span."""
    ctx = trace.get_current_span().get_span_context()
    if ctx == INVALID_SPAN_CONTEXT or not ctx.is_valid:
        event_dict["trace_id"] = "0" * 32
        event_dict["span_id"] = "0" * 16
    else:
        event_dict["trace_id"] = format(ctx.trace_id, "032x")
        event_dict["span_id"] = format(ctx.span_id, "016x")
    return event_dict  # every path must return the dict


structlog.configure(
    processors=[
        structlog.contextvars.merge_contextvars,
        structlog.processors.add_log_level,
        extract_otel_context,
        structlog.processors.TimeStamper(fmt="iso"),
        structlog.processors.JSONRenderer(),
    ],
    wrapper_class=structlog.make_filtering_bound_logger(20),
    logger_factory=structlog.PrintLoggerFactory(),
    cache_logger_on_first_use=True,  # skip chain rebuild on every call
)

structlog.get_logger().info("request_processed", user_id="usr_992", status=200)

Expected Output:

{"level": "info", "trace_id": "00000000000000000000000000000000", "span_id": "0000000000000000", "timestamp": "2024-01-15T10:30:00.123456Z", "event": "request_processed", "user_id": "usr_992", "status": 200}

Step 2 — Configure Loguru with a custom JSON sink. Loguru has no processor chain, so the equivalent logic lives inside a callable sink. Extract the span context there, build a flat payload, and write to stdout with colorization disabled so no ANSI bytes corrupt the JSON. The sink-authoring rules — buffering, drop policy, error containment — are covered in implementing custom sinks in Loguru.

import sys
import json
from datetime import datetime, timezone
from loguru import logger
from opentelemetry import trace
from opentelemetry.trace import INVALID_SPAN_CONTEXT


def otel_json_sink(message):
    record = message.record
    ctx = trace.get_current_span().get_span_context()
    if ctx == INVALID_SPAN_CONTEXT or not ctx.is_valid:
        trace_id, span_id = "0" * 32, "0" * 16
    else:
        trace_id = format(ctx.trace_id, "032x")
        span_id = format(ctx.span_id, "016x")
    payload = {
        "timestamp": datetime.now(timezone.utc).isoformat(),
        "level": record["level"].name,
        "message": record["message"],
        "trace_id": trace_id,
        "span_id": span_id,
        "module": record["name"],
        "extra": record["extra"],
    }
    sys.stdout.write(json.dumps(payload, default=str) + "\n")
    sys.stdout.flush()


logger.remove()
logger.add(otel_json_sink, level="INFO", colorize=False,
           backtrace=False, diagnose=False)
logger.bind(request_id="req_881").info("cache_hit")

Expected Output:

{"timestamp": "2024-01-15T10:30:00.456789+00:00", "level": "INFO", "message": "cache_hit", "trace_id": "00000000000000000000000000000000", "span_id": "0000000000000000", "module": "__main__", "extra": {"request_id": "req_881"}}

Step 3 — Route third-party library logs through the same schema. In a real microservice most records do not come from your code: uvicorn, SQLAlchemy, Celery, botocore and the HTTP client all emit through the standard library. If they bypass the pipeline you get two record shapes in one stream and your aggregator's parser silently drops half of them. structlog bridges them with ProcessorFormatter, sharing one processor list between native events and foreign records so the two paths cannot drift apart — the same discipline used when migrating from standard logging to structlog.

import logging
import structlog

shared_processors = [
    structlog.contextvars.merge_contextvars,
    structlog.processors.add_log_level,
    extract_otel_context,                     # reused from Step 1
    structlog.processors.TimeStamper(fmt="iso"),
]

structlog.configure(
    processors=shared_processors + [
        structlog.stdlib.ProcessorFormatter.wrap_for_formatter,
    ],
    logger_factory=structlog.stdlib.LoggerFactory(),
    cache_logger_on_first_use=True,
)

formatter = structlog.stdlib.ProcessorFormatter(
    foreign_pre_chain=shared_processors,      # stdlib records get identical fields
    processors=[
        structlog.stdlib.ProcessorFormatter.remove_processors_meta,
        structlog.processors.JSONRenderer(),
    ],
)

handler = logging.StreamHandler()
handler.setFormatter(formatter)
root = logging.getLogger()
root.handlers = [handler]
root.setLevel(logging.INFO)

logging.getLogger("sqlalchemy.engine").info("connection acquired")

Expected Output:

{"event": "connection acquired", "level": "info", "trace_id": "00000000000000000000000000000000", "span_id": "0000000000000000", "timestamp": "2024-01-15T10:30:00.789012Z"}

Loguru solves the same problem from the other direction, with a handler that intercepts standard library records and replays them into the Loguru pipeline. The depth walk is what keeps the originating module and line number correct instead of attributing every record to the handler itself.

import logging
from loguru import logger


class InterceptHandler(logging.Handler):
    def emit(self, record):
        try:
            level = logger.level(record.levelname).name
        except ValueError:
            level = record.levelno          # unmapped custom level
        frame, depth = logging.currentframe(), 2
        while frame and frame.f_code.co_filename == logging.__file__:
            frame = frame.f_back            # skip logging's own frames
            depth += 1
        logger.opt(depth=depth, exception=record.exc_info).log(
            level, record.getMessage()
        )


logging.basicConfig(handlers=[InterceptHandler()], level=logging.INFO, force=True)
logging.getLogger("uvicorn.access").info("GET /orders 200")

Note the level-name mapping: Loguru's default levels do not include the standard library's WARN alias or any custom numeric level you registered with logging.addLevelName, which is why the except ValueError branch falls back to the numeric value. Aligning the two scales before you ship is the same exercise described in mapping Python log levels to syslog.

Step 4 — Measure per-record cost against a null sink. Throughput claims are worth nothing without a measurement on your hardware and your chain, so time both pipelines with everything except the I/O held constant.

import os
import timeit
import structlog
from loguru import logger

devnull = open(os.devnull, "w")

structlog.configure(
    processors=[
        structlog.processors.add_log_level,
        structlog.processors.TimeStamper(fmt="iso"),
        structlog.processors.JSONRenderer(),
    ],
    wrapper_class=structlog.make_filtering_bound_logger(20),
    logger_factory=structlog.PrintLoggerFactory(file=devnull),
    cache_logger_on_first_use=True,
)
slog = structlog.get_logger()

logger.remove()
logger.add(devnull, level="INFO", colorize=False, serialize=True,
           backtrace=False, diagnose=False)

runs = 20_000
s = timeit.timeit(lambda: slog.info("request_processed", user_id="usr_992"),
                  number=runs)
l = timeit.timeit(lambda: logger.bind(user_id="usr_992").info("request_processed"),
                  number=runs)
print(f"structlog {s / runs * 1e6:6.2f} us/record")
print(f"loguru    {l / runs * 1e6:6.2f} us/record")

Expected Output:

structlog   6.14 us/record
loguru     18.37 us/record

Those absolute numbers are representative, not authoritative — they move with CPU, Python version, and how much work your own processors do. The ratio is the durable part: Loguru's record construction, level lookup and sink routing cost several times a structlog render, because structlog is doing little more than dict mutation followed by one json.dumps. At a few hundred records per second the difference is invisible; at tens of thousands, on a service already CPU-bound on request handling, it becomes a real budget line. Note also that logger.bind allocates a new bound logger on every call in the loop above; hoisting it out of the hot path recovers part of the gap.

Configuration options

Dimension structlog Loguru
Async context propagation contextvars, implicit per coroutine bind is explicit; thread it through
Where transformation runs processor chain inside a callable sink
JSON output JSONRenderer() processor serialize=True or callable sink
Trace context injection dedicated processor logic inside the sink
stdlib bridging structlog.stdlib.ProcessorFormatter InterceptHandler on the root logger
Non-blocking dispatch QueueHandler under the stdlib handler enqueue=True on the sink
Sustained-throughput overhead lower; deferred formatting higher; record formatting plus routing
Choosing structlog or Loguru for a microservice Four questions asked in order. If request context must cross await points implicitly, choose structlog for contextvars binding. Otherwise, if uvicorn and SQLAlchemy records need the same JSON schema, choose structlog for its ProcessorFormatter bridge. Otherwise, if the sustained log rate is above roughly ten thousand records per second, choose structlog for lower CPU per record. Otherwise, if in-process rotation, retention and enqueue are needed, choose Loguru. If none apply, Loguru wins on one-line setup and ergonomics. Must request context cross await points implicitly? Do uvicorn / SQLAlchemy records need the same JSON schema? Sustained log rate above ~10k records per second? Need in-process rotation, retention and enqueue? yes yes yes yes no no no no structlog contextvars bind per coroutine structlog ProcessorFormatter bridges stdlib structlog lower CPU per rendered record Loguru rotation, retention, enqueue built in Loguru one-line setup, developer ergonomics
Ask the questions in order and stop at the first yes; only a service that answers no to all four is choosing on ergonomics alone.

The serialization detail that bites teams: structlog's JSONRenderer operates on a plain dict it controls, while Loguru's serialize=True wraps a full record and nests everything under text and record keys, which complicates schema validation downstream. Writing a callable sink, as in Step 2, sidesteps that nesting entirely.

The practical decision rule that follows from the table: choose structlog when the service is async-heavy and you want trace context to propagate implicitly across coroutines without threading a logger object through every call, and when you will route library logs through the same JSON pipeline using its stdlib bridge. Choose Loguru when developer ergonomics and a one-line setup matter more than ambient async context, when most logging is synchronous, or when you value the built-in rotation, retention and enqueue-based background dispatch that structlog leaves to the stdlib handler underneath it — where the equivalent is non-blocking logging with QueueHandler.

Neither choice is reversible cheaply once a fleet of services standardizes on one schema, so settle the JSON field names — trace_id, span_id, severity_number, body versus message — before either library ships to production. Mixed field names across services are the most common cause of broken aggregator queries during a partial rollout, and they are invisible in single-service testing because every query you write during development matches the one shape you have.

Verification

Two properties decide whether this configuration is correct: the trace fields have the exact widths the aggregator expects, and per-request context stays isolated when requests interleave.

First confirm field widths. Both snippets above emit a single line of valid JSON with zero-valued trace identifiers when run outside an active span, which is the correct fallback rather than a missing key.

Exact widths of the trace_id and span_id fields trace_id is a 32 hexadecimal character string, drawn twice as wide as span_id, which is 16 hexadecimal characters. Each is annotated with the length assertion the verification snippet makes. Outside an active span both fields are emitted as all zeros rather than omitted. trace_id 4bf92f3577b34da6a3ce929d0e0e4736 32 hex characters len(trace_id) == 32 span_id 00f067aa0ba902b7 16 hex characters len(span_id) == 16 Outside an active span both fields are all zeros — present at full width, never omitted.
The aggregator matches on width as well as name: a truncated or omitted identifier breaks the pivot from a log line to its trace.
import json

line = '{"trace_id": "00000000000000000000000000000000", "span_id": "0000000000000000"}'
parsed = json.loads(line)
assert len(parsed["trace_id"]) == 32
assert len(parsed["span_id"]) == 16
print("trace field widths valid")

Expected Output:

trace field widths valid

Then prove isolation across await. Run two coroutines that bind different request ids and yield to each other before logging; if the pipeline is correct, neither line carries the other's id.

import asyncio
import structlog

structlog.configure(
    processors=[
        structlog.contextvars.merge_contextvars,
        structlog.processors.JSONRenderer(),
    ],
    logger_factory=structlog.PrintLoggerFactory(),
)
log = structlog.get_logger()


async def handle(request_id):
    structlog.contextvars.clear_contextvars()
    structlog.contextvars.bind_contextvars(request_id=request_id)
    await asyncio.sleep(0.01)          # force interleaving with the other task
    log.info("handled")


async def main():
    await asyncio.gather(handle("req_a"), handle("req_b"))


asyncio.run(main())

Expected Output:

{"request_id": "req_a", "event": "handled"}
{"request_id": "req_b", "event": "handled"}

If both lines show the same id, the context is living somewhere shared rather than in a contextvar. The identical test under Loguru only passes when the bound logger is passed explicitly into handle, which is precisely the difference this page turns on.

Common mistakes

  • json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0). Cause: Loguru colorization left ANSI escape codes in the stream feeding a strict JSON parser. Remediation: set colorize=False on the sink and verify raw JSON output with od -c before blaming the collector.
  • A structlog processor returning None. Symptom: KeyError: trace_id downstream or silently dropped records. Cause: a processor that returns event_dict only inside an else branch returns None on the other path. Remediation: ensure every code path ends with return event_dict.
  • TypeError: Object of type datetime is not JSON serializable. Cause: relying on implicit datetime serialization. Remediation: pass default=str to json.dumps, or coerce with structlog.processors.TimeStamper(fmt="iso") before the renderer.
  • Trace fields present on your own events but absent on library records. Symptom: a trace jumps from your handler line straight to the next service, with the SQLAlchemy or uvicorn lines in between uncorrelated. Cause: the standard library bridge from Step 3 is missing, so foreign records skip the extraction processor entirely. Remediation: share one shared_processors list between structlog.configure and ProcessorFormatter(foreign_pre_chain=...), or install the Loguru InterceptHandler on the root logger with force=True.

Frequently Asked Questions

Which library has lower CPU overhead for JSON serialization in high-throughput microservices?

structlog typically shows lower CPU overhead because its pre-compiled processor chain avoids wrapping a standard logging record and defers formatting to the final render. Loguru adds measurable latency under sustained high request rates due to internal record formatting and sink routing. Measure on your own hardware with your real processor chain before treating the gap as decisive.

How do I propagate OpenTelemetry trace IDs without blocking the async event loop?

Use contextvars with structlog so trace context attaches per coroutine automatically. For Loguru, extract the span context synchronously at the middleware entry point before yielding to the async handler, and never perform blocking I/O inside the logging pipeline.

Can I run both libraries side by side during a gradual migration?

Yes, but route both to the same stdout sink and standardize the JSON schema, including consistent W3C trace context field names. Otherwise your aggregator sees two incompatible record shapes for the same service.

Is bind() in Loguru async-safe the way structlog contextvars are?

logger.bind returns an immutable bound logger, which is safe to pass explicitly, but it does not propagate implicitly across await points the way structlog's contextvars-based binding does. For ambient async context you must thread the bound logger through or patch in the context yourself.

Related Guides