Loguru Configuration and Sinks: Production-Ready Setup for Python Observability

Backend engineers and SREs adopting Loguru need a configuration that survives high request rates, never blocks a hot path, and emits machine-parseable JSON that joins cleanly to distributed traces. This guide details production-grade sink architecture, serialization overhead, rotation and retention policy, and tracing integration, and it is part of the Modern Python Logging Libraries Deep Dive. When you are still deciding which library to standardize on, weigh the trade-offs in structlog vs Loguru vs standard library logging, and when you need bespoke routing to external backends, follow the patterns in implementing custom sinks in Loguru.

Loguru replaces the stdlib handler hierarchy with a flat list of sinks, each a destination registered through a single logger.add call. Every sink carries its own level, filter, format, rotation, retention, and dispatch policy. That flat model is simpler than the stdlib graph, but it shifts responsibility onto you: synchronous sinks block the caller, unbounded files exhaust disk, and the wrong serialization mode produces JSON your aggregator cannot index.

Prerequisites

Pin Loguru to a compatible range. The callable-sink and OTLP examples assume the standard library plus an OpenTelemetry API for span-context extraction.

pip install \
  "loguru>=0.7.0,<0.8.0" \
  "opentelemetry-api>=1.30.0,<2.0.0" \
  "orjson>=3.10.0,<4.0.0"

Confirm the import before wiring sinks. Loguru exposes a single pre-configured logger singleton; you reconfigure it rather than instantiating your own.

python -c "from loguru import logger; print('loguru ready')"

Expected Output:

loguru ready

Concept and architecture

A sink is any destination Loguru can write to: a file path string, a file-like stream such as sys.stderr, or a callable that receives a Message. Each logger.add returns an integer sink id you can later pass to logger.remove. The dispatcher evaluates the level and filter of every registered sink for each record, formats the record per sink, and writes it. Because the default configuration ships with one stderr sink already attached, the first production step is always to call logger.remove() so no record escapes through a destination you did not configure.

Records carry structured context in a record["extra"] dict, populated by logger.bind or by logger.contextualize. This is where correlation identifiers live. To keep logs joinable to traces you inject trace_id and span_id into extra and ensure your JSON sink promotes them to top-level fields that match the OpenTelemetry logs data model. The W3C Trace Context specification fixes the formats: a 32-character hex trace id and a 16-character hex span id.

Two production defaults matter for every sink. Set diagnose=False and backtrace=False so exception logging does not serialize local variable values into your logs, which both inflates volume and risks leaking secrets. Set enqueue=True so the record is handed to a background thread through an internal multiprocessing-safe queue, decoupling the caller from sink I/O.

The flat-list model has one subtle consequence worth internalizing before you write any sink: because every record is offered to every registered sink, level and filter are not global switches but per-sink gates. A record logged at DEBUG reaches a DEBUG console sink and is silently dropped by an INFO file sink in the same call, with no duplication and no extra cost beyond the level comparison. This is what makes the multi-sink topology in the examples below cheap. It also means there is no notion of handler propagation up a logger hierarchy the way the standard library models it; there is exactly one logger and a list of destinations. If you are migrating from the stdlib hierarchy, that mental shift — from a tree of loggers and handlers to one logger and a flat sink list — is the single largest conceptual change, and the cause of most early confusion when records appear in more or fewer destinations than expected.

Step-by-step implementation

Step 1 — Reset and add a structured file sink. Remove the default sink, then register a rotating JSON file sink. serialize=True makes Loguru emit one JSON object per line; rotation, retention, and compression bound disk usage; enqueue=True moves the write off the request thread.

import sys
from loguru import logger

# Step 1: clear the implicit stderr sink so nothing leaks unconfigured
logger.remove()

# Step 2: structured JSON file sink for the observability pipeline
logger.add(
    "logs/app.jsonl",
    rotation="50 MB",        # roll when the file crosses 50 MB
    retention="30 days",     # prune files older than 30 days
    compression="gz",        # gzip rolled files to save disk
    serialize=True,          # one JSON object per line
    enqueue=True,            # background-thread dispatch
    level="INFO",
    backtrace=False,
    diagnose=False,
)

Step 2 — Add a console sink for local development. Attach a colorized stderr sink at a lower level. In containers you typically drop this and let the JSON sink write to stdout so the platform collector ingests it.

# Step 3: human-readable console sink, development only
logger.add(
    sys.stderr,
    format="<green>{time:YYYY-MM-DD HH:mm:ss}</green> | <level>{level}</level> | {message}",
    level="DEBUG",
    colorize=True,
)

Step 3 — Bind correlation context and emit. Use logger.bind to attach trace identifiers to the record's extra dict. Bound loggers are immutable copies, so this is safe across concurrent requests.

# Step 4: attach W3C trace context and log
logger.bind(
    trace_id="0af7651916cd43dd8448eb211c80319c",
    span_id="b7ad6b7169203331",
).info("Service initialized")

Expected Output (console):

2024-01-15 10:30:00 | INFO | Service initialized

Expected Output (logs/app.jsonl, Loguru's serialize=True envelope):

{"text": "2024-01-15 10:30:00.000 | INFO | __main__:<module>:22 - Service initialized\n", "record": {"elapsed": {"repr": "0:00:00.012345", "seconds": 0.012345}, "exception": null, "extra": {"trace_id": "0af7651916cd43dd8448eb211c80319c", "span_id": "b7ad6b7169203331"}, "file": {"name": "app.py", "path": "/srv/app.py"}, "function": "<module>", "level": {"name": "INFO", "no": 20, "icon": "ℹ️"}, "line": 22, "message": "Service initialized", "module": "__main__", "name": "__main__", "process": {"id": 41, "name": "MainProcess"}, "thread": {"id": 140, "name": "MainThread"}, "time": {"repr": "2024-01-15 10:30:00.000000+00:00", "timestamp": 1705314600.0}}}

Configuration reference

These are the logger.add parameters that govern production behavior. All apply to file and stream sinks; format is ignored by callable sinks, which receive the full Message object instead.

Observability pipeline integration

Configuration is only half the job; the records have to be useful to the system that ingests them. The non-negotiable field is the correlation identifier. A log line without a trace_id is an island — searchable in isolation but impossible to place on a request's timeline. Inject the active span's identifiers into record["extra"] so your JSON sink can promote them to top-level fields, and format them exactly as the W3C Trace Context specification requires: 32 hex characters for the trace id, 16 for the span id, lower-case, zero-padded. The cleanest place to do this is once per request at the framework boundary, binding the identifiers so every subsequent log call inside that request inherits them.

from loguru import logger
from opentelemetry import trace


def bound_logger_for_request():
    """Return a logger pre-bound with the active span's W3C identifiers."""
    ctx = trace.get_current_span().get_span_context()
    if ctx.is_valid:
        return logger.bind(
            trace_id=format(ctx.trace_id, "032x"),
            span_id=format(ctx.span_id, "016x"),
        )
    return logger.bind(trace_id="0" * 32, span_id="0" * 16)


log = bound_logger_for_request()
log.info("checkout_completed", order_id="ord_5521", amount_cents=4999)

Expected Output (controlled JSON sink):

{"timestamp":"2024-01-15T10:30:00+00:00","level":"INFO","message":"checkout_completed","trace_id":"0af7651916cd43dd8448eb211c80319c","span_id":"b7ad6b7169203331","order_id":"ord_5521","amount_cents":4999}

Standardize the JSON schema once and enforce it across every service so Datadog, Splunk, Loki, or Elasticsearch index the same field names everywhere. The field names matter more than the values: an aggregator query that filters on trace_id breaks the moment one service emits traceId instead. Pick names that align with the OpenTelemetry logs data model — severity_number, severity_text, body, trace_id, span_id — and keep arbitrary context under a single attributes object rather than spraying it across the top level, which keeps cardinality predictable for the indexer. Align your WARNING and ERROR thresholds with the SLO alerts that fire on them so the log level itself carries operational meaning rather than being a developer's gut feeling.

Async and concurrency considerations

enqueue=True is the foundation of safe concurrency: it serializes record dispatch through one background thread and a queue that is safe across processes, which removes the GIL contention and interleaved-write corruption you get when multiple threads write a file sink directly. The trade-off is back-pressure. By default the queue is unbounded and the producer never blocks, but if the sink cannot keep up, memory grows. For deeper coverage of bounded queues, drop-on-full policies, and graceful drain on shutdown, see async and non-blocking logging with Loguru enqueue.

Serialization cost lands on the queue thread, but a slow serializer still caps throughput. For high-volume endpoints, replace the default json module with orjson inside a callable sink; it is markedly faster on large payloads and emits bytes you can write directly. Pre-building the context dict once per request and binding it, rather than merging keys on every call, also reduces per-record overhead. If you are comparing this dispatch model against an alternative structured pipeline, review Structlog Architecture and Setup.

A second concurrency subtlety concerns process forks. With enqueue=True the queue and its worker thread are bound to the process that created the sink. Under a pre-fork server such as Gunicorn or uWSGI, sinks added before the fork do not carry a live worker thread into the children, because threads do not survive fork. The robust pattern is to add sinks inside a post-fork hook — Gunicorn's post_fork, for example — so each worker process owns its own queue thread and file handles. Sharing one file sink across forked workers without per-process handles invites interleaved writes and duplicated rotation. If your deployment uses spawn rather than fork, configuration runs fresh in each child and this concern disappears, but you still want one sink set per process rather than an inherited one.

Shutdown is the other place records vanish. When the process exits, any records still sitting in the enqueue queue are lost unless you flush. Call logger.remove() (or logger.complete() in async contexts) during graceful shutdown so the worker drains before the interpreter tears down. In a web framework this belongs in the application's shutdown lifecycle hook, paired with whatever drains your tracing exporter.

Production code examples

Example 1 — Controlled JSON schema with a callable sink

When serialize=True produces more nesting than your aggregator wants, write a callable sink and emit exactly the flat schema you control. Build the JSON with orjson for speed and write to stdout for container collection.

import sys
import orjson
from loguru import logger


def structured_json_sink(message) -> None:
    """Callable sink emitting a flat, controlled JSON schema to stdout."""
    record = message.record
    extra = record["extra"]
    payload = {
        "timestamp": record["time"].isoformat(),
        "level": record["level"].name,
        "severity_number": record["level"].no,
        "message": record["message"],
        "module": record["name"],
        "trace_id": extra.get("trace_id"),
        "span_id": extra.get("span_id"),
    }
    # orjson.dumps returns bytes and serializes datetimes natively
    sys.stdout.buffer.write(orjson.dumps(payload) + b"\n")
    sys.stdout.buffer.flush()


logger.remove()
logger.add(structured_json_sink, level="INFO", enqueue=True,
           backtrace=False, diagnose=False)

logger.bind(
    trace_id="0af7651916cd43dd8448eb211c80319c",
    span_id="b7ad6b7169203331",
).info("Service initialized")

Expected Output:

{"timestamp":"2024-01-15T10:30:00+00:00","level":"INFO","severity_number":20,"message":"Service initialized","module":"__main__","trace_id":"0af7651916cd43dd8448eb211c80319c","span_id":"b7ad6b7169203331"}

Example 2 — Level-routed multi-sink topology

Route errors to a dedicated file while everything informational flows to the main pipeline, using a filter predicate. This keeps an alert-ready error stream separate from the noisy info stream without duplicating records.

from loguru import logger

logger.remove()

# Main pipeline: INFO and above, structured, rotated
logger.add("logs/app.jsonl", level="INFO", serialize=True,
           enqueue=True, rotation="100 MB", retention="14 days",
           compression="gz", backtrace=False, diagnose=False)

# Dedicated error stream: only WARNING and above
logger.add("logs/errors.jsonl", level="WARNING", serialize=True,
           enqueue=True, rotation="00:00", retention="30 days",
           compression="gz", backtrace=False, diagnose=False)

logger.bind(request_id="req_881").info("cache_hit", )
logger.bind(request_id="req_882").error("payment_gateway_timeout")

Expected Output (logs/errors.jsonl, only the error record):

{"text": "... | ERROR | __main__:<module>:18 - payment_gateway_timeout\n", "record": {"extra": {"request_id": "req_882"}, "level": {"name": "ERROR", "no": 40}, "message": "payment_gateway_timeout", "name": "__main__"}}

Both sinks see both records, but the level="WARNING" gate on the error sink drops the cache_hit info record before it is written, so errors.jsonl contains only the timeout. The info record still lands in app.jsonl. This is the level-routing pattern: one logger call, two destinations, each filtering independently. To split by category rather than severity — for example, an audit stream that captures only records bound with an audit=True key — replace the level gate with a filter callable that inspects record["extra"]. When the routing logic outgrows a simple predicate and you need to forward records to a network backend with retries and a dead-letter path, graduate to a callable sink as described in implementing custom sinks in Loguru.

Common mistakes

• Enabling diagnose=True and backtrace=True in production. Symptom: log lines balloon with rendered local variables and secrets appear in tracebacks. Root cause: Loguru's enhanced exception formatting is on by default for the implicit sink. Remediation: set both to False on every production sink and restrict the verbose mode to local development.
• Omitting enqueue=True in a multi-threaded service. Symptom: request latency spikes and occasionally interleaved, corrupted lines in a file sink under concurrent load. Root cause: synchronous writes contend on the GIL and the file handle. Remediation: set enqueue=True so a single background thread serializes all writes.
• Passing format= to a callable sink and seeing it ignored. Symptom: your format string has no effect. Root cause: format applies only to string and stream sinks; a callable receives the full Message. Remediation: format inside the callable using message.record.
• Configuring rotation without retention. Symptom: disk fills with rolled files until the volume is exhausted. Root cause: rotation rolls but never prunes. Remediation: always pair rotation with retention or ship and delete via an external agent.
• Using the stdlib json module on a hot path. Symptom: the enqueue queue grows and dispatch lags under load. Root cause: json.dumps is CPU-bound on large payloads. Remediation: serialize with orjson inside a callable sink and write the resulting bytes directly.

Frequently Asked Questions