Logging Configuration and dictConfig in Python

Hand-wiring loggers, handlers, and formatters with imperative calls works for a script but collapses into unmaintainable setup code once a service has multiple sinks, per-module levels, and environment-specific routing. logging.config.dictConfig replaces that boilerplate with a single declarative dictionary that describes the entire logging tree and is applied once at startup. This guide is part of the Python Logging and Structured Data reference, and it is the assembly layer for the three subsystems around it: the routing decisions in handler architecture, the serialization contract in formatter configuration, and the severity model in log levels and severity mapping. For a single end-to-end walkthrough that ships a YAML file and an entry point, see configuring Python logging with dictConfig; for the hardened variant tuned to a running service, see how to configure Python logging for production.

The principles below shape every decision in this guide:

• Configure logging exactly once, as early as possible, before any module logs its first record.
• Reference formatters, filters, and handlers by name so each definition lives in exactly one place.
• Set propagate and disable_existing_loggers explicitly rather than inheriting defaults that surprise you.
• Keep secrets and environment-specific values out of static config files by resolving them in code immediately before the call.
• Treat the config as code you test: a broken dictionary fails at startup, but a silently wrong one only shows up when you need the logs.

Prerequisites

dictConfig ships with the standard library, so the core feature needs nothing beyond CPython. Python 3.12 is worth targeting because it added native QueueHandler and QueueListener support to the schema plus logging.getHandlerByName, both used later in this guide. Loading configuration from a YAML file is the only piece that requires a third-party dependency; JSON needs only the stdlib json module.

# 3.12+ unlocks queue handler configuration inside the schema itself
python --version

# Optional: only if config lives in YAML rather than JSON or Python
pip install "pyyaml>=6.0,<7.0"

Two environment variables drive the examples below. Keeping them out of the config file is what lets one container image run in staging and production unchanged.

export LOG_LEVEL=INFO            # applied to the application logger and root
export LOG_FORMAT=json           # 'json' for collectors, anything else for humans
export PYTHONUNBUFFERED=1        # flush stdout per line so a SIGKILL cannot swallow records

Concept and Architecture

logging.config.dictConfig accepts a dictionary conforming to a versioned schema. The version key is mandatory and must equal the integer 1 — it exists so a future revision can change behaviour without breaking existing configs, and any other value raises ValueError. Unrecognized top-level keys are rejected outright, which is deliberate: a typo like handler instead of handlers fails loudly rather than silently configuring nothing.

The remaining sections form a small dependency graph. formatters, filters, and handlers each map a name to a definition. Handlers reference formatters and filters by name, and loggers plus root reference handlers by name. This indirection is the entire point of the schema: you define a JSON serializer once and attach it to three handlers, or define one context filter and share it across every sink, without repeating a single field.

Internally the configurator resolves this graph in dependency order rather than dictionary order — formatters and filters are instantiated first, then handlers (which may be deferred if one handler references another, as a MemoryHandler targeting a file handler does), then loggers, then the root. That ordering is why you can write the sections in any order in YAML and why a handler may name a formatter defined "below" it in the file.

Loggers, the root, and propagation

The loggers section maps logger names — the dotted hierarchy you pass to getLogger — to a configuration with level, handlers, filters, and propagate. The special root key configures the root logger and accepts the same fields minus a name. Because a record travels up the hierarchy to the root unless propagation is stopped, the interaction between a named logger's handlers and the root's handlers is the single most common source of duplicate output, covered in the mistakes section below.

Levels behave differently on loggers and handlers, and mixing them up costs hours. The logger's level decides whether a record is created and dispatched at all; the handler's level decides whether an already-dispatched record is emitted to that sink. A handler at DEBUG attached to a logger at INFO will never see a debug record, because it was discarded before dispatch. The practical rule is to set the logger level to the most verbose severity any sink needs, then use handler levels to restrict each individual destination. The severity semantics behind those choices are covered in log levels and severity mapping.

Lifecycle flags

Two flags govern what happens to the tree that already exists when the call runs. disable_existing_loggers (default True) disables every logger that exists and is not named in the new config. Since libraries call getLogger(__name__) at import time, and imports almost always precede your configuration call, the default silences your dependencies. Setting it to False is the right choice for essentially every real application.

incremental (default False) tells dictConfig to only adjust level and propagate on already-configured loggers and handlers instead of rebuilding the tree. It exists because handler and formatter objects cannot be meaningfully merged into a running configuration — sockets and file descriptors are already open, and a queue may hold undelivered records. Incremental mode is how you safely bump verbosity in a live process; it is not a way to add a sink.

The ext:// and cfg:// prefixes

Two string prefixes turn plain config values into live objects. ext:// resolves a dotted name against importable objects, so ext://sys.stdout becomes the actual stream rather than the literal string. cfg:// references another part of the same configuration dictionary, so cfg://handlers.stdout.formatter reads a value already declared elsewhere, letting a YAML file avoid duplicating a shared path or hostname. Both are resolved by the configurator at apply time, which is what makes a pure-data file capable of describing object wiring at all.

Step-by-Step Implementation

Step 1 — Anchor the schema. Start with the version key and the lifecycle flag. This is the minimal valid config: applied on its own it resets the root logger and, critically, keeps every already-imported library logger alive.

import logging.config

config = {
    "version": 1,                       # required, must be the integer 1
    "disable_existing_loggers": False,  # keep import-time library loggers alive
}
logging.config.dictConfig(config)

Step 2 — Declare formatters. A formatter definition supports format, datefmt, the style placeholder syntax (%, {, or $), and validate, which makes Python check the format string against the chosen style at construction time. Define a human-readable console formatter and a JSON-shaped one so the same process can serve a terminal or a collector.

config["formatters"] = {
    "console": {
        "format": "%(asctime)s %(levelname)-8s %(name)s | %(message)s",
        "datefmt": "%Y-%m-%dT%H:%M:%S",
        "validate": True,                # fail fast on a bad placeholder
    },
    "json": {
        # A compact JSON line; see formatter configuration for a real encoder.
        "format": '{"ts":"%(asctime)s","level":"%(levelname)s",'
                  '"logger":"%(name)s","msg":"%(message)s"}',
        "datefmt": "%Y-%m-%dT%H:%M:%S",
    },
}

The inline JSON string above is fine for a demo but breaks on any message containing a quote or newline. Production services should point the () key at a real encoder subclass instead — the deterministic field projection and escaping rules are in structured logging with the Python standard library.

Step 3 — Add filters. Filters are referenced by both handlers and loggers. The () key marks a custom callable to instantiate; every other key in the definition is passed to that callable as a keyword argument. This is the general "user-defined object" escape hatch in the schema and works identically for formatters and handlers.

config["filters"] = {
    "below_error": {
        "()": "myapp.logging_filters.MaxLevelFilter",  # dotted path to the class
        "max_level": "WARNING",                        # kwarg passed to __init__
    },
    "request_context": {
        "()": "myapp.logging_filters.ContextFilter",   # attaches contextvars data
    },
}

Filter placement matters as much as filter logic. A filter that reads ambient request state must run on the thread that produced the record, so attach it to the logger or to the QueueHandler rather than to a handler owned by a background listener — the reasoning is in context variables and thread safety.

Step 4 — Define handlers. Each handler names its class, an optional level, the formatter to use, a filters list, and any class-specific keyword arguments such as stream, filename, maxBytes, or backupCount. Here INFO-and-below goes to stdout while ERROR-and-above goes to a rotating file.

config["handlers"] = {
    "stdout": {
        "class": "logging.StreamHandler",
        "level": "DEBUG",                     # gated further by the logger level
        "formatter": "console",
        "filters": ["below_error", "request_context"],
        "stream": "ext://sys.stdout",         # resolved to the real stream object
    },
    "error_file": {
        "class": "logging.handlers.RotatingFileHandler",
        "level": "ERROR",
        "formatter": "json",
        "filename": "errors.log",
        "maxBytes": 10_485_760,               # 10 MiB before rollover
        "backupCount": 5,                     # keep five generations
        "encoding": "utf-8",                  # never rely on the platform default
        "delay": True,                        # open the file on first ERROR only
    },
}

Rotation parameters deserve their own thought, especially under multiple processes where two workers rotating the same file will truncate each other's output; sizing and the multi-process hazards are covered in best practices for log rotation in Python.

Step 5 — Attach loggers and the root. Map your application's package logger to both handlers and set propagate: False so records do not also reach the root's handlers. Configure the root as the catch-all for every namespace you do not own, and quiet noisy dependencies by name.

config["loggers"] = {
    "myapp": {
        "level": "INFO",
        "handlers": ["stdout", "error_file"],
        "propagate": False,           # prevents duplicate emission via root
    },
    "sqlalchemy.engine": {
        "level": "WARNING",           # quiet a chatty dependency
        "propagate": True,            # no handlers here; let root emit it
    },
}
config["root"] = {
    "level": "WARNING",
    "handlers": ["stdout"],
}

Note the asymmetry: myapp owns handlers and therefore stops propagation, while sqlalchemy.engine sets only a level and deliberately propagates to the root. Setting a level without attaching handlers is the cleanest way to tune a third-party namespace, because it changes verbosity without duplicating routing.

Step 6 — Apply once at startup. Call dictConfig before anything logs, ideally as the first statement in your entry point or application factory, and before importing modules that log during import.

logging.config.dictConfig(config)
logging.getLogger("myapp").info("logging configured")

Step 7 — Start the listener if you configured a queue. On Python 3.12+ the schema can build a QueueHandler and its QueueListener, but it never starts the listener thread — that is your job, and so is stopping it so buffered records flush at exit.

import atexit
import logging

queue_handler = logging.getHandlerByName("queue")   # 3.12+, name from the config
if queue_handler is not None and queue_handler.listener is not None:
    queue_handler.listener.start()                  # dictConfig builds it, never starts it
    atexit.register(queue_handler.listener.stop)    # drain before the process exits

Loading Configuration from YAML and JSON

A static file keeps routing editable by operators without a code change, and puts logging configuration into review alongside the rest of your deployment manifests. YAML and JSON both deserialize to exactly the dictionary dictConfig consumes, so nothing about the schema changes.

import json
import logging.config
from pathlib import Path

import yaml  # from pyyaml>=6.0,<7.0


def load_logging(path: str) -> None:
    text = Path(path).read_text(encoding="utf-8")
    if path.endswith((".yaml", ".yml")):
        config = yaml.safe_load(text)   # never yaml.load: it can construct objects
    else:
        config = json.loads(text)
    logging.config.dictConfig(config)

The safe_load call is not a stylistic preference. Plain yaml.load can instantiate arbitrary Python objects named in the document, so a config file that an operator or a config-map can edit becomes a code execution path. Combined with the () key — which by design imports and calls a dotted path — a logging config file should be treated as trusted, deployment-controlled input, never as user input.

The durable production shape is a static file for structure plus a small code overlay for environment. Structure (which sinks exist, what each formatter emits, which namespaces are quieted) rarely changes between stages; levels, filenames, and destinations always do.

import os
from pathlib import Path

import yaml

config = yaml.safe_load(Path("logging.yaml").read_text(encoding="utf-8"))

# Overlay only what genuinely varies per environment.
level = os.environ.get("LOG_LEVEL", "INFO").upper()
config["root"]["level"] = level
config["loggers"]["myapp"]["level"] = level
chosen = "json" if os.environ.get("LOG_FORMAT") == "json" else "console"
config["handlers"]["stdout"]["formatter"] = chosen

logging.config.dictConfig(config)

Resolving secrets and hostnames in code rather than in the file has a second benefit beyond hygiene: a missing environment variable produces a KeyError at startup, next to the code that needed it, rather than a syslog handler quietly pointing at the string ${SYSLOG_HOST}.

For the complete file-plus-entry-point version of this pattern, including the YAML document itself and a verification run, follow configuring Python logging with dictConfig.

Configuration Reference

Two entries are easy to miss. The . key sets plain attributes on an object after it is constructed, which is how you configure something that exposes a settable attribute but no constructor argument. And respect_handler_level defaults to False, meaning a queue listener will hand every record to every downstream handler regardless of that handler's level — almost never what you want once a debug sink and an error sink share a queue.

Async and Concurrency Considerations

dictConfig itself is a one-shot setup call, not a hot path, but the handlers it builds run on whatever thread or task emits the record. The default StreamHandler and FileHandler perform synchronous, lock-protected I/O, so under asyncio they block the event loop for the full duration of the write — a stalled stdout pipe or a slow network filesystem becomes latency in every unrelated request handler. The standard remedy is to declare a QueueHandler as the only handler attached to your application logger and let a background listener drain it into the real sinks; the mechanics, sentinel shutdown, and drop policy are in non-blocking logging with QueueHandler.

From Python 3.12 the whole arrangement is expressible in the schema:

config["handlers"]["queue"] = {
    "class": "logging.handlers.QueueHandler",
    "handlers": ["stdout", "error_file"],  # listener drains into these
    "respect_handler_level": True,         # keep per-sink levels after the queue
    # "queue": "ext://myapp.logging_queue.QUEUE"  # optional: a bounded queue
}
config["loggers"]["myapp"]["handlers"] = ["queue"]  # queue is the only attachment

Omitting queue gives you an unbounded queue.Queue, which trades memory for never blocking; naming a bounded queue via ext:// caps memory but means a saturated queue blocks the caller unless you subclass QueueHandler to drop records instead. On Python 3.11 and earlier the listener is not part of the schema at all, so construct and start it in code after dictConfig returns, using logging.getLogger("myapp").handlers[0].queue as the shared queue.

Thread and process boundaries change the picture again. Records crossing to a QueueListener in the same process are never pickled, but a multiprocessing.Queue requires picklable records, and QueueHandler.prepare() strips exc_info for exactly that reason. Configuration itself is not inherited across a fork in every scenario either — a worker spawned with the spawn start method re-imports your module and must call dictConfig again in its own initializer. The full set of boundary rules is in thread-safe and multiprocessing-safe logging.

Finally, an incremental: True reload is applied in-process and is cheap enough to call from a signal handler, but it mutates levels on live objects while other threads are logging. That is safe — level assignment is atomic under the module lock — yet it cannot swap handler objects, so a queue-based handler stays in place across reloads and no records are lost mid-change.

Production Code Examples

The first example assembles a complete, runnable configuration that adapts to environment variables and emits both a console line and a JSON error record.

import logging.config
import os


def build_config() -> dict:
    level = os.environ.get("LOG_LEVEL", "INFO").upper()
    console_fmt = "json" if os.environ.get("LOG_FORMAT") == "json" else "console"
    return {
        "version": 1,
        "disable_existing_loggers": False,      # keep library loggers alive
        "formatters": {
            "console": {
                "format": "%(asctime)s %(levelname)-8s %(name)s | %(message)s",
                "datefmt": "%Y-%m-%dT%H:%M:%S",
            },
            "json": {
                "format": '{"ts":"%(asctime)s","level":"%(levelname)s",'
                          '"logger":"%(name)s","msg":"%(message)s"}',
                "datefmt": "%Y-%m-%dT%H:%M:%S",
            },
        },
        "handlers": {
            "stdout": {
                "class": "logging.StreamHandler",
                "level": level,
                "formatter": console_fmt,       # swapped by LOG_FORMAT
                "stream": "ext://sys.stdout",
            },
            "error_file": {
                "class": "logging.handlers.RotatingFileHandler",
                "level": "ERROR",               # only failures reach the file
                "formatter": "json",
                "filename": "errors.log",
                "maxBytes": 10_485_760,         # 10 MiB
                "backupCount": 5,
                "encoding": "utf-8",
                "delay": True,                  # no empty file on a healthy run
            },
        },
        "loggers": {
            "payment": {
                "level": level,
                "handlers": ["stdout", "error_file"],
                "propagate": False,             # no duplicate via root
            },
            "urllib3": {"level": "WARNING"},    # level only; propagates to root
        },
        "root": {"level": "WARNING", "handlers": ["stdout"]},
    }


if __name__ == "__main__":
    logging.config.dictConfig(build_config())
    log = logging.getLogger("payment")
    log.info("charge accepted")
    log.error("gateway timeout after 3 retries")

Expected Output (console, with LOG_LEVEL=INFO):

2026-06-19T12:04:51 INFO     payment | charge accepted
2026-06-19T12:04:51 ERROR    payment | gateway timeout after 3 retries

The error_file handler simultaneously writes only the ERROR record, as a JSON line, to errors.log:

{"ts":"2026-06-19T12:04:51","level":"ERROR","logger":"payment","msg":"gateway timeout after 3 retries"}

The second example adds live verbosity control: a SIGHUP handler flips the application logger between INFO and DEBUG using an incremental config, without rebuilding handlers or dropping a single buffered record.

import logging
import logging.config
import signal


def set_verbosity(level: str) -> None:
    logging.config.dictConfig({
        "version": 1,
        "incremental": True,        # only level/propagate are applied
        "loggers": {
            "payment": {"level": level},
        },
        "root": {"level": level},
    })
    logging.getLogger("payment").warning("log level switched to %s", level)


def _toggle(signum, frame) -> None:
    current = logging.getLogger("payment").level
    set_verbosity("INFO" if current == logging.DEBUG else "DEBUG")


signal.signal(signal.SIGHUP, _toggle)   # kill -HUP <pid> to toggle

Expected Output (after kill -HUP <pid> twice):

2026-06-19T12:07:03 WARNING  payment | log level switched to DEBUG
2026-06-19T12:07:11 WARNING  payment | log level switched to INFO

Note that the incremental dictionary omits formatters, handlers, and disable_existing_loggers entirely. Including them would not raise — they are simply ignored in incremental mode — which is precisely the trap described next.

Common Mistakes

Silent third-party loggers after configuration Error signature: a dependency that logged normally before your setup goes completely quiet, with no error anywhere. Root cause: disable_existing_loggers defaults to True, disabling every logger created during import — which is nearly all library loggers, since they call getLogger at module scope before your dictConfig runs. Remediation: set disable_existing_loggers: False, or name the libraries explicitly in the loggers section so they are reconfigured rather than disabled.

Duplicate log lines from propagation Error signature: every message appears exactly twice in stdout, often with two different formats. Root cause: a named logger and the root both have handlers attached, and the named logger propagates records upward by default. Remediation: set propagate: False on any logger that owns handlers, or attach handlers only at the root and use named loggers purely to set levels. Adjusting levels per log levels and severity mapping does not fix duplication — only propagation control does.

Treating incremental config as a full rebuild Error signature: a new handler or formatter declared in an incremental: True config never takes effect, and no exception is raised. Root cause: incremental mode only mutates level and propagate on objects that already exist; handler, formatter, and filter definitions are ignored outright. Remediation: use incremental mode purely for live verbosity changes, and run a full non-incremental dictConfig whenever handler topology must change.

Calling dictConfig after the first log record Error signature: early startup messages use the wrong format or land in the wrong sink, then everything after a certain line looks correct. Root cause: modules imported before the config call already grabbed loggers and emitted records under the default lastResort handler at WARNING. Remediation: call dictConfig at the very top of your entry point, before importing modules that log at import time, and never at module scope in a library.

A queue handler whose listener was never started Error signature: the process runs cleanly and logs nothing at all, while memory creeps upward. Root cause: dictConfig on 3.12+ constructs the QueueListener but does not start its thread, so records accumulate in the queue and nobody drains them. Remediation: fetch the handler with logging.getHandlerByName after the config call, start the listener, and register listener.stop with atexit or your framework's shutdown event so buffered records flush at exit.

Configuration silently half-applied after an import error Error signature: a ValueError: Unable to configure handler 'x' at startup, and after catching it the application logs through a partially built tree. Root cause: the configurator instantiates objects as it walks the graph, so a bad class path or a handler naming a nonexistent formatter fails mid-apply rather than atomically. Remediation: never swallow the exception at startup — let the process exit — and add a test that calls dictConfig(build_config()) so a broken dotted path fails in CI rather than in production.

Related

• Python Logging and Structured Data — the parent reference covering the record schema, handler graph, and context propagation this configuration assembles.
• Configuring Python logging with dictConfig — the end-to-end walkthrough with a complete YAML file and entry point.
• Handler architecture for Python logging — the sink topology and queue decoupling that the handlers section wires up.
• Formatter configuration for Python logging — what each handler does with a record once the config has attached a formatter to it.
• Log levels and severity mapping — how to choose the level values that appear throughout this schema.
• Context variables and thread safety in Python logging — where context-reading filters must be attached when a queue sits in the middle.
• Logging configuration in Django settings — the three configuration passes, the framework loggers, and the gunicorn boundary.
• Configuring logging for FastAPI and Uvicorn — one JSON pipeline across uvicorn.error, the access log and your own records.

Frequently Asked Questions