Configuring Python Logging with dictConfig

This walkthrough builds one complete, copy-ready logging setup: a single YAML file with a JSON formatter, a console handler and a rotating file handler, per-module loggers, and a filter, all applied with one logging.config.dictConfig call at startup. It is the concrete companion to the logging configuration and dictConfig overview and part of the Python Logging Fundamentals and Structured Data guide.

logging.config.dictConfig consumes a plain dictionary that describes the entire logging tree at once: formatters, filters, handlers, and loggers, plus a small set of top-level switches. Building that dictionary in YAML rather than in Python keeps operations knobs (levels, file paths, rotation sizes) out of code and lets you ship different files per environment. The schema is versioned by the mandatory version: 1 key; that anchor is what tells dictConfig to use the dictionary schema rather than the older fileConfig INI format.

Prerequisites

# YAML parsing only; the json and logging modules are stdlib.
pip install "pyyaml>=6.0,<7.0"

Optional environment override used below:

export LOG_LEVEL=DEBUG   # falls back to INFO when unset

Implementation

The dictionary schema has five sections you will fill in: formatters, filters, handlers, loggers, and the root logger, sitting under three top-level switches (version, disable_existing_loggers, and optionally incremental). dictConfig resolves them in dependency order — formatters and filters first, then handlers that reference them by name, then loggers that attach handlers — so a typo in a name surfaces as a build-time error rather than a silent miswire.

Step 1 — Write a custom JSON formatter via the () factory. The stdlib Formatter interpolates a format string but does not emit valid JSON when messages contain quotes. A tiny json.dumps-based formatter is safer and needs no third-party package. The () key tells dictConfig to treat the value as a factory: it imports the dotted path and calls it, passing any sibling keys as keyword arguments. That means your formatter can accept configuration straight from YAML. Place it in an importable module, here app/log_json.py.

# app/log_json.py
import json
import logging


class JsonFormatter(logging.Formatter):
    """Serialize each record as a single JSON object."""

    def __init__(self, fields: list[str] | None = None) -> None:
        # fields arrives from the YAML sibling keys via the () factory
        super().__init__()
        self.fields = fields or ["ts", "level", "logger", "msg"]

    def format(self, record: logging.LogRecord) -> str:
        base = {
            "ts": self.formatTime(record, "%Y-%m-%dT%H:%M:%S"),
            "level": record.levelname,
            "logger": record.name,
            "msg": record.getMessage(),
        }
        payload = {k: base[k] for k in self.fields if k in base}
        if record.exc_info:                       # attach traceback if present
            payload["exc"] = self.formatException(record.exc_info)
        return json.dumps(payload, ensure_ascii=False)

Step 2 — Write a filter to drop noisy records. A filter is any callable or object with a filter(record) method; returning falsy discards the record before it reaches the handler. Filters are also configured through the () factory, and a handler attaches them by name. Here a filter suppresses health-check log lines that would otherwise flood the file.

# app/log_filters.py
import logging


class DropHealthChecks(logging.Filter):
    """Discard records whose message mentions the health endpoint."""

    def filter(self, record: logging.LogRecord) -> bool:
        # returning False drops the record; True lets it through
        return "/healthz" not in record.getMessage()

Step 3 — Declare the configuration in YAML. The file names two formatters, one filter, two handlers, and two module loggers. The custom formatter and filter are wired with the () key, which dictConfig instantiates by importing the dotted path. propagate: false on each named logger prevents records from also reaching the root.

# logging.yaml
version: 1
disable_existing_loggers: false

formatters:
  console:
    format: "%(asctime)s %(levelname)-8s %(name)s | %(message)s"
    datefmt: "%Y-%m-%dT%H:%M:%S"
  json:
    "()": app.log_json.JsonFormatter      # custom class, instantiated by dictConfig
    fields: [ts, level, logger, msg]      # passed as a kwarg to __init__

filters:
  no_health:
    "()": app.log_filters.DropHealthChecks

handlers:
  stdout:
    class: logging.StreamHandler
    level: DEBUG
    formatter: console
    stream: ext://sys.stdout
  file:
    class: logging.handlers.RotatingFileHandler
    level: INFO
    formatter: json
    filters: [no_health]                   # attach the filter by name
    filename: app.log
    maxBytes: 10485760                     # 10 MiB per file
    backupCount: 5                         # keep five rolled files

loggers:
  app.api:
    level: DEBUG
    handlers: [stdout, file]
    propagate: false
  app.db:
    level: WARNING                         # quieter than the rest of the app
    handlers: [stdout, file]
    propagate: false

root:
  level: WARNING
  handlers: [stdout]

Step 4 — Load and apply it once at startup. Read the file, deserialize with yaml.safe_load, optionally override the level from the environment, then call dictConfig before importing any module that logs. Reading the YAML in code (rather than via the LOGGING setting some frameworks expose) keeps the load path explicit and testable.

# app/bootstrap.py
import logging.config
import os
from pathlib import Path

import yaml  # pyyaml>=6.0,<7.0


def configure_logging(path: str = "logging.yaml") -> None:
    config = yaml.safe_load(Path(path).read_text(encoding="utf-8"))
    # Optional env-driven override applied before the config is built.
    level = os.environ.get("LOG_LEVEL")
    if level:
        config["loggers"]["app.api"]["level"] = level.upper()
    logging.config.dictConfig(config)

Step 5 — Emit from two modules. Each module fetches its own named logger. Because the names match the YAML keys, app.api runs at DEBUG and app.db is held at WARNING. Per-module loggers are the reason dotted names matter: a logger named app.db.pool with no explicit entry inherits the app.db level through the hierarchy, so you configure a whole subtree with one key.

# app/main.py
from app.bootstrap import configure_logging

configure_logging()                  # MUST run before the loggers are used

import logging

api_log = logging.getLogger("app.api")
db_log = logging.getLogger("app.db")

api_log.debug("request received: GET /orders/42")
api_log.info("order 42 served in 12ms")
api_log.info("GET /healthz ok")                    # dropped by the file filter
db_log.debug("SELECT * FROM orders WHERE id=42")   # suppressed at WARNING
db_log.warning("connection pool at 90% capacity")

incremental vs disable_existing_loggers

Two top-level switches change how dictConfig treats loggers that already exist when it runs.

disable_existing_loggers (default true) disables every logger created before the call that is not explicitly named in the new config. Libraries imported earlier grab their loggers at import time, so the default silently mutes them. Setting it to false leaves those loggers enabled and is almost always what you want in an application.

incremental (default false) changes the whole semantics: when true, dictConfig ignores formatters, filters, and handler/logger construction, and only adjusts the level and propagate of objects that already exist. It is meant for tuning a live configuration — for example raising a single logger to DEBUG at runtime — without tearing down and rebuilding handlers (which would close open file descriptors). You cannot add a new handler or formatter incrementally; attempting to do so is simply skipped. Most startups use a full, non-incremental config and reserve incremental updates for an admin endpoint that bumps levels.

Configuration options

Verification

Run python -m app.main with the default level. The console (via the console formatter) shows readable lines:

Expected Output (stdout):

2026-06-19T12:11:03 DEBUG    app.api | request received: GET /orders/42
2026-06-19T12:11:03 INFO     app.api | order 42 served in 12ms
2026-06-19T12:11:03 INFO     app.api | GET /healthz ok
2026-06-19T12:11:03 WARNING  app.db | connection pool at 90% capacity

The app.db DEBUG line is absent because that logger is pinned to WARNING. The rotating app.log simultaneously receives JSON-formatted records. The file handler is at INFO, so the DEBUG line from app.api is excluded there, and the no_health filter strips the /healthz line that did reach the console:

Expected Output (app.log):

{"ts": "2026-06-19T12:11:03", "level": "INFO", "logger": "app.api", "msg": "order 42 served in 12ms"}
{"ts": "2026-06-19T12:11:03", "level": "WARNING", "logger": "app.db", "msg": "connection pool at 90% capacity"}

Each record appears once per destination, confirming propagate: false prevents the root logger from re-emitting it, and the health-check line is present on the console but absent from the file, confirming the filter is attached to the file handler only.

Common mistakes

Configuring after the loggers are already in use Importing modules that log at import time before calling configure_logging means those early records use the default root handler and the wrong format. Call the bootstrap function as the first statement in your entry point.

Pointing () at a class that cannot be imported A ValueError during dictConfig usually means the dotted path under () is wrong or the module is not on sys.path. Verify the path resolves with a quick python -c "import app.log_json" before wiring it into the YAML.

Forgetting disable_existing_loggers: false Leaving it at the default disables every logger created before the call, silencing dependencies. Keep it false unless you have a deliberate reason to mute pre-existing loggers, as detailed in the logging configuration and dictConfig overview.

Expecting incremental: true to add handlers An incremental config silently ignores any new formatter, filter, or handler and only touches the levels of objects that already exist. If a fresh handler never appears, you left incremental on from an earlier runtime tweak; use a full config to construct objects and reserve incremental updates for level changes.

Frequently Asked Questions