Python Observability

Logging Configuration in Django Settings

Django ships its own logging configuration, applies yours on top of it, and gives several of its loggers behaviour that surprises people the first time they see duplicate 500s or a gigabyte of SQL. This page covers the LOGGING dict as Django actually processes it, the framework loggers worth configuring explicitly, and the gunicorn boundary. It builds on logging configuration and dictConfig, part of the Python logging fundamentals and structured data section.

Three configuration passes, in this order A Django deployment configures logging three times before the first request. First gunicorn starts and configures its own error and access loggers from its command line or logconfig dict; at this point Django's settings have not been imported at all. Second, Django imports settings and applies DEFAULT_LOGGING, which sets up the django logger, the django.server console handler and the mail_admins handler on django.request. Third, Django applies the project's LOGGING dict through dictConfig, which merges on top of the defaults rather than replacing them. Anything the project's dict does not mention keeps whatever the previous pass configured, which is why removing a logger from the dict does not switch it off, and why disable_existing_loggers set to True is disruptive: it silences every logger created during the first two passes. who configures logging, and when 1 · gunicorn starts gunicorn.error gunicorn.access Django settings have not been imported yet 2 · DEFAULT_LOGGING django · django.server mail_admins on django.request Django's own baseline, applied before yours 3 · your LOGGING applied through dictConfig merged, not substituted what you omit keeps whatever pass 2 set the two consequences everybody meets omitting a logger does not disable it — it inherits from pass 2, which is why django.request still mails admins disable_existing_loggers: True silences every logger created in passes 1 and 2, including third-party module-level ones
Three passes, and only the third one is in your settings file. Most "Django ignores my logging config" reports are really pass two still being in effect.

Prerequisites

pip install "django>=5.0,<6.0" \
            "python-json-logger>=2.0.7,<4.0.0" \
            "gunicorn>=21.2.0,<23.0.0"

export DJANGO_LOG_LEVEL=INFO
export DJANGO_SETTINGS_MODULE=project.settings

Implementation

Step 1 — Write the dict with disable_existing_loggers off. Django applies your dict through logging.config.dictConfig, so every rule from configuring logging with dictConfig applies. The one setting to get right first is disable_existing_loggers: at True it switches off every logger created before the config ran, which in Django means Django's own and every third-party library that made a module-level getLogger call.

# settings.py
LOGGING = {
    "version": 1,
    "disable_existing_loggers": False,
    "filters": {
        "request_context": {"()": "observability.filters.RequestContextFilter"},
    },
    "formatters": {
        "json": {
            "()": "pythonjsonlogger.jsonlogger.JsonFormatter",
            "format": "%(asctime)s %(levelname)s %(name)s %(message)s",
        },
    },
    "handlers": {
        "stdout": {
            "class": "logging.StreamHandler",
            "formatter": "json",
            "filters": ["request_context"],
        },
    },
    "root": {"handlers": ["stdout"], "level": os.environ.get("DJANGO_LOG_LEVEL", "INFO")},
}

Step 2 — Configure the framework's loggers explicitly. Four of Django's loggers behave in ways worth deciding about rather than inheriting.

LOGGING["loggers"] = {
    "django": {
        "handlers": ["stdout"],
        "level": "INFO",
        "propagate": False,          # own handler + root handler = duplicates
    },
    "django.request": {
        "handlers": ["stdout"],
        "level": "WARNING",          # 4xx at WARNING, 5xx at ERROR — Django's own mapping
        "propagate": False,
    },
    "django.server": {
        "handlers": ["stdout"],
        "level": "INFO",
        "propagate": False,          # runserver's access log; irrelevant under gunicorn
    },
    "django.db.backends": {
        "handlers": ["stdout"],
        "level": "INFO",             # DEBUG logs every statement, with parameters
        "propagate": False,
    },
}

django.request is the one that produces duplicate 500s: it has handlers of its own from DEFAULT_LOGGING and propagates to the root by default, so a root handler sees it twice. Setting propagate: False alongside an explicit handler list resolves it. django.db.backends at DEBUG emits every SQL statement including bound parameters, which is both an enormous volume problem and a straightforward way to put customer data in a log index — and it only emits at all when settings.DEBUG is True, so it is a development tool, not a production one.

Step 3 — Attach the request filter where every record passes. The filter on the handler covers records from Django, from your apps, and from third-party libraries alike, because they all end up at the same handler. That is one of the few cases where a handler filter is the right placement — it is enriching, not redacting, so running late is harmless.

# observability/filters.py
import logging
from .context import request_id_var         # a contextvars.ContextVar

class RequestContextFilter(logging.Filter):
    def filter(self, record: logging.LogRecord) -> bool:
        rid = request_id_var.get()
        if rid is not None:
            record.request_id = rid
        return True

Set the value in middleware, as early in the stack as possible so it covers the rest of the chain:

# observability/middleware.py
import uuid
from .context import request_id_var

class RequestIDMiddleware:
    def __init__(self, get_response):
        self.get_response = get_response

    def __call__(self, request):
        rid = request.headers.get("X-Request-ID") or uuid.uuid4().hex[:12]
        token = request_id_var.set(rid)
        try:
            response = self.get_response(request)
            response["X-Request-ID"] = rid
            return response
        finally:
            request_id_var.reset(token)     # never leak into the next request on this thread

The reset(token) in finally matters under gunicorn's threaded workers: without it a contextvar set by one request stays visible to the next request that reuses the thread.

Step 4 — Configure gunicorn separately. Gunicorn configures logging before Django's settings are imported, so its gunicorn.error and gunicorn.access loggers are outside your LOGGING dict entirely. Give it the same shape through logconfig_dict.

# gunicorn.conf.py
logconfig_dict = {
    "version": 1,
    "disable_existing_loggers": False,
    "formatters": {"json": {"()": "pythonjsonlogger.jsonlogger.JsonFormatter"}},
    "handlers": {"stdout": {"class": "logging.StreamHandler", "formatter": "json"}},
    "loggers": {
        "gunicorn.error":  {"handlers": ["stdout"], "level": "INFO", "propagate": False},
        "gunicorn.access": {"handlers": ["stdout"], "level": "INFO", "propagate": False},
    },
}
accesslog = "-"     # without this, the access logger emits nothing at all
Django's logger tree and where duplicates come from Django's logger hierarchy drawn as a tree under the root logger. The django logger sits below root and has children: django.request, which handles 4xx at WARNING and 5xx at ERROR and carries a mail_admins handler from Django's defaults; django.server, which is the development server's access log; django.db.backends, which logs SQL only when settings.DEBUG is true; and django.security. Alongside them sit your own application loggers and third-party library loggers, all of which propagate to root by default. The diagram marks the duplication path: a logger that both has its own handler and propagates delivers each record twice, once to its own handler and once to the root handler, which is the cause of every duplicated 500 in a Django log. Setting propagate to False on a logger that carries handlers resolves it. the tree, and the one edge that duplicates records root your stdout handler django the framework's namespace django.request 4xx WARNING · 5xx ERROR has its own handler django.server runserver access log unused under gunicorn django.db.backends every statement at DEBUG and only when DEBUG is on your app loggers no handlers of their own propagate to root — correct the dashed edge is the duplicate: django.request handles the record itself, then propagates it to root, which handles it again
Handlers plus propagation equals duplicates. Either give a logger handlers and set propagate: False, or give it none and let the root do the work.
One request ID, every record on the way through A request ID follows a request across a Django stack. It arrives either as an inbound X-Request-ID header from an upstream proxy or is generated in middleware when absent. The middleware sets it on a context variable before calling the rest of the stack, so every layer beneath sees it: the view's own log records, the ORM's records, a third-party HTTP client's records, and any exception logged by django.request. On the way out the same value is written to the response header so the caller can quote it. The filter attached to the handler is what puts it on each record, which is why records from libraries that know nothing about the request still carry it. A footer marks the two places the chain breaks: a thread or process started without copying the context, and a middleware that fails to reset the variable, which leaks the previous request's id onto the next request served by that thread. r-9f3c — set once, attached to every record by the handler's filter inbound header or generated here middleware contextvar set view · ORM · HTTP client none of them know about it response header echoed back the filter on the stdout handler reads the contextvar, sets record.request_id — for every logger alike two places the chain breaks a thread or process started without copying the context · a middleware that never resets the token, leaking the id into the next request
The library records are the point. Nothing in the ORM or the HTTP client knows a request exists, and both carry the ID anyway, because the filter runs where they all converge.

Configuration options

Setting Where Default Recommended
LOGGING_CONFIG settings logging.config.dictConfig leave alone unless configuring in code
disable_existing_loggers LOGGING — False, always
django.request propagate LOGGING True False when it has handlers
django.db.backends level LOGGING DEBUG INFO in production
django.server LOGGING console irrelevant under gunicorn
logconfig_dict gunicorn.conf.py none mirror the Django handlers
accesslog gunicorn off "-" to emit at all

Verification

python manage.py shell -c "import logging; logging.getLogger('django.request').error('probe')"

Expected Output:

{"asctime": "2026-08-02 12:41:07,332", "levelname": "ERROR",
 "name": "django.request", "message": "probe", "request_id": null}

Then check the real path end to end — one request, one record, one request ID shared between the access log and the application log:

curl -s -H 'X-Request-ID: r-9f3c' localhost:8000/orders/1 > /dev/null

Expected Output:

{"levelname": "INFO", "name": "gunicorn.access", "message": "GET /orders/1 200", "request_id": "r-9f3c"}
{"levelname": "INFO", "name": "orders.views", "message": "order fetched", "request_id": "r-9f3c", "order_id": 1}

If the second line is missing its request ID, the filter is on the wrong handler. If the first line is missing entirely, accesslog is unset.

Common mistakes

Every 500 appears twice

Error signature: two identical ERROR records from django.request per failure. Root cause: the logger has handlers and also propagates to the root, which has its own. Remediation: set propagate: False on any logger you give handlers to.

Third-party libraries go silent after adding LOGGING

Error signature: boto3, celery or urllib3 stop logging entirely the moment the project's config lands. Root cause: disable_existing_loggers defaults to True in dictConfig, and those loggers were created at import time. Remediation: set it to False. There is no situation in a Django project where True is the better choice.

SQL floods the log after a deploy

Error signature: log volume multiplies and each record contains a full statement with parameters. Root cause: django.db.backends at DEBUG with settings.DEBUG accidentally true in a deployed environment. Remediation: pin django.db.backends to INFO in the config, and assert settings.DEBUG is False in a startup check. For query visibility in production, instrument the driver instead — see tracing SQLAlchemy async queries.

Environments and management commands

A Django project runs the same settings module in several very different contexts, and a configuration tuned for the web server is usually wrong for at least two of them.

Development. A human is reading the output, so JSON is a hindrance. Switch the formatter rather than the structure, so the same handler graph is exercised in both environments and a configuration bug shows up locally rather than on deploy.

LOGGING["formatters"]["console"] = {
    "format": "{levelname:8} {name:32} {message}",
    "style": "{",
}
LOGGING["handlers"]["stdout"]["formatter"] = "json" if not DEBUG else "console"

Management commands. manage.py runs the full settings module, so a command inherits the web configuration including any handler that ships to a network sink — which means a data migration run from a laptop can emit records into the production log stream. Two useful habits: give commands their own logger namespace so they are distinguishable, and check sys.argv at configuration time when a command genuinely should log elsewhere.

import sys

IS_MANAGEMENT_COMMAND = len(sys.argv) > 1 and sys.argv[0].endswith("manage.py")
if IS_MANAGEMENT_COMMAND:
    LOGGING["root"]["level"] = os.environ.get("COMMAND_LOG_LEVEL", "INFO")

Tests. Django's test runner does not disable logging, so a suite that exercises error paths writes every one of them to the configured handler. That is noisy and, if a handler does network I/O, slow. Point the handlers at a null sink in the test settings rather than raising levels, which keeps the filters and formatters under test.

Celery workers. They import the same settings and then reconfigure logging themselves unless told not to — the mechanics are in structlog in Celery workers, and the same setup_logging signal applies whether or not structlog is involved.

Context What differs How to handle it
Development a human reads it swap the formatter, keep the graph
Management command may run from anywhere own logger namespace, level from the environment
Test suite volume and speed null handler in test settings
Celery worker Celery configures logging connect setup_logging
Migration runs during deploy keep at INFO; migrations are audit-relevant

Request logging without a middleware

Django emits a record for every request through django.request, but only for 4xx and 5xx — successful requests produce nothing, because the WSGI server owns the access log. That surprises people who expect a per-request record and find only failures.

The two options are to let gunicorn's access log serve that purpose, in which case make sure it uses the same format and carries the request ID, or to emit an access record from your own middleware, which gives you the route template and any business fields the framework does not know about. The second is more work and produces something more useful; both are better than the common third option, which is a logger.info at the top of every view.

Verifying the configuration applies

The single most useful diagnostic in a Django project is a startup dump of the resolved logger table, because Django's three configuration passes make it genuinely hard to predict the outcome by reading the settings file.

# observability/apps.py
from django.apps import AppConfig
import logging

class ObservabilityConfig(AppConfig):
    name = "observability"

    def ready(self) -> None:
        if os.environ.get("LOG_DIAGNOSTIC_DUMP") == "1":
            for name in sorted(logging.Logger.manager.loggerDict):
                log = logging.getLogger(name)
                logging.getLogger("startup").info(
                    "logger", extra={"target": name,
                                     "level": logging.getLevelName(log.getEffectiveLevel()),
                                     "handlers": len(log.handlers),
                                     "propagate": log.propagate},
                )

Run it once in staging after any change to LOGGING, and compare the output against what you intended. It takes a minute and reliably finds the entry that was silently ignored.

Frequently Asked Questions

Does Django's LOGGING replace the default configuration or merge with it?

It merges, in a specific order: Django first applies DEFAULT_LOGGING, then applies your LOGGING dict on top through dictConfig. That is why a django.request entry in your dict changes that logger rather than removing it, and why setting disable_existing_loggers to True is disruptive — it switches off every logger created before your config ran, including the ones Django just configured.

Why do my 500 errors appear twice?

Because django.request has a handler of its own and also propagates to the root, which has another handler. Set propagate to False on django.request if you attach handlers to it, or attach none and let the root handle everything. The double entry is always a propagation question, never a Django bug.

Should I set LOGGING_CONFIG to None?

Only when you want to call dictConfig yourself, typically because you need to build the configuration in code rather than as a literal. Setting it to None means Django will not configure logging at all, so you must call logging.config.dictConfig explicitly at the top of settings — and if you forget, the service runs with the root logger's default lastResort handler and no structure at all.

Why is django.db.backends silent even at DEBUG?

It only emits when settings.DEBUG is True. That is a deliberate guard in Django's own code, not a logging configuration issue: the check happens before the log call. In production the correct source for query timing is database instrumentation rather than SQL logging, which is covered by the SQLAlchemy and psycopg tracing pages.

Related Guides