Capturing Unhandled Exceptions and Warnings in Python

Everything on the error path assumes an except block ran. This page covers the failures where none did: a crash on the main thread, a worker thread that vanishes, a task nobody awaited, a DeprecationWarning that goes to stderr and is lost the moment the container is replaced. It is for engineers who have a working log pipeline and want to be sure nothing routes around it. It builds on the exception and traceback logging guide, part of the Python logging fundamentals and structured data section.

There are four separate mechanisms, and installing one of them tells you nothing about the other three.

Prerequisites

Standard library only. The one pin below is for the asyncio-heavy example at the end.

pip install "uvicorn>=0.30.0,<1.0.0"

export PYTHONWARNINGS=default::DeprecationWarning   # Python hides these outside __main__

Implementation

Step 1 — Replace sys.excepthook. It receives the three-element triple directly, which is exactly what exc_info accepts, so forwarding is a one-liner. Delegate KeyboardInterrupt to the original hook: an operator pressing Ctrl-C does not want a CRITICAL record and a stack trace.

import logging
import sys

logger = logging.getLogger("service")

def _excepthook(exc_type, exc_value, exc_tb):
    if issubclass(exc_type, KeyboardInterrupt):
        sys.__excepthook__(exc_type, exc_value, exc_tb)   # keep Ctrl-C quiet
        return
    logger.critical("uncaught exception", exc_info=(exc_type, exc_value, exc_tb))

sys.excepthook = _excepthook

CRITICAL is deliberate. The process is unwinding, nothing downstream will handle this, and the record must survive a level filter someone raised to ERROR during an incident. The reasoning behind that mapping is in log levels and severity mapping.

Step 2 — Replace threading.excepthook. This is a different attribute with a different signature: it receives a single ExceptHookArgs namedtuple carrying exc_type, exc_value, exc_traceback and thread. sys.excepthook is never consulted for a thread, which is why a dead worker thread is one of the most common silent failures in Python services.

import threading

def _thread_excepthook(args: threading.ExceptHookArgs) -> None:
    if issubclass(args.exc_type, SystemExit):
        return
    logger.critical(
        "uncaught exception in thread",
        exc_info=(args.exc_type, args.exc_value, args.exc_traceback),
        extra={"thread_name": args.thread.name if args.thread else "unknown"},
    )

threading.excepthook = _thread_excepthook

Step 3 — Install the asyncio loop handler. An exception on a task you never awaited is stored on the task object and surfaces only when it is collected, through the loop's exception handler. The default handler prints Task exception was never retrieved to stderr. Replace it, and pull the exception out of the context dict.

import asyncio

def _loop_exception_handler(loop: asyncio.AbstractEventLoop, context: dict) -> None:
    exc = context.get("exception")
    logger.error(
        context.get("message", "unhandled error in the event loop"),
        exc_info=exc if exc is not None else False,
        extra={"asyncio_future": repr(context.get("future") or context.get("task"))},
    )

async def main() -> None:
    asyncio.get_running_loop().set_exception_handler(_loop_exception_handler)
    ...

Note the timing: this fires at garbage-collection time, potentially long after the request that created the task has finished. Any request-scoped contextvars are gone by then, which is why the extra above records the future's repr rather than relying on ambient context. The pattern for keeping context alive across task boundaries is in using contextvars for request tracing.

Step 4 — Redirect warnings. logging.captureWarnings(True) monkey-patches warnings.showwarning so anything that passes the filters is emitted as a WARNING record on the py.warnings logger. It does not change which warnings fire — that is still the filter's job, and Python ignores DeprecationWarning outside __main__ by default, which is why a library deprecation can vanish for a year.

import warnings

logging.captureWarnings(True)
warnings.simplefilter("default")                     # let deprecations through
warnings.filterwarnings("ignore", category=ResourceWarning, module="botocore")
logging.getLogger("py.warnings").setLevel(logging.WARNING)

Step 5 — Make sure the last record is flushed. logging.shutdown() is registered with atexit when the logging module is imported, and atexit runs hooks in reverse order of registration. Anything you register after configuring logging therefore runs before shutdown, which is the order you want for a queue drain.

import atexit
import logging.config

logging.config.dictConfig(CONFIG)     # logging imported and configured first
atexit.register(listener.stop)        # registered later → runs earlier → drains first

Configuration options

Verification

Exercise all four paths in one script and confirm four records arrive on the same sink with the same format.

import asyncio, logging, sys, threading, warnings

logging.basicConfig(level=logging.INFO, format="%(levelname)s %(name)s %(message)s")
logger = logging.getLogger("service")
install_hooks()                                # steps 1-4 from above

def boom():
    raise RuntimeError("thread died")

threading.Thread(target=boom, name="worker-1").start()

async def orphan():
    raise ValueError("nobody awaited me")

async def main():
    asyncio.get_running_loop().set_exception_handler(_loop_exception_handler)
    asyncio.create_task(orphan())
    await asyncio.sleep(0.1)

asyncio.run(main())
warnings.warn("old api", DeprecationWarning)
raise SystemError("main thread")

Expected Output:

CRITICAL service uncaught exception in thread
Traceback (most recent call last):
  File "verify.py", line 11, in boom
RuntimeError: thread died
ERROR service Task exception was never retrieved
Traceback (most recent call last):
  File "verify.py", line 16, in orphan
ValueError: nobody awaited me
WARNING py.warnings verify.py:24: DeprecationWarning: old api
CRITICAL service uncaught exception
Traceback (most recent call last):
  File "verify.py", line 25, in <module>
SystemError: main thread

Four failures, four records, one stream. Without the hooks, the first three print to stderr in three different formats and the fourth is the only one your log backend ever sees.

Common mistakes

The hook is installed but never fires

Error signature: crashes still print a bare traceback to stderr. Root cause: something replaced sys.excepthook after you did — pytest, a debugger, sentry_sdk, or a framework's own error handling — or the failure was in a thread. Remediation: install the hooks as the last step of startup, assert sys.excepthook is _excepthook in a smoke test, and install threading.excepthook alongside it. If a chained hook is what you want, capture the previous value and call it after logging.

The final record never reaches the file

Error signature: the log ends mid-incident; the CRITICAL record from the excepthook is missing. Root cause: logging.shutdown() closed the handlers before the queue listener drained, because the drain was registered with atexit before logging was configured. Remediation: register the drain after dictConfig, or call listener.stop() explicitly inside the excepthook before returning.

captureWarnings produces nothing

Error signature: no py.warnings records ever appear, in a codebase that is definitely using deprecated APIs. Root cause: the filter stage, not the routing stage — DeprecationWarning is ignored by default outside __main__. Remediation: set warnings.simplefilter("default") or PYTHONWARNINGS=default::DeprecationWarning, then silence individual noisy modules with filterwarnings rather than turning the category off again.

What the hooks cannot cover

Installing all four hooks closes the Python-level escape routes, and it is worth being explicit about what remains open, because the remaining cases are the ones that produce a container that vanished with no log line at all.

Signals. A SIGKILL — from an out-of-memory kill, from a container runtime that ran out of termination patience, from an operator — cannot be intercepted at all. SIGTERM can be, and should be: a handler that logs the signal and then flushes is the difference between "the pod restarted" and "the pod restarted because the orchestrator asked it to at 03:14". Keep the handler tiny, because it runs between bytecodes on the main thread and anything that acquires a lock there can deadlock against whatever held it when the signal arrived.

import signal

def _on_term(signum, frame):
    logger.warning("received signal", extra={"signal": signal.Signals(signum).name})
    listener.stop()                       # drain, then let the default behaviour run
    raise SystemExit(0)

signal.signal(signal.SIGTERM, _on_term)

Interpreter-level crashes. A segmentation fault in a C extension, a stack overflow, or an abort from a native library ends the process without running any Python code. Nothing in the logging system helps here; faulthandler.enable() does, because it installs a native handler that writes a Python traceback to a file descriptor from inside the signal handler itself.

import faulthandler
import sys

faulthandler.enable(file=sys.stderr, all_threads=True)

Exceptions during interpreter shutdown. Once shutdown begins, module globals are being torn down, and an exception raised in a __del__ method or an atexit hook may find that the objects it needs are already None. Those produce the characteristic Exception ignored in: messages that go to stderr and cannot be routed anywhere, because the logging module itself may already be partially dismantled. The remedy is not to catch them but to avoid needing to: keep __del__ methods free of logging, and do cleanup in explicit close() methods called from a context manager.

Work that never started. A thread that could not be created, a task that was cancelled before running, a subprocess that failed to exec — none of these raise where you can see them. The general remedy is a heartbeat: a component that is supposed to be running should say so periodically, and its absence should alert. That is a monitoring design rather than an exception-handling one, and it is the only thing that catches a component which silently never started.

The practical conclusion is that the four hooks plus a SIGTERM handler and faulthandler cover everything a Python process can report about its own death. What remains is covered by noticing that something stopped reporting, which is a different mechanism and belongs in the metrics pipeline rather than in the logging one.

Related

• Exception and traceback logging in Python — the parent guide: exc_info, chains, and the queue boundary.
• Logging exceptions and tracebacks in Python — turning the captured triple into structured fields.
• Redacting sensitive data in log records — the filter that must run before any of these records ships.
• Log levels and severity mapping — why an uncaught exception is CRITICAL and a warning is not.
• Thread-safe logging in multiprocessing — installing the same hooks inside worker processes.

Frequently Asked Questions