Logging Exception Groups and Notes in Python

asyncio.TaskGroup reports the failure of several concurrent tasks as a single ExceptionGroup, and add_note lets code further up the stack attach context to an exception after it was raised. Both are rendered correctly by Python's traceback formatting and both are routinely flattened into uselessness by structured log formatters, which record the outer type and message and drop everything inside. This page covers keeping every sub-exception searchable and every note visible. It is a task article under exception and traceback logging, part of the Python logging fundamentals and structured data section, and it extends logging exceptions and tracebacks in Python.

Prerequisites

Exception groups and notes are part of the language from Python 3.11 onward. A JSON formatter provides the structured output.

pip install "python-json-logger>=2.0.7,<4.0.0"

Implementation

Step 1 — Log the group with the logging module's exception support. Calling log.exception inside an except block, or passing exc_info=True, gives the formatter the full exception. Python's traceback rendering handles groups correctly: each sub-exception appears with its own traceback, indented under the group, and notes appear after the relevant exception's message. That rendered text is the baseline and should always be recorded.

import asyncio
import logging

log = logging.getLogger("checkout")

async def assemble_quote(order):
    try:
        async with asyncio.TaskGroup() as tg:
            tg.create_task(fetch_inventory(order))
            tg.create_task(fetch_pricing(order))
            tg.create_task(fetch_shipping(order))
    except* Exception:
        log.exception("quote assembly failed", extra={"order_id": order.id})
        raise

Step 2 — Walk the group and record the leaves. The structured record should carry the leaf exceptions — the ones that are not themselves groups — each with its type and message. Groups can nest, so the walk is recursive. With this field present, a query for a specific exception type finds the record even when that exception was one of several inside a group.

def leaf_exceptions(exc: BaseException) -> list[BaseException]:
    if isinstance(exc, BaseExceptionGroup):
        leaves = []
        for sub in exc.exceptions:
            leaves.extend(leaf_exceptions(sub))
        return leaves
    return [exc]

Step 3 — Preserve notes in their own field. Notes live in the exception's __notes__ attribute and are the easiest context to lose: they appear in the rendered traceback, and a formatter that records only type and message drops them from the structured fields. Recording them per leaf keeps the context — which attempt, which request, which shard — queryable.

from pythonjsonlogger import jsonlogger
import collections

class GroupAwareFormatter(jsonlogger.JsonFormatter):
    MAX_LEAVES = 10

    def add_fields(self, target, record, message_dict):
        super().add_fields(target, record, message_dict)
        if not record.exc_info or not record.exc_info[1]:
            return
        exc = record.exc_info[1]
        target["exception"] = self.formatException(record.exc_info)
        target["exception.type"] = type(exc).__name__
        leaves = leaf_exceptions(exc)
        if len(leaves) > 1 or isinstance(exc, BaseExceptionGroup):
            counts = collections.Counter(type(e).__name__ for e in leaves)
            target["exception.leaf_count"] = len(leaves)
            target["exception.leaf_types"] = dict(counts)     # 1. searchable by type
            target["exception.leaves"] = [
                {"type": type(e).__name__, "message": str(e)[:300],
                 "notes": list(getattr(e, "__notes__", []))}   # 2. notes kept
                for e in leaves[: self.MAX_LEAVES]
            ]
        elif getattr(exc, "__notes__", None):
            target["exception.notes"] = list(exc.__notes__)
        target.pop("exc_info", None)

Expected Output: a record that answers "was there a timeout" and "which attempt" as field queries.

{
  "message": "quote assembly failed",
  "order_id": "ord_7",
  "exception.type": "ExceptionGroup",
  "exception.leaf_count": 4,
  "exception.leaf_types": {"TimeoutError": 1, "ConnectionError": 1, "HTTPError": 2},
  "exception.leaves": [
    {"type": "TimeoutError", "message": "inventory read timed out", "notes": []},
    {"type": "ConnectionError", "message": "pricing unreachable", "notes": ["attempt 3/3"]},
    {"type": "HTTPError", "message": "503 from provider A", "notes": []},
    {"type": "HTTPError", "message": "503 from provider B", "notes": []}
  ]
}

Step 4 — Summarise large groups. A fan-out over fifty items that all time out produces a group of fifty identical exceptions. Rendering every traceback produces an enormous record with no information after the first. Counting leaves by type, capping the list of leaves recorded, and keeping the rendered traceback bounded — as the formatter above does — preserves the information at a sensible size.

Step 5 — Handle groups close to where they are created. except* lets code handle each exception type in a group separately. Handling near the TaskGroup — retrying the timeouts, logging the connection errors with their context, re-raising the rest — produces more useful records than letting an unexamined group propagate to a generic top-level handler, where the only possible log line is "something went wrong in several places".

try:
    async with asyncio.TaskGroup() as tg:
        ...
except* TimeoutError as eg:
    log.warning("dependencies timed out", extra={"count": len(eg.exceptions)})
    raise
except* ConnectionError as eg:
    for exc in eg.exceptions:
        exc.add_note(f"order {order.id}")          # context for whoever logs it
    raise

Notes as a context mechanism

Exception notes are worth using deliberately, not only handling when they appear, because they solve a real problem in how context reaches logs.

The usual difficulty is that the code which knows the context — which request, which item in a batch, which retry attempt — is often not the code that logs the exception. A retry wrapper knows the attempt number; a batch processor knows the item identifier; the top-level handler that eventually logs knows neither. Before notes, the options were to log at every level (producing duplicate records), to wrap the exception in a new one (losing its type), or to thread context through every call (intrusive).

Notes let each layer add what it knows to the exception itself as it passes through, and the final handler logs one record that carries all of it. The retry wrapper adds "attempt 3 of 3"; the batch processor adds "item 4812"; the request handler logs once. Combined with the group-aware formatter above, those notes become queryable fields rather than text buried in a traceback.

def with_retries(fn, attempts=3):
    for attempt in range(1, attempts + 1):
        try:
            return fn()
        except ConnectionError as exc:
            if attempt == attempts:
                exc.add_note(f"attempt {attempt}/{attempts}")
                raise

The discipline to keep is that notes carry context, not data. A note naming a request identifier is useful; a note containing a request body reintroduces every concern from redacting sensitive data in log records, and redaction filters need to be applied to the notes field as well as to the message.

Groups in traces as well as logs

An exception group that reaches a span needs the same care, because span exception recording has the same flattening problem.

When a span records an exception, the standard attributes are the exception type, the message and the stack trace. For a group, that means exception.type is ExceptionGroup and the message is the group's own summary — the same loss of information as a naive log formatter. A trace backend searching for spans that failed with TimeoutError will not find it.

Two approaches help. The first is recording each leaf as its own span event, so the span carries one exception event per underlying failure, each with its own type, message and stack. This preserves searchability at the cost of more events on the span, and it is reasonable for groups with a handful of members. The second, better suited to large groups, is to record the group once and set span attributes with the leaf type counts, mirroring the log record's fields so the two signals agree.

The more structural answer is that the tasks inside the TaskGroup should each have their own spans. If each concurrent call is instrumented, each failing task's span records its own exception directly, with the right type, and the group on the parent span is a summary rather than the only record. That is the arrangement described in tracing asyncio gather and task groups, and it makes group handling in the parent a much smaller concern.

Configuration options

Verification

Raise a nested group with notes and check every leaf and note reaches the structured record.

import io, json, logging
stream = io.StringIO()
h = logging.StreamHandler(stream); h.setFormatter(GroupAwareFormatter("%(message)s"))
log = logging.getLogger("eg"); log.handlers = [h]; log.propagate = False

inner = ExceptionGroup("shipping", [RuntimeError("A down"), RuntimeError("B down")])
conn = ConnectionError("pricing unreachable"); conn.add_note("attempt 3/3")
try:
    raise ExceptionGroup("quote", [TimeoutError("inventory"), conn, inner])
except* Exception:
    log.exception("quote failed")

rec = json.loads(stream.getvalue())
print(rec["exception.leaf_types"], rec["exception.leaves"][1]["notes"])

Expected Output:

{'TimeoutError': 1, 'ConnectionError': 1, 'RuntimeError': 2} ['attempt 3/3']

Common mistakes

Recording only the outer type. Error signature: searches for a specific exception miss every occurrence inside a group. Root cause: exception.type is ExceptionGroup. Remediation: record leaf types as a field.

Dropping notes. Error signature: the context that explains a failure present in the traceback text and absent from every field. Root cause: the formatter ignores __notes__. Remediation: include notes per leaf.

Rendering huge groups in full. Error signature: records split or rejected by the pipeline. Root cause: fifty tracebacks in one record. Remediation: summarise by type and cap the leaves.

Not recursing into nested groups. Error signature: leaves counted as one group instead of several exceptions. Root cause: only the top level walked. Remediation: recurse through exceptions.

Logging at every layer instead of adding notes. Error signature: the same failure appearing as three records with different context each. Root cause: each layer logging what it knows. Remediation: add notes on the way up, log once.

Frequently Asked Questions