Python Observability

Defining Custom Log Levels in Python

Python's logging module supports adding levels beyond the standard five, and the facility is used far more often than it should be. A TRACE level below DEBUG is a genuine severity distinction and a reasonable addition; an AUDIT or SECURITY level is a distinction of kind masquerading as severity, and it causes problems for every consumer downstream. This page covers when a custom level earns its place, how to register one properly, and the mapping that keeps it intelligible to syslog, OpenTelemetry and the log store. It is a task article under log levels and severity mapping, part of the Python logging fundamentals and structured data section.

Severity is a ladder; kind is a different axis The standard Python log levels are drawn as a vertical ladder from DEBUG at ten through INFO, WARNING and ERROR to CRITICAL at fifty. A custom TRACE level at five is added below DEBUG; it fits naturally because it is simply more verbose than debug, and every comparison — is this record at or above the configured level — continues to work. Beside the ladder, a second axis shows record kinds: operational, security and audit. An audit event can be informational in severity while being audit in kind, and a security event can be a warning or an error. Forcing kind onto the severity ladder as a level called AUDIT at, say, twenty-five would mean an audit record is suppressed whenever the level is set to warning, which is exactly wrong. The note records that kind belongs in a field or a dedicated logger, where it is independent of level filtering. two independent questions: how severe, and what kind severity CRITICAL 50 ERROR 40 WARNING 30 INFO 20 DEBUG 10 TRACE 5 — fits just more verbose; comparisons still work kind — a separate axis operational security — WARNING or ERROR audit — usually INFO an AUDIT level at 25 would be suppressed whenever the level is set to WARNING kind belongs in a field or a dedicated logger, where level filtering cannot touch it
A custom level is appropriate when it extends the severity ladder. When it describes a kind of record, it breaks the one property levels exist for.

Prerequisites

The standard library is all that is needed.

pip install "python-json-logger>=2.0.7,<4.0.0"

Implementation

Step 1 — Decide whether the distinction is severity or kind. The test is simple: should raising the configured level suppress these records? For extremely verbose diagnostic output, yes — that is severity, and a level is correct. For audit events, security events or business events, no — an operator raising the level to quieten a noisy service must not suppress them. That is kind, and it belongs in a field such as record.class or in a dedicated logger with its own configuration, as in audit logging in Python applications.

Step 2 — Choose a number between existing levels. Level filtering is a numeric comparison, so the number determines behaviour. TRACE at 5 is below DEBUG, so it is off whenever debug is off and on only when explicitly requested. NOTICE at 25 sits between INFO and WARNING for events that are normal but noteworthy. Numbers that collide with standard levels, or that sit outside the zero-to-fifty range, produce surprising filtering.

import logging

TRACE = 5
NOTICE = 25

Step 3 — Register the names. Without registration, a record at level 5 renders as Level 5, which is unreadable in output and unrecognisable to any consumer. addLevelName fixes the rendering and makes the name resolvable in configuration, so "level": "TRACE" works in a dictConfig.

logging.addLevelName(TRACE, "TRACE")
logging.addLevelName(NOTICE, "NOTICE")

Step 4 — Expose the level without patching the Logger class. The common recipe adds a trace method to logging.Logger. That modifies the class for every library in the process, conflicts when two libraries do the same thing differently, and makes behaviour depend on import order. A small module-level helper, or calling log.log(TRACE, …) directly, gives the same ergonomics with no global side effect.

# mylogging.py
import logging

TRACE = 5
logging.addLevelName(TRACE, "TRACE")

def trace(logger: logging.Logger, msg: str, *args, **kwargs) -> None:
    if logger.isEnabledFor(TRACE):
        logger._log(TRACE, msg, args, **kwargs)

# usage
log = logging.getLogger("wire")
trace(log, "frame received %d bytes", len(frame))

Step 5 — Map the level for every downstream consumer. Syslog has its own severity numbers, OpenTelemetry has a severity scale from 1 to 24, and a log store's dashboards expect a known set of names. A custom level unmapped at any of these boundaries arrives as something unrecognised. The formatter is the place to translate, and the mapping should be explicit for every custom level. The standard-level mappings are covered in mapping Python log levels to syslog.

# Python level -> (OpenTelemetry severity number, OpenTelemetry severity text, syslog severity)
SEVERITY = {
    TRACE:            (1,  "TRACE", 7),    # debug-level in syslog; there is no lower
    logging.DEBUG:    (5,  "DEBUG", 7),
    logging.INFO:     (9,  "INFO",  6),
    NOTICE:           (10, "INFO2", 5),    # syslog "notice" exists and fits exactly
    logging.WARNING:  (13, "WARN",  4),
    logging.ERROR:    (17, "ERROR", 3),
    logging.CRITICAL: (21, "FATAL", 2),
}

class SeverityMappingFormatter(jsonlogger.JsonFormatter):
    def add_fields(self, target, record, message_dict):
        super().add_fields(target, record, message_dict)
        number, text, syslog = SEVERITY.get(record.levelno, (9, "INFO", 6))
        target["severity_number"] = number
        target["severity_text"] = text
        target["level"] = record.levelname

Expected Output: a custom level that every consumer can interpret.

{"message": "frame received 1482 bytes", "level": "TRACE", "severity_number": 1, "severity_text": "TRACE"}
Unmapped levels arrive as noise A record at a custom TRACE level of five travels from a Python process through a collector to a log store. Without explicit mapping, the formatter emits the level as the string Level 5, the collector cannot map it to a standard severity and assigns an unknown or default severity, and the store's dashboards — which group by a fixed set of severity names — place it in an unrecognised bucket or treat it as informational. Filters set to show only errors may even include it, depending on how the unknown value is sorted. With explicit mapping, the formatter emits the level name TRACE together with the OpenTelemetry severity number one and severity text TRACE, the collector passes them through, and the store groups and filters it correctly as the least severe category. The note records that the mapping is a few lines in one place and that every custom level needs an entry. a TRACE record on its way to the store unmapped "levelname": "Level 5" collector: unknown severity store: odd bucket, bad filters mapped TRACE · severity_number 1 collector: passes through store: least severe, correct every consumer has its own severity vocabulary syslog 0–7 · OpenTelemetry 1–24 · the store's fixed set of names the formatter is the one place that can translate, and it needs an entry per custom level
Python knows what level 5 means because it was told. Nothing downstream was, unless the formatter translates.

Why most proposed custom levels should be fields

Requests for custom levels usually come from a real need expressed in the wrong mechanism, and it helps to recognise the common ones.

"We need an AUDIT level." The need is for audit records to be separately routable, never suppressed and retained differently. A level achieves none of those: it is filtered by the same threshold as everything else and routed through the same handlers. A dedicated logger with propagation disabled achieves all three.

"We need a SECURITY level." The need is for security-relevant records to be findable and alertable. Their severity varies — a failed login is a warning, a detected intrusion is critical — so collapsing them into one level loses the severity information that should drive alert priority. A field such as record.class: security alongside the real severity preserves both.

"We need a BUSINESS or METRIC level." The need is for business events to be countable. These are informational in severity and distinct in kind; an event field with a bounded vocabulary makes them queryable, and a counter metric usually serves the counting need better than logs at all.

"We need a SUCCESS level." The need is usually for positive outcomes to stand out when reading logs. That is a presentation concern and an outcome field — outcome: success — serves it without inventing a severity for good news.

The pattern is consistent: when a proposed level describes what a record is about rather than how urgently it needs attention, it belongs in a field. The genuine severity extensions — TRACE, occasionally NOTICE — are few, and they extend the ladder rather than bending it.

Custom levels across libraries and the fleet

A custom level defined in one service is a local convention, and two problems appear as soon as it spreads beyond that service.

The first is collision. Two libraries in the same process that each register a level at number 5 with different names — TRACE in one, VERBOSE in another — silently overwrite each other, because the level-name registry is global. Whichever registers last wins, and records from the other library are rendered with the wrong name. Libraries should therefore not register custom levels at all; a library that wants extremely verbose output should log at DEBUG under a dedicated child logger, which an application can enable or silence precisely. Custom levels are an application decision, made once, in the application's own logging setup.

The second is inconsistency across a fleet. If some services use TRACE at 5, others at 1, and others do not use it at all, a dashboard or alert rule that filters by severity behaves differently per service. Standardising the set of custom levels — ideally none, or TRACE alone — in a shared logging package, together with its severity mapping, keeps the meaning of a level consistent everywhere it appears. The same shared package is the natural home for the schema conventions in designing a log schema for a service fleet.

There is also a cost to consider on the reading side. Every additional level is one more value engineers must understand when filtering, one more entry in every dashboard's severity breakdown, and one more mapping to keep correct. The standard five cover the overwhelming majority of needs, and each addition should clear a higher bar than "it would be convenient".

Where custom levels break A table of four consumers of log levels and what each does with a custom level such as TRACE at 5 or NOTICE at 25. The standard library handles it correctly once addLevelName is called. JSON formatters emit the level name, so the custom name appears. Log backends and shippers map known level names to severities; an unknown name is often indexed as unknown or treated as INFO. OpenTelemetry's severity mapping converts level numbers, so TRACE at 5 maps to TRACE severity and NOTICE at 25 falls between INFO and WARN. Alert rules filtering on level=ERROR or above ignore custom names they do not know. The note says every consumer must learn the new level, which is why a field is usually better. consumer with TRACE=5 or NOTICE=25 standard library fine once addLevelName is called JSON formatter emits the custom name shipper and backend unknown name → unknown or INFO OpenTelemetry severity mapped by number; NOTICE sits between alert rules on level ignore names they do not know every consumer has to learn the new level — usually a field is simpler
A custom level works in Python immediately and nowhere else until each downstream consumer is taught it.

Configuration options

Level Number Syslog OpenTelemetry Justified when
TRACE 5 7 debug 1 TRACE output too verbose even for debug
DEBUG 10 7 debug 5 DEBUG standard
INFO 20 6 info 9 INFO standard
NOTICE 25 5 notice 10 INFO2 normal but noteworthy, if the team will use it
WARNING 30 4 warning 13 WARN standard
ERROR 40 3 err 17 ERROR standard
CRITICAL 50 2 crit 21 FATAL standard

Verification

Confirm the level filters correctly and renders by name.

import io, logging
stream = io.StringIO()
h = logging.StreamHandler(stream); h.setFormatter(logging.Formatter("%(levelname)s %(message)s"))
log = logging.getLogger("wire"); log.handlers = [h]; log.propagate = False

log.setLevel(logging.DEBUG);  trace(log, "hidden at DEBUG")
log.setLevel(TRACE);          trace(log, "visible at TRACE")
print(stream.getvalue().strip())

Expected Output: only the second record, rendered by name.

TRACE visible at TRACE

Common mistakes

A level for a kind of record. Error signature: audit or security events suppressed when an operator raises the level. Root cause: kind modelled as severity. Remediation: a field or a dedicated logger.

Patching logging.Logger. Error signature: a method that exists or not depending on import order, or behaves differently between libraries. Root cause: a global class modification. Remediation: a helper function or log.log(level, …).

No addLevelName. Error signature: Level 5 in output. Root cause: the number never registered with a name. Remediation: register every custom level at import.

Unmapped downstream. Error signature: custom-level records in an unknown severity bucket in the store. Root cause: no translation in the formatter. Remediation: an explicit severity mapping per level.

Numbers that collide. Error signature: a custom level filtered identically to a standard one. Root cause: reusing 10, 20, 30, 40 or 50. Remediation: choose a number strictly between standard levels.

Frequently Asked Questions

When is a custom log level justified?

When the distinction is genuinely about severity and none of the standard five fits — most commonly a TRACE level below DEBUG for extremely verbose output that should be off even in debug builds. Distinctions about the kind of record, such as audit or security, belong in fields or dedicated loggers.

What numbers do the standard levels use?

DEBUG is 10, INFO 20, WARNING 30, ERROR 40 and CRITICAL 50. A custom level should sit between two of these so ordinary level comparisons treat it sensibly, for example 5 for TRACE or 25 for NOTICE.

Why not add a method like logger.trace to the Logger class?

Patching logging.Logger changes the class for every library in the process. It works, and it creates a global side effect, conflicts if two libraries define the same method differently, and makes the codebase depend on import order. A small helper function or calling log with the level number avoids all of that.

How do custom levels appear in a log store?

However the formatter renders them. Without an explicit mapping, a store expecting standard severity names receives Level 5 or an unfamiliar string, which dashboards and alert rules do not recognise. The mapping to standard severities has to be written deliberately.

Related Guides