Snapshot Testing Structured Log Output

A structured log record's shape — its field names, their types, which ones are present — is a contract with every dashboard, alert rule and saved query that reads it. A snapshot test serialises a representative record through the real formatter and compares the result with a stored example, so any change to that shape appears as a diff in review rather than as a broken dashboard in production. It is the cheapest logging test to write and the one that catches the most. This page covers building deterministic records, capturing the real output, and treating snapshot changes as the schema changes they are. It is a task article under log testing and verification, part of the Python logging fundamentals and structured data section.

Prerequisites

pip install "pytest>=8.0.0,<9.0.0" \
            "syrupy>=4.6.0,<5.0.0" \
            "python-json-logger>=2.0.7,<4.0.0"

Implementation

Step 1 — Build records deterministically. A LogRecord captures the time, the process and thread identifiers, and the source location when it is created. All of these differ between runs and would make every snapshot fail. They are ordinary attributes and can be set explicitly, which produces output that is identical on every run and on every machine.

import logging

FIXED_TIME = 1789000000.123

def make_record(level=logging.INFO, msg="order accepted", exc_info=None, **fields):
    record = logging.LogRecord(
        name="orders", level=level, pathname="orders/service.py", lineno=88,
        msg=msg, args=(), exc_info=exc_info)
    record.created = FIXED_TIME
    record.msecs = 123.0
    record.relativeCreated = 0.0
    record.process, record.processName = 1, "MainProcess"
    record.thread, record.threadName = 1, "MainThread"
    for key, value in fields.items():
        setattr(record, key, value)
    return record

Step 2 — Format through the real handler chain. The snapshot must show what leaves the process, which means the production formatter and the production filters — redaction, enrichment, field renaming. Taking the formatter and filters from the actual configuration, rather than constructing a similar one in the test, is what makes the test meaningful: a test with its own formatter tests that formatter, not yours.

import json
import logging.config
import pytest

from myservice.logging_config import build_config

@pytest.fixture
def production_handler(tmp_path, monkeypatch):
    monkeypatch.setenv("LOG_DIR", str(tmp_path))
    logging.config.dictConfig(build_config("production", queue=False))
    handler = logging.getLogger().handlers[0]      # the stdout handler production uses
    yield handler
    logging.config.dictConfig({"version": 1, "disable_existing_loggers": False})


def render(handler, record) -> dict:
    for f in handler.filters:                       # run filters exactly as emit() would
        result = f.filter(record) if hasattr(f, "filter") else f(record)
        if not result:
            return {"__dropped__": True}
    return json.loads(handler.format(record))

Step 3 — Compare parsed output. Comparing the raw string makes the test fail on key order and whitespace, neither of which any consumer depends on. Comparing the parsed dictionary keeps it sensitive to exactly what matters — names, values, types, presence — and a snapshot library then shows a readable diff when something changes.

def test_order_accepted_shape(production_handler, snapshot):
    record = make_record(event="order_accepted", order_id="ord_7",
                         duration_ms=41.2, tenant="acme")
    assert render(production_handler, record) == snapshot

Expected Output: the stored snapshot, which becomes the reviewed definition of this record's shape.

{
  "asctime": "2026-09-09T18:13:20.123Z",
  "levelname": "INFO",
  "name": "orders",
  "message": "order accepted",
  "event": "order_accepted",
  "order_id": "ord_7",
  "duration_ms": 41.2,
  "tenant": "acme",
  "service": "checkout",
  "environment": "production"
}

Step 4 — Snapshot the shapes that differ. A formatter has several paths and a single snapshot exercises one. At minimum: a plain record, a record with an exception, and a record carrying context fields such as a trace identifier. The exception path is where formatters most often surprise people — multi-line tracebacks, missing exception types, truncation — and the context path is where enrichment either works or silently does nothing.

def test_exception_shape(production_handler, snapshot):
    try:
        raise TimeoutError("read timed out")
    except TimeoutError:
        import sys
        record = make_record(level=logging.ERROR, msg="charge failed",
                             exc_info=sys.exc_info(), order_id="ord_9")
    out = render(production_handler, record)
    # 1. The traceback's line numbers vary with the test file; normalise them.
    out["exception"] = out["exception"].split("\n")[-1]
    assert out == snapshot

Step 5 — Review snapshot updates as schema changes. The mechanics of updating a snapshot are one command. The value is in what happens next: the updated snapshot appears in the pull request as a diff, and the reviewer sees that a field has been renamed or retyped. That is the moment to ask which dashboards, alerts and saved queries use it — while the author is still available and the change is still easy to adjust.

Snapshots as documentation

A side effect of snapshot testing is worth drawing out, because it changes how a team thinks about its logs.

The stored snapshots are a precise, always-current description of what the service emits. They answer the question every consumer of the logs eventually asks — what fields does this service produce, and what types are they — without anybody having to read the formatter or run the service. A new engineer building a dashboard can open the snapshot directory and see exactly what is available. An engineer on another team writing an alert can check the field name against the snapshot instead of guessing from an example they found in the log store.

That also makes snapshots the natural input to the fleet-level schema work described in designing a log schema for a service fleet. Validating each service's snapshots against the shared schema in continuous integration turns a document that everyone agrees with and nobody checks into an enforced contract, and it catches divergence at the moment a service introduces it rather than when two services' records collide in a shared index.

The discipline this requires is small: keep the snapshots representative — a real business event rather than a trivial "hello" record — and review their changes with the same care as a change to an API response. Teams that do this find that the number of broken dashboards after releases drops sharply, and that the conversation about renaming a field happens before the rename rather than after.

Choosing what goes into a representative record

A snapshot is only as useful as the record it captures, and three choices decide how much it protects.

Use a real business event. A record representing an order acceptance, a payment failure or a login is what dashboards and alerts actually query. A generic "test message" exercises the formatter's mechanics without exercising the fields anybody depends on, so a change to those fields slips past.

Include every field type the formatter handles differently. Strings, integers, floats, booleans, nested dictionaries, lists, None, and a datetime if the service logs one. Formatters handle these through different branches, and type coercion bugs — a number serialised as a string, a datetime rendered in local time, a nested dictionary flattened unexpectedly — appear only when the type is present. One record with one field of each type covers all of them.

Include a field that redaction should remove. A snapshot that shows a redacted value — the placeholder rather than the secret — documents that redaction runs, and fails the moment it stops. This is the cheapest possible regression test for the property that matters most, and it pairs naturally with redacting sensitive data in log records.

These three choices usually produce a record with ten or twelve fields, which is a comfortable size to read in a diff and large enough to exercise every path that matters.

Configuration options

Verification

Confirm the snapshot fails on the changes it exists to catch, by making one deliberately.

# rename a field in the formatter's rename map, then run the snapshot tests
sed -i 's/"duration_ms"/"duration"/' myservice/logging_config.py
pytest tests/test_log_snapshots.py -q; git checkout myservice/logging_config.py

Expected Output: a readable diff naming the renamed field.

FAILED tests/test_log_snapshots.py::test_order_accepted_shape
  -   'duration_ms': 41.2,
  +   'duration': 41.2,

Common mistakes

A formatter constructed in the test. Error signature: green snapshots and different production output. Root cause: the test's formatter is a copy, not the real one. Remediation: take the formatter and filters from the production configuration.

Unstable fields in the snapshot. Error signature: snapshots that fail on every run or every machine. Root cause: real timestamps, process ids or absolute paths. Remediation: fix them on the record before formatting.

Comparing strings. Error signature: failures on key order after a harmless library upgrade. Root cause: raw output compared byte for byte. Remediation: compare parsed JSON.

Only the happy path. Error signature: a broken traceback serialisation discovered during an incident. Root cause: no snapshot of an exception record. Remediation: snapshot plain, exception and context shapes.

Updating snapshots without review. Error signature: a renamed field in production and a broken dashboard. Root cause: snapshot updates treated as routine. Remediation: review every snapshot change as a schema change, naming its consumers.

Frequently Asked Questions