Python Observability

The OpenTelemetry Log Data Model in Python

OpenTelemetry defines a log record with a small, fixed structure: when it happened, how severe it was, a body, attributes, the trace context it belongs to, and the resource that produced it. That structure is shared in spirit with spans and metrics, which is why logs emitted this way join traces without any correlation work. This page covers what each part of the model is for, how Python's LogRecord maps onto it, and the handful of decisions — body versus attributes, resource versus record — that determine whether the result is useful. It is a task article under JSON log schemas and conventions, part of the modern Python logging libraries deep dive section, and it pairs with exporting Python logs through OpenTelemetry.

The shape of the record An OpenTelemetry log record is drawn as a set of fields, each connected to its source in a Python process. The resource — service name, version and environment — comes from the logger provider's configuration and is shared with every span and metric the process emits; it is serialised once per batch. The instrumentation scope comes from the Python logger's name. The timestamp and observed timestamp come from the LogRecord's creation time and the moment the handler processed it. Severity number and text come from the Python level through a fixed mapping. The body comes from the formatted message. Attributes come from extra fields and exception information. Trace identifier, span identifier and trace flags come from the span active when the log call was made, and are filled automatically by the SDK. The note records that trace context is the one field the application never has to supply, and it is the one that joins the record to everything else. an OpenTelemetry log record, and where each part comes from resource: service.name, version, env provider config · shared with spans and metrics scope: instrumentation name the Python logger's name timestamp, observed_timestamp record.created · when the handler saw it severity_number, severity_text the Python level, through a fixed map body, attributes the message · extra fields and exceptions trace_id, span_id, trace_flags the active span — filled automatically the last row is the one the application never supplies, and the one that joins the record to its trace
Most of the record comes from Python's own LogRecord. The resource comes from configuration, and trace context comes from the active span without the call site doing anything.

Prerequisites

pip install "opentelemetry-sdk>=1.27.0,<2.0.0" \
            "opentelemetry-exporter-otlp-proto-grpc>=1.27.0,<2.0.0"

Implementation

Step 1 — Route standard logging through the OpenTelemetry handler. The SDK provides a logging handler that converts each standard library record into an OpenTelemetry log record and passes it to a processor for export. Attaching it to the root logger means every record from every library follows the model without changing any call site.

import logging
import os

from opentelemetry._logs import set_logger_provider
from opentelemetry.sdk._logs import LoggerProvider, LoggingHandler
from opentelemetry.sdk._logs.export import BatchLogRecordProcessor
from opentelemetry.exporter.otlp.proto.grpc._log_exporter import OTLPLogExporter
from opentelemetry.sdk.resources import Resource

resource = Resource.create({
    "service.name": os.environ["OTEL_SERVICE_NAME"],
    "service.version": os.environ.get("SERVICE_VERSION", "unknown"),
    "deployment.environment": os.environ.get("ENVIRONMENT", "dev"),
})
provider = LoggerProvider(resource=resource)
provider.add_log_record_processor(BatchLogRecordProcessor(OTLPLogExporter(timeout=5)))
set_logger_provider(provider)

logging.getLogger().addHandler(LoggingHandler(level=logging.INFO, logger_provider=provider))

Step 2 — Put identity on the resource. Service name, version and environment describe the process, not the event. As resource attributes they are sent once per export batch, apply to logs, spans and metrics identically, and cannot drift between signals. Putting them on each record as attributes instead duplicates them in every record and invites the three signals to disagree. The details of resource construction are in configuring resource attributes for Python services.

Step 3 — Keep the body a message and details in attributes. The body is intended to be the human-readable description of the event. Variable data belongs in attributes, named with semantic conventions where one exists, so it is queryable and so the body remains constant across occurrences and groups correctly.

log = logging.getLogger("checkout.payments")

log.warning(
    "payment declined",                          # body: fixed, groupable
    extra={
        "payment.provider": "acme-pay",          # attributes: queryable
        "payment.decline_code": "insufficient_funds",
        "http.response.status_code": 402,
    },
)

Step 4 — Log inside an active span and let the SDK attach context. When a log call is made while a span is current, the handler records that span's trace identifier, span identifier and flags on the log record. Nothing at the call site supplies them. This is the property that makes the model valuable: logs and traces join on the trace identifier automatically, and a trace viewer can show a span's log records alongside it.

from opentelemetry import trace
tracer = trace.get_tracer("checkout")

with tracer.start_as_current_span("charge"):
    log.info("charging card", extra={"payment.amount_cents": 4200})

Expected Output: a record with the span's context attached, as the collector receives it.

{
  "timeUnixNano": "1789740131408000000",
  "severityNumber": 9,
  "severityText": "INFO",
  "body": {"stringValue": "charging card"},
  "attributes": [
    {"key": "payment.amount_cents", "value": {"intValue": "4200"}},
    {"key": "code.filepath", "value": {"stringValue": "checkout/payments.py"}},
    {"key": "code.lineno", "value": {"intValue": "88"}}
  ],
  "traceId": "9f2a71c4f0b84c2e9d5f1a7b3c8e6d02",
  "spanId": "4b1e77a2c9de5013",
  "flags": 1
}

Step 5 — Confirm the severity mapping. Python's five standard levels map to the model's severity numbers in fixed ranges — DEBUG to 5, INFO to 9, WARNING to 13, ERROR to 17, CRITICAL to 21 — and consumers group by those numbers. Custom levels need a deliberate place in the scale, as covered in defining custom log levels in Python, or they arrive at an arbitrary severity.

Resource versus record attributes A batch of five hundred log records is shown twice. In the first, service name, version and environment are set as attributes on every record, so the batch carries them five hundred times, and a typo in one code path produces records attributed to a slightly different service name than the spans from the same process. In the second, they are set once on the resource, which the exporter serialises once for the whole batch; every record, span and metric from the process shares exactly the same values, and they cannot disagree. The note records that the resource form is both smaller on the wire and structurally unable to drift between signals, which is why identity belongs there. a batch of 500 records, identity carried two ways on every record × 500 copies · free to drift from the spans on the resource once shared by every log, span and metric from the process identity describes the process, so it belongs to the process smaller on the wire, and structurally unable to disagree with the traces attributes are for what differs between events — the decline code, the amount, the route not for what every event from this process shares
The resource is the process's identity, stated once. Repeating it on every record costs bytes and creates a way for logs and traces to disagree about who produced them.

Emitting the model without the SDK

Adopting the data model does not require exporting logs through OpenTelemetry. Many services write JSON to standard output and have a collector parse it, and the model is just as useful as a target shape for that JSON.

A formatter that emits a record with body, severity_text, severity_number, attributes, trace_id and span_id fields — and that leaves resource attributes to the collector, which knows the pod, the namespace and the service from its environment — produces output the collector can convert to OpenTelemetry records with a simple parsing rule. The application keeps the robustness of logging to standard output, described in standard output versus file logging in containers, and the store receives records in the same shape as those from services using the SDK.

The trade-off is trace context. The SDK handler reads the active span automatically; a JSON formatter must do the same explicitly, reading the current span in a filter on the calling thread. That is a few lines and it is the same code needed for any trace-correlated JSON logging, so in practice the difference is small.

The choice between the two paths is therefore mostly about delivery rather than about shape. Services already exporting spans over OTLP gain the most from the SDK path, because logs travel the same connection with the same batching and retry behaviour. Services where standard output is the established path can adopt the model's shape without changing how logs leave the process.

Events, and records that describe something happening

The model distinguishes, loosely, between a log record — a statement written by code for a human to read — and an event, a record with a name and a defined set of attributes that describes something that happened and is meant to be processed by machines. Both are log records in the data model; an event carries an event name and its attributes follow a known shape.

The distinction is useful in Python services because it maps onto a real difference in how records are used. Most log statements are written for engineers reading output during an investigation, and their structure is loose. A smaller number describe business or security occurrences — a payment declined, a login failed, a permission granted — that dashboards count, alerts fire on and audit reviews read. Those benefit from being treated as events: a stable name, a fixed set of attributes with defined types, and no reliance on the body text.

In practice this means giving such records an explicit event name attribute alongside the body, and treating the attribute set for each event name as a contract, tested as described in snapshot testing structured log output. The body stays a readable sentence for humans; the event name and attributes are what machines key on. A consumer that filters on event.name = "payment.declined" keeps working when somebody rewords the body, which is the stability that dashboards and alerts need.

Where each standard library field lands A table showing where a standard library log record's parts land in the OpenTelemetry log data model. created becomes Timestamp. The level number becomes SeverityNumber on the OpenTelemetry scale, and levelname becomes SeverityText. getMessage() becomes Body. extra fields become record Attributes. The active span's context becomes TraceId, SpanId and TraceFlags. The service name and host do not belong on each record; they come from the Resource, attached once per provider. The logger name becomes the InstrumentationScope. The note says separating resource from record attributes is what keeps each record small. standard library OpenTelemetry log model created Timestamp levelno · levelname SeverityNumber · SeverityText getMessage() Body extra={...} Attributes active span TraceId · SpanId · TraceFlags logger name InstrumentationScope service, host Resource — once per provider resource attributes are attached once, not repeated on every record
Each part of a familiar LogRecord has a named home in the model. Service identity moves out of the record entirely.

Configuration options

Model field Python source Note
Resource LoggerProvider(resource=…) once per process; shared with traces
Scope logger name fine-grained logger names help here
Timestamp record.created when the event happened
Observed timestamp handler time when the SDK saw it
Severity fixed level map custom levels need an explicit entry
Body formatted message keep it constant across occurrences
Attributes extra, exception info semantic-convention names
Trace context active span automatic with the SDK handler

Verification

Confirm that a log call inside a span carries that span's identifiers.

from opentelemetry.sdk._logs.export import InMemoryLogExporter, SimpleLogRecordProcessor

memory = InMemoryLogExporter()
provider.add_log_record_processor(SimpleLogRecordProcessor(memory))

with tracer.start_as_current_span("verify") as span:
    log.warning("inside a span")

rec = memory.get_finished_logs()[-1].log_record
print(f"{rec.trace_id:032x}" == f"{span.get_span_context().trace_id:032x}", rec.severity_text)

Expected Output:

True WARN

A False means the log call happened outside the span's context — usually because it ran on a thread or task the context did not propagate to.

Common mistakes

Identity on every record. Error signature: logs attributed to a slightly different service name than the spans. Root cause: service name set as a record attribute in some code paths. Remediation: set it on the resource only.

Variable data in the body. Error signature: every record's body unique, so nothing groups. Root cause: formatting values into the message. Remediation: fixed body, values in attributes.

Logging outside the span. Error signature: records with empty trace context despite being part of a traced request. Root cause: the call made on a thread or task where the span is not current. Remediation: propagate context, or log inside the span's scope.

Handler added twice. Error signature: each record exported twice. Root cause: the SDK handler attached both explicitly and by an auto-instrumentation option. Remediation: attach it in exactly one place.

Relying on the observed timestamp. Error signature: records ordered by when the pipeline processed them rather than when events happened. Root cause: a consumer reading the observed timestamp as the event time. Remediation: use the event timestamp for ordering and keep the observed one for measuring pipeline delay.

Unmapped custom levels. Error signature: records at an unexpected severity in the store. Root cause: a custom level with no place in the severity scale. Remediation: map it explicitly.

Frequently Asked Questions

What fields does an OpenTelemetry log record have?

Two timestamps — when the event occurred and when it was observed — a severity number and text, a body, a set of attributes, trace and span identifiers with trace flags, and the resource and instrumentation scope that produced it. It is deliberately minimal, with everything else carried in attributes.

How does a Python LogRecord map onto it?

The formatted message becomes the body, the level becomes severity number and text, extra fields become attributes, the logger name becomes the instrumentation scope, and exception information becomes exception attributes following semantic conventions. Trace context comes from the active span, not from the LogRecord.

What is the difference between the resource and attributes?

The resource describes the entity producing telemetry — the service, its version, its host — and is shared by every log, span and metric from that process. Attributes describe the individual event. Putting service identity on the resource means it is sent once per batch, not once per record, and matches the other signals exactly.

Do I need the OpenTelemetry logs pipeline to use this model?

No. A JSON formatter can emit the same structure to standard output, and a collector can parse it into OpenTelemetry records. Using the SDK's handler is simpler when the service already exports traces over OTLP, because the logs travel the same way and trace context is attached automatically.

Related Guides