Exporting Python Logs Through OpenTelemetry

The OpenTelemetry logs signal exists to make log records first-class telemetry: same resource attributes as your spans, same exception field names, same transport, same Collector pipeline. This page wires the standard library into it, covers what the bridge does to each record, and is honest about when writing JSON to stdout remains the better choice. It builds on correlating logs, traces and metrics, part of the distributed tracing and OpenTelemetry in Python section.

Prerequisites

pip install "opentelemetry-sdk>=1.27.0,<2.0.0" \
            "opentelemetry-exporter-otlp-proto-grpc>=1.27.0,<2.0.0"

export OTEL_SERVICE_NAME=checkout-api
export OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317
export OTEL_BLRP_MAX_QUEUE_SIZE=2048
export OTEL_BLRP_SCHEDULE_DELAY=5000

The Collector needs a logs pipeline to receive them:

service:
  pipelines:
    logs:
      receivers: [otlp]
      processors: [memory_limiter, batch]
      exporters: [otlphttp/backend]

Implementation

Step 1 — Build the provider with the shared resource. The same Resource object the tracer and meter use, so all three signals agree.

import logging
from opentelemetry import _logs
from opentelemetry.sdk._logs import LoggerProvider, LoggingHandler
from opentelemetry.sdk._logs.export import BatchLogRecordProcessor
from opentelemetry.exporter.otlp.proto.grpc._log_exporter import OTLPLogExporter
from observability.otel import RESOURCE          # the one shared object

logger_provider = LoggerProvider(resource=RESOURCE)
logger_provider.add_log_record_processor(
    BatchLogRecordProcessor(
        OTLPLogExporter(insecure=True),
        max_queue_size=2048,
        schedule_delay_millis=5000,
        max_export_batch_size=512,
    )
)
_logs.set_logger_provider(logger_provider)

Step 2 — Attach the bridge to the root logger. As a root handler it receives records from your code and from every dependency that propagates, which is the whole point of using the standard library as the front end.

handler = LoggingHandler(level=logging.INFO, logger_provider=logger_provider)
root = logging.getLogger()
root.addHandler(handler)
root.setLevel(logging.INFO)

Step 3 — Keep a stdout handler alongside it. A logs pipeline that silently drops is indistinguishable from a service that has nothing to say. Run both until the OTLP path has proven itself for a week.

import sys
from pythonjsonlogger import jsonlogger

stdout = logging.StreamHandler(sys.stdout)
stdout.setFormatter(jsonlogger.JsonFormatter("%(asctime)s %(levelname)s %(name)s %(message)s"))
root.addHandler(stdout)                            # belt and braces during the transition

Step 4 — Keep the trace-context filter too. The bridge captures the span itself, but the filter puts the IDs on the LogRecord, where the stdout formatter can also see them.

from observability.otel import TraceContextFilter

root.addFilter(TraceContextFilter())               # on the logger, so both handlers benefit

Step 5 — Flush on shutdown. The log processor has the same bounded queue and the same shutdown obligation as the span processor.

import atexit

atexit.register(logger_provider.shutdown)          # drains the queue, then closes

Configuration options

Verification

Emit one record inside a span and read both the stdout copy and the Collector's debug output.

import logging
from opentelemetry import trace

configure()                                        # tracing + logs, sharing the resource
tracer = trace.get_tracer("probe")
log = logging.getLogger("orders")

with tracer.start_as_current_span("probe-span"):
    try:
        raise ValueError("payment declined")
    except ValueError:
        log.exception("order failed", extra={"order_id": 8812})

Expected Output (stdout copy):

{"asctime": "2026-08-02 14:31:07,220", "levelname": "ERROR", "name": "orders",
 "message": "order failed", "order_id": 8812,
 "trace_id": "4bf92f3577b34da6a3ce929d0e0e4736", "span_id": "00f067aa0ba902b7"}

Expected Output (Collector debug exporter):

LogRecord #0
Severity: ERROR (17)
Body: order failed
Attributes:
     -> order_id: Int(8812)
     -> exception.type: Str(ValueError)
     -> exception.message: Str(payment declined)
     -> exception.stacktrace: Str(Traceback (most recent call last)…)
Trace ID: 4bf92f3577b34da6a3ce929d0e0e4736
Span ID: 00f067aa0ba902b7
Resource attributes:
     -> service.name: Str(checkout-api)
     -> service.version: Str(2026.8.1)

Three things to check: the severity number is 17 rather than 40, the exception is expanded into three attributes under the semantic-convention names, and the resource attributes are present without appearing anywhere in the logging call.

Common mistakes

Records arrive with no trace ID

Error signature: Trace ID is empty on the Collector side although the code runs inside a span. Root cause: the record was created outside the span's context — commonly on a background thread, or in a task created before the span started. Remediation: check where the record is produced. The bridge reads the active span at conversion time, so a record produced where no span is current genuinely has none.

The logs pipeline silently drops everything

Error signature: no log records at the backend, no errors anywhere, and the service is clearly running. Root cause: the Collector has no logs pipeline, so it accepts the records on the OTLP receiver and has nowhere to send them. Remediation: add the logs pipeline to service.pipelines, and keep the stdout handler until it is confirmed.

Duplicate records after adding the bridge

Error signature: every line appears twice. Root cause: the bridge handler was added to a named logger that also propagates to a root logger carrying the same handler. Remediation: attach the bridge once, at the root, and let propagation do the rest — the same rule as any root-handler setup, described in taming third-party library loggers.

Deciding whether to adopt it

The logs signal is the newest of the three and the one where "should we" is a real question rather than a formality. Four considerations decide it, and the answer differs by deployment more than by preference.

Do your logs already reach a backend that ingests all three signals? If the destination is a system that stores traces, metrics and logs together and joins them for you, sending logs over OTLP puts them in the same pipeline with the same resource attributes and the same redaction — which is most of the value. If logs go somewhere else entirely, the OTLP path adds a transport without adding a join.

Is central redaction worth having? A Collector attribute processor that hashes db.statement covers spans today; extending it to log records is one pipeline entry once the records arrive as OTLP. In a mixed-language fleet that single policy point is often the strongest argument on this page.

Can you tolerate an in-process failure mode? stdout has none: the write goes to a pipe and the platform owns the rest. OTLP export introduces a bounded queue that drops under pressure, which is the same trade the span pipeline makes and a new one for logs.

Is anything depending on the file or stream format? An on-host agent, a security tool tailing a file, a support workflow that reads container logs — all of them break if stdout stops carrying the records. That is the most common practical blocker, and it is also the reason the transition period with both paths running is worth the duplication.

Severity, and what to do about custom levels

The mapping is fixed and mostly unsurprising, with one wrinkle worth planning around: a project-defined level between two standard ones maps to the nearest lower standard severity number and keeps its own name in severity_text. An AUDIT level at 25 therefore arrives as severity 9 — the same number as INFO — with the text AUDIT.

That is usually the right behaviour, and it means anything keyed on the numeric severity groups audit records with informational ones. If audit records need to be distinguishable by a numeric filter, the practical options are to place the custom level at a standard number's value or to carry the distinction as an attribute rather than as a level. The second is generally better: a level is a coarse severity axis, and "this is an audit event" is a category rather than a severity.

logger.info("permission granted", extra={"event_class": "audit", "principal": principal})

What this does not replace

Two things the OTLP logs path is sometimes expected to do and does not.

It is not a durability mechanism. The bounded queue drops under pressure and nothing is written to disk, so a record that matters for audit or compliance needs a path that does not lose data quietly — a file with an agent, or a write to a store, rather than a telemetry pipeline whose design principle is to shed load rather than block.

And it is not a substitute for structured fields. A record whose body is a formatted sentence arrives as a formatted sentence with a resource attached; the fields that make it queryable still have to be set at the call site through extra. The transport improves correlation and identity; it does nothing for a log line that was never structured in the first place, which is the subject of structured logging with the Python standard library.

Related

• Correlating logs, traces and metrics — the parent guide: the three joins and the shared resource.
• Linking metrics to traces with exemplars — the third signal's join.
• Adding trace IDs to log records — the filter that serves the stdout path.
• Exporters and the OpenTelemetry Collector — the pipeline these records travel through.
• Structured logging with the Python standard library — the stdout half of the transition.

Frequently Asked Questions