Propagating Trace Context Across Celery Tasks

When a web request fans out work to a Celery worker, the span that the worker creates becomes a disconnected root unless the producer's traceparent rides along in the message headers; this guide shows the exact instrumentation that keeps trace_id continuity intact across the broker. It is a focused task within the context propagation and baggage reference, part of the Distributed Tracing and OpenTelemetry in Python guide.

Reliable continuity depends on three things being true at once. The instrumentation package must patch Celery's signals so headers are written and read automatically. The global propagator must serialize W3C TraceContext into the message payload. And the worker must restore the parent context before your task body runs so child spans attach correctly.

Prerequisites

Pin the instrumentation and exporter so the producer and worker agree on the wire format. A version skew between opentelemetry-api and the instrumentation package is the most common source of silent propagation failures.

pip install \
  "opentelemetry-sdk>=1.30.0,<2.0.0" \
  "opentelemetry-instrumentation-celery>=0.51b0,<1.0.0" \
  "opentelemetry-exporter-otlp-proto-grpc>=1.30.0,<2.0.0" \
  "celery>=5.3.0,<6.0.0"

Set the export endpoint and service name through environment variables so the same module runs unchanged in the producer and the worker. Define OTEL_SERVICE_NAME distinctly per role so producer and consumer spans are attributable.

export OTEL_EXPORTER_OTLP_ENDPOINT="http://localhost:4317"
export OTEL_SERVICE_NAME="payments-worker"

Implementation

Establish a baseline SDK configuration before instrumenting Celery. Initializing the TracerProvider after worker startup triggers NoOpTracer fallbacks and severs the trace chain, so every step below runs at module import time.

1. Create the provider and register a batch processor. Build the TracerProvider, attach a BatchSpanProcessor wrapping the OTLP exporter, and set it as the global provider. The batch processor flushes on a background thread so span export never blocks task execution.

2. Register the W3C propagator globally. Call set_global_textmap before the Celery app is instantiated. This is the same propagator the rest of your services use, which is what keeps trace_id consistent end to end; it mirrors the standard context propagation and baggage mechanics.

3. Instrument the Celery app once. Call CeleryInstrumentor().instrument() in the main process. The instrumentation hooks before_task_publish to inject traceparent, tracestate, and baggage into the message headers, and hooks task_prerun to extract them and start a child span. Manual header manipulation is unnecessary and breaks extraction.

4. Set baggage before dispatch. Any baggage you set inside an active span before calling .delay() is serialized into the baggage header and restored on the worker. This is how request-scoped values such as a tenant identifier travel to the worker without being passed as task arguments.

import os
from celery import Celery
from opentelemetry import trace, baggage
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import OTLPSpanExporter
from opentelemetry.propagate import set_global_textmap
from opentelemetry.propagators.composite import CompositePropagator
from opentelemetry.trace.propagation.tracecontext import TraceContextTextMapPropagator
from opentelemetry.baggage.propagation import W3CBaggagePropagator
from opentelemetry.instrumentation.celery import CeleryInstrumentor

# 1. Provider + batch export (non-blocking background flush)
provider = TracerProvider()
exporter = OTLPSpanExporter(
    endpoint=os.getenv("OTEL_EXPORTER_OTLP_ENDPOINT", "http://localhost:4317")
)
provider.add_span_processor(BatchSpanProcessor(exporter))
trace.set_tracer_provider(provider)

# 2. Global propagator: trace context + baggage, registered BEFORE Celery()
set_global_textmap(CompositePropagator([
    TraceContextTextMapPropagator(),
    W3CBaggagePropagator(),
]))

# 3. Instrument once in the main process
app = Celery("worker_tasks", broker="redis://localhost:6379/0")
CeleryInstrumentor().instrument()

tracer = trace.get_tracer(__name__)


@app.task(bind=True, max_retries=3)
def process_payment(self, order_id: str):
    # Parent context already restored by CeleryInstrumentor on task_prerun
    current_baggage = baggage.get_all()
    print(f"Processing {order_id} with baggage: {current_baggage}")
    return {"status": "completed", "order_id": order_id}

On the producer side, set baggage inside a span and dispatch the task; the instrumentation injects the headers automatically.

# producer.py
from opentelemetry import baggage, context
from worker_tasks import process_payment, tracer

with tracer.start_as_current_span("checkout"):
    ctx = baggage.set_baggage("tenant_id", "acme_corp")
    ctx = baggage.set_baggage("request_source", "api_gateway", context=ctx)
    token = context.attach(ctx)
    try:
        process_payment.delay("order_12345")
    finally:
        context.detach(token)

Configuration options

Verification

Run the worker and dispatch a task, then confirm the worker restores the baggage that the producer set. The presence of the baggage values proves the headers crossed the broker and were extracted before the task body ran.

Expected Output (Worker Execution):

Processing order_12345 with baggage: {'tenant_id': 'acme_corp', 'request_source': 'api_gateway'}

For wire-level confirmation, run the worker with OTEL_LOG_LEVEL=debug and look for the inject and extract events in stdout. Absence of these lines confirms a signal-hook failure rather than a broker problem.

Expected Output (debug log):

[celery] Injecting trace context into headers traceparent=00-4bf92f3577b34da6a3ce929d0e0e4736-00f067aa0ba902b7-01
[celery] Extracting trace context from headers traceparent=00-4bf92f3577b34da6a3ce929d0e0e4736-00f067aa0ba902b7-01

Finally, query your backend: the worker span for process_payment must share the producer's trace_id (4bf92f3577b34da6a3ce929d0e0e4736 above) and carry a non-null parent_id. Filtering for parent_id = null on the worker service surfaces any orphaned spans that indicate broken propagation.

Common mistakes

Missing instrument() call. Worker logs emit NoOpTracer warnings and spans appear as isolated roots. Call CeleryInstrumentor().instrument() in the main process immediately after Celery() and before celery -A startup so the publish and prerun signals are patched before any task runs.

Manually rewriting task headers. Overwriting kwargs['trace_id'] or stamping your own header produces KeyError: 'traceparent' during extraction and broken parent_id links. Remove all manual header assignments and let the propagator serialize W3C TraceContext into headers.

Instrumenting only one side. If the producer injects but the worker process never calls instrument() (or vice versa), the worker span detaches into a new root. Both the dispatching process and the worker process must configure the global propagator and call instrument().

Frequently Asked Questions