Using Baggage for Tenant and Feature Context

Some context belongs to the whole request rather than to any single service: which tenant it is for, which experiment arm it is in, whether a support engineer has asked for it to be traced in full. OpenTelemetry baggage carries such values from the service that first knows them to every service downstream, through the same propagation as trace context. It does not, by itself, put them anywhere useful — baggage is carried, not recorded — so the practical pattern has two halves: set it once at the edge, and copy it onto spans and logs where it is needed. This page covers both, plus the size and privacy limits that decide what belongs in baggage at all. It is a task article under context propagation and baggage, part of the distributed tracing and OpenTelemetry in Python section.

Prerequisites

pip install "opentelemetry-api>=1.27.0,<2.0.0" \
            "opentelemetry-sdk>=1.27.0,<2.0.0"

The default propagator set includes both trace context and baggage, so HTTP instrumentation carries baggage without further configuration.

export OTEL_PROPAGATORS=tracecontext,baggage

Implementation

Step 1 — Set baggage at the edge. The first service that authenticates a request knows the tenant; it sets baggage and makes the resulting context current for the rest of the request. Everything downstream — in-process and across HTTP or message boundaries with instrumented clients — receives it.

from opentelemetry import baggage, context

async def tenant_middleware(request, call_next):
    tenant = await authenticate(request)
    ctx = baggage.set_baggage("tenant.id", tenant.id)
    ctx = baggage.set_baggage("tenant.plan", tenant.plan, context=ctx)
    token = context.attach(ctx)                    # 1. current for this request
    try:
        return await call_next(request)
    finally:
        context.detach(token)

Step 2 — Copy chosen keys onto spans. Baggage is not recorded on spans. A span processor that reads specific baggage keys when each span starts, and sets them as attributes, makes them queryable in the trace store in every service that installs the processor. Copying an allow-list of keys, rather than everything, keeps unexpected baggage from untrusted callers out of the trace store.

from opentelemetry.sdk.trace import SpanProcessor

class BaggageToAttributes(SpanProcessor):
    KEYS = ("tenant.id", "tenant.plan", "debug.capture")

    def on_start(self, span, parent_context=None):
        for key in self.KEYS:
            value = baggage.get_baggage(key, parent_context)
            if value is not None:
                span.set_attribute(key, value)

provider.add_span_processor(BaggageToAttributes())

Step 3 — Copy the same keys onto log records. A logging filter that reads baggage from the current context stamps each record with the tenant, so logs can be filtered by tenant in every service without any call site passing it. It must run on the calling thread — before any queue handler — for the same reason trace identifiers must, as covered in adding trace IDs to log records.

import logging

class BaggageFilter(logging.Filter):
    def filter(self, record: logging.LogRecord) -> bool:
        record.tenant_id = baggage.get_baggage("tenant.id") or "-"
        return True

Expected Output: a span in a downstream service carrying the tenant it never received as an argument.

{"name": "SELECT inventory", "attributes": {"tenant.id": "acme", "tenant.plan": "enterprise", "db.system": "postgresql"}, "traceId": "9f2a71c4f0b84c2e9d5f1a7b3c8e6d02"}

Step 4 — Strip baggage at trust boundaries. Automatic HTTP client instrumentation injects baggage into every outbound request, including requests to third-party APIs. A tenant identifier sent to an external shipping provider is data leaving the organisation. The outbound hook of the HTTP client instrumentation, or a context with baggage cleared around the external call, keeps it inside.

from opentelemetry import baggage, context

def call_external(fn, *args):
    ctx = baggage.clear()                          # 2. no baggage beyond this point
    token = context.attach(ctx)
    try:
        return fn(*args)
    finally:
        context.detach(token)

The reverse applies to inbound requests from untrusted callers: baggage arriving at a public edge was set by the caller and should be discarded, not trusted, since a caller could claim to be any tenant.

Step 5 — Keep it small. Baggage is a request header sent on every downstream call. Identifiers and short flags cost little; a serialised object repeated across twenty hops costs bandwidth, may exceed proxy header limits, and gets truncated or dropped in ways that break propagation for the whole request.

What belongs in baggage

The mechanism makes it easy to put anything in baggage, and the constraints — every value on every hop, visible to every service, potentially sent outside — make it important to be selective.

Good candidates are short, request-wide identifiers and flags that many services need and that are not sensitive on their own: a tenant identifier, a plan or tier, an experiment arm, a debug capture flag, a request priority. Each is a few bytes, is useful for filtering and routing in several services, and is safe for any internal service to see.

Poor candidates are anything large, anything sensitive, and anything only one service needs. User email addresses and names are sensitive and travel everywhere, including potentially to third parties if stripping is missed; the arguments in logging personal data safely apply with more force here, because baggage fans out. Serialised objects are large and fragile. Values only one downstream service uses belong in that call's arguments, not in context that every service receives.

Never put authorisation decisions in baggage. A downstream service that trusts a role=admin value from baggage has delegated its access control to whatever set the header, and at a public edge that is the caller. Baggage is for observability and routing hints, not for security.

A useful discipline is to maintain the list of permitted baggage keys in one place — the same allow-list the span processor uses — and to treat an addition to it as a small design review: who sets it, who reads it, and where it must be stripped.

Baggage across message queues and background work

HTTP instrumentation carries baggage automatically. Other boundaries do so only if their instrumentation injects the full context, and it is worth checking each one a request crosses.

Message queues. Producer instrumentation for queues typically injects trace context into message headers, and whether it includes baggage depends on the propagators configured and on the instrumentation. With the default propagator set, a producer that injects through the global propagator carries baggage too, and a consumer that extracts through it recovers the tenant. A consumer that extracts only the trace parent by hand loses it. The mechanics are the same as in propagating trace context across Celery tasks, and the check is simply whether the tenant attribute appears on consumer spans.

Metrics. Baggage can inform metric attributes, but only bounded values belong there. A tenant plan with three values is a reasonable metric dimension; a tenant identifier with ten thousand values is a cardinality explosion, for the reasons in controlling label cardinality in Prometheus. Copying baggage onto spans and logs is safe in a way copying it onto metrics is not.

Work that outlives the request. A background task started during a request inherits its context, baggage included. That is correct when the work belongs to the tenant, and misleading when it does not — a shared cache refresh triggered by one tenant's request should not carry that tenant's identifier. Clearing baggage at the start of such work, as with any context that should not be inherited, keeps its telemetry correctly attributed.

Configuration options

Verification

Confirm baggage arrives downstream and is recorded, with a two-service test harness or a single process that injects and extracts.

from opentelemetry.propagate import inject, extract

ctx = baggage.set_baggage("tenant.id", "acme")
carrier: dict = {}
inject(carrier, context=ctx)
print(carrier.get("baggage"))
print(baggage.get_baggage("tenant.id", extract(carrier)))

Expected Output: the header carrying the value, and the value recovered on the other side.

tenant.id=acme
acme

Common mistakes

Expecting baggage on spans automatically. Error signature: a tenant set at the edge and absent from every downstream span. Root cause: baggage is propagated, not recorded. Remediation: a span processor that copies chosen keys.

Baggage sent to third parties. Error signature: tenant identifiers visible in an external provider's logs. Root cause: automatic client instrumentation injecting baggage into every request. Remediation: clear baggage around external calls.

Trusting inbound baggage at a public edge. Error signature: requests attributed to tenants they do not belong to. Root cause: caller-supplied baggage accepted. Remediation: discard it at the edge and set it from authentication.

Large or sensitive values. Error signature: header size errors from proxies, or personal data in every service. Root cause: baggage used as a general data channel. Remediation: short identifiers only.

Copying every key. Error signature: unexpected attributes in the trace store from arbitrary callers. Root cause: a processor that copies all baggage. Remediation: an explicit allow-list.

Frequently Asked Questions