Choosing Histogram Buckets for Latency SLOs

A latency histogram answers exactly the questions its bucket boundaries allow. Get one boundary right and the SLO compliance figure is an exact count; get it wrong and the same query returns an interpolation whose error nobody can see. This page covers choosing the ladder, sizing it against the real distribution, and the cardinality budget it consumes. It builds on metric types and cardinality, part of the Python metrics and instrumentation section.

Prerequisites

pip install "prometheus-client>=0.20.0,<1.0.0" \
            "opentelemetry-sdk>=1.27.0,<2.0.0"

Implementation

Step 1 — Measure the distribution before choosing a ladder. A ladder chosen from assumptions usually puts most observations in one bucket, which is a counter with extra storage cost.

import time
from statistics import quantiles

samples = []
for _ in range(2000):
    started = time.perf_counter()
    handle_request()
    samples.append(time.perf_counter() - started)

p50, p90, p99 = quantiles(samples, n=100)[49], quantiles(samples, n=100)[89], quantiles(samples, n=100)[98]
print(f"p50 {p50*1000:.1f} ms · p90 {p90*1000:.1f} ms · p99 {p99*1000:.1f} ms")

Expected Output:

p50 41.2 ms · p90 138.6 ms · p99 412.9 ms

Step 2 — Build the ladder around three anchors. The SLO threshold, the observed median, and roughly twice the observed p99 so the tail is bracketed rather than dumped in +Inf.

from prometheus_client import Histogram

REQUEST_LATENCY = Histogram(
    "http_request_duration_seconds",
    "Request latency in seconds",
    ["route", "method"],
    buckets=(
        0.010, 0.025, 0.050,      # around and below the p50 of 41 ms
        0.100, 0.150,             # the p90 region
        0.250,                    # the SLO objective — an exact boundary
        0.400, 0.600,             # around the p99 of 413 ms
        1.000, 2.500,             # the tail, bracketed
    ),
)

Ten boundaries. Every one of them earns its place: three describe the body of the distribution, one is the objective, two bracket the p99, and two catch the tail.

Step 3 — Write the SLO query against the boundary.

# exact: two counters divided by each other
sum(rate(http_request_duration_seconds_bucket{le="0.25"}[30d]))
  /
sum(rate(http_request_duration_seconds_count[30d]))

No histogram_quantile, no interpolation. The compliance figure is as exact as the counters themselves, which is what an SLO report should be.

Step 4 — Budget the series. Buckets multiply with labels, and the product is the number that appears on a storage bill.

The last row is why request identity belongs in an exemplar rather than a label, as set out in controlling label cardinality in Prometheus.

Step 5 — Give each metric family its own ladder. A cache lookup, a web request and a nightly export have nothing in common distributionally.

CACHE_LATENCY = Histogram(
    "cache_operation_duration_seconds", "Cache latency", ["operation"],
    buckets=(0.0001, 0.00025, 0.0005, 0.001, 0.0025, 0.005, 0.01, 0.05),
)

EXPORT_DURATION = Histogram(
    "report_export_duration_seconds", "Report export duration", ["report"],
    buckets=(1.0, 5.0, 15.0, 30.0, 60.0, 120.0, 300.0, 600.0),
)

For OpenTelemetry, the same ladder is expressed as a View, which is covered in configuring views and aggregation in OpenTelemetry metrics.

Configuration options

Verification

Confirm the SLO query against a directly counted number.

from prometheus_client import REGISTRY

under_slo = REGISTRY.get_sample_value(
    "http_request_duration_seconds_bucket",
    {"route": "/orders/{id}", "method": "GET", "le": "0.25"},
)
total = REGISTRY.get_sample_value(
    "http_request_duration_seconds_count", {"route": "/orders/{id}", "method": "GET"},
)
print(f"compliance {under_slo / total:.4%}")

Expected Output:

compliance 99.2140%

Then compare that against a count computed from the raw samples in the same run. They must match exactly — if they differ, the boundary is not where you think it is, most often because the le label is formatted differently from the literal you wrote (0.25 versus 0.250).

Also check that the top bucket is not saturated:

rate(http_request_duration_seconds_bucket{le="+Inf"}[5m])
  - ignoring(le) rate(http_request_duration_seconds_bucket{le="2.5"}[5m])

A non-trivial value means requests are landing beyond the ladder, and every quantile above that point is a guess.

Common mistakes

The SLO number moves when nobody changed anything

Error signature: monthly compliance shifts by a percentage point between reports with no code or traffic change. Root cause: the objective sits between boundaries, so the figure is an interpolation and moves as the distribution's shape inside that bucket shifts. Remediation: add a boundary at exactly the threshold and compute the ratio from counters.

Every observation is in the first bucket

Error signature: all bucket counts are equal, and quantile queries return the lowest boundary. Root cause: the ladder was copied from a metric with a different distribution — usually the library default applied to a sub-millisecond operation. Remediation: measure the distribution and build the ladder around its percentiles.

One histogram is most of the storage bill

Error signature: a single metric name accounts for the majority of series. Root cause: boundaries multiplied by a high-cardinality label. Remediation: cut boundaries first, then labels. Move request identity to an exemplar — see linking metrics to traces with exemplars.

Changing a ladder that is already deployed

Bucket boundaries are part of a metric's identity in Prometheus — each one is its own series — so changing them is not a transparent operation, and doing it carelessly breaks the queries that motivated the change.

What happens. Old boundaries stop receiving data and new ones start from zero. A histogram_quantile query spanning the change reads both sets and produces a result that is wrong in an interesting way: the old buckets have counts and no recent increase, the new ones have a partial history. Rate queries over a window shorter than the time since the change are fine; anything longer is not.

The safe sequence. Add boundaries without removing any, wait out the longest query window you care about, then remove the ones you do not want. Adding is purely additive — new series appear, old ones keep working — and the intermediate state is queryable throughout. Removing afterwards is a cleanup rather than a change.

When the whole ladder must change, for example moving from a millisecond-scale to a second-scale distribution, a new metric name is cleaner than mutating the existing one. Dashboards move at their own pace, the old series ages out with retention, and no query ever spans two incompatible ladders.

That last row deserves emphasis: changing a histogram from milliseconds to seconds without renaming it produces a metric whose historical values are a thousand times off and whose queries return plausible nonsense. The unit belongs in the name — _seconds — precisely so that this mistake is visible in review.

Native and exponential histograms

Both Prometheus and OpenTelemetry now offer a form of histogram whose buckets are generated from a scale factor rather than enumerated: exponential-bucket histograms in OpenTelemetry, native histograms in Prometheus. They remove the bucket-choosing problem entirely — resolution is uniform in relative terms across the whole range — and they are attractive for exactly the metrics where nobody knows the distribution in advance.

The trade for an SLO metric is specific: you cannot place a boundary exactly on the objective, because the boundaries are generated. A compliance figure from an exponential histogram is therefore an interpolation, with an error bounded by the bucket width at that point — small at a reasonable scale, and not zero.

The practical position that follows is a split. Use exponential or native histograms for exploratory and general-purpose latency metrics, where the freedom from bucket choice is worth an interpolation nobody is auditing. Keep explicit buckets for the handful of metrics that back an SLO, where the exact count is the point. Two mechanisms, chosen per metric, rather than one applied everywhere.

A checklist before shipping a new histogram

Five questions, all answerable in a few minutes, and between them they prevent every failure described on this page. What is the distribution — measured, not assumed? Where is the objective, and is there a boundary on it? How many series does this create, boundaries multiplied by label combinations? Is the unit in the name? And is the top boundary high enough that the +Inf bucket is not routinely receiving traffic? A histogram that answers all five is one nobody has to revisit.

Related

• Metric types and cardinality — the parent guide: instrument choice and the series budget.
• Choosing between Counter, Gauge, Histogram and Summary — why latency is a histogram at all.
• Controlling label cardinality in Prometheus — the other half of the series multiplication.
• Configuring views and aggregation in OpenTelemetry metrics — the same ladder, declared as a View.
• Measuring asyncio event loop lag — a metric that needs a very different ladder from request latency.

Frequently Asked Questions