Instrumenting FastAPI with Prometheus Metrics

A FastAPI service needs three request metrics to support dashboards and SLOs: a counter of requests by route and status, a histogram of latency by route, and a gauge of requests in flight. Recording them correctly takes a little care — route templates rather than raw paths as labels, the real status code even for streaming responses, and aggregation when Uvicorn runs several workers. This article builds a small pure ASGI middleware that does all three. It is part of Prometheus client instrumentation in the Python metrics and instrumentation section, alongside the Flask and Django equivalents.

Prerequisites

pip install "fastapi>=0.110.0,<1.0.0" "uvicorn>=0.29.0,<1.0.0" "prometheus-client>=0.20.0,<1.0.0"

Implementation steps

Step 1 — Define the metrics. Buckets include each latency threshold an SLO will use — here 0.3 seconds — so the latency indicator in defining SLIs from Python request metrics is exact.

# myservice/metrics.py
from prometheus_client import Counter, Gauge, Histogram

REQUESTS = Counter("http_requests_total", "HTTP requests", ["route", "method", "status"])
LATENCY = Histogram("http_request_duration_seconds", "HTTP request latency", ["route", "method"],
                    buckets=(0.01, 0.025, 0.05, 0.1, 0.2, 0.3, 0.5, 1.0, 2.5, 5.0))
IN_FLIGHT = Gauge("http_requests_in_flight", "Requests in flight",
                  multiprocess_mode="livesum")

Step 2 — Write the middleware. A pure ASGI middleware wraps send to observe the response messages directly.

# myservice/middleware.py
import time
from myservice.metrics import REQUESTS, LATENCY, IN_FLIGHT

EXCLUDED = {"/metrics", "/health"}

class PrometheusMiddleware:
    def __init__(self, app):
        self.app = app

    async def __call__(self, scope, receive, send):
        if scope["type"] != "http" or scope["path"] in EXCLUDED:
            return await self.app(scope, receive, send)

        status = 500
        t0 = time.perf_counter()
        IN_FLIGHT.inc()

        async def send_wrapper(message):
            nonlocal status
            if message["type"] == "http.response.start":
                status = message["status"]
            await send(message)

        try:
            await self.app(scope, receive, send_wrapper)
        finally:
            IN_FLIGHT.dec()
            route = getattr(scope.get("route"), "path", None) or "unmatched"
            method = scope["method"]
            LATENCY.labels(route=route, method=method).observe(time.perf_counter() - t0)
            REQUESTS.labels(route=route, method=method, status=str(status)).inc()

The status defaults to 500, so an exception that escapes before any response is sent is counted as a server error, which is what the client will see from the server's error handling.

Step 3 — Read the route template. Starlette's router places the matched route object in scope["route"] once routing has happened, and its path attribute is the template. Because the middleware reads it in finally, after the inner application has run, the value is present for every matched request. Unmatched requests — 404s for random paths — get the fixed value unmatched, so scanners cannot create series.

Step 4 — Wire it up and expose metrics.

# myservice/app.py
from fastapi import FastAPI
from prometheus_client import make_asgi_app
from myservice.middleware import PrometheusMiddleware

app = FastAPI()
app.add_middleware(PrometheusMiddleware)
app.mount("/metrics", make_asgi_app())

@app.get("/orders/{order_id}")
async def get_order(order_id: int):
    return {"id": order_id}

With several Uvicorn workers, make_asgi_app needs a multiprocess registry, and PROMETHEUS_MULTIPROC_DIR must be set before start, as described in metrics in multi-process Python servers:

from prometheus_client import CollectorRegistry, multiprocess
registry = CollectorRegistry()
multiprocess.MultiProcessCollector(registry)
app.mount("/metrics", make_asgi_app(registry=registry))

Streaming responses, background tasks and mounted apps

Three FastAPI features interact with the middleware in ways worth knowing.

Streaming responses send many body messages. The middleware above measures until the application returns, which for a StreamingResponse is after the last chunk — so latency includes the whole stream. A long-lived stream such as server-sent events will record a latency of minutes, which distorts the route's histogram. Excluding streaming routes from the latency histogram, or giving them a separate metric, keeps the main histogram meaningful.

Background tasks run after the response is sent but before the application call returns. Their time is therefore included in the measured latency, even though the client already has its response. When background tasks are significant, capturing the end time on the final body message — more_body false — rather than in finally measures what the client experienced. Tracing FastAPI background tasks covers the same boundary for spans.

Mounted sub-applications have their own routers. For a request to a mounted app, scope["route"] is the Mount itself, whose path is the mount prefix — so all requests to the sub-application share a label. That is often acceptable; if not, the sub-application can carry its own instance of the middleware with a prefix label.

Which status to record, and when

The status code seems like the simplest label, and three situations complicate it.

Handled exceptions raised as HTTPException, or converted by a registered exception handler, produce a normal response with the handler's status, and the middleware records it like any other. This is the common case for 4xx responses and for deliberate 503s from a service shedding load.

Unhandled exceptions escape the application before any response exists. Starlette's outermost error middleware turns them into a 500 for the client, but that layer sits outside user middleware, so this middleware never sees a start message. The default of 500 records what the client received. Re-raising in finally — which the try/finally does implicitly — keeps the exception flowing to the error handler and to any tracing instrumentation.

Client disconnects happen mostly on slow or streaming responses. The status already sent is recorded, which is technically accurate and hides the fact that the client never received the whole response. For services where disconnects indicate latency problems — users giving up — a separate counter incremented on asyncio.CancelledError makes them visible without distorting the status breakdown.

Recording business metrics next to request metrics

Request metrics describe the service's behaviour. Business metrics — orders placed, payments failed, searches with no results — describe what the service is for, and they often reveal problems that request metrics cannot: every request returns 200, and orders have stopped because a downstream pricing service returns zeros.

Business metrics belong in the endpoint or service layer, recorded where the event is known, with labels drawn from bounded sets — payment method, not customer; product category, not product. The same multiprocess considerations apply, and the same naming rules, covered in naming metrics and choosing units. A dashboard that puts orders per minute beside request rate makes a silent business failure visible within minutes, and an alert on the ratio of orders to checkout requests is often the most valuable alert a commerce service has.

Overhead and where it comes from

The middleware adds a handful of operations to each request: two clock reads, a gauge increment and decrement, a histogram observation and a counter increment, plus two label lookups. With the in-memory backend that totals a few microseconds. With multiprocess mode each operation writes to a memory-mapped file, and the histogram writes one value per bucket up to the observed one, bringing the total closer to ten microseconds. Against a FastAPI endpoint that takes even a millisecond, both are negligible.

The cost that does grow is label lookup. labels() hashes the label values to find the child metric, and a high number of distinct combinations makes that lookup and the memory it uses grow too. Keeping labels bounded — routes, methods, status codes — keeps the cost flat. Caching the child objects for the hottest routes is possible and rarely worth the complexity. Profiling a real service, as in profiling a live Python process with py-spy, almost always shows the metrics middleware well below one percent of CPU time.

Configuration options

Verification

for i in $(seq 200); do curl -s -o /dev/null localhost:8000/orders/$i; done
curl -s localhost:8000/metrics | grep '^http_requests_total{'

Expected Output: one series for the route, counting exactly 200, regardless of how many distinct order identifiers were requested.

http_requests_total{method="GET",route="/orders/{order_id}",status="200"} 200.0

Requesting a nonexistent path should add a single route="unmatched",status="404" series, and requesting a hundred more nonexistent paths should increment it rather than creating new series.

Common mistakes

request.url.path as the label. Error signature: series count in the hundreds of thousands. Root cause: identifiers in paths. Remediation: the route template from the scope.

Reading the route before calling the app. Error signature: every request labelled unmatched. Root cause: routing has not happened yet. Remediation: read it after the inner application returns.

BaseHTTPMiddleware with streaming. Error signature: streaming responses buffered or background tasks behaving oddly. Root cause: the wrapper's response handling. Remediation: a pure ASGI middleware.

No multiprocess mode with --workers. Error signature: counters that jump between scrapes. Root cause: one registry per worker. Remediation: the multiprocess directory and registry.

Server-sent events in the latency histogram. Error signature: a p99 of several minutes on an otherwise fast service. Root cause: streams recorded as requests. Remediation: exclude them or give them their own metric.

Status label as an integer in one place and a string in another. Error signature: two series for the same status, and SLI ratios that do not add up. Root cause: inconsistent label types. Remediation: always convert with str() in the one place that records.

Frequently Asked Questions