API rate limiting

What a rate limit actually protects

Two different jobs get called "rate limiting" and they want different settings. Capacity protection stops well-meaning clients from saturating a shared resource: a database connection pool, a downstream vendor with its own quota, a queue that grows faster than workers drain it. Abuse prevention stops deliberate misuse: credential stuffing, scraping, enumeration of identifiers. Capacity limits should be generous, transparent and clearly signalled so good clients can adapt. Abuse limits should be quiet, strict and give an attacker no feedback about where the boundary sits. Building one mechanism to do both produces a limit that is too tight for your customers and too loose for your attackers.

There is a third thing that is not rate limiting at all, covered at the end of this section: concurrency limiting. Rate is requests per unit of time; concurrency is requests in flight simultaneously. Ten requests per second that each take a minute will pile up six hundred open connections while never breaching the rate limit.

The algorithms

Fixed window counter

Divide time into fixed windows — each clock minute, say — keep one integer per client per window, increment it on each request, and reject once it exceeds the limit. Memory is one counter per active client, the operation is a single atomic increment, and the reset time is trivially explainable to users. It is by far the cheapest thing to build.

Its flaw is the boundary problem. Suppose the limit is 100 requests per minute. A client sends 100 requests between 10:00:59 and 10:01:00, filling the 10:00 window. The counter resets at 10:01:00, and the client immediately sends another 100 between 10:01:00 and 10:01:01. Both windows are within their limits and no request was rejected — yet the server absorbed 200 requests in about two seconds, twice the nominal rate, in the shortest possible span. In the worst case a fixed window permits 2× the limit across any window-length interval that straddles a boundary. If your limit exists to protect a fragile downstream, that factor of two is exactly the thing you were trying to prevent.

Sliding window log

Store a timestamp for every request in a per-client sorted set. On each new request, discard entries older than the window, count what remains, and allow the request if the count is under the limit. This is exact: at no instant can more than N requests exist in any trailing window, and the boundary problem disappears entirely.

The price is memory and CPU proportional to traffic. A limit of 5,000 per hour means up to 5,000 timestamps retained per client, and the trimming work grows with the request rate — precisely when you are busiest and least want extra work. It is a fine choice for low-volume, high-value limits (password resets, expensive report generation) and a poor one for a general-purpose edge limiter.

Sliding window counter

The standard compromise, and what most production limiters actually run. Keep only two counters — the current window and the previous one — and estimate the rate over a trailing window by taking the whole of the current window's count plus a weighted fraction of the previous window's count. The weight is the proportion of the trailing window that still overlaps the previous window: if you are 25% of the way into the current window, then 75% of the trailing window lies in the previous one, so you count 75% of its requests.

Concretely, with a limit of 100 per minute: the 10:00 window recorded 80 requests, the 10:01 window has recorded 20 so far, and it is now 10:01:15 — 25% into the current minute. The estimate is 20 + 80 × 0.75 = 80, so 20 requests remain before the limit bites. At 10:01:45 the same counts give 20 + 80 × 0.25 = 40, and headroom has grown as the old window ages out. Memory is two integers per client, accuracy is close enough that the 2× boundary spike is gone, and the approximation only misbehaves when traffic is extremely uneven inside a window. That trade is why this algorithm is so widely deployed.

Token bucket

Give each client a bucket with capacity B tokens, refilled at a steady rate R tokens per second up to the capacity. Every request removes one token; if the bucket is empty the request is rejected (or waits). Because unused allowance accumulates, an idle client builds up to B tokens and can then spend them all at once — so the algorithm permits bursts up to B while bounding the long-run average at R.

With B = 100 and R = 10 per second, a client that has been quiet for a while can fire 100 requests immediately, after which it is paced at 10 per second until the bucket refills. That is usually exactly the behaviour you want from a public API: it forgives the natural clumpiness of real clients — a page load that fans out into twenty calls, a nightly job that syncs a batch — while still capping sustained throughput. State is two values per client, the token count and the timestamp of the last refill, and refill is computed lazily on read rather than by a background timer. This is the algorithm most public APIs actually use.

Leaky bucket

A leaky bucket is a queue of finite size drained at a constant rate. Requests arrive and join the queue; the queue leaks one request every 1/R seconds regardless of how full it is; when the queue is full, arrivals are dropped. The output is therefore perfectly smooth — a fixed rate, always — no matter how ragged the input was.

The distinction from token bucket matters and is constantly muddled. Token bucket limits the average and lets bursts through: a burst of 100 arrives at the server as a burst of 100. Leaky bucket enforces a constant outflow and does not let bursts through: a burst of 100 is buffered and released at R per second, or dropped if the buffer is full. Choose leaky bucket when the thing downstream genuinely cannot absorb a spike — a legacy system, a hardware device, a vendor that charges per second of peak. Choose token bucket when you care about total volume and would rather let a short spike through than add latency. Note that queueing means a leaky bucket adds delay, and a queue that fills faster than it drains only postpones the rejection while making every response slower.

Concurrency limiting

Distinct from all of the above: cap the number of requests in flight per client, rather than requests started per interval. A semaphore of size 5 means a client may have five open requests and the sixth waits or is rejected. This is the right control for long-running or resource-heavy work — report exports, large uploads, streaming responses — where the cost is measured in occupied workers and open connections rather than in call count. Most robust APIs run both: a rate limit on cheap calls and a concurrency limit on expensive ones.

Comparison

How a limiter answers

The verdict comes back as a status code, and picking the right one tells the client whether to wait, retry elsewhere, or give up permanently.

• 429 Too Many Requests is the correct answer when an identified client has exceeded its rate. It says: your request was fine, you sent too many, come back later. Include Retry-After whenever you can compute it, and keep the body small — a limiter that returns a fat error document under load is doing the attacker's work.
• 503 Service Unavailable is for shedding load when the service is in trouble, not when a particular client misbehaved. It signals a server-side condition, so a client that receives it should back off but need not assume it did anything wrong. Retry-After is valid and useful here too.
• 403 Forbidden fits a hard quota or a ban: the key has burned its monthly allowance, or has been blocked. Waiting will not help, and saying 429 would invite a retry loop that never succeeds.

You may also see 402 Payment Required used for exhausted paid plans; it is rare and its semantics are still loosely defined, so most APIs use 403 instead.

Retry-After

Retry-After is defined in RFC 9110 and takes two forms, both legal, and a client must handle either:

Note that a bare number is always seconds, never a Unix timestamp — a value like Retry-After: 1793491200 means "wait fifty-six years", which is a genuine bug people ship. The header is not limited to 429: RFC 9110 defines it for 503 and for 3xx redirects as well, where it hints how long a temporary condition will last. When present it is authoritative and beats any delay your client calculated.

Rate-limit headers, and what is actually standardised

The familiar trio — X-RateLimit-Limit, X-RateLimit-Remaining and X-RateLimit-Reset — is a de facto convention, not a standard. No RFC defines it, nothing enforces consistency, and implementations genuinely differ. The worst divergence is Reset: some APIs send a Unix epoch timestamp, others send the number of seconds remaining in the window, and the values are indistinguishable without knowing which. A client that assumes wrongly either hammers the API immediately or sleeps effectively forever. There is no substitute for reading each API's documentation and then confirming against a real response.

There is IETF work to standardise unprefixed RateLimit-Limit, RateLimit-Remaining, RateLimit-Reset and RateLimit-Policy — the "RateLimit header fields for HTTP" specification. It is still a draft. It has not been ratified, and it may change again before it is: successive revisions have reshaped the design significantly, including a move toward a combined RateLimit structured field carrying the values as parameters rather than separate headers. That churn is itself the argument for not building client logic that depends on it yet. Emit the headers if you like, read them defensively, and rely on Retry-After — which genuinely is standardised — for anything load-bearing. The HTTP headers guide covers how these sit among the rest of the response metadata.

Client-side handling

This is where most of the value is. A server-side limiter is a few dozen lines; a client that behaves well when limited is the difference between a degraded minute and a cascading failure.

Respect Retry-After first

If the response carries Retry-After, use it. The server knows when its window resets and your client does not. Override your computed delay with the server's value — but clamp it against a sanity ceiling, because a misconfigured server sending a huge value should not park your worker for a week.

Capped exponential backoff

When there is no Retry-After, wait base × 2^attempt: with a 200 ms base, that is 200 ms, 400 ms, 800 ms, 1.6 s and so on. Two limits keep it sane. Apply a cap — truncated exponential backoff — so delays stop growing at, say, 30 seconds instead of climbing into minutes. Apply a maximum attempt count, typically three to five, so a permanently broken call surfaces as an error rather than retrying forever and hiding the fault.

Jitter, and why it is not optional

Plain exponential backoff has a subtle and severe failure mode. When a server throttles a thousand clients at the same instant, every one of them computes the same delay from the same formula and retries at the same instant. The retry arrives as a synchronised spike, which overloads the server again, re-triggers the limit, and throttles all thousand clients simultaneously once more — now perfectly phase-locked. This is the thundering herd, or retry storm, and backoff alone does not break it; it merely lengthens the interval between identical spikes.

Jitter fixes it by randomising each client's delay so the retries decorrelate and the load arrives as a smooth ramp instead of a wall. Two standard forms:

• Full jitter: sleep = random(0, min(cap, base × 2^n)). Each client picks uniformly from zero up to its computed ceiling. It spreads load extremely well and — counter-intuitively — tends to finish the whole batch of work sooner than plain backoff, because clients that draw a short delay make progress early instead of everyone idling for the full interval.
• Decorrelated jitter: sleep = min(cap, random(base, previous_sleep × 3)), seeded with previous_sleep = base. Each delay is drawn relative to the last one, so the sequence wanders upward rather than doubling in lockstep, keeping spread while avoiding the very short draws that full jitter sometimes produces.

Either is vastly better than none. If you take one thing from this page, take this: exponential backoff without jitter is a synchronisation mechanism wearing the costume of a safety mechanism.

Only retry what is safe to retry

A retry re-sends the request, so it must be harmless to have sent it twice. Safe and idempotent methods qualify: GET, HEAD, PUT and DELETE leave the same state whether sent once or five times — see the HTTP methods reference for exactly which properties each method carries, and PUT vs PATCH for why PATCH is not idempotent in general. POST and PATCH should only be retried when the server supports an Idempotency-Key, which lets it recognise the repeat and return the original result rather than performing the work twice. Blindly retrying a POST /payments after a timeout is how a customer gets charged twice. Note that a 429 is comparatively safe in this respect: the request was rejected before processing, so it almost certainly had no effect — but "almost certainly" is doing real work in that sentence, and an idempotency key removes the doubt.

Never hit the limit in the first place

Retrying is damage control. The better posture is proactive:

• Client-side pacing. Run your own token bucket just under the server's advertised rate and queue outbound calls through it. You control the latency, and you never see a 429.
• Read the remaining count. If the API reports how much allowance is left, slow down as it approaches zero rather than sprinting into the wall.
• Circuit breakers. After a run of failures, open the circuit and fail fast for a cool-down period instead of queueing thousands of doomed requests. Probe with a single call before closing it again.
• Retry budgets. Cap retries as a fraction of total traffic — for example, no more than 10% of requests may be retries. This is what stops a retry policy from tripling your load against an already struggling dependency.
• Batch and cache. The cheapest request is the one you did not send. Conditional requests with ETag and bulk endpoints both cut call volume without any limiter involvement.

A retry loop

MAX_ATTEMPTS = 5
BASE        = 0.2      # seconds
CAP         = 30.0     # seconds

attempt = 0
loop:
    response = send(request)

    if response.status is not 429 and response.status is not 503:
        return response                      # success, or a real error worth surfacing

    attempt = attempt + 1
    if attempt >= MAX_ATTEMPTS:
        fail("rate limited after " + attempt + " attempts")

    # Server guidance wins over any local calculation.
    if response has header "Retry-After":
        if value is an integer:
            delay = that many seconds
        else:
            delay = parse_http_date(value) - now()
        delay = clamp(delay, 0, CAP)
    else:
        ceiling = min(CAP, BASE * (2 ** attempt))
        delay   = random_uniform(0, ceiling)   # full jitter

    sleep(delay)
    goto loop

Retry only if the request is idempotent, or carries an idempotency key. The same loop is worth applying to connection timeouts and 502/504 responses, which are similarly transient.

Designing the server side

What to key on

The limiter needs an identity, and the quality of that identity determines the quality of the limit. In descending order of usefulness: API key or client ID is best, because it maps to a contract and a billing tier and cannot be shared accidentally. User ID is next, for authenticated per-user quotas. IP address is the last resort, and it is unreliable in both directions — an entire office, mobile carrier or CGNAT pool shares one address, so limiting by IP punishes thousands of innocent users together, while IPv6 gives a single attacker effectively unlimited addresses. If you must use IP behind a proxy, take it from a trusted position in X-Forwarded-For, counting from the right-hand side where your own infrastructure appended it; the left-hand entries are attacker-supplied and trivially spoofed. Getting this wrong lets anyone bypass your limiter by adding a header. See API authentication for how the identity is established in the first place.

Tiered, per-endpoint and cost-weighted limits

One global number rarely fits. Tier by plan so paying customers get more headroom. Limit per endpoint so a slow search cannot be called as freely as a cached lookup. Best of all, make requests cost different numbers of tokens: a simple read costs 1, a full-text search costs 10, a report export costs 100. A cost-weighted token bucket lets you express "you may do a lot of cheap things or a few expensive ones" in a single limit that is honest about what actually consumes your capacity.

This becomes essential rather than optional with GraphQL, where a single request can be arbitrarily expensive — one query can fan out across nested collections and touch thousands of rows. Counting requests is meaningless there; you need query-complexity or depth analysis assigning a cost before execution and charging that cost against the budget. The REST vs GraphQL comparison covers why the request-count model does not transfer, and you can experiment with query shapes in the GraphQL tester.

Distributed counters

With more than one server, a per-process counter means the real limit is your limit times your instance count, and it changes every time you autoscale. Shared state — typically Redis — fixes that, but only if the read-modify-write is atomic. Reading a counter, adding one and writing it back is a textbook race: under concurrency several requests read the same value and all conclude they are within the limit. Use an atomic increment with an expiry set on creation, or run the whole check-and-update as a single server-side script so it executes indivisibly. Token buckets need the same care, since refill and consume must happen together.

Every shared-state limiter also adds a network hop to every request. Common mitigations are approximate local buckets that sync periodically, or sharding clients across limiter nodes so each key has one owner — both trading a little accuracy for latency, which is nearly always the right trade.

Fail open or fail closed

Decide in advance what happens when the limiter itself is unavailable. Fail open — allow the request — is right for capacity protection, because turning a Redis blip into a total outage is worse than briefly running unmetered. Fail closed — reject — is right where the limit guards something that must not be unbounded: authentication attempts, payment operations, anything that costs money per call. Whichever you pick, alarm loudly on it, because a limiter that has silently failed open is indistinguishable from one that is working until the day it matters.

Seeing rate limits in practice

This site applies its own limit: 60 requests per minute per IP, which keeps the request proxy responsive for everyone. That is plenty for interactive use and deliberately not enough to drive a load test through.

To inspect any API's limits, send a request from the online API tester and read the response headers pane. Look for a Limit/Remaining/Reset family and note which spelling the API uses and, critically, whether Reset looks like a Unix timestamp (ten digits, starting with 17 for the present decade) or a small number of seconds. Then send the same call a few times in a row from the REST API tester and watch Remaining count down; if you can reach the limit safely, capture the 429 and check whether Retry-After is present and which form it uses. Those two observations are everything your client needs.

One browser-specific trap: JavaScript cannot read most response headers cross-origin unless the server names them in Access-Control-Expose-Headers, so rate-limit headers that are plainly on the wire can appear missing in a front-end app. The CORS guide explains the exposure rules, and the HTTP headers reference lists the fields themselves.

Frequently asked questions