What you'll learn

  1. Quick answer
  2. Why it exists
  3. Fixed window: simple, with a known flaw
  4. Token bucket: the one most systems use
  5. How to reject properly
  6. Being a good client
  7. FAQ

Quick Answer

Rate limiting caps how many requests a client can make in a period. Token bucket is the usual choice because it allows short bursts while capping the sustained rate. Return 429 with a Retry-After header, and clients should back off exponentially.

Why it exists

Four separate reasons, and they call for slightly different limits:

  • Protecting capacity. One client in an accidental infinite loop can consume the resources every other client needs. This is the most common real incident, and it is usually a bug rather than an attack.
  • Cost. If each request triggers a paid downstream call, unlimited requests mean an unlimited bill.
  • Abuse prevention. Rate limiting the login endpoint is what makes password guessing impractical. An unlimited login endpoint is a brute-force service you host for attackers.
  • Fair use. Enforcing plan tiers, where a free tier gets fewer requests than a paid one.

The login case is worth separating out. It needs a much tighter limit than ordinary endpoints, and it should be keyed on both the account and the source address so an attacker cannot spread attempts across many accounts.

Fixed window: simple, with a known flaw

Count requests per client per clock interval. Reset at the boundary.

key = f"rate:{client_id}:{current_minute}"
count = redis.incr(key)
if count == 1:
    redis.expire(key, 60)
if count > LIMIT:
    return 429

Easy to implement and cheap to store. The flaw is the boundary: with a limit of 100 per minute, a client can send 100 at 10:00:59 and another 100 at 10:01:00 — 200 requests in one second, entirely within the rules.

Sliding window log fixes it by storing timestamps and counting those within the last 60 seconds, at the cost of storing every request. Sliding window counter approximates it by weighting the previous window, which is the usual compromise.

Token bucket: the one most systems use

Each client has a bucket holding up to N tokens, refilled at a steady rate. Each request removes one. An empty bucket means rejection.

A bucket of 100 refilling at 10 per second allows a burst of 100 immediately, then a sustained 10 per second. If the client goes quiet, the bucket refills and another burst is available.

This matches how clients genuinely behave. A user opening a page might legitimately fire twenty requests at once, then nothing for a minute. A fixed window either blocks that legitimate burst or has to be set so high it fails to limit anything. Token bucket allows the burst and still caps the long-run rate.

The related leaky bucket smooths output to a constant rate instead, which suits queue-like processing where you want an even flow rather than bursts.

How to reject properly

The status code is 429 Too Many Requests. Alongside it, tell the client what happened and when to return:

HTTP/1.1 429 Too Many Requests
Retry-After: 30
X-RateLimit-Limit: 100
X-RateLimit-Remaining: 0
X-RateLimit-Reset: 1755000000

Retry-After is the important one — it turns guessing into knowing. Returning a bare 429 leaves well-intentioned clients retrying immediately, which makes the situation worse.

Publishing the remaining allowance lets good clients pace themselves before hitting the limit at all, which is better for everyone.

Do not use 403, and do not silently drop the request. Both leave the client unable to distinguish a rate limit from a bug.

Being a good client

The other half, and the one students meet first when an API starts refusing them.

Respect Retry-After. If it says 30 seconds, wait 30 seconds.

Use exponential backoff with jitter when there is no header — wait 1s, then 2s, then 4s, plus a small random amount. The jitter matters: without it, many clients rate-limited simultaneously all retry at the same instant and cause a second spike.

Do not retry 4xx errors other than 429. A 400 means your request was wrong, and it will be wrong again.

Cap total retries and fail cleanly rather than looping forever.

Two design notes for the server side. Key limits on the authenticated user rather than IP where possible, since many users share an IP behind NAT. And apply limits at a shared layer — a gateway or load balancer — so the count is correct across all your servers rather than per instance.

Frequently Asked Questions

What status code should a rate-limited request return? 429 Too Many Requests, ideally with a Retry-After header telling the client how long to wait. A bare 429 leaves clients guessing and retrying too soon.
What is the problem with fixed window rate limiting? Requests can cluster at the window boundary, letting a client send the full allowance at the end of one window and again at the start of the next — double the intended rate in a moment.
Why is token bucket preferred? It allows short bursts, which is how real clients behave, while still capping the sustained rate. Fixed windows must either block legitimate bursts or be set too loosely to help.
Should I rate limit by IP or by user? By authenticated user where possible, since many users can share one IP behind NAT. Use IP for unauthenticated endpoints such as login, ideally combined with the account being targeted.
What is jitter in retry backoff? A small random delay added to each retry interval. Without it, many clients limited at the same moment retry simultaneously and cause another spike.