On this page
Rate limiting strategies
Techniques for controlling the rate of traffic sent or received by a network interface controller to prevent resource exhaustion.
What it is
Rate limiting is a defensive mechanism that restricts the number of requests a client can make to a system within a specific timeframe. When a client exceeds the limit, the system rejects subsequent requests, typically returning an HTTP 429 Too Many Requests status code, until the timeframe resets.
It is implemented at the edge (API Gateways, Load Balancers) or within the application logic itself, and is essential for maintaining service availability under heavy load.
Why it matters
Without rate limiting, an API is completely exposed to resource exhaustion. A single rogue client - whether it is a malicious attacker attempting a Denial of Service (DoS) or a well-intentioned partner who accidentally wrote a script with an infinite loop - can overwhelm the server’s CPU, database connection pool, or network bandwidth.
By enforcing a rate limit (e.g., 100 requests per minute per IP address), the system guarantees that no single user can consume all available resources, ensuring fair usage and protecting the platform’s overall uptime.
How to implement
There are several standard algorithms used to calculate rate limits, each with different performance characteristics.
Token Bucket
The most common algorithm. Imagine a bucket that holds a maximum of 10 tokens. Every second, the system adds 1 token to the bucket. When a request arrives, the system removes 1 token. If the bucket is empty, the request is rejected.
- Advantage: It allows for sudden bursts of traffic (up to the maximum bucket size) while enforcing a steady long-term average rate.
Leaky Bucket
Similar to the token bucket, but requests enter the bucket and leak out of the bottom at a strictly constant rate. If the bucket is full, new incoming requests are discarded.
- Advantage: It smooths out bursty traffic into a perfectly steady stream, which is ideal for protecting legacy backend databases that cannot handle concurrency spikes.
Fixed Window Counters
The system tracks the number of requests in fixed time windows (e.g., 12:00 to 12:01). If the limit is 100 requests per minute, and a user makes 100 requests at 12:00:01, they are blocked for the rest of the minute.
- Advantage: Extremely simple to implement using a fast in-memory store like Redis (
INCRwith a TTL). - Disadvantage: Vulnerable to boundary spikes. A user can make 100 requests at 12:00:59, and another 100 requests at 12:01:01, effectively pushing 200 requests in 2 seconds and bypassing the intended physical limits.
Common mistakes
Global vs Local State
Implementing rate limiting strictly in the memory of a single web server fails in a distributed environment. If you have 10 web servers behind a load balancer, and the limit is 100 requests per minute, a client could potentially make 1,000 requests per minute if the traffic is perfectly distributed across all 10 servers. Rate limit counters must be stored in a centralized, low-latency database (like Redis) so all servers share the same state.
Using the Wrong Key
Rate limiting by IP address is common but dangerous. If 500 employees at a corporate office all share the same public NAT IP address, and one employee triggers the rate limit, the other 499 employees are instantly blocked from using your service. Rate limits should ideally be keyed by an authenticated User ID or API Token. IP-based limits should only be used as a coarse-grained defense mechanism against unauthenticated abuse.
Missing Retry-After Headers
When returning a 429 Too Many Requests, a well-designed API must include the Retry-After HTTP header. This tells the client exactly how many seconds they must wait before trying again. Without this header, poorly written clients will simply enter a tight loop, aggressively hitting the server with retries and consuming network bandwidth even though they are being rejected.
Verification
To verify a rate limit is functioning, write a simple bash script that fires requests faster than the limit using curl:
for i in {1..110}; do
curl -s -o /dev/null -w "%{http_code}\n" https://api.example.com/data &
doneIf the limit is 100 requests per minute, you should see exactly 100 200 status codes and exactly 10 429 status codes.
Inspect the response headers of the 429 error:
curl -I https://api.example.com/data