Design a Distributed Rate Limiter

Foundations & Core PatternsMediumdesign-rate-limiter

Design a low-latency API rate-limiting tier that protects downstream microservices from traffic spikes, DDoS, and noisy neighbors across global regions.

Token BucketSliding WindowFail-OpenRedis Lua Atomicity

Traffic & Capacity Estimates:

1,000,000 requests/second · <2ms overhead · Multi-region synchronization

Functional Requirements

•Limit client requests based on User ID, IP Address, or API Key (e.g., max 100 req/min).
•Return HTTP 429 Too Many Requests with Retry-After header when limit is exceeded.
•Support configurable tiered rules per client plan (Free vs Pro).

Non-Functional Requirements

•Minimal latency overhead (<2ms per request check).
•Accurate distributed tracking without race conditions across multiple server instances.
•Graceful degradation: If rate limiter fails, allow requests through (fail-open) rather than blocking all traffic.

Back-of-the-Envelope Math

1M incoming API checks per second.
Memory footprint: 100M active clients * 64 bytes = ~6.4 GB RAM (easily fits in Redis cluster).

Key Architectural Trade-offs

Token Bucket vs Leaky Bucket vs Sliding Window Counter algorithms.
In-memory Redis cluster with Lua scripts vs local memory cache with asynchronous batch sync.
Fail-open vs Fail-closed resilience policies during Redis partition events.

Click or drag a component onto the canvas, then connect the handles to draw the data flow.

3 nodes · 2 edges

Components · 35

Client & Edge4

Compute & Gateway7

Storage & Caching11

Messaging & Streaming6

Coordination & Ops5

Intelligence2

React Flow