Design an API Gateway / Backend-for-Frontend

Foundations & Core PatternsMediumdesign-api-gateway

Design an API Gateway / Backend-for-Frontend

Design the single front door for a microservice fleet: authentication, routing, aggregation, and versioning, without becoming the bottleneck or the outage.

Gateway AggregationBackend-for-FrontendRequest CollapsingEdge Auth

Traffic & Capacity Estimates:

300k RPS · 40 upstream services · 3 client types (web, iOS, Android)

Functional Requirements

•Terminate TLS, authenticate the caller, and inject a verified identity context into upstream requests.
•Route by path, host, and version, with weighted traffic shifting for canary releases.
•Aggregate several upstream calls into one client response to cut mobile round-trips.
•Enforce per-client quotas and emit per-route metrics, logs, and trace spans.

Non-Functional Requirements

•Added latency under 5ms at p99 for pass-through routes.
•The gateway must not be a single point of failure — it is horizontally scalable and stateless.
•A slow upstream must not exhaust gateway connections available to other routes.

Back-of-the-Envelope Math

300k RPS across 3 regions = 100k RPS/region; at 5k RPS per instance that's 20 instances plus headroom.
Token verification cached for 60s cuts auth-service traffic from 300k to under 5k RPS.

Key Architectural Trade-offs

One shared gateway (uniform policy, contended roadmap) vs a BFF per client type (tailored payloads, duplicated logic).
JWT verified locally at the edge (fast, hard to revoke) vs opaque tokens introspected against the auth service (revocable, one more hop).
Aggregating in the gateway keeps clients simple but puts business logic in infrastructure — the classic path to a distributed monolith.

Click or drag a component onto the canvas, then connect the handles to draw the data flow.

3 nodes · 2 edges

Components · 35

Client & Edge4

Compute & Gateway7

Storage & Caching11

Messaging & Streaming6

Coordination & Ops5

Intelligence2

React Flow