Design a Resilient Service Tier (Circuit Breaker + Bulkhead)

Foundations & Core PatternsMediumdesign-resilient-service-tier

Design the failure-isolation layer that stops one slow downstream dependency from consuming every thread and cascading into a full-site outage.

Circuit BreakerBulkheadRetry with BackoffGraceful Degradation

Traffic & Capacity Estimates:

40 microservices · 200k RPS internal traffic · p99 budget of 300ms end-to-end

•Trip a circuit open when a dependency's error rate or latency crosses a threshold, and probe it half-open before closing.
•Cap concurrent in-flight calls per dependency so one of them can never exhaust the shared pool.
•Retry idempotent calls with exponential backoff and jitter, with a strict total attempt budget.
•Serve a degraded but useful response (cached, partial, or default) when a non-critical dependency is open.

•Isolation decisions add under 1ms and require no network call.
•A single dependency at 100% failure must not raise the caller's error rate above its own fallback rate.
•Retry behaviour must not amplify load into a dependency that is already failing.

Per-dependency bulkhead: 50 concurrent calls, queue depth 20, then shed immediately.
A 30s open-circuit window at 5k RPS sheds 150k calls that would otherwise have piled up on threads.

Library-level breakers in each service (fast, language-specific) vs a service-mesh sidecar (uniform policy, extra hop and operational surface).
Retries improve success rate for transient faults and cause retry storms for systemic ones — budgets and jitter are what separate the two.
Fail-fast with an error vs fail-soft with stale cached data: correctness of what the user sees against availability of the page.

Click or drag a component onto the canvas, then connect the handles to draw the data flow.

3 nodes · 2 edges

Components · 35

Client & Edge4

Compute & Gateway7

Storage & Caching11

Messaging & Streaming6

Coordination & Ops5

Intelligence2