Phase 5
Operations
Resilience, reliability, and how to run systems in production.
Pause & reflect
Before you read on, answer this in your head — or on paper:
Service A calls Service B. Service B starts failing — slow or timeout.
What happens to Service A? If it keeps trying, does it make things worse? How do you stop a cascading failure?
Circuit Breaker
Prevents cascading failures when a remote service is slow or down. Closed: requests pass through. Failures increment counter. Open: after threshold, fail fast — no calls to the service. After timeout, move to half-open. Half-open: allow a few test requests. Success → closed; failure → open. Protects callers and downstream systems.
Rate Limiting Algorithms
Leaky bucket: queue requests; process at fixed rate. Full queue = drop. Token bucket: refill tokens over time; consume one per request; reject when empty. Fixed window: count per N-second window; simple but bursts at boundaries. Sliding log: store timestamps; count in last N seconds — accurate but heavy. Sliding window: hybrid of fixed + sliding for better boundaries with lower cost. Distributed: use Redis with atomic ops; avoid race conditions with get-then-set.
Retries & backoff
Exponential backoff: 1s, 2s, 4s, … Jitter to avoid thundering herd. Idempotency keys for writes so retries are safe. Circuit breaker complements retries — stop if service is clearly down.
Security & auth
OAuth 2.0 (authorization), OIDC (identity), SSO (single sign-on), mTLS (service-to-service auth). Important for production—covered in depth in security-focused resources. For interviews, mention OAuth for user auth, mTLS for service mesh.