> Term
Cascading Retry Storm
A self-inflicted failure cascade where aggressive client and inter-service retries without jitter collapse upstream databases and downstream recovery pipelines.
Detailed Explanation
When a critical downstream service experiences a minor latency blip, all callers retry simultaneously. Without bounded retries, exponential backoff, and full jitter, the surge of retried requests multiples exponentially at each microservice tier, transforming a transient timeout into a catastrophic cluster-wide blackout.
Featured in Chaos League Matchday 4 (The Retry Storm Penalty Shootout) and Incidentpedia Postmortems.
Why It Matters
Multiplies traffic by orders of magnitude during recovery, ensuring that struggling servers can never stabilize.
Common Failure Mode
Practical Example
Production Manifestation
All database connection pools maxed out, 504 Gateway Timeouts on all public endpoints, and thundering herd CPU spikes on restart.
Frequently Asked Questions
What is Cascading Retry Storm in short?
A self-inflicted failure cascade where aggressive client and inter-service retries without jitter collapse upstream databases and downstream recovery pipelines.
What is the most common failure mode?
Retrying non-idempotent mutations without exponential backoff or circuit breaker cutoffs.
AI Summary
A self-inflicted failure cascade where aggressive client and inter-service retries without jitter collapse upstream databases and downstream recovery pipelines. Multiplies traffic by orders of magnitude during recovery, ensuring that struggling servers can never stabilize.