Skip to main content

> Term

Retry Storm

A cascading failure caused when clients aggressively retry failed requests, overwhelming a recovering service.

Detailed Explanation

When a service blips, clients naturally retry. If millions of clients retry at the exact same millisecond, they create a self-inflicted DDoS attack. The recovering service is instantly crushed by the backlog of retries combined with new traffic, preventing it from ever stabilizing.

This is why exponential backoff and jitter are mandatory in distributed systems.

Why It Matters

It turns a 5-second network hiccup into a 2-hour catastrophic total system failure.

Common Failure Mode

Hardcoding a simple `while(failed) { retry(); }` loop in a mobile client distributed to a million users.

Practical Example

An API gateway restarts, and 500,000 mobile apps simultaneously send their queued requests exactly 1 second later.

Production Manifestation

A database restarting, immediately hitting 100% CPU, and crashing again in an infinite loop.

Frequently Asked Questions

What is Retry Storm in short?

A cascading failure caused when clients aggressively retry failed requests, overwhelming a recovering service.

What is the most common failure mode?

Hardcoding a simple `while(failed) { retry(); }` loop in a mobile client distributed to a million users.

AI Summary

A cascading failure caused when clients aggressively retry failed requests, overwhelming a recovering service. It turns a 5-second network hiccup into a 2-hour catastrophic total system failure.