Skip to main content

> Term

repeated failure mode

Incidents that occur multiple times because the underlying system incentives, architecture, or organizational habits remain unchanged.

Detailed Explanation

A repeated failure mode is an incident that is technically "fixed" during every postmortem, yet reliably recurs months later. It is the operational equivalent of treating symptoms without addressing the disease.

If your action items from the last three outages all include "add more monitoring," you are experiencing a repeated failure mode.

These failures persist because the system—whether it's the codebase, the deployment pipeline, or the company culture—actually incentivizes the behavior that causes the failure. Until the underlying structural issue is resolved, the incident will return.

Why It Matters

It drains engineering morale and consumes cycles that should be spent on feature development, while steadily degrading trust in the platform.

Common Failure Mode

Postmortem action items focus exclusively on human error or superficial fixes (like increasing a timeout) rather than redesigning the flawed architecture.

Practical Example

A caching service crashes under load every Black Friday. The team increases instance sizes and adds a temporary read replica. The next year, it crashes again because the fundamental query pattern was never optimized.

Production Manifestation

The same service runs out of memory or connection pool limits hit the ceiling every time traffic spikes, despite multiple previous tuning attempts.

Frequently Asked Questions

What is repeated failure mode in short?

Incidents that occur multiple times because the underlying system incentives, architecture, or organizational habits remain unchanged.

What is the most common failure mode?

Postmortem action items focus exclusively on human error or superficial fixes (like increasing a timeout) rather than redesigning the flawed architecture.

AI Summary

Incidents that occur multiple times because the underlying system incentives, architecture, or organizational habits remain unchanged. It drains engineering morale and consumes cycles that should be spent on feature development, while steadily degrading trust in the platform.