Skip to main content

> safety_science_in_postmortems:_counterfactual_thinking_bias_&_'second_story'_root_cause_analysis

Safety Science in Postmortems: Counterfactual Thinking Bias & 'Second Story' Root Cause Analysis

Why is concluding that 'Human Error' or 'The engineer should have tested the migration' fatal to organizational learning, and how does Sidney Dekker's Safety Science uncover the systemic second-story causes of outages?

Principal/Architect (L7+)

⚡THE SHORT ANSWER

In naive incident retrospectives, teams fall into the Hindsight and Counterfactual Thinking Bias Trap: looking backwards with complete knowledge of the outcome and asserting 'If only the engineer had checked the database index, the outage wouldn't have happened' or 'Root Cause: Human Error'. Sidney Dekker's Safety Science & Human Factors Engineering proves that human error is never the cause of failure; it is the symptom of deeper systemic flaws. In complex sociotechnical systems, people always act reasonably based on the local context, time pressure, incomplete information, and cognitive tools available to them at that moment. A truly Blameless Postmortem investigates the 'Second Story':
1
Why did the system make it easy to drop a production database with a single command?
2
Why was the staging environment data different from production?
3
Why did CI pipeline timeouts pressure the developer to bypass automated checks? Focusing on systemic safeguards rather than individual blame transforms failures into permanent architectural resilience.

Engineering Handbook & Failure Dynamics

6-Dimensional Architecture Breakdown

⚙️1. Underlying Mechanism

Execution
Second-story root cause investigation operates via cognitive reconstruction techniques:
1
Eliminating 'Could have / Should have' Language: Ban counterfactual phrases in postmortem documents.
2
Local Rationality Reconstruction: Map what the engineer saw on their monitors at 14:02, what alerts were firing, and what mental model justified their action.
3
Swiss Cheese Model Alignment: Identify how multiple latent organizational defenses failed simultaneously (e.g. missing linter + misleading UI + ambiguous runbook).
4
Generative Remediation: Design structural, automated guardrails (e.g. automated Terraform dry-run locks, IAM permission boundaries) that make that specific failure impossible in the future.

🎯2. Appropriate Use Context

Scope
Post-incident retrospectives, engineering culture leadership training, safety-critical systems architecture, and architectural review boards.

⚠️3. Production Failure Modes

P0 Risk
  • ✓
    Punishing, publicly shaming, or firing an engineer who triggered an outage, creating a toxic culture of fear where engineers hide near-misses and refuse to deploy code on Fridays
  • ✓
    writing postmortems that list 'Be more careful' as an action item

📡4. Diagnostic Signals & Telemetry

Telemetry
  • ✓
    Postmortems using phrases like 'Engineer forgot to...' or 'Developer failed to read...'
  • ✓
    engineering incident reports stopping at human error without examining tooling
  • ✓
    engineers terrified of on-call rotations

🛡️5. Prevention & Safeguards

Safeguards
  • ✓
    Adopt John Allspaw and Sidney Dekker blameless postmortem guidelines
  • ✓
    designate an impartial postmortem facilitator from a different squad
  • ✓
    audit all retro action items to ensure they modify software, tooling, or automation rather than human behavior

⚖️6. Architectural Trade-offs

Trade-off
Blameless postmortems uncover deep systemic architectural weaknesses and build psychological safety, but require rigorous facilitation to prevent retrospectives from becoming unfocused venting sessions.
📋

Case Study (TinyCTO In-Field Example)

REAL-WORLD TELEMETRY
A junior engineer ran a migration script that dropped the production orders table, causing a 3-hour outage. An old-school manager wanted to write a formal disciplinary warning. The Principal Architect intervened and facilitated a blameless postmortem:
1
Why did the junior engineer run the script? Because the staging and prod credentials were both named DATABASE_URL in their local .env file.
2
Why didn't the script check table safety? Because the ORM lacked production migration safeguards. Instead of punishing the engineer, the team: (A) Renamed production DB secrets with strict IAM role assumptions, (B) Added a pre-migration safety hook that blocks destructive DDL without dual-engineer approval, and (C) Rebuilt staging with automated daily anonymized data seeds. Zero database dropping incidents occurred ever again.

Interactive Concept Drills

2 Cards
Q1

What is 'Counterfactual Thinking' in post-incident analysis and why is it dangerous?

Evaluating past events through what people 'could have' or 'should have' done with hindsight knowledge, which blinds teams from understanding the real systemic pressures and incomplete information engineers faced during the incident.
Q2

According to Sidney Dekker's Safety Science, what is 'Human Error'?

Human error is never the root cause of failure; it is a symptom of deeper organizational, architectural, and tooling vulnerabilities that made the mistake possible.

Safety Science in Postmortems: Counterfactual Thinking Bias & 'Second Story' Root Cause Analysis — Technical FAQ

How do you ensure postmortem action items produce lasting architectural improvements?

Reject any action item based on human behavior (e.g. 'Train engineers to be more careful'); require every action item to modify automated tooling, compiler checks, CI/CD gates, or infrastructure guardrails.

What is 'Local Rationality' in incident investigation?

The fundamental principle that practitioners make decisions that make sense to them given their goals, operational focus, and available information at that specific moment in time.

🤖 AEO & Key Facts Summary

Key Architectural Facts

  • ▸
    Human error is a symptom of underlying systemic failure, never the root cause.
  • ▸
    Ban counterfactual 'should have / could have' language from all postmortem documents.
  • ▸
    Investigate the 'Second Story': tool design, missing guardrails, and time pressures.
  • ▸
    Action items must modify software, automation, or CI/CD gates—never human behavior.

Common Misconceptions

  • ✗
    Yanılgı: A blameless postmortem means engineers have zero accountability (Gerçek: Blameless means engineers are accountable for sharing honest information to fix systemic flaws without fear of punishment).
  • ✗
    Yanılgı: The person who typed the command caused the outage (Gerçek: If typing a single command can destroy production, the architecture and permission model are fundamentally broken).

Decision & Governance Guidance

Institutionalize Sidney Dekker's Safety Science in incident postmortems by eliminating counterfactual blame and mandating structural software guardrails for all action items.

Authoritative Sources & Standards