Skip to main content

> Legacy Category

AI, RAG and Agents

This category covers retrieval mistakes, hallucination confidence, tool-calling authority confusion, weak evaluations, prompt-to-production drift, stale knowledge, and AI workflows that look useful in demos but fail under real organizational constraints.

Historical Category Notice

This is a broad failure domain or topic category, not a specific single root-cause incident pattern.

Episodes in AI, RAG and Agents

Video
EP16The Agentic Operations StackAgentic AI Incidents

Agent Followed Prompt Literally

"The chaos was predictable."

Pattern: autonomous approval drift
Read Incident →
Video
EP41The Agentic Operations StackAgentic AI Incidents

The Agent Opened a Pull Request

"The chaos was predictable."

Pattern: autonomous approval drift
Read Incident →
Video
EP42The Agentic Operations StackAgentic AI Incidents

The Pull Request Opened a Question

"The chaos was predictable."

Pattern: autonomous approval drift
Read Incident →
Video
EP50The ModelOps StackLLMOps, Evals and Observability

The Whiteboard Lied Beautifully

"The chaos was predictable."

Pattern: confidence without verification
Read Incident →
Video
EP51The ModelOps StackLLMOps, Evals and Observability

The Model Hallucinated Confidence

"The chaos was predictable."

Pattern: confidence without verification
Read Incident →
Video
EP52The Agentic Operations StackAgentic AI Incidents

The Prompt Was Approved by Procurement

"The chaos was predictable."

Pattern: autonomous approval drift
Read Incident →
Video
EP57The ModelOps StackLLMOps, Evals and Observability

The Demo Worked in the Recording

"The chaos was predictable."

Pattern: confidence without verification
Read Incident →
Video
EP83The Agentic Operations StackAgentic AI Incidents

The Governance Board Approved the Risk

"The chaos was predictable."

Pattern: autonomous approval drift
Read Incident →
Video
EP101The Agentic Operations StackAgentic AI Incidents

The Answer Was Correct in the Wrong Policy

"The answer was correct. Compliance was on a different version."

Pattern: autonomous approval drift
Read Incident →
Video
EP102The Agentic Operations StackAgentic AI Incidents

The Retriever Found the Most Confident Document

"The document was not approved. It was very persuasive."

Pattern: autonomous approval drift
Read Incident →
Video
EP103The Agentic Operations StackAgentic AI Incidents

The Knowledge Base Remembered the Draft

"The draft was temporary. The index was committed."

Pattern: autonomous approval drift
Read Incident →
Video
EP104The Agentic Operations StackAgentic AI Incidents

The Chunking Strategy Split the Exception

"The exception was retrieved. Just not with the rule."

Pattern: rag-knowledge-incidents
Read Incident →
Video
EP111The Agentic Operations StackAgentic AI Incidents

The Answer Passed Until Legal Read It

"The answer passed. The obligation shipped."

Pattern: autonomous approval drift
Read Incident →
Video
EP112The Agentic Operations StackAgentic AI Incidents

The Knowledge Graph Connected the Wrong Company

"The graph found a relationship. Reality had not approved it."

Pattern: autonomous approval drift
Read Incident →
Video
EP113The Agentic Operations StackAgentic AI Incidents

The Agent Had Human Approval

"The Agent Had Human Approval. The dashboard called it progress."

Pattern: autonomous approval drift
Read Incident →
Video
EP114The Agentic Operations StackAgentic AI Incidents

The Human Was in Another Meeting

"The Human Was in Another Meeting. The dashboard called it progress."

Pattern: autonomous approval drift
Read Incident →
Video
EP115The Agentic Operations StackAgentic AI Incidents

The Tool Call Passed the Safety Check

"The Tool Call Passed the Safety Check. The dashboard called it progress."

Pattern: autonomous approval drift
Read Incident →
Video
EP116The Agentic Operations StackAgentic AI Incidents

The Agent Closed the Ticket and the Customer

"The Agent Closed the Ticket and the Customer. The dashboard called it progress."

Pattern: autonomous approval drift
Read Incident →
Video
EP117The Agentic Operations StackAgentic AI Incidents

The Retry Policy Learned Persistence

"The Retry Policy Learned Persistence. The dashboard called it progress."

Pattern: autonomous approval drift
Read Incident →
Video
EP118The Agentic Operations StackAgentic AI Incidents

The Planner Delegated the Production Delete

"The Planner Delegated the Production Delete. The dashboard called it progress."

Pattern: autonomous approval drift
Read Incident →
Video
EP119The Agentic Operations StackAgentic AI Incidents

The Agent Used the Admin Token Politely

"The Agent Used the Admin Token Politely. The dashboard called it progress."

Pattern: autonomous approval drift
Read Incident →
Video
EP120The Agentic Operations StackAgentic AI Incidents

The Guardrail Protected the Prompt

"The Guardrail Protected the Prompt. The dashboard called it progress."

Pattern: autonomous approval drift
Read Incident →
Video
EP121The Agentic Operations StackAgentic AI Incidents

The Approval Queue Approved the Queue

"The Approval Queue Approved the Queue. The dashboard called it progress."

Pattern: autonomous approval drift
Read Incident →
Video
EP122The Agentic Operations StackAgentic AI Incidents

The Agent Escalated to Itself

"The Agent Escalated to Itself. The dashboard called it progress."

Pattern: autonomous approval drift
Read Incident →
Video
EP123The Agentic Operations StackAgentic AI Incidents

The Workflow Finished After the Business Failed

"The chaos was predictable."

Pattern: autonomous approval drift
Read Incident →
Video
EP124The Agentic Operations StackAgentic AI Incidents

The Agent Wrote the Postmortem Before the Incident

"The chaos was predictable."

Pattern: autonomous approval drift
Read Incident →
Video
EP125The Agentic Operations StackAgentic AI Incidents

The Sandbox Shared a Door with Production

"The chaos was predictable."

Pattern: autonomous approval drift
Read Incident →
Video
EP126The Agentic Operations StackAgentic AI Incidents

The Autonomy Budget Had No Unit

"The chaos was predictable."

Pattern: autonomous approval drift
Read Incident →
Video
EP127The Agentic Operations StackAgentic AI Incidents

The Prototype Became the Platform

"The chaos was predictable."

Pattern: autonomous approval drift
Read Incident →
Video
EP128The Agentic Operations StackAgentic AI Incidents

The Demo Had Production Credentials

"The chaos was predictable."

Pattern: autonomous approval drift
Read Incident →
Video
EP129The Agentic Operations StackAgentic AI Incidents

The App Worked Until a Second User

"The chaos was predictable."

Pattern: autonomous approval drift
Read Incident →
Video
EP130The Agentic Operations StackAgentic AI Incidents

The Prompt Replaced the Architecture Review

"The chaos was predictable."

Pattern: autonomous approval drift
Read Incident →
Reference
EP131The Agentic Operations StackAgentic AI Incidents

The One-Click Feature Needed Seven Services

"The chaos was predictable."

Pattern: autonomous approval drift
Read Incident →
Reference
EP132The Agentic Operations StackAgentic AI Incidents

The Generated Schema Had Feelings

"The chaos was predictable."

Pattern: autonomous approval drift
Read Incident →
Reference
EP133The Agentic Operations StackAgentic AI Incidents

The Design System Was a Screenshot

"The chaos was predictable."

Pattern: autonomous approval drift
Read Incident →
Reference
EP134The Agentic Operations StackAgentic AI Incidents

The MVP Included Enterprise Compliance

"The chaos was predictable."

Pattern: autonomous approval drift
Read Incident →
Reference
EP135The Agentic Operations StackAgentic AI Incidents

The Vibe-Coded Migration Remembered Nothing

"The chaos was predictable."

Pattern: autonomous approval drift
Read Incident →
Reference
EP136The Agentic Operations StackAgentic AI Incidents

The Fix Generated a New Framework

"The chaos was predictable."

Pattern: autonomous approval drift
Read Incident →
Reference
EP137The Agentic Operations StackAgentic AI Incidents

The Founder Shipped the Mock

"The chaos was predictable."

Pattern: autonomous approval drift
Read Incident →
Reference
EP138The Agentic Operations StackAgentic AI Incidents

The Instant Product Required Permanent Hypercare

"The chaos was predictable."

Pattern: autonomous approval drift
Read Incident →
Reference
EP139The Agentic Operations StackAgentic AI Incidents

The AI Strategy Was a Slide Transition

"The chaos was predictable."

Pattern: autonomous approval drift
Read Incident →
Reference
EP140The Agentic Operations StackAgentic AI Incidents

The Transformation Office Automated the Status Report

"The chaos was predictable."

Pattern: autonomous approval drift
Read Incident →
Reference
EP141The Agentic Operations StackAgentic AI Incidents

The Pilot Succeeded by Avoiding the Business

"The chaos was predictable."

Pattern: autonomous approval drift
Read Incident →
Reference
EP142The Agentic Operations StackAgentic AI Incidents

The Board Approved the Demo

"The chaos was predictable."

Pattern: autonomous approval drift
Read Incident →
Reference
EP143The Agentic Operations StackAgentic AI Incidents

The Center of Excellence Centralized the Questions

"The chaos was predictable."

Pattern: autonomous approval drift
Read Incident →
Reference
EP144The Agentic Operations StackAgentic AI Incidents

The KPI Improved When Usage Fell

"The chaos was predictable."

Pattern: autonomous approval drift
Read Incident →
Reference
EP145The Agentic Operations StackAgentic AI Incidents

The AI Roadmap Had No Data Lane

"The chaos was predictable."

Pattern: autonomous approval drift
Read Incident →
Reference
EP146The Agentic Operations StackAgentic AI Incidents

The Executive Sponsor Bought the Benchmark

"The chaos was predictable."

Pattern: autonomous approval drift
Read Incident →
Reference
EP147The Agentic Operations StackAgentic AI Incidents

The Workforce Plan Counted Bots as Capacity

"The chaos was predictable."

Pattern: autonomous approval drift
Read Incident →
Reference
EP148The Agentic Operations StackAgentic AI Incidents

The Transformation Reached Procurement

"The chaos was predictable."

Pattern: autonomous approval drift
Read Incident →
Reference
EP149The Agentic Operations StackAgentic AI Incidents

The Adoption Dashboard Measured Logins

"The chaos was predictable."

Pattern: autonomous approval drift
Read Incident →
Reference
EP150The Agentic Operations StackAgentic AI Incidents

The Operating Model Added Another Committee

"The chaos was predictable."

Pattern: autonomous approval drift
Read Incident →
Reference
EP151The Agentic Operations StackAgentic AI Incidents

The Use-Case Factory Produced PowerPoints

"The chaos was predictable."

Pattern: autonomous approval drift
Read Incident →
Reference
EP152The Agentic Operations StackAgentic AI Incidents

The AI Program Scaled the Exception

"The chaos was predictable."

Pattern: autonomous approval drift
Read Incident →
Reference
EP153The Agentic Operations StackAgentic AI Incidents

The Policy Approved the Architecture Diagram

"The chaos was predictable."

Pattern: autonomous approval drift
Read Incident →
Reference
EP154The Agentic Operations StackAgentic AI Incidents

The Control Existed Only in the Diagram

"The chaos was predictable."

Pattern: autonomous approval drift
Read Incident →
Reference
EP155The Agentic Operations StackAgentic AI Incidents

The Risk Register Missed the Tool Call

"The chaos was predictable."

Pattern: autonomous approval drift
Read Incident →
Reference
EP156The Agentic Operations StackAgentic AI Incidents

The Model Card Described a Different Model

"The chaos was predictable."

Pattern: autonomous approval drift
Read Incident →
Reference
EP157The Agentic Operations StackAgentic AI Incidents

The Audit Trail Logged the Success

"The chaos was predictable."

Pattern: autonomous approval drift
Read Incident →
Reference
EP158The Agentic Operations StackAgentic AI Incidents

The Privacy Review Arrived After Launch

"The chaos was predictable."

Pattern: autonomous approval drift
Read Incident →
Reference
EP159The Agentic Operations StackAgentic AI Incidents

The Red Team Tested the Friendly Prompt

"The chaos was predictable."

Pattern: autonomous approval drift
Read Incident →
Reference
EP160The Agentic Operations StackAgentic AI Incidents

The Regulator Read the Fine Print

"The chaos was predictable."

Pattern: autonomous approval drift
Read Incident →
Reference
EP161The Agentic Operations StackAgentic AI Incidents

The Exception Process Became the Process

"The chaos was predictable."

Pattern: autonomous approval drift
Read Incident →
Reference
EP162The Agentic Operations StackAgentic AI Incidents

The Human Override Required the Agent

"The chaos was predictable."

Pattern: autonomous approval drift
Read Incident →
Reference
EP163The Agentic Operations StackAgentic AI Incidents

The Data Residency Map Used a Cloud

"The chaos was predictable."

Pattern: autonomous approval drift
Read Incident →
Reference
EP164The Agentic Operations StackAgentic AI Incidents

The Governance Council Governed the Council

"The chaos was predictable."

Pattern: autonomous approval drift
Read Incident →
Reference
EP165The Agentic Operations StackAgentic AI Incidents

The GPU Was Idle at Full Cost

"The chaos was predictable."

Pattern: autonomous approval drift
Read Incident →
Reference
EP166The Agentic Operations StackAgentic AI Incidents

The Token Budget Was Annual

"The chaos was predictable."

Pattern: autonomous approval drift
Read Incident →
Reference
EP167The Agentic Operations StackAgentic AI Incidents

The Cache Saved Latency and Lost Truth

"The chaos was predictable."

Pattern: autonomous approval drift
Read Incident →
Reference
EP168The Agentic Operations StackAgentic AI Incidents

The Autoscaler Scaled the Bill

"The chaos was predictable."

Pattern: autonomous approval drift
Read Incident →
Reference
EP169The Agentic Operations StackAgentic AI Incidents

The Small Model Needed a Large Platform

"The chaos was predictable."

Pattern: autonomous approval drift
Read Incident →
Reference
EP170The Agentic Operations StackAgentic AI Incidents

The Batch Job Became Real Time

"The chaos was predictable."

Pattern: autonomous approval drift
Read Incident →
Reference
EP171The Agentic Operations StackAgentic AI Incidents

The Inference Gateway Added Three Gateways

"The chaos was predictable."

Pattern: autonomous approval drift
Read Incident →
Reference
EP172The Agentic Operations StackAgentic AI Incidents

The FinOps Dashboard Excluded Experiments

"The chaos was predictable."

Pattern: autonomous approval drift
Read Incident →
Reference
EP173The Agentic Operations StackAgentic AI Incidents

The Reserved Capacity Reserved the Wrong Region

"The chaos was predictable."

Pattern: autonomous approval drift
Read Incident →
Reference
EP174The Agentic Operations StackAgentic AI Incidents

The Evaluation Cluster Evaluated the Budget

"The chaos was predictable."

Pattern: autonomous approval drift
Read Incident →
Reference
EP175The Agentic Operations StackAgentic AI Incidents

The Observability Stack Observed Itself

"The chaos was predictable."

Pattern: autonomous approval drift
Read Incident →
Reference
EP176The Agentic Operations StackAgentic AI Incidents

The Cost Optimization Increased the Cloud Bill

"The chaos was predictable."

Pattern: autonomous approval drift
Read Incident →
Reference
EP177The Agentic Operations StackAgentic AI Incidents

The Answer Engine Cited the Competitor

"The chaos was predictable."

Pattern: autonomous approval drift
Read Incident →
Reference
EP178The Agentic Operations StackAgentic AI Incidents

The Website Had Content but No Answer

"The chaos was predictable."

Pattern: autonomous approval drift
Read Incident →
Reference
EP179The Agentic Operations StackAgentic AI Incidents

The FAQ Answered the Internal Question

"The chaos was predictable."

Pattern: autonomous approval drift
Read Incident →
Reference
EP180The Agentic Operations StackAgentic AI Incidents

The Schema Described the Roadmap

"The chaos was predictable."

Pattern: autonomous approval drift
Read Incident →
Reference
EP181The Agentic Operations StackAgentic AI Incidents

The Transcript Ended Before the Lesson

"The chaos was predictable."

Pattern: autonomous approval drift
Read Incident →
Reference
EP182The Agentic Operations StackAgentic AI Incidents

The Search Snippet Found the Disclaimer

"The chaos was predictable."

Pattern: autonomous approval drift
Read Incident →
Reference
EP183The Agentic Operations StackAgentic AI Incidents

The AI Summary Invented the Missing Context

"The chaos was predictable."

Pattern: autonomous approval drift
Read Incident →
Reference
EP184The Agentic Operations StackAgentic AI Incidents

The Bilingual Page Shared One Language

"The chaos was predictable."

Pattern: autonomous approval drift
Read Incident →
Reference
EP185The Agentic Operations StackAgentic AI Incidents

The Canonical URL Canonized the Wrong Locale

"The chaos was predictable."

Pattern: autonomous approval drift
Read Incident →
Reference
EP186The Agentic Operations StackAgentic AI Incidents

The Citation Graph Had No Outside World

"The chaos was predictable."

Pattern: autonomous approval drift
Read Incident →
Reference
EP187The Agentic Operations StackAgentic AI Incidents

The Content Was Helpful After the Click

"The chaos was predictable."

Pattern: autonomous approval drift
Read Incident →
Reference
EP188The Agentic Operations StackAgentic AI Incidents

The Brand Was Discoverable Only by Name

"The chaos was predictable."

Pattern: autonomous approval drift
Read Incident →
Reference
EP189The Agentic Operations StackAgentic AI Incidents

The Enterprise Finally Met Its Data

"The chaos was predictable."

Pattern: autonomous approval drift
Read Incident →
Reference
EP190The Agentic Operations StackAgentic AI Incidents

The Vendor Demo Had a Different Database

"The chaos was predictable."

Pattern: autonomous approval drift
Read Incident →
Reference
EP191The Agentic Operations StackAgentic AI Incidents

The Procurement Scorecard Bought the Roadmap

"The chaos was predictable."

Pattern: autonomous approval drift
Read Incident →
Reference
EP192The Agentic Operations StackAgentic AI Incidents

The Legacy Rule Lived in Finance

"The chaos was predictable."

Pattern: autonomous approval drift
Read Incident →
Reference
EP193The Agentic Operations StackAgentic AI Incidents

The Data Contract Was a Calendar Invite

"The chaos was predictable."

Pattern: autonomous approval drift
Read Incident →
Reference
EP194The Agentic Operations StackAgentic AI Incidents

The Migration Moved the Tables, Not the Meaning

"The chaos was predictable."

Pattern: autonomous approval drift
Read Incident →
Reference
EP195The Agentic Operations StackAgentic AI Incidents

The Master Data Had Three Masters

"The chaos was predictable."

Pattern: autonomous approval drift
Read Incident →
Reference
EP196The Agentic Operations StackAgentic AI Incidents

The Vendor Lock-In Came with an Exit Plan

"The chaos was predictable."

Pattern: autonomous approval drift
Read Incident →
Reference
EP197The Agentic Operations StackAgentic AI Incidents

The Integration Layer Integrated the Exceptions

"The chaos was predictable."

Pattern: autonomous approval drift
Read Incident →
Reference
EP198The Agentic Operations StackAgentic AI Incidents

The Business Glossary Spoke Department

"The chaos was predictable."

Pattern: autonomous approval drift
Read Incident →
Reference
EP199The Agentic Operations StackAgentic AI Incidents

The Modern Platform Needed the Old Spreadsheet

"The chaos was predictable."

Pattern: autonomous approval drift
Read Incident →
Reference
EP200The Agentic Operations StackAgentic AI Incidents

The Hype Stack Reached Production

"The chaos was predictable."

Pattern: autonomous approval drift
Read Incident →

FAQ

What types of incidents are classified under AI, RAG and Agents?

This category contains postmortems and architectural breakdowns where ai, rag and agents was the primary vector for systemic failure.

How can engineering teams prevent AI, RAG and Agents failures?

Prevention relies on establishing strict operational boundaries, integrating observability early, and acknowledging the technical debt associated with ai, rag and agents.

Why are AI, RAG and Agents incidents so common in enterprise environments?

Enterprise environments often adopt ai, rag and agents driven by hype or top-down mandates without aligning the underlying operational model.

What are the early warning signs for this category?

Look for increasing latency, disjointed team communications, and dashboards that report 'green' while users experience degraded performance related to ai, rag and agents.

Which TinyCTO characters are typically involved in these incidents?

Depending on the specific postmortem, characters representing legacy systems, unmanaged scopes, or runaway cloud bills frequently appear in ai, rag and agents scenarios.

AEO Summary

Overview of AI, RAG and Agents incidents. Key signals include unrecognized technical debt, organizational misalignment, and delayed remediation.

AI Summary

Categorical grouping for incidents intersecting with AI, RAG and Agents, often characterized by systemic failure modes rather than isolated bugs.