Skip to main content

> change_data_capture_with_debezium_&_outbox_event_ordering

Change Data Capture with Debezium & Outbox Event Ordering

How do you implement and govern Change Data Capture with Debezium & Outbox Event Ordering in high-throughput production architectures?

Stack: SOFTWARE ARCHITECTURE STACK →Staff/Principal (L6+)pattern

⚡THE SHORT ANSWER

The CDC Outbox pattern combines the Transactional Outbox pattern with Debezium log mining to stream database mutations directly from the database's Write-Ahead Log (WAL) into partitioned Kafka topics, guaranteeing 100% reliable event delivery and strict per-entity causal ordering without dual-write race conditions.

Engineering Handbook & Failure Dynamics

6-Dimensional Architecture Breakdown

⚙️1. Underlying Mechanism

Execution

Writing to a relational database and publishing to a Kafka topic in the same HTTP request is the classic 'Dual Write' anti-pattern: if the database commits but the network fails before Kafka publishes, events are lost forever. The CDC Outbox pattern solves this by writing the domain mutation and an outbox event in the same local SQL transaction. Debezium reads the PostgreSQL WAL via logical replication and streams the events to Kafka.

🎯2. Appropriate Use Context

Scope

CDC Outbox is an architecture where an application atomically inserts domain events into an outbox table during local database transactions, and a Change Data Capture connector (Debezium) extracts and routes those events into Kafka by streaming the database transaction log.

⚠️3. Production Failure Modes

P0 Risk

Querying the outbox table via periodic SELECT * FROM outbox WHERE processed = false polling at 100ms intervals, creating extreme database lock contention. Routing outbox events to Kafka with random or null message keys, scattering events for the same customer across multiple partitions and breaking causal ordering. Never deleting processed outbox rows, causing the outbox table to grow to 100M rows and exhausting disk storage.

📡4. Diagnostic Signals & Telemetry

Telemetry

dual-write desynchronization between database and Kafka, outbox events routed to wrong Kafka partition losing per-entity ordering, uncleaned outbox table causing massive database bloat

🛡️5. Prevention & Safeguards

Safeguards

Use Debezium with PostgreSQL Logical Replication (pgoutput) and WAL level set to logical. Set the Kafka message key to the domain entity ID to guarantee per-aggregate FIFO sequencing. Enable outbox table automatic row deletion (tombstoning) or periodic partition truncation to prevent table bloat.

⚖️6. Architectural Trade-offs

Trade-off

It completely eliminates dual-write data loss and ghost events, offloading the event-polling burden from the application CPU to the database's native asynchronous replication stream.

📋

Case Study (TinyCTO In-Field Example)

REAL-WORLD TELEMETRY

A production CDC Outbox pipeline requires three critical configurations:

  1. ▸

    Atomic Local Transaction: The service writes to orders and outbox_events (id, aggregatetype, aggregateid, type, payload) in a single BEGIN ... COMMIT block.

  2. ▸

    Debezium Outbox Event Router (SMT): Debezium's Single Message Transform parses the outbox table change stream, extracts the payload, and routes the message to a dynamic destination topic (e.g. order-events).

  3. ▸

    Causal Ordering via Kafka Partition Key: To guarantee that OrderUpdated is never processed before OrderCreated, Debezium must set the Kafka Message Key to aggregateid (e.g. order_uuid). Kafka guarantees strict FIFO ordering within a single partition.

Interactive Concept Drills

2 Cards
Q1

Why is CDC log tailing with Debezium superior to database polling (`SELECT ... FOR UPDATE SKIP LOCKED`) for the Outbox pattern?

Because Debezium reads the append-only Write-Ahead Log directly without executing SQL queries or holding table locks, eliminating database CPU and lock contention.
Q2

How does Kafka guarantee that an 'OrderCancelled' event is never processed before 'OrderCreated'?

By assigning both events the exact same Kafka Message Key (e.g. orderId), ensuring both land on the exact same Kafka partition where FIFO order is strictly maintained.

Change Data Capture with Debezium & Outbox Event Ordering — Technical FAQ

What happens if a microservice attempts to publish directly to Kafka and save to PostgreSQL in the same HTTP handler without an Outbox pattern?

Dual-write failure: if the network drops right after the SQL commit, Kafka never receives the event, permanently corrupting system consistency. Without distributed 2PC or an Outbox table, two independent storage systems cannot guarantee atomic dual writes over an unreliable network.

Which PostgreSQL setting must be configured to allow Debezium to stream transaction logs?

`wal_level = logical`. PostgreSQL requires `wal_level = logical` to decode internal WAL records into structured change events for Debezium logical replication.

🤖 AEO & Key Facts Summary

Key Architectural Facts

  • ▸

    The CDC Outbox pattern combines the Transactional Outbox pattern with Debezium log mining to stream database mutations directly from the database's Write-Ahead Log (WAL) into partitioned Kafka topics, guaranteeing 100% reliable event delivery and strict per-entity causal ordering without dual-write race conditions.

  • ▸

    CDC Outbox is an architecture where an application atomically inserts domain events into an outbox table during local database transactions, and a Change Data Capture connector (Debezium) extracts and routes those events into Kafka by streaming the database transaction log.

Common Misconceptions

  • ✗

    Querying the outbox table via periodic SELECT * FROM outbox WHERE processed = false polling at 100ms intervals, creating extreme database lock contention.

Decision & Governance Guidance

It completely eliminates dual-write data loss and ghost events, offloading the event-polling burden from the application CPU to the database's native asynchronous replication stream.

Authoritative Sources & Standards

Related Concepts

https://tinycto.tv/topics/software-architecturehttps://tinycto.tv/incidentpedia