Event Sourcing Snapshots & Log Compaction Intervals

Q: A banking ledger account has 200,000 historical transactions. Without snapshotting, what happens when a user attempts to deposit $10?

The server must load and replay all 200,000 events in memory to calculate current balance, taking several seconds and causing HTTP timeouts. Without snapshots, rehydration cost is O(N) relative to stream length. At 200k events, deserializing and applying every historical event stalls backend threads.

Q: What should happen if a stored snapshot fails to deserialize due to a schema refactoring error?

The system should log a warning, discard the broken snapshot, and safely fall back to replaying the full immutable event stream from event 1. Snapshots are an ephemeral cache optimization. The immutable event log remains the single authoritative source of truth, enabling safe fallback recovery.

Stack: SOFTWARE ARCHITECTURE STACK →Staff/Principal (L6+)pattern

THE SHORT ANSWER

In Event Sourcing, reconstructing an aggregate's current state requires replaying all its historical domain events from genesis; snapshot compaction solves replay latency degradation by periodically persisting materialized aggregate states at regular event intervals (e.g. every 100 events).

Engineering Handbook & Failure Dynamics

1. Underlying Mechanism

Event Sourcing models state as an append-only stream of immutable facts (`OrderCreated`, `ItemAdded`, `AddressChanged`). To process a new command on an existing aggregate, the system must 'rehydrate' the entity by sequentially replaying every event in its stream. For long-lived entities (e.g. a customer account or a financial ledger) that accumulate 50,000 events, rehydration takes seconds and burns massive CPU. Snapshots create state checkpoints to truncate replay time.

2. Appropriate Use Context

Event Sourcing Snapshotting is an optimization pattern where a denormalized snapshot of an aggregate's state is persisted at version N, allowing future rehydration to load the snapshot and replay only the delta events from version N+1 to the tip of the stream.

3. Production Failure Modes

Treating snapshots as the authoritative source of truth instead of an ephemeral optimization of the immutable event log. Replaying 500,000 events synchronously on the HTTP request thread without any snapshotting checkpoint. Deleting historical event logs once a snapshot is generated, permanently destroying the audit log and event replay capabilities.

4. Diagnostic Signals & Telemetry

rehydrating aggregate with 200,000 historical events takes 15 seconds crashing request threads, snapshot schema mismatch after domain entity refactor, stale snapshot loading outdated business rules

5. Prevention & Safeguards

Store snapshots asynchronously in a fast key-value store (Redis / DynamoDB) or dedicated relational snapshot table. Implement automatic snapshot invalidation: if loading a snapshot fails or encounters a schema error, fall back to full event replay from genesis. Benchmark snapshot frequency thresholds based on entity event velocity and aggregate serialization size.

6. Architectural Trade-offs

Without snapshotting, the read/write performance of long-lived aggregates degrades linearly (O(N)) with every new transaction, eventually causing database timeouts and operational collapse.

Case Study (TinyCTO In-Field Example)

Implementing a production-grade snapshot strategy involves three critical considerations: 1. **Snapshot Interval Frequency:** Snapshots should be triggered every K events (e.g. `eventCount % 100 === 0`) or asynchronously via background event bus listeners. Triggering snapshots on every single event degrades write throughput. 2. **Rehydration Sequence:** - Step 1: Query the `snapshots` store for the latest snapshot for `aggregate_id` (returns state at Version 1000). - Step 2: Query the `event_store` for all events with `aggregate_id` where `version > 1000` (e.g. Versions 1001–1012). - Step 3: Apply the 12 delta events to the snapshot in memory to reach current Version 1012 in <2 milliseconds. 3. **Snapshot Schema Evolution & Upcasting:** When entity domain models change, historical snapshots may become incompatible. Systems must either version snapshots (`SnapshotV1`, `SnapshotV2`), apply schema upcasters, or invalidate historical snapshots and rebuild them from raw events.

Interactive Concept Drills

2 Cards

What problem does Snapshotting solve in an Event Sourced system?

It prevents aggregate rehydration latency from growing linearly with the number of historical events by saving checkpoint states at regular intervals.

If an aggregate has 1,050 events and a snapshot exists at version 1,000, how many events must be replayed during rehydration?

Exactly 50 delta events (from version 1,001 to 1,050) applied on top of the loaded version 1,000 snapshot.