Two-Tier Caching (L1 Local Memory + L2 Redis Sync)

Stack: SOFTWARE ARCHITECTURE STACK →Staff/Principal (L6+)pattern

⚡THE SHORT ANSWER

Two-Tier Caching pairs an ultra-fast L1 in-process memory cache (Caffeine/GoCache) with a centralized L2 distributed cache (Redis); when mutations occur, the mutating pod updates Redis and broadcasts an invalidation message over Redis Pub/Sub so all peer pods evict their local L1 entries.

Engineering Handbook & Failure Dynamics

6-Dimensional Architecture Breakdown

⚙️1. Underlying Mechanism

Execution

While Redis provides centralized caching, reading from Redis across the network still incurs 0.5–2ms of latency and consumes network interface bandwidth. For extreme read-heavy hot keys (e.g. global feature flags, tenant permissions, homepage metadata), querying Redis 100,000 times per second saturates the Redis CPU. An L1 in-process memory cache provides nanosecond-level access (0.0001ms), but introduces the cache coherence dilemma: how do you invalidate L1 across 50 independent pods?

🎯2. Appropriate Use Context

Scope

▸
Two-Tier Caching is an architectural caching hierarchy where requests first check local process heap memory (L1)
▸
on miss, query a shared remote Redis cluster (L2)
▸
on miss, query the database, synchronizing cross-pod L1 invalidations via lightweight messaging

⚠️3. Production Failure Modes

P0 Risk

Adding L1 in-process caching across 20 pods without any invalidation mechanism, serving inconsistent data for 24 hours. Broadcasting the full data payload over Pub/Sub on every write instead of sending lightweight eviction keys (causing network broadcast amplification). Setting L1 cache size to unbounded, leading to JVM/Node.js process OutOfMemory heap crashes.

📡4. Diagnostic Signals & Telemetry

Telemetry

stale data served from L1 process memory across different backend pods, redis network bandwidth saturated by high-frequency read loops, missing pub/sub invalidation causing persistent split-brain state

🛡️5. Prevention & Safeguards

Safeguards

Combine L1 size-bounded LRU eviction (max 10,000 entries) with short TTLs (30s) alongside Redis Pub/Sub eviction broadcasting. Use Redis Client-Side Caching (Redis 6+ RESP3 Tracking) where Redis automatically tracks which keys each client has cached in L1. Measure L1 vs L2 cache hit ratios separately in Prometheus to optimize memory allocation.

⚖️6. Architectural Trade-offs

Trade-off

It reduces Redis cluster costs by 80%+ and delivers sub-millisecond API response times while guaranteeing zero stale-data reads across horizontal Kubernetes pod replicas.

📋

Case Study (TinyCTO In-Field Example)

REAL-WORLD TELEMETRY

The lifecycle of a mutation in a coordinated Two-Tier Cache operates as follows:

▸
Read Path: Request checks local L1 (Caffeine). If Hit -> Return in 100 nanoseconds. If Miss -> Query L2 (Redis). If Hit -> Populate L1 and return. If Miss -> Lock database row, fetch from SQL, write to L2, populate L1, and return.
▸
Write Path & Invalidation Broadcast: When a user updates their profile, Pod A writes to PostgreSQL and invalidates the L2 Redis key (DEL user:123). Pod A then immediately publishes an eviction event (PUBLISH cache:invalidations user:123) over Redis Pub/Sub.
▸
Peer Pod Eviction: Pods B, C, and D are listening to cache:invalidations. Upon receiving user:123, each pod deletes user:123 from its local L1 memory cache within 1 millisecond.

Short TTL Safety Net: To protect against lost Pub/Sub messages during network blips, all L1 entries have a maximum short TTL (e.g. 30 to 60 seconds).

Interactive Concept Drills

2 Cards

What problem does Two-Tier Caching solve compared to querying Redis alone?

It eliminates network latency and Redis CPU/bandwidth saturation for extreme read-heavy hot keys by serving them in nanoseconds from local process RAM.

How do peer application pods know to invalidate their local L1 cache when Pod A updates a user record?

Pod A publishes the mutated key ID to a Redis Pub/Sub invalidation topic; all peer pods receive the message and evict that key from their local memory.