---
title: "Multi-Tenant Segmented Storage Streaming | Distributed Systems Architecture Canon"
description: "Decoupled compute-and-storage streaming architecture utilizing Apache Pulsar brokers and Apache BookKeeper ledgers for infinite retention and millions of topics."
image: "https://tinycto.tv/assets/distributed-systems/distributed_systems_architectures_og.jpg"
canonicalUrl: "https://tinycto.tv/distributed-systems/architectures/pulsar-segmented-lakehouse"
locale: "en"
---

# Multi-Tenant Segmented Storage Streaming (`arch-pulsar-segmented-lakehouse`)

> **Architectural Pillar**: STREAMING_LOGS | **Archetype**: LOG_STREAMING_PARTITIONING
> **PACELC**: PC/EC | **Consensus Protocol**: Quorum

Decoupled compute-and-storage streaming architecture utilizing Apache Pulsar brokers and Apache BookKeeper ledgers for infinite retention and millions of topics.

### 3 Maturity Target Configurations

#### 1. Initial Target: 20,000 msg/sec

- **Guarantees**: At-Least-Once Delivery
- **Infrastructure Topology**: 3 broker pods + 3 BookKeeper bookie pods.
- **Operational Trade-Off**: Dual-layer operations (managing both brokers and bookies).

#### 2. Scaled Target: 350,000 msg/sec

- **Guarantees**: Strict Deduplication (Broker Deduplication Engine)
- **Infrastructure Topology**: 6 stateless brokers + 12 bookies with dedicated journal and ledger NVMe disks.
- **Operational Trade-Off**: Higher component operational complexity.

#### 3. Ultra-Scale Target: 2,000,000 msg/sec

- **Guarantees**: Global Geo-Replication across 3 Continents with Automatic Failover
- **Infrastructure Topology**: Multi-datacenter mesh across US, EU, and APAC with asynchronous geo-replication.
- **Operational Trade-Off**: Cross-continental network latency on geo-replicated namespaces.

### Handled Failure Modes

- `DS-FAIL-03: Unbounded Consumer Lag Spiral`
- `DS-FAIL-12: Hot Partition Skew`


```json
{
  "@context": "https://schema.org",
  "@type": "TechArticle",
  "headline": "Multi-Tenant Segmented Storage Streaming",
  "description": "Decoupled compute-and-storage streaming architecture utilizing Apache Pulsar brokers and Apache BookKeeper ledgers for infinite retention and millions of topics.",
  "inLanguage": "en",
  "url": "https://tinycto.tv/distributed-systems/architectures/pulsar-segmented-lakehouse"
}
```