Sharded Database
System Analysis
Normal Behavior
Analyzes incoming SQL queries or document requests through a query routing layer (coordinator). When a query includes the defined shard key, the router directs the read or write operation exclusively to the single physical shard that owns that key range or hash, maintaining linear performance scaling and low latency as the overall dataset grows.
Failure Behavior
When an inappropriate shard key is selected (such as a monotonically increasing timestamp or a low-cardinality status field), a 'hot shard' condition develops where a single node absorbs the vast majority of all read and write traffic. That single shard crashes under resource exhaustion, while the remaining database nodes remain virtually idle, bottlenecking the entire application.
Business Consequence
A catastrophic failure in a sharded database—such as a lost shard or corrupted routing table—results in partial, randomized data unavailability. A subset of customers will experience total data loss or login failures while others remain unaffected. Rebalancing or recovering a broken shard takes immense time, bleeding revenue by the minute.
Visual Manifestation
"Query routing errors, specific user IDs constantly timing out, and cross-shard queries locking up the entire cluster."
Satirical Behavior
"The architectural decision to chop your data into tiny pieces across multiple servers, ensuring that when it breaks, you have absolutely no idea where anything is."
Known Aliases
Technical Terminology
Failure Indicators
System Architecture (Graph)
FAQ
How does it normally behave?
Analyzes incoming SQL queries or document requests through a query routing layer (coordinator). When a query includes the defined shard key, the router directs the read or write operation exclusively to the single physical shard that owns that key range or hash, maintaining linear performance scaling and low latency as the overall dataset grows.
How does it fail?
When an inappropriate shard key is selected (such as a monotonically increasing timestamp or a low-cardinality status field), a 'hot shard' condition develops where a single node absorbs the vast majority of all read and write traffic. That single shard crashes under resource exhaustion, while the remaining database nodes remain virtually idle, bottlenecking the entire application.
What is the business consequence?
A catastrophic failure in a sharded database—such as a lost shard or corrupted routing table—results in partial, randomized data unavailability. A subset of customers will experience total data loss or login failures while others remain unaffected. Rebalancing or recovering a broken shard takes immense time, bleeding revenue by the minute.
What architectural pitfalls cause severe hot shard hotspots in a sharded database?
Hotspots occur when the chosen shard key does not distribute read and write traffic uniformly across the cluster. For instance, sharding by creation timestamp directs all active write traffic to the single shard assigned to the current time range. Similarly, sharding multi-tenant databases by tenant ID without salting causes massive enterprise customers to overwhelm individual shards while smaller tenant shards sit underutilized.
Why do cross-shard scatter-gather queries degrade distributed database performance?
When a query does not specify the shard key in its WHERE clause, the query coordinator must broadcast the request to every shard in the cluster (scatter) and merge the individual result sets in memory (gather). This operation ties up connection pools across all nodes and causes query response times to be dictated by the slowest-performing shard in the cluster, multiplying tail latency exponentially.
Explore the system
AI Summary
Sharded Database is a DATA_AND_STORAGE system in TinyCTO.tv. Analyzes incoming SQL queries or document requests through a query routing layer (coordinator). When a query includes the defined shard key, the router directs the read or write operation exclusively to the single physical shard that owns that key range or hash, maintaining linear performance scaling and low latency as the overall dataset grows.