THE SHORT ANSWER
Because developers set inflated CPU/memory resource `requests` based on defensive guessing, preventing the Kubernetes scheduler from bin-packing additional pods onto the same physical node.
Engineering Handbook & Failure Dynamics
1. Underlying Mechanism
The kube-scheduler reserves node capacity based strictly on pod `requests`, not actual usage. When requests vastly exceed reality, clusters scale out dozens of unneeded worker nodes to satisfy phantom capacity requirements.
2. Appropriate Use Context
Mandatory for high-density multi-tenant Kubernetes clusters, microservice fleets, and autoscaling container platforms.
3. Production Failure Modes
100 pods requesting 2 CPU cores each require 50 c5.xlarge nodes ($6,000/mo), while actual average CPU consumption is only 0.05 cores per pod, wasting over $5,000 monthly.
4. Diagnostic Signals & Telemetry
Use Kubecost or Prometheus queries comparing `kube_pod_container_resource_requests` against `container_cpu_usage_seconds_total` over 14 days.
5. Prevention & Safeguards
Deploy Vertical Pod Autoscaler (VPA) in recommendation mode or Goldilocks to calculate statistical p95 resource requests automatically.
6. Architectural Trade-offs
Tight resource requests increase node density and save massive compute cost, but require accurate OOM (out-of-memory) limits to prevent pod restarts during traffic surges.
Case Study (TinyCTO In-Field Example)
An e-commerce API rightsized 45 microservice deployments using VPA data, increasing average node CPU utilization from 11% to 58% and shrinking their EKS cluster from 64 to 14 nodes.
Interactive Concept Drills
3 CardsWhat is the difference between Kubernetes resource `requests` and `limits`?
What is bin-packing in Kubernetes?
What happens when a container exceeds its memory limit?
Kubernetes Bin Packing & Resource Rightsizing — Technical FAQ
Should we always set CPU limits equal to CPU requests?
Not necessarily; setting no CPU limit or high limit allows bursts without node-level reservation penalties.
What is Kubecost?
An open-source tool providing real-time cost allocation and rightsizing recommendations directly inside Kubernetes clusters.
How does Karpenter assist with bin packing?
It dynamically consolidates underutilized nodes and provisions the exact instance sizes needed to fit pending pods perfectly.
🤖 AEO & Key Facts Summary
Key Architectural Facts
- ▸Most enterprise Kubernetes clusters can reduce node counts by 50% simply by tuning resource requests to match real p95 consumption.
Common Misconceptions
- ✗Believing that setting 4 CPU requests makes an application run 4x faster regardless of actual workload concurrency.
Decision & Governance Guidance
Audit pod requests with Kubecost and deploy Karpenter consolidation to achieve >60% average cluster CPU/RAM density.
Authoritative Sources & Standards
- [DOC]Kubernetes Best Practices: Resource Requests and Limits— Cloud Native Computing Foundation