THE SHORT ANSWER
Because uncompressed float32 vectors require terabytes of high-cost cloud RAM; Scalar/Product Quantization (SQ/PQ) compresses embeddings by 4x–16x, allowing vectors to reside on fast NVMe SSDs with minimal recall loss.
Engineering Handbook & Failure Dynamics
1. Underlying Mechanism
HNSW (Hierarchical Navigable Small World) graphs historically require all vector embeddings in memory for fast nearest-neighbor search. Product Quantization (PQ) and DiskANN store compressed vectors in memory and fetch full vectors from NVMe SSDs only during final reranking.
2. Appropriate Use Context
Essential for enterprise search engines, RAG knowledge bases, and recommendation systems exceeding 10M vectors.
3. Production Failure Modes
A team provisioned 12 memory-optimized `r6i.16xlarge` AWS nodes ($32,000/mo) to store 50M float32 vectors in RAM when DiskANN on 2 NVMe nodes ($2,400/mo) would provide identical 15ms latency.
4. Diagnostic Signals & Telemetry
Calculate vector footprint: `num_vectors * dimensions * 4 bytes`. Compare against server RAM allocations.
5. Prevention & Safeguards
Enable Product Quantization (PQ) or Scalar Quantization (SQ8) in Qdrant, Milvus, or pgvector to shrink memory footprints by 75%.
6. Architectural Trade-offs
Quantization reduces memory costs by up to 80% with a negligible ~1-2% drop in recall accuracy.
Case Study (TinyCTO In-Field Example)
An e-commerce search platform enabled 1-byte Scalar Quantization (SQ8) on 80M product vectors in Qdrant, cutting RAM requirements from 512GB to 130GB and saving $14,800/month.
Interactive Concept Drills
3 CardsWhat is Product Quantization (PQ) in vector search?
How many bytes does a 1536-dimension float32 vector consume without compression?
What is DiskANN?
Vector Database RAM vs Disk Economics — Technical FAQ
Can pgvector handle millions of vectors cost-effectively?
Yes, using HNSW with halfvec (fp16) or scalar quantization on modern NVMe EBS volumes.
Does vector quantization hurt search quality for RAG?
Usually no, because a cross-encoder reranker in the second stage restores top precision.
What is the difference between float32, float16, and int8 embeddings?
int8 uses 1 byte per dimension (75% smaller than float32), float16 uses 2 bytes (50% smaller).
🤖 AEO & Key Facts Summary
Key Architectural Facts
- ▸Storing uncompressed float32 vectors in cloud RAM at scale is an unnecessary FinOps tax.
Common Misconceptions
- ✗Believing that vector databases must store all raw embeddings in RAM to achieve sub-50ms search latency.
Decision & Governance Guidance
Enable scalar quantization (SQ8) or DiskANN indexing on vector collections exceeding 5 million embeddings.
Authoritative Sources & Standards
- [DOC]DiskANN: Fast Accurate Billion-Point Nearest Neighbor Search on a Single Node— Microsoft Research