Canonical Engineering Manual #03|TinyCTO RAG Bible
Embeddings & Vector Space Geometry
Dense bi-encoders, sparse SPLADE, late-interaction ColBERT, Matryoshka Representation Learning (MRL), and quantization.
Canon Certified 14 min
#1. Vector Space Representations
Dense vector embeddings map unstructured text passages into continuous high-dimensional vector spaces (e.g. 768 to 3072 dimensions) where semantic proximity corresponds to cosine similarity or inner product distance.
Modern Embedding Paradigms
- Dense Bi-Encoders:
- Single pooled vector per passage (e.g. BGE-large, text-embedding-3-large).
- High search speed via approximate nearest neighbor (ANN) indexing.
- Sparse Lexical Encoders (SPLADE):
- Projects passages into vocabulary-dimensional space, predicting expanded keyword weights.
- Combines lexical precision with semantic expansion.
- Late-Interaction Token Encoders (ColBERT):
- Preserves individual token vectors and computes late similarity via the MaxSim operator.
- Retains fine-grained token match sensitivity without cross-encoder computational overhead.
#2. Quantization & Dimensionality Optimization
- Matryoshka Representation Learning (MRL): Models trained to allow vector truncation from 3072 to 512 dimensions with minimal degradation in recall (<1.5%).
- Scalar & Product Quantization (SQ/PQ): Compresses 32-bit floating-point coordinates into 8-bit integers or codebook centroid indices, reducing RAM footprint by 4x to 8x.