> ML_DATASET // FINEWEB-DATASET_v1.0
FineWeb (15 Trillion High-Quality Web Tokens)
Hugging Face (Lozhkov et al.) · Foundation LLM Pretraining · 15 trillion tokens (44 TB of compressed text across 96 Common Crawl snapshots)
Foundation LLM PretrainingOpen Data Commons Attribution License (ODC-By)15 trillion tokens (44 TB of compressed text across 96 Common Crawl snapshots)open
Dataset Profile & Characteristics
Label Type:Unsupervised pre-training text with synthetic quality annotations
Languages:en
License Tier:permissive-open-source
Modalities:text
Intended Use
- Open-weights state-of-the-art foundation language model pretraining
Prohibited / Discouraged Use
- Training without safety post-training and alignment
Bias, Leakage & Privacy Risk Analysis
Privacy / Sensitive Data Risks:
PII anonymization filters applied to strip phone numbers, emails, and IP addresses.
Known Bias:
Web crawl distribution biased towards English and commercial web content.
Known Benchmark Leakage:
Heuristic minhash deduplication and benchmark decontamination applied.