Skip to main content

> ML_LIBRARY // WHYLOGS_v1.0

whylogs

WhyLabs — Lightweight, privacy-preserving statistical data profiling library powered by Apache DataSketches.

evaluation-observabilityv1.4.11Apache-2.0qualified
Official DocumentationSource Repository

Model Training

Not Supported

This library is a dedicated runtime engine for inference serving and does not train models.

Model Inference

Supported
Inference Accelerators:
CPU
Deployment Targets:server, edge

What It Does

  • +Ultra-lightweight statistical profiling with sub-millisecond per-row overhead
  • +Privacy-preserving: stores mergeable statistical sketches (quantiles, cardinality, types) without recording raw PII records
  • +Mathematically mergeable profiles: combine hourly edge profiles across 1,000 workers into an exact global profile without data loss
  • +LangKit integration for LLM prompt and response telemetry (toxicity, sentiment, regex patterns)

What It Does Not Do

  • -Store raw database records or audio/video pixels
  • -Train predictive machine learning models
  • -Serve model inference endpoints

>Suitable Work Types

  • High-throughput streaming ML inference logging (10,000+ requests/sec) with minimal CPU overhead
  • Regulated banking and healthcare logging where transmitting raw customer data to monitoring tools violates GDPR/HIPAA
  • Distributed data profiling across Spark/Ray clusters without shuffling raw data

>Unsuitable Work Types

  • Explaining individual predictions (use SHAP or LIME)
  • Model training loops
Data Residency Implications

Zero raw customer data is retained. Only anonymous mathematical sketches are persisted locally or exported.

Security Considerations

Apache-2.0 license. Provides guaranteed PII protection by design through one-way mathematical sketching.

Operational Profile & Known Limitations

Maturity:mature
Learning Curve:low
Ops Complexity:low
Cost Tier:free-oss
> Known Limitations:
  • Sketches are approximate statistical representations; exact raw values cannot be reconstructed from whylogs profiles.

Associated Incident Patterns (Incidentpedia)

Enforce safeguards and monitoring to guard against these documented real-world failure modes:

training-serving-skew

> Primary Evidence & Benchmark Citations

whylogs Documentationofficial-docs • >=1.4.0, <=1.4.x
2026-09-25HIGH