Drift Monitor
System Analysis
Normal Behavior
The drift monitor ingests streaming inferences from production serving nodes, computes statistical summary profiles over rolling windows (e.g. hourly or daily), and compares the distribution of each feature against the training baseline. If divergence exceeds a configured threshold, it dispatches warnings to on-call engineers.
Failure Behavior
If the reference baseline is outdated, sample windows are too small, or features with high natural volatility lack adjusted thresholds, the drift monitor either generates overwhelming false-positive alert fatigue or fails to detect silent concept drift until business revenue drops.
Business Consequence
Unknown
Known Aliases
Technical Terminology
Failure Indicators
System Architecture (Graph)
FAQ
How does it normally behave?
The drift monitor ingests streaming inferences from production serving nodes, computes statistical summary profiles over rolling windows (e.g. hourly or daily), and compares the distribution of each feature against the training baseline. If divergence exceeds a configured threshold, it dispatches warnings to on-call engineers.
How does it fail?
If the reference baseline is outdated, sample windows are too small, or features with high natural volatility lack adjusted thresholds, the drift monitor either generates overwhelming false-positive alert fatigue or fails to detect silent concept drift until business revenue drops.
What is the business consequence?
Unknown
What is the difference between data drift and concept drift?
Data drift (or feature drift) occurs when the statistical distribution of the input features $P(X)$ changes over time, while concept drift occurs when the underlying mathematical relationship between features and the target label $P(Y|X)$ changes, causing the model's learned logic to become invalid even if input features look familiar.
How does Population Stability Index (PSI) quantify drift?
PSI divides the feature distribution into bins and calculates $\sum (Actual\% - Expected\%) \times \ln(Actual\% / Expected\%)$. A PSI value below 0.1 indicates minimal shift, 0.1 to 0.2 indicates moderate change requiring monitoring, and above 0.2 represents significant distribution drift requiring model retraining.
Explore the system
AI Summary
Drift Monitor is a undefined system in TinyCTO.tv. The drift monitor ingests streaming inferences from production serving nodes, computes statistical summary profiles over rolling windows (e.g. hourly or daily), and compares the distribution of each feature against the training baseline. If divergence exceeds a configured threshold, it dispatches warnings to on-call engineers.