> ML_ALGORITHM // XGBOOST-LIGHTGBM-CATBOOST_v1.0
Modern Histogram & Symmetric Boosting (XGBoost / LightGBM / CatBoost)
State-of-the-art tabular learning engines leveraging second-order Hessian optimizations, histogram binning, and GPU acceleration.
Tree & Rule Ensemblesclassical-supervisedmoderate-posthoclarge (>100k)
Back to All AlgorithmsComputational Complexity
Training Complexity:O(n_trees * p * n_bins) via histogram splitting
Inference Complexity:Sub-millisecond O(n_trees)
Hardware Profile
CPU Friendly:Yes
Requires GPU:Yes
Memory Footprint:moderate
Interpretability & Data
Interpretability Tier:moderate-posthoc
Training Data Needs:large (>100k)
Interpretability Assessment
Fast TreeSHAP algorithms compute exact Shapley values in milliseconds.
Suitable Tasks & Supported Modalities
Suitable Tasks:
binary classificationmulticlass classificationregressionranking
Supported Modalities:
tabular
Implementing Libraries
Foundational Literature
XGBoost: A Scalable Tree Boosting SystemTianqi Chen, Carlos Guestrin (2016) · ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD)
LightGBM: A Highly Efficient Gradient Boosting Decision TreeGuolin Ke, Qi Meng (2017) · Advances in Neural Information Processing Systems (NeurIPS)
CatBoost: unbiased boosting with categorical featuresLiudmila Prokhorenkova, Gleb Gusev (2018) · Advances in Neural Information Processing Systems (NeurIPS)
Common Pitfalls & Warnings
- Target encoding leakage across cross-validation folds in CatBoost
- Tree depth setting causing memory exhaustion on high-cardinality features