CatBoost: unbiased boosting with categorical features

Ordered target statistics for categorical features and ordered boosting to eliminate target leakage and prediction shift.

State-of-the-art out-of-the-box accuracy on tabular datasets rich in categorical features without manual encoding pipelines.

Assumptions

• Target statistics calculated on past samples according to random permutations prevent target leakage

Limitations

• Slower training on pure numerical datasets compared to LightGBM due to symmetric oblivious tree constraints