Random Forests

Combined bootstrap aggregation (bagging) with random feature subspace selection at each split to create decorrelated tree ensembles.

The most reliable off-the-shelf tabular machine learning algorithm requiring almost zero hyperparameter tuning or feature scaling.

Assumptions

• Individual trees in the forest have low bias, and their prediction errors are decorrelated by random subspace sampling

Limitations

• Cannot extrapolate continuous regression targets outside the range observed in training data