Regression Tree Ensembles

Random forests, boosted and bagged regression trees

A regression tree ensemble is a predictive model composed of a weighted combination of multiple regression trees. In general, combining multiple regression trees increases predictive performance. To boost regression trees using LSBoost, use fitrensemble. To bag regression trees or to grow a random forest [11], use fitrensemble or TreeBagger. To implement quantile regression using a bag of regression trees, use TreeBagger.

For classification ensembles, such as boosted or bagged classification trees, random subspace ensembles, or error-correcting output codes (ECOC) models for multiclass classification, see Classification Ensembles.

Apps

Regression LearnerTrain regression models to predict data using supervised machine learning

Functions

expand all

fitrensembleFit ensemble of learners for regression
predictPredict responses using ensemble of regression models
oobPredictPredict out-of-bag response of ensemble
TreeBaggerCreate bag of decision trees
fitrensembleFit ensemble of learners for regression
predictPredict responses using ensemble of bagged decision trees
oobPredictEnsemble predictions for out-of-bag observations
quantilePredictPredict response quantile using bag of regression trees
oobQuantilePredictQuantile predictions for out-of-bag observations from bag of regression trees
crossvalCross validate ensemble
plotPartialDependenceCreate partial dependence plot (PDP) and individual conditional expectation (ICE) plots
predictorImportanceEstimates of predictor importance for regression ensemble

Classes

expand all

RegressionEnsembleEnsemble regression
CompactRegressionEnsembleCompact regression ensemble class
RegressionPartitionedEnsembleCross-validated regression ensemble
TreeBaggerBag of decision trees
CompactTreeBaggerCompact ensemble of decision trees grown by bootstrap aggregation
RegressionBaggedEnsembleRegression ensemble grown by resampling

Topics

Ensemble Algorithms

Learn about different algorithms for ensemble learning.

Framework for Ensemble Learning

Obtain highly accurate predictions by using many weak learners.

Train Regression Ensemble

Train a simple regression ensemble.

Test Ensemble Quality

Learn methods to evaluate the predictive quality of an ensemble.

Select Predictors for Random Forests

Select split-predictors for random forests using interaction test algorithm.

Ensemble Regularization

Automatically choose fewer weak learners for an ensemble in a way that does not diminish predictive performance.

Bootstrap Aggregation (Bagging) of Regression Trees Using TreeBagger

Create a TreeBagger ensemble for regression.

Use Parallel Processing for Regression TreeBagger Workflow

Speed up computation by running TreeBagger in parallel.

Detect Outliers Using Quantile Regression

Detect outliers in data using quantile random forest.

Conditional Quantile Estimation Using Kernel Smoothing

Estimate conditional quantiles of a response given predictor data using quantile random forest and by estimating the conditional distribution function of the response using kernel smoothing.

Tune Random Forest Using Quantile Error and Bayesian Optimization

Tune quantile random forest using Bayesian optimization.