variance in predicted values of decision trees

Hello,

how to know the variance on the predicted values of a decision tree?

Let us take the decision tree in the example https://nl.mathworks.com/help/stats/templatetree.html#bug9bel-1

The command pMPG = predict(Mdl1,[4 200 150 3000])

returns the predicted value of the tree ensamble.

Is there a way to know the variance for each predicted value?

In other words: the value returned by 'predict' is the mean of the values returned by the different trees in the ensamble. Is there a way to get the variance of the values returned by these trees?

Python allows to do this, see https://blog.datadive.net/prediction-intervals-for-random-forests/

Answer 1

[Akshat]

Hi @Pierluigi Frisco,

I see you are trying to find the variance of outputs from an ensemble of decision trees.

I suggest that you use "fitrensemble" (https://www.mathworks.com/help/stats/fitrensemble.html).

In a "fitrensemble" object, there is a property called "Trained", which lets you access each of the decision trees in the ensemble. Using this, you can make a prediction for each of the trees, and then find out the variance. The "Trained" property is mentioned here: https://www.mathworks.com/help/stats/fitrensemble.html#:~:text=The%20software%20composes%20the%20ensemble%20using%20all%20trained%20learners%20and%20stores%20them%20in%20Mdl.Trained.

You can use this boilerplate code to get the variance:

% Assuming Mdl1 is your ensemble model
% Obtain the number of trees in the ensemble
numTrees = Mdl1.NumTrained;

individualPredictions = zeros(numTrees, 1);

inputData = [4 200 150 3000];

for i = 1:numTrees
    tree = Mdl1.Trained{i};
    individualPredictions(i) = predict(tree, inputData);
end

varianceOfPredictions = var(individualPredictions);

Hope this helps!