Not reproducibility in Random Forest with Hyperparameters

Hello, I tried many ways to apply random forest with hyperparameters and reproduce the results and no chance. I have added as the manual says: rng('default') before the cross validations and the classifier. If I do not use hyperparameters, it works. The problem is when I use grid search. I use expected-improvement-plus for reproducibility. But it does not work. For svm or any other machine learning method I did not have this issue. Only in random forest.

Thank you

myopts = struct('Optimizer','gridsearch','UseParallel',true,'AcquisitionFunctionName','expected-improvement-plus', 'ShowPlots',false);
                        rng('default');
                        classificationRF = fitcensemble(...
                            predictors, ...
                            response, ...
                            'Method', 'Bag', ...
                            'NumLearningCycles', 30, ...
                            'Learners', template, ...
                            'ClassNames', [1; 2; 3; 4; 5],'OptimizeHyperparameters',{'NumLearningCycles','MaxNumSplits'},'HyperparameterOptimizationOptions',myopts);  

Answer 1

[Alan Weiss]

In creating your template for training, did you set the Reproducible name-value argument to true? Also, for your parallel options, to compute reproducibly I think that you need to set UseSubstreams to true and set the stream to a type that works with substreams, as shown in the documentation.

Alan Weiss

MATLAB mathematical toolbox documentation

[Esmeralda Ruiz Pujadas]

Thank you very much. Yes I used Reproducible but not UseSubstreams. I will try that. Thank you very much