Feature selection / Dimensionality reduction for tall array

Hi everyone!

I work with a tall array of more than 2 M observations and about 3000 numerical predictor variables. My response variable is binary (no / yes). I would like to know how and what algorithms I can use to select (or rank) the best features to develop a predictive model.

Thanks.

Answer 1

[Kumar Pallav]

Hi,

Please look at the various feature selection techniques available in Statistics and Machine Learning Toolbox. As an example, you can use fscmrmr function for classification problems. Alternatively, you can use pca to reduce the dimensionality of the feature space.

Hope this helps!

[Santiago Cepeda]

Thank you Kumar, I tried fscmrmr(gather(X), gather(Y)) and it works. But I can’t run the sintax fscmrmr(Tbl,ResponseVarName)...

[Kumar Pallav]

Hi,

As an example shown here, if 'salary' is the response variable in the table 'adultdata',you could try the following command:

[idx,scores] = fscmrmr(adultdata,'salary')

Also,the data type supported for Tbl is 'table', so that may be the reason you are not able to run the syntax directly.

[Santiago Cepeda]

I’m working with tall arrays so, how should I write the command?