Dimensional reduction by PCA

I have an vector x whose dimension has to be reduced to 5 x 50 by PCA.

 [5,500]=size(x);

For that I use the code http://crcv.ucf.edu/source/dimension%20reduction/pca.m

while giving the no_dimns=50 I faced a problem

 Warning: Target dimensionality reduced to 5.

Is it possible to reduce the dimension of matrix from 5 x 500 to 5 x 50??

Answer 1

[Shashank Prasanna]

I think that the code expects you provide x' or x transpose where the rows are 500 and columns are 5

[Berbia]

This code reduces only the column dimension of the matrix. If I give x' instead of x, the my input dimension remains 5 and hence the input and output are of same size (no reduction in dimension)

[Shashank Prasanna]

What you are saying is, that you have 500 hundred variables and 5 observations of those variables. And you would like to reduce the dimensions of those 500 variables.

This seems highly doubtful. Are you very sure that you don't have 500 observations of 5 variables? 5 is too few observations.

Or atleast please provide more information on where you got the data and what the colums and rows mean and we can assist you better.

[Tom Lane]

What are you providing as no_dimns? I didn't try it, but the comment in the code implies that you should specify the number of dimensions you want. If you specify this as 5, you'll get no reduction.

[Berbia]

yes I have 500 variables(as I specified as no_dims) and I need to reduce those dimensions to 50. For training I have more than 500 observations,so it works. I'm working in video processing, in that the features are extracted by appending the [5 500] observations within a loop, and finally it is reduced to [(no_of_times_loop*5) 50] by pca. But in some case(eg,for some videos) the no_of_observations is less than 50, that's why I cant able to perform dimensional reduction before classification. Is it possible to reduce the dimension of variables more than the number of observations??

[Shashank Prasanna]

It is quite possible to reduce the dimensions of variables when there are fewer observations. I recommend you use the PCARES function from the statistic toolbox. http://www.mathworks.com/help/stats/pcares.html

The code you shared does something funny to get the covariance matrix as below when the num of dimensions are larger than the number of observations

    C = (1 / size(X, 1)) * (X * X');

Tom may be able to shed more light on this.

[Berbia]

But while I use this

 residuals = pcares(X,ndim)

the dimension of X and residuals remains same. Is this similar to princomp?Shall I use residuals(:,1:50) as a reduced variables??

[Shashank Prasanna]

It appears the file in your link doesn't reconstruct but just gives you the reduced scores, which means you can do:

 [pc,score,latent,tsquare] = princomp(X);
  red_dim = score(:,1:50);

PCARES actually reconstructs the scores back to the original basis.

[Berbia]

I thank you for your prompt response.

Answer 2

[Nikos Mp]

2 questions: 1.So we take the scores or the reconstructed data? 2.P.Component = feature ? So we choose the best PComponents?