memory pca vs pcacov
5 vues (au cours des 30 derniers jours)
Afficher commentaires plus anciens
Dear all,
I'm running a pca on a large matrix (33*500,000) and with pcacov I get a memory error, but pca gives me no trouble. Could anyone explain this to me? Is the matrix somehow being reduced before computing the covariance matrix in pca?
Thanks!
Best
Hans
0 commentaires
Réponses (3)
Hiro Yoshino
le 2 Nov 2020
In PCA, your matrix (p x q) will be once converted into the variance-covariance matrix (q x q).
This would reqiure huge memory comsumption. Meanwhile, pcacov accepts a variance-covariance matrix as an input and, therefore the argument (input) should be a square matrix though.
As for big data anaysis, you may want to use tall array - this can be a solution.
0 commentaires
Hans van der Horn
le 2 Nov 2020
2 commentaires
Hiro Yoshino
le 2 Nov 2020
I do not believe pcacov works with your matrix in the first place since the shape of your matrix is unacceptable. I do not know what the error will be like.
Hans van der Horn
le 2 Nov 2020
2 commentaires
Hiro Yoshino
le 2 Nov 2020
I got your point now!!
OK, actually to avoid memory problem, pca normally takes a different approach to calculate eigen vectors - SVD. This is not a direct method and produces some by-product. This is a well-known fact - you may find the this in your text book too, I'm sure.
Voir également
Catégories
En savoir plus sur Dimensionality Reduction and Feature Extraction dans Help Center et File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!