Direct LDA and PCA+LDA

Implementation of LDA, Direct LDA and PCA+LDA. See description for details.

This implements the algorithm described in
Hua Yu and Jie Yang "A direct LDA algorithm for high-dimensional data -
with application to face recognition"
Read it. It's short.

Basically what we have here is LDA (linear discriminant analysis), Direct LDA, which does LDA but better (basically when your within class scatter matrix, Sw, is singular, it does not throw away the null space of Sw, which contains most of the discriminating information), and PCA+LDA, which does PCA on the data and then does LDA on the dimensionally reduced data.

There is also an implementation of PCA (principal component analysis) using eigendecomposition and another using SVD in here.

INPUT

X : nt x n where nt examples of feature vectors of size n
y : vector of size nt of class labels
eg:
X = [2 3 4 2; 8 2 3 4; 3 9 2 3; 8 4 2 3; 9 9 2 8];
y = [3; 2; 4; 3; 2];
k : the number of features we want at the end

method : either pcalda or directlda
if method = directlda then we are maximizing |A'*Sb*A|/|A'*Sw*A| except
that the null space of Sw, which apparently has the most discriminating
information, is not thrown away
if method = pcalda then we are maximizing |A'*St*A|/|A'*Sw*A|

m :
if method = directlda then m = the no. dims. we want from the Sb scatter matrix
if method = pcalda then m = the no. of dims. we want from the pca part
if method = pcalda and m = inf, then this is just regular lda

OUTPUT

A : the projection A which maximizes between class / within class scatter
T : transformation that spheres the data, for classification/comparison

In the demo, I compared this to http://www.mathworks.com/matlabcentral/fileexchange/38950-fischer-linear-dicriminant-analysis for checking correctness.