Fisher Linear Dicriminant Analysis

find the discriminative susbspace for samples using fisher linear dicriminant analysis

Scope:
Given a number of samples, each one represented by a point in the R^d dimensional space, together with a label associated to each sample, this function returns a d x matrix of size n x d that allows to represent each sample by a point in a R^n dimensional space, so that points of different labels are still distinguishable (classification problem)

The implemented function supports two variations of the Fisher criterion, one based on generalised eigenvalues (ratio trace criterion) and another based on an iterative solution of a standard eigenvalue problem (trace ratio criterion). The later implementation, is based on

Lei-Hong Zhang, Li-zhi Liao and Michael K. Ng, "Fast Algorithms for the generalised foley-sammon discriminant analysis", SIAM Journal on Matrix Analysis and Applications, vol 31, p.1584, 2010

which also provides a good analysis of the criteria.

A further variant allows to orient the subspace found by the trace ratio criterion according to the ration trace criterion.

An optional PCA-like preprocessing is available to increase robustness, when the number of features is larger to the number of samples

Options:

Syntax [ V, eigvalueSum ] = fld( X, L, n, crit, qrf, r, e, M )

Input arguments:
X: the s x d samples matrix (s samples x d features each )
L: the s x 1 labels matrix
n: the dimension the subspace required
Optional arguments:
crit: 1 (ratio trace criterion - default)
2 (trace ratio criterion)
3 (tr criterion for subspace + rt criterion for axis)
qrf: preprocess with qr decomposition, for robustness (default value: false)
r: the regularisation parameter (default value: 0)
e: epsilon value to end iterations for tr criterion (defaut value: 0.001)
M: maximum number of iterations for tr criterion (defaut value: 100)

Output:
V: the optimal fiher ( n x d ) orthonormal matrix
ratio: the sum of generalised eigenvalues or the optimal value of the trace ratio
% eigvals: the eigenvalues of the (B - r * W) matrix