Regularization is the process of finding a small set of predictors
that yield an effective predictive model. For linear discriminant
analysis, there are two parameters, γ and δ,
that control regularization as follows. cvshrink
helps
you select appropriate values of the parameters.
Let Σ represent the covariance matrix of the data X,
and let be the centered data (the data X minus
the mean by class). Define
The regularized covariance matrix is
Whenever γ ≥ MinGamma
, is nonsingular.
Let μk be the
mean vector for those elements of X in class k,
and let μ0 be the
global mean vector (the mean of the rows of X).
Let C be the correlation matrix of the data X,
and let be the regularized correlation
matrix:
where I is the identity matrix.
The linear term in the regularized discriminant analysis classifier
for a data point x is
The parameter δ enters into this equation
as a threshold on the final term in square brackets. Each component
of the vector is set to zero
if it is smaller in magnitude than the threshold δ.
Therefore, for class k, if component j is
thresholded to zero, component j of x does
not enter into the evaluation of the posterior probability.
The DeltaPredictor
property is a vector related
to this threshold. When δ ≥ DeltaPredictor(i)
, all classes k have
Therefore, when δ ≥ DeltaPredictor(i)
, the regularized
classifier does not use predictor i
.