Finding the knee point in an eigenvalue plot

Hi

I want to automatically find the "knee" point of the eigenvalue plot. I.e. I have a vector of eigenvalues (sorted from highest to lowest) and I want some heuristic to find the "knee" point.

Is there some heuristic for doing that?

[John D'Errico]

Your definition of where the "knee" lies may not be mine. You need to define carefully what that point is, and even then I think you may have problems to find that point consistently.

My point is, unless you can define (mathematically) what the knee looks like, you cannot program an algorithm to do so, and if you get some code from someone else, their idea may not coincide with yours. (Yeah, while even a supreme court justice says that he knows "it" when he sees it, writing a computer algorithm is not quite so clear.) So expend some thought, and define carefully that location to be found. Once you do, the algorithm is often trivial.

Answer 1

[the cyclist]

This is a complex problem, and there is some academic literature on it. I suggest you Google some combination of the keywords: scree elbow automatic.

(These eigenvalue plots are often called "scree plots", and the "kink" is usually referred to as the "elbow", not the "knee".)

[Sepp]

Thank you for your answers. I will google for the scree elbow and look what I find. An other option is to learn the number of eigenvalues (i.e. the point of the elbow) by cross-validation globally.

I found two other easy suggestions:

1. Setting a threshold, say 0.99, or 0.95 and keep m of n eigenvalues when

T(m-1) < 0.99 *T(n) <= T(m)

where

T(m) = sum(i=1:m){lambda(i)}

2. The knee is located at a point where the radius of curvature. is a local minimum. For a curve y = f(x) the curvature is

k = y''/(1+(y')^2)^(3/2).

Just repace the derivatives with finite differences.

What do you think of these two proposals?

How can I implement the second one? I don't understand how to replace the derivatives with the differences.