How can I do a simple binary image classification?

Hi,

I'm not really new to MATLAB, just new to this whole Machine Learning thing.

I have to do a simple binary image classification. I don't care if it's a toolbox or just code, I just need to do it. I tried a couple of classification codes I found online on Github or on other sites, but most of them worked randomly and some of them worked for pre-defined images.

Those that worked on pre-defined images were neat (e.g.: http://www.di.ens.fr/willow/events/cvml2011/materials/practical-classification/), but I had issues applying on a new set of images, just because there were some .txt files (vectors of the name of the images, which was easy to replicate) and some .mat files (with both name and histogram).

I had issues creating the name and histogram in the same order, the piece of code that I use is:

for K = 1 : 4
  filename = sprintf('image_%04d.jpg', K);
  I = imread(filename);
  IGray = rgb2gray(I);
  H = hist(Igray(:), 32);
end
save('ImageDatabase.mat', 'I', 'H');

But for one reason or another, only the name and path of the last image remains stored (e.g. in this case, only image_0004 is stored in the name slot).

Another code that I found and it seemed easy was: https://github.com/rich-hart/SVM-Classifier , but the output is really random (for me) so if someone could explain to me what is happening I'd be grateful. There are 19 training images and 20 for test. Yet, if I remove one of the test images, 2 entries disappear from the Support Vector Structure?

Anyway, if you have a toolbox, or a more easy to adapt code or some explanations to the above codes, I'd be grateful.

Cheers!

Answer 1

[Walter Roberson]

http://matlab.wikia.com/wiki/FAQ#How_can_I_process_a_sequence_of_files.3F

[Dorian Diaconu]

Hi Waller, I will try your solution in a couple of minutes, I found a "easier" SVM algorithm floating around: http://dipwm.blogspot.ro/2013/01/svm-support-vector-machine-with-matlab.html , but after using 30 test/train images of 100x100 I get the following errors:

Error using svmtrain (line 253) Y and TRAINING must have the same number of rows.

Error in Untitled (line 74) SVMStruct = svmtrain(Training_Set , train_label, 'kernel_function', 'linear');

Thanks!

[Walter Roberson]

Use the debugger

 dbstop if error

Run the program. When it stops, give the command

dbup

and then check size(Training_Set) and size(train_label) . The first elements of the respective sizes (number of rows) must be the same. Note that train_label must not be a row vector: it would typically be a column vector.

[Dorian Diaconu]

Thank you for the fast answer. The whole issue with this is that my colleagues didn't do any work until now and I have to learn everything kind of over-night to finish the project. So I'm sorry if I'm bothering you that much.

Here is the output I got:

error(message('stats:svmtrain:DataGroupSizeMismatch'))
K>> dbup
In workspace belonging to Untitled at 74

74: SVMStruct = svmtrain(Training_Set , train_label, 'kernel_function', 'linear');

K>> size(Training_Set)

ans =

             3       10000

K>> size(train_label) 

ans =

      30     1

K>>

[Dorian Diaconu]

I managed to work out the error. It was due to the absolute path, using relative path worked. No idea why.

I used the same images in both of the sets, just for a test.

I attached my results. Why in one place the value is 18 instead of 30? Where can I see in which of the classes they belong to the most?

[Walter Roberson]

svmtrain() is defined to use a subset of the training data as the support vectors. See https://www.mathworks.com/help/stats/svmtrain.html#outputarg_SVMStruct

At the moment it is not clear to me how it determines how many support vectors to use.

Looking through the code I see that for SMO and QP, the code runs minimizations to determine which support vectors to use; the minimizer used is quite different for SMO than it is for QP. In other case, the support vectors chosen are the ones with non-zero coefficients after minimizing. For LS (least squared) then all of the points are used.

[Dorian Diaconu]

So basically the svmtrain gets a subset of vectors (in this case 18 I think?) and I get 18 because 18 of them fit identically with them?

[Walter Roberson]

Yes, svmtrain decided that 18 of the vectors were important enough to use as support vectors.

The comparison used is to eps rather than exactly 0, to account for round-off error. The ones with small absolute alpha value are discarded.

I would not say "fit identically" for the ones that are discarded: the minimization process involved is a bit complicated. For SMO you can look at the code at toolbox/stats/stats/private/seqminopt.m . For QP, a call to quadprog() is made about 530 lines into svmtrain.m, something related to Hessian.

[Dorian Diaconu]

I cannot thank you enough for your help. So as an interpretation to the SVMStruct is:

SupportVectorIndices: The vectors that were used as support (from the training set, 18 images were selected). SupportVectors (same as above). KernelFunction (linear classification). GroupNames (all thetrainign data).

But what about alpha and bias? Actually, where can I see exactly what vector to what class it belongs to?

[Walter Roberson]

Those are described at https://www.mathworks.com/help/stats/svmtrain.html#outputarg_SVMStruct

• Alpha — Vector of weights for the support vectors. The sign of the weight is positive for support vectors belonging to the first group, and negative for the second group.
• Bias — Intercept of the hyperplane that separates the two groups in the normalized data space (according to the 'AutoScale' argument).

For the support vectors, look at the Alpha values to determine which class it was.