Neural Network Performance function for classification problem

My neural network seems to be selecting a non-optimal solution, and I think I know the problem but I don't know how to fix it.

My problem is a classification problem between 13 classes, 1-13, but the performance functions for neural networks seem to be based on MSE, so they represent mistaking class 1 for 13 as worse than mistaking class 1 for 2.

For classification problems, I would like to train my neural network to want to increase its overall accuracy, or possibly its balanced error rate or average f-score, but I do not wish to train my neural network on a regression-based criteria.

Is this possible?

Also, I see that I have parameters for the percent of values used for training, validation and testing. Are the training and validation values resampled at every epoch while the testing values held constant? If this is the case then I assume I understand that every epoch, the data is resampled, then the network is training another iteration, then the network is trained on the validation data to estimate if it is improving or regressing.

I am not used to knowing so little about the innerworkings of a feature I am attempting to use, and would appreciate any comments or guidance someone can give me.

[Lee]

I believe the equal class weightings is the 'normalization', 'standard' attribute of the mse class, but I'd still like a clarification on if I understand that correctly.

Also I really have no way I know of to confirm my understanding of the training vs validation structure and of train(neuralnetwork,X,Y) and would really appreciate an explanation if someone can clarify.

Answer 1

[Greg Heath]

It is always best to start with the examples in

 help patternnet

 doc patternnet

Then I can respond to specific questions. One obvious mistake is your choice of target matrix format. The target matrix for c classes should be c-dimensional {0,1} unit vectors that are related to the target class indices via

 targets           = ind2vec(trueclassindices)
 trueclassindices = vec2ind(targets)

Then, the 0/1 errors row vector obtained from the trained net outputs is given by

 errors = (outputs~=targets)

I have posted many, many, examples. Search in NEWSGROUP and ANSWERS using

 greg patternnet

Hope this helps.

Thank you for formally accepting my answer

Greg

[Lee]

Thank you Greg, your other answers helped me a lot as well already, I'm reading through them now.