Improve accuracy of small data set using Neural Network

Hello ,i have an input matrix 4x24( 4features of 24 patients which 10 are positive and 14 negative)and target 2x24 in eye(2) form for a binary problem classification .I use patternnet and network node topology 4-3-2 with the transfer functions in default(tansig) .The total accuracy is 66% .If i dont use the validation set the accuracy goes up 90%.Is that correct or overfitting? Is it possible to have unbalanced data? How can i improve accuracy in this tiny data set?I use this code for k fold cross validation for 10 repetitions .

Answer 1

[Greg Heath]

 [I N ] = input; % [ 4 24 ]
 [O N ] = target;% [ 1 24 ]
 Ntrn   = N -2*round(0.15N) % 16 default
 Ntrneq = Ntrn*O            % 16 No. of training equations
 Nw     = (I+1)*H+(H+1)O = (I+O+1)*H+O % No. of unknown weights
 % NO OVERFITTING Ntrneq >= Nw   <==>  H <= (Ntrneq-O)/(I+O+1)=  2.5

 ==> H = 3 is a slight overfitting ==> Using 15% validation set is  justified .

 But have you tried 10 trials each of 

 H = 2 
 or
 MSEREG
 or 
 TRAINBR ?

    or 

 10-FOLD X-VALIDATION?

Hope this helps.

Greg

[Lia]

Thank you for your answer ,i use your tutorial for 10 fold X Val for my classification problem using patternnet instead of fitnet with training function 'trainlm' and H=3 . For 10 repetitions i have a mean accuracy 66%. If i change H=2 mean accuracy falls and the same happens if i use trainbr and trainscg.The change mse to Msereg in matlabR2012b seems to be ignored .I have attached the relevant code in CV.mat .Does it look ok with differentions that i made?

[Greg Heath]

The point of using MSEREG and/or TRAINBR is that you can use H >> 2.

Hope this helps.

Greg