neural network validation accuracy on Test. Images

Hi All Professionals,

I have this code works fine!!

What I am trying to observe is the performance of the test images, can some direct me to a link or an example to see this process visually?

How is validation done on convolution networks after the training?

How can I see the validation performance and visual how accurate the network was at this phase?

Can someone assist me please?

thank you in advance have a great day!

%% Turen Of PNG Warnings
[~, warnid] = lastwarn; %get identifier of warning
warning('off', warnid); %turn warning off

% clc
% close all
% clear 

%% Training The R-CNN Detector On Gun Dataset
%% Step 1 Loading Training Data & Network Layers.
load('ReviewSim264.mat');
load('layers40.mat');
save new.mat ReviewSim layers;
load('new.mat', 'ReviewSim','layers');
summary(ReviewSim);


%% Step 2 Specifing Image Location
imDir = fullfile(matlabroot,'Revims');
addpath(imDir);

%% Step 3 Accessing Content of Folder TrainingSet Using Datastore
imds =imageDatastore(imDir,'IncludeSubFolders',true,'LabelSource','Foldernames');
tbl = countEachLabel(imds);
%imds.Labels
% 
%% Step 4 Splitting Inputs Into Training and Testing Sets
[imdsTrain,imdsValidation] = splitEachLabel(imds,0.7,'randomized');
size(imdsTrain);

%% Step 5 Specifying Input Size OF 1st Network Layer
inputSize = layers(1).InputSize;
labelData = ReviewSim.imageFilename;

%% Step 6 Replacing Final Layer/Last 3 Configure For Classes
% Finetuning these 3 layers for new classification
% Extracting all Layers except the last 3
layersTransfer = layers(1:end-3);
% Stipulating Amount Of Classes 
numClasses = numel(categories(imdsTrain.Labels));

% Adding Newly Edited Layers
Tlayers = [layersTransfer
  fullyConnectedLayer(numClasses,'Name','fullyConn')
  softmaxLayer('Name','softmax')
  classificationLayer('Name','classoutput','Classes', 'auto')];

%% Step 7 Warping Images For Added Accuracy
pixelRange = [-30 30];
imageAugmenter = imageDataAugmenter(...
'RandRotation',[-40 40],...
'RandXReflection',true,...
'RandYReflection',true,...
'RandXShear',[-15 15],...
'RandYShear',[-10 10],...
'RandXTranslation',pixelRange, ... 
'RandYTranslation',pixelRange);

%% Step 8 Deploying Augmentation Preventing Overfitting   
augimdsValidation = augmentedImageDatastore(inputSize,imdsValidation,...
'ColorPreprocessing','gray2rgb','DataAugmentation',imageAugmenter);

augmentedTrainingSet = augmentedImageDatastore(inputSize,imdsTrain,...
'ColorPreprocessing', 'gray2rgb','DataAugmentation',imageAugmenter);

%% Step 9 Specifying Option Features
% automatically drop the learn rate during training using a piecewise
% learn rate schedule
options = trainingOptions('sgdm',... 
    'Momentum',0.8,... 
    'InitialLearnRate', 1e-3,... 
    'LearnRateSchedule', 'piecewise', ... 
    'LearnRateDropFactor', 0.1, ... 
    'Shuffle','every-epoch', ... 
    'LearnRateDropPeriod', 14, ... 
    'L2Regularization', 1e-4, ... 
    'MaxEpochs',8,... 
    'MiniBatchSize',20,...
    'Verbose', true);

%% Step 10 Combining All Network Variables For Training Sequence
netTransfer = trainNetwork(augmentedTrainingSet,Tlayers,options);

%% Step 11 Training The R-CNN Detector/Display Network Layers 
rcnn = trainRCNNObjectDetector(ReviewSim, netTransfer, options, 'NegativeOverlapRange', [0 0.3]);
rcnn.RegionProposalFcn;
network = rcnn.Network;
layers = network.Layers;

%% Step 12 Displaying RCNN Class Names
rcnn.ClassNames;


%% Step 13 Displaying Strongest Detection Result.
img = imread('3.jpg');

%loop through all images of the augmented dataset and predict guns
%locations in them
%compare the output of the rcnn with the truth location of the image guns.

[bbox, score, label] = detect(rcnn, img, 'MiniBatchSize', 8,'SelectStrongest',true);
[score, idx] = max(score);
bbox = bbox(idx, :); 
annotation = sprintf('%s: (Confidence = %f)', label(idx), score);
detectedImg = insertObjectAnnotation(img, 'rectangle', bbox, annotation);
figure
imshow(detectedImg);

%test the network over the unknown validation dataset 

[Kai Ketelhut]

by C++ CalculatorTutorial within inf because of Plank and Kelvin program by devision with 0 your 0.15°K upon physics that this is inf

[Matpar]

Hi Kai Ketelhut, I don't get it!!

please elaborate!

Answer 1

[Bhargavi Maganuru]

trainRCNNObjectDetector doesn’t support “ValidationData”, “ValidationFrequency” and “ValidationPatience” options as of now. Hence, it is not possible to directly feed the validation data to the trainingOptions parameter.

Possible workaround for the k-fold validation could be to run a loop “k” times.

• In each loop, split the train set randomly into two sets, where one set will be used for training and the other for validation. You can use crossvalind or cvpartition functions for this purpose.
• Use this new train set for training the model and after the training is done, use the validation set to evaluate the model.

This above step is done “k” times.

For more information of k-fold cross validation, you can refer to the following links:

1) https://www.mathworks.com/help/bioinfo/ref/crossvalind.html

2) https://www.mathworks.com/help/stats/cvpartition.html

Answer 2

[Saira]

Hi,

I have 5600 training images. I have extracted features using Principal Component Analysis (PCA). Then I am applying CNN on extracted features. My training accuracy is 30%. How to increase training accuracy?

Feature column vector size: 640*1

My training code:

% Convolutional neural network architecture

layers = [

imageInputLayer([1 640 1]);

reluLayer

fullyConnectedLayer(7);

softmaxLayer();

classificationLayer()];

options = trainingOptions('sgdm', 'Momentum',0.95, 'InitialLearnRate',0.0001, 'L2Regularization', 1e-4, 'MaxEpochs',5000, 'MiniBatchSize',8192, 'Verbose', true);

[Matpar]

Look at the options section of the code above, try those settings!

or simple add more training data!

if that doesn't work try augmentation, shearing, reflecting images which is also in the code above!

hope that helps

[Vinay Chawla]

Try a greater Initial learn rate say '0.001' and then add a dropping factor.

[Ullah Nadeem]

Your network seems really brief.

[Rayan Matlob]

Hi @Saira, Iam trying to apply PCA on 5 folders classes (each folder contain about 200 images), which is similar to what you are doing..

Can you share the code please ?