How to split training, validation and testing from ground truth file generated by ground truth label application

Hi guys

i want to know how to split training, validation and testing from ground truth file generated by ground truth labeler application i need for example 70% for training 20% for validation and 10% for testing

And then how can i evaluate the created detector with validation data and testing data using evaluateDetectionPrecision

Herein the code

load('gTruth.mat')
fruits = selectLabels(gTruth,{'apple','orange'});
if isfolder(fullfile('TrainingData'))
    cd TrainingData
else
    mkdir TrainingData
end 
addpath('TrainingData');

options = trainingOptions('sgdm', ...
    'MiniBatchSize', 32, ...
    'InitialLearnRate', 1e-3, ...
    'MaxEpochs', 250, ...
    'CheckpointPath', tempdir);

classes = {'first', 'second'};
outputs = 1+numel(classes); % +1 for background class

layers = [
    imageInputLayer([32 32 3],"Name","imageinput")
    convolution2dLayer([3 3],32,"Name","conv_1","Padding",[1 1 1 1],"WeightsInitializer","narrow-normal")
    reluLayer("Name","relu_1")
    convolution2dLayer([3 3],32,"Name","conv_2","Padding",[1 1 1 1],"WeightsInitializer","narrow-normal")
    reluLayer("Name","relu_2")
    maxPooling2dLayer([2 2],"Name","maxpool","Stride",[2 2])
    fullyConnectedLayer(64,"Name","fc_1","WeightsInitializer","narrow-normal")
    reluLayer("Name","relu_3")
    fullyConnectedLayer(outputs)
    softmaxLayer("Name","softmax")
    classificationLayer("Name","classoutput")];    
trainingData = objectDetectorTrainingData(fruits,'SamplingFactor',1,...
'WriteLocation','TrainingData');
detector = trainFastRCNNObjectDetector(trainingData, layers, options, ...
    'NegativeOverlapRange', [0 0.1], ...
    'PositiveOverlapRange', [0.7 1]);
save('Detector.mat','detector');

Answer 1

[Uttiya Ghosh]

Hi Abdussalam,

From my understanding, you would like to know the function that can be used to split a dataset into training, validation and testing sub datasets. I am assuming a random dataset of size 1000*10 and providing you the solution below.

% Define the datasize 
dataSize=1000; 
featureDim=10; 
% Create the dataset 
data=rand(dataSize,featureDim); 
% Define the split sizes 
trainSize=0.7*dataSize; 
valSize=0.2*dataSize; 
testSize=0.1*dataSize; 
% Create the training data 
cv=cvpartition(dataSize,'HoldOut',valSize+testSize); 
idx=cv.test; 
dataTrain = data(~idx,:); 
dataValTest  = data(idx,:); 
% Create validation and test data 
cv=cvpartition(valSize+testSize,'HoldOut',testSize); 
idx=cv.test; 
dataVal = data(~idx,:); 
dataTest  = data(idx,:); 

For the second part of your question, from my undersanding you want to know the usage of usage of evaluateDetectionPrecision. I have used 'vehicleTrainingData.mat' file to explain the usage, This file is available in MATLAB.

% Load vehicleTrainingData 
vehicleData = load('vehicleTrainingData.mat'); 
trainingData = data.vehicleTrainingData; 
% Add fullpath to the local vehicle data folder. 
dataDir = fullfile(toolboxdir('vision'), 'visiondata'); 
trainingData.imageFilename = fullfile(dataDir, trainingData.imageFilename); 
% Create an imageDatastore using the files from the table 
imds = imageDatastore(trainingData.imageFilename); 
% Create a boxLabelDatastore using the label columns from the table 
blds = boxLabelDatastore(trainingData(:,2:end)); 
% Load YOLOv2 Detector 
vehicleDetector = load('yolov2VehicleDetector.mat'); 
detector = vehicleDetector.detector; 
% Run the detector with imageDatastore. 
results = detect(detector, imds); 
% Evaluate the results against the ground truth data. 
[ap, recall, precision] = evaluateDetectionPrecision(results, blds); 

For more information, refer to the links given below:

https://www.mathworks.com/help/stats/cvpartition.html 

https://www.mathworks.com/help/vision/ref/evaluatedetectionprecision.html

[Abdussalam Elhanashi]

Hi Uttiya Ghosh

Thanks for your reply

I how can i implement the split in this code as i amusing ground truth file

load('gTruth.mat')
fruits = selectLabels(gTruth,{'apple','orange'});
if isfolder(fullfile('TrainingData'))
    cd TrainingData
else
    mkdir TrainingData
end 
addpath('TrainingData');
options = trainingOptions('sgdm', ...
    'MiniBatchSize', 32, ...
    'InitialLearnRate', 1e-3, ...
    'MaxEpochs', 250, ...
    'CheckpointPath', tempdir);
classes = {'first', 'second'};
outputs = 1+numel(classes); % +1 for background class
layers = [
    imageInputLayer([32 32 3],"Name","imageinput")
    convolution2dLayer([3 3],32,"Name","conv_1","Padding",[1 1 1 1],"WeightsInitializer","narrow-normal")
    reluLayer("Name","relu_1")
    convolution2dLayer([3 3],32,"Name","conv_2","Padding",[1 1 1 1],"WeightsInitializer","narrow-normal")
    reluLayer("Name","relu_2")
    maxPooling2dLayer([2 2],"Name","maxpool","Stride",[2 2])
    fullyConnectedLayer(64,"Name","fc_1","WeightsInitializer","narrow-normal")
    reluLayer("Name","relu_3")
    fullyConnectedLayer(outputs)
    softmaxLayer("Name","softmax")
    classificationLayer("Name","classoutput")];    
trainingData = objectDetectorTrainingData(fruits,'SamplingFactor',1,...
'WriteLocation','TrainingData');
detector = trainFastRCNNObjectDetector(trainingData, layers, options, ...
    'NegativeOverlapRange', [0 0.1], ...
    'PositiveOverlapRange', [0.7 1]);
save('Detector.mat','detector');

[Uttiya Ghosh]

Hi Abdussalam,

Could you share the gTruth.mat. file?

[Abdussalam Elhanashi]

Hi Uttiya Ghosh

attached is MAT file

looking for your support

[bekal]

what fis featureDM in this case ?