Data augmentation for YOLOv2 isn't work

Hello,

I try to trained a YOLOv2 objectdetection network, but I encounter some problems about data preprocessing(data augmentation and resize ) before training process.

I use the support function augmentData and preprocessData providing in following example to preprocessing my trainingdata .

https://www.mathworks.com/help/vision/examples/train-an-object-detector-using-you-only-look-once.html

the code and function is below：

function B = augmentData(A)
% Apply random horizontal flipping, and random X/Y scaling. Boxes that get
% scaled outside the bounds are clipped if the overlap is above 0.25. Also,
% jitter image color.
B = cell(size(A));

I = A{1};
sz = size(I);
if numel(sz)==3 && sz(3) == 3
    I = jitterColorHSV(I,...
        'Contrast',0.2,...
        'Hue',0,...
        'Saturation',0.1,...
        'Brightness',0.2);
end

% Randomly flip and scale image.
tform = randomAffine2d('XReflection',true,'Scale',[1 1.1]);
rout = affineOutputView(sz,tform,'BoundsStyle','CenterOutput');
B{1} = imwarp(I,tform,'OutputView',rout);

% Apply same transform to boxes.
[B{2},indices] = bboxwarp(A{2},tform,rout,'OverlapThreshold',0.25);
B{3} = A{3}(indices);

% Return original data only when all boxes are removed by warping.
if isempty(indices)
    B = A;
end
end

function data = preprocessData(data,targetSize)
% Resize image and bounding boxes to the targetSize.
scale = targetSize(1:2)./size(data{1},[1 2]);
data{1} = imresize(data{1},targetSize(1:2));
data{2} = bboxresize(data{2},scale);
end

----------------------------------------------------------------------------------------------------------------------------------

% data augmentation
% The trainingData is a datastore created by groundtruth object of video labeler
augmentedTrainingData = transform(trainingData,@augmentData);

% preprocess training data
inputSize=[224 224 3];
preprocessedTrainingData = transform(augmentedTrainingData,@(data)preprocessData(data,inputSize));

% Train the network
detector= trainYOLOv2ObjectDetector(preprocessedTrainingData,detector,options);

But I got a error message：

Invalid transform function defined on datastore.

The cause of the error was:

Error using bboxwarp>iParseInputs (line 324)
The value of 'bboxA' is invalid. Expected input number 1, bboxA, to be integer-valued.

Error in bboxwarp (line 81)
params = iParseInputs(bboxA,tform,ref,varargin{:});

Error in augmentData (line 23)
[B{2},indices] = bboxwarp(A{2},tform,rout,'OverlapThreshold',0.25)

Then I checked my training data and found some values of bboxes are not integer, like [ 8.8000 75.0000 39.6000 101.4000 ]

so I try to transform them to integer. However, the label data are for read only, I don’t know how deal with this problem.

Also, I make some experiments and found if I use the video labeler to create a groundtruth, some value of bboxes will not be integer.

if I use the image labeler to create a groundtruth, the value of bboxes will be integer and they can be augmented and preprocessed using the functions I mentioned above.

I think that maybe the algorithm used to automately create bbox make the value of bbox non-integer. (I used "Temporal interpolator algorithm" to label my video)

On the other hand, I read one image from the training datastore(created by video) and using the method in the following example

https://www.mathworks.com/help/deeplearning/ug/bounding-box-augmentation-using-computer-vision-toolbox.html

I want to know whether it can be preprocessed or not. The result is that I can not rotate or rescale for only one image also.

Hope someone could help me to solve this problem, Thx.

Answer 1

[Mona Al-Kharraz]

you can try this code:

function B = augmentData(A)

% Apply random horizontal flipping, and random X/Y scaling. Boxes that get

% scaled outside the bounds are clipped if the overlap is above 0.25. Also,

% jitter image color.

B = cell(size(A));

I = A{1};

sz = size(I);

if numel(sz)==3 && sz(3) == 3

I = jitterColorHSV(I,...

'Contrast',0.2,...

'Hue',0,...

'Saturation',0.1,...

'Brightness',0.2);

end

% Randomly flip and scale image.

tform = randomAffine2d('XReflection',true,'Scale',[1 1.1]);

rout = affineOutputView(sz,tform,'BoundsStyle','CenterOutput');

B{1} = imwarp(I,tform,'OutputView',rout);

% Apply same transform to boxes.

boxEstimate=round(A{2});

boxEstimate(:,1)=max(boxEstimate(:,1),1);

boxEstimate(:,2)=max(boxEstimate(:,2),1);

[B{2},indices] = bboxwarp(boxEstimate,tform,rout,'OverlapThreshold',0.25);

B{3} = A{3}(indices);

% Return original data only when all boxes are removed by warping.

if isempty(indices)

B = A;

end

end

function data = preprocessData(data,targetSize)

scale = targetSize(1:2)./size(data{1},[1 2]);

data{1} = imresize(data{1},targetSize(1:2));

boxEstimate=round(data{2});

boxEstimate(:,1)=max(boxEstimate(:,1),1);

boxEstimate(:,2)=max(boxEstimate(:,2),1);

data{2} = bboxresize(boxEstimate,scale);

end

[Zheng Yuan]

Thanks very much

[Bishal Paudel]

Thank you very much. It worked and thanks to the questionare so every one is benefitted.

Answer 2

[Massimo Del Guasta]

Thank You very much Mona !!!

I replaced the @augmentData function suggested by Mathworks with your function.. and I finally receive no errors when training my Fast-RCNN with the augmented datastore. I see that the unvalid boxes I had before with the original augmentData disappeare from the augmented ds as obtained by your function. I added random rotations and translations to my Imagedatastore and it still works (original @augmentData worked for me only with X-Y mirror and color adjustments) .

I admit that , even if I am succesfully using MATLAB from the past 'century' , but being a beginner in MATLAB databases and related functions, I don't find Mathwork very helpy in this field: The explanations for many functions / properties are very very chaotic for beginners, and some of them apparently do not work going behind the Mathwork examples. As a beginner in this field, I spend much more time to adapt some ds functions etc. to my real-world-ds than training the CNNs.

Thanks again