how to crop images within a DataStore with labeled bounding boxes

I have a set of labeled images created with imageLabeler (Bounding boxes, not pixel-based segmentation). The images are too large to use as-is for training and the target objects are too smal to simply resize the images so I want to crop the images and create a training set with these cropped images and the corresponding bounding boxes they contain. As reported in a previous post (https://www.mathworks.com/matlabcentral/answers/525484-augmentedimagedatastore-center-crop-does-not-return-datastore-with-labels) using augmentedImageDataStore returns only the cropped images without the bounding boxes. Using pixelLabelImageDataStore as suggested in that post does not provide a solution since I am interested in bounding boxes and not segmentic segmentation. Any suggestion?

Answer 1

[Aylin]

Does BoxLabelDatastore (R2019b and newer) work for you? See: https://www.mathworks.com/help/vision/ref/boxlabeldatastore.html

[Raphael]

Rylan, Can you elaborate a little? This is what I have a the moment:

data = load('FrontLabels.mat'); % create by exporting the labels from imageLabeler Session

GT_data=data.gTruth;

GT_files=GT_data.DataSource.Source;

GT_labels=GT_data.LabelData;

GT_files_table=cell2table(GT_files);

FrontDataset=[GT_files_table GT_labels];

rng(0);

shuffledIndices = randperm(height(FrontDataset));

idx = floor(0.6 * length(shuffledIndices) );

trainingIdx = 1:idx;

trainingDataTbl = FrontDataset(shuffledIndices(trainingIdx),:);

imdsTrain = imageDatastore(trainingDataTbl{:,'GT_files'});

bldsTrain = boxLabelDatastore(trainingDataTbl(:,2:end));

trainingData = combine(imdsTrain,bldsTrain);

As you see I am already using boxLabelDatastore but I don't see how this could somehow help me create a DataStore that contains only some region of each images and the corresponding labeled bounding bxoes

[Aylin]

To clarify, are you trying to crop each image in the ImageDatastore using the bldsTrain data? I think you can use datastore transform to apply imcrop to each image:

function data = myTransformFcn(image, rect)
    data = imcrop(image, rect); % Note, this might need to be modified slightly if image and rect are cells.
    data = {data rect}; % Add the bounding boxes back to the result if you want it.
end

trainingData = transform(imdsTrain, bldsTrain, @myTransformFcn);

Note that you might need to change the myTransformFcn function depending on the box label syntax you're using.

If you're trying to generate new files out of this, you can use the writeall function for datastores: https://www.mathworks.com/help/matlab/ref/matlab.io.datastore.writeall.html?s_tid=doc_ta

I hope this helps!

[Raphael]

Thanks

Answer 2

[Dominique]

Not sure if you ever sorted this out, but I just stumbled upon this question when asking a different question about a function that I believe may provide a solution to your issue:

balanceBoxLabels

This function will essentially randomly sample blocks of a specified size (i.e. your network input size) that contain bounding boxes from your larger labelled images that are too large to use "as is" for training. It will create a blockLocationSet that you can then use to 'extract' the randomly sampled blocks in both a blockedImageDatastore and corresponding boxLabelDatastore. It will work even if you have just a single label class. The potential drawbacks are that it performs a random sampling of your images whereas you may prefer a systematic sampling, and there seems to be a lot going on 'under the hood' that isn't explained in the documentation, hence my question:

https://www.mathworks.com/matlabcentral/answers/2114871-how-does-the-overlapthreshold-argument-in-balanceboxlabels-get-passed-along-to-the-resulting-blocklo