Network doesn't work on test image

Hi all.

I'm trying to use and train a pretrained network (YOLO v2) on my own dataset (80 images).

I divided my set in training, validation and test, but network works correctly only on these images and not on other images that aren't in these set.

What can I do? I have to test my network only on the images that I have labeled?

Thanks

Answer 1

[arushi]

Hi Mario,

When using a pretrained network like YOLO v2 on your own dataset, it's important to ensure that your model generalizes well to new, unseen data. If your network performs well only on the images in your training, validation, and test sets, but not on other images, there are several factors and strategies you can consider to improve generalization:

1. Increase Dataset Size:

- Data Augmentation: Use techniques such as rotation, scaling, flipping, cropping, and color jittering to artificially increase the size and diversity of your dataset. This can help the model learn more robust features.

- Collect More Data: If possible, gather more labeled images that cover a wider variety of scenarios and conditions.

2. Improve Labels and Annotations:

- Ensure that your labels are accurate and consistent. Poor labeling can lead to poor model performance.

3. Fine-Tuning the Model:

- Adjust Learning Rate: Experiment with different learning rates. A learning rate that is too high might cause the model to converge too quickly to a suboptimal solution.

- More Epochs: Train for more epochs to allow the model to learn better representations, but watch out for overfitting.

4. Regularization Techniques:

- Use techniques like dropout, weight decay, or early stopping to prevent overfitting.

5. Evaluate and Adjust the Model:

- Validation: Regularly validate the model on a separate validation set to monitor its performance and adjust hyperparameters accordingly.

- Error Analysis: Analyze where the model is failing on new images. This can give insights into what features or scenarios the model is not capturing well.

6. Test on New Data:

- Ideally, you should test your model on a completely separate dataset that was not used during training or validation. This can give you a better indication of how well your model generalizes to new data.

By applying these strategies, you should be able to improve the generalization of your YOLO v2 model and achieve better performance on new, unseen images.

Hope this helps.

[Mario]

Thank you. I will do other tests soon.

[Mario]

Do you think that there are few images? How many images I have to use? Labeling images is time-expensive!

Answer 2

[Saurav]

Hi Mario,

As I understand it, you are using a pretrained network (YOLO v2) on your dataset in MATLAB; however, it fails to work for images not in the dataset.

Training a YOLO v2 model on a limited dataset, consisting of only 80 photos, can be a problem due to the potential for overfitting. To enhance your model's performance using MATLAB, consider the following major step:

• Data Augmentation:

1. Data augmentation refers to the process of artificially increasing the diversity and size of a training dataset by applying various transformations to the existing data, especially when dealing with limited data.
2. Augmentation effectively increases the size of the dataset without the need for additional data collection. Refer to the following documentation to learn more about this concept: https://www.mathworks.com/help/deeplearning/ref/imagedataaugmenter.html

Additional steps that can be addressed include:

• Label your Dataset:

1. Ensure that your dataset is accurately labeled. You can also use MATLAB's Image Labeler app to create bounding box annotations for each image. https://www.mathworks.com/help/vision/ug/get-started-with-the-image-labeler.html

• Dividing the Dataset:

1. A common split of the dataset is 70% training, 15% validation, and 15% test. However, with a small dataset, you might need to adjust these ratios to ensure enough data for training.https://www.mathworks.com/help/deeplearning/ug/divide-data-for-optimal-neural-network-training.html

• Modify & Train the Network:

1. Configure training options to prevent overfitting. Use a lower learning rate and consider using dropout if available. Refer: https://www.mathworks.com/help/deeplearning/ref/trainingoptions.html
2. Train the model using the modified network and augmented data. Experiment with different learning rates, batch sizes, and augmentation techniques.

By following these steps and iteratively refining your approach, you can improve the accuracy of your YOLO v2 model on new, unseen images.

Let me know if this works or if you need further help!

[Mario]

I've defined an augment data function (that I find on this website). I don't know if it works, because the training time is very long: I've started it about 40 minutes ago, but it's still loading. I will publish my code soon.

Answer 3

[Image Analyst]

Make sure your new images are resized to the required size for your model (the same size as your training, validation, and test set images). Are you sure they're all the same size?

What exactly does not "works correctly" mean? Does an error get thrown and it not give any result, or it gives a result but you just don't believe/like the result?

[Mario]

I have to control if images are the same size. The network doesn't give any result on new images.

[Image Analyst]

"I have to control if images are the same size." Yes, you do. Did you?

So it gives no result whatsoever? Did it hang (lock up the computer, never finish)?

[Mario]

I controlled my images. They aren't the same size, but I use a function that I find here to resize images, based on an input size that I give.

This is my code (augment data, resize image and label, helper sanitize boxes are function that I find here).

arch = imageDatastore("###MYFOLDER");

% HERE I CREATE THE GROUND TRUTH OBJECT, WITH 2 LABELS: "GATTO", "CANE"

data = load("gTruth80.mat") % THIS IS A TABLE

archCaniGatti = data.gTruth80

archCaniGatti.imageFilename = fullfile(archCaniGatti.imageFilename)

rng(0)

shuffledIndices = randperm(height(archCaniGatti));

idx = floor(0.6 * height(archCaniGatti));

trainingIdx = 1:idx;

trainingDataTbl = archCaniGatti(shuffledIndices(trainingIdx),:);

validationIdx = idx+1 : idx + 1 + floor(0.1 * length(shuffledIndices) );

validationDataTbl = archCaniGatti(shuffledIndices(validationIdx),:);

testIdx = validationIdx(end)+1 : length(shuffledIndices);

testDataTbl = archCaniGatti(shuffledIndices(testIdx),:);

imdsTrain = imageDatastore(trainingDataTbl{:,"imageFilename"});

bldsTrain = boxLabelDatastore(trainingDataTbl(:,["Cane","Gatto"]));

imdsValidation = imageDatastore(validationDataTbl{:,"imageFilename"});

bldsValidation = boxLabelDatastore(validationDataTbl(:,["Cane","Gatto"]));

imdsTest = imageDatastore(testDataTbl{:,"imageFilename"});

bldsTest = boxLabelDatastore(testDataTbl(:,["Cane","Gatto"]))

pretrainedDetector = yolov2ObjectDetector("tiny-yolov2-coco");

inputSize = [416 416 3];

gT = load("#GTRUTH_FILE")

[imds, blds] = objectDetectorTrainingData(gT.gTruth)

dataR = readall(blds)

ds = combine(imds,blds);

preprocessedData = transform(ds,@(dataR)resizeImageAndLabel(dataR,inputSize));

data = preview(preprocessedData);

rng(0);

preprocessedData = shuffle(preprocessedData);

dsTrain = subset(preprocessedData,trainingIdx);

dsVal = subset(preprocessedData,validationIdx);

dsTest = subset(preprocessedData,testIdx);

augmentedTrainingData = transform(dsTrain,@augmentData);

opts = trainingOptions("rmsprop", ...

InitialLearnRate=0.001, ...

MiniBatchSize=8, ...

MaxEpochs=10, ...

LearnRateSchedule="piecewise", ...

LearnRateDropPeriod=5, ...

VerboseFrequency=30, ...

L2Regularization=0.001, ...

ValidationData=dsVal, ...

ValidationFrequency=50, ...

OutputNetwork="best-validation-loss");

featureLayer = "leaky_relu_5";

numAnchorBoxes = 5; % I DON'T KNOW THE CORRECT VALUE

aboxes = estimateAnchorBoxes(preprocessedData,numAnchorBoxes);

numClasses = 2;

pretrainedNet = pretrainedDetector.Network;

lgraph = yolov2Layers(inputSize, numClasses, aboxes, pretrainedNet, featureLayer);

[detector,info] = trainYOLOv2ObjectDetector(augmentedTrainingData,lgraph,opts);

testData = combine(imdsTest, bldsTest)

% HERE THE NETWORK WORKS

data = read(testData)

i = data{1}

bbox = data{2}

label = data{3}

imgBis = insertObjectAnnotation(i, "rectangle", bbox, label)

figure

imshow(imgBis)

% HERE THE NETWORK DOESN'T WORK

data = imread("#UNSEEN_IMAGE")

data = imresize(data,inputSize(1:2));

[bboxes, labels] = detect(detector, data)

data = insertObjectAnnotation(data,"rectangle",bboxes,labels);

figure

imshow(data)

[Image Analyst]

OK, but it seems you forgot to answer the second paragraph of my reply. Here it is again:

What exactly does not "works correctly" mean? Does an error get thrown and it not give any result, or it gives a result but you just don't believe/like the result?

So does it ever get past the call to detect()? If so, what's in bboxes and labels? Are they just null, [ ]?

[Mario]

Bboxes and labels are empty.

[Image Analyst]

It's hard to debug your code without your model and your images. Can you at least post one image that works (non-null results) showing the boject that it finds the bounding box for, and one images that it doesn't find the bounding box for. Also say what is it finding? What is in the bounding box? A person, cat, vehicle, or something else???

[Mario]

My network has to find a cat or a dog ("Cane" and "Gatto" in my model).

This is what network should do:

But when I use the detect function on unseen images, this is the result:

Now it doesn't work on any image.

[Mario]

These are my folder and my gTruth file:

f = fullfile("gTruth80.mat")

f = "gTruth80.mat"

folder = fullfile("images.zip")

folder = "images.zip"

[Mario]

Hi, I did other tests, but it doesn't work. Labels and bboxes are always empty. I don't know what I can do.

Answer 4

[Vivek Akkala]

Hi Mario,

Training YOLO v2 with fewer than 80 images (considering you are dividing the total into training, validation, and test sets) is not feasible. I suggest increasing the size of your training dataset. While an ideal number of images cannot be precisely determined due to factors like object size, noise, lighting, and other elements, if you plan to train YOLO v2 to detect a single class, using between 300 to 400 images should yield optimal results. As mentioend in Arushi's suggestion it's good to have validation data. Ensure you have a sufficient amount of validation data (around 100 images) and regularly monitor validation performance to understand how the model performs on unseen data. Ideally, you can use the trained YOLO v2 network for inference once the validation accuracy exceeds 95%.