Problem: Image segmentation of forest area using CNN and MATLAB's BLOCKPROC function.

Hello everyone, how are you?

I trained a Convolutional Neural Network - CNN (Resnet-18) to separate two classes (Palms / Non-Palms [background]) from a forest area.

Now, I need to generate the segmentation of the area from the trained network's result using the command:

C = semanticseg(I, network);

However, with the image being too large (21800x38480x3 uint8), the result shows the following error: "Maximum variable size allowed on the device is exceeded." Therefore, I decided to use a resource suggested by the Matlab website itself, which is to use the command BLOCKPROC, which processes the image in parts, which apparently would solve this problem of processing large images. So, I used the following commands:

I = imread('imagem.tif');
fun = @(block_struct) semanticseg(block_struct.data, network);
C = blockproc(I,[1000 1000],fun);

The result shows the following errors:

"Error using blockprocInMemory Invalid output class. The user function, FUN, returned an invalid result. The class of the result was categorical.

Error in blockproc (line 251) result_image = blockprocInMemory(source,fun,options);"

I didn't understand, as the result shouldn't be of the categorical class. I tried to modify the function to:

fun = @(block_struct) uint8(semanticseg(block_struct.data, net));

without success! :-(

Could someone please help me?

===================================================================================================

Information about my Computer:

===================================================================================================

Processor 12th Gen Intel(R) Core(TM) i7-12700H 2.30 GHz Installed RAM 32.0 GB (usable: 31.7 GB) System type 64-bit operating system, x64-based processor

===================================================================================================

Information about my Graphics Card:

===================================================================================================

CUDADevice with properties:

Name: 'NVIDIA GeForce RTX 3070 Ti Laptop GPU'

Index: 1

ComputeCapability: '8.6'

SupportsDouble: 1

DriverVersion: 12

ToolkitVersion: 11.2000

MaxThreadsPerBlock: 1024

MaxShmemPerBlock: 49152 (49.15 KB)

MaxThreadBlockSize: [1024 1024 64]

MaxGridSize: [2.1475e+09 65535 65535]

SIMDWidth: 32

TotalMemory: 8589410304 (8.59 GB)

AvailableMemory: 6583049776 (6.58 GB)

MultiprocessorCount: 46

ClockRateKHz: 1485000

ComputeMode: 'Default'

GPUOverlapsTransfers: 1

KernelExecutionTimeout: 1

CanMapHostMemory: 1

DeviceSupported: 1

DeviceAvailable: 1

DeviceSelected: 1

[Matt J]

I would implement fun as a non-anonymous function. Then, you can stop inside the function and inspect the result:

C = blockproc(I,[1000 1000],@(b)fun(b,net));

function out = fun(block_struct,net)

   out = uint8(semanticseg(block_struct.data, net));
   
   whos out
   keyboard

end

[Image Analyst]

Did you train your network with 1000*1000 images? It must match the blockproc tile size.

[Airton Gaio Junior]

Hello, how are you?

I trained the network with 4,660 image files (80% of the database) each with a size of 512x512 pixels, and tested it with 1,165 image files (20% of the database) of the same size (Figure 1). Here's the configuration used for training the network:

...

Figure 1 - Example of training images.

miniBatchSize = 16; 
learnRate=0.05; 

options = trainingOptions('sgdm', ...
    'Momentum',0.9,...
    'ExecutionEnvironment','gpu',...
    'LearnRateSchedule','piecewise',...
    'InitialLearnRate',learnRate, ...
    'LearnRateDropFactor',0.5, ...
    'LearnRateDropPeriod',5, ...
    'MaxEpochs',10, ...  
    'MiniBatchSize',miniBatchSize, ...
    'Shuffle','every-epoch', ...
    'GradientThreshold',0.05,...
    'VerboseFrequency',2,...
    'ValidationPatience', 4,...
    'Verbose',1,...
    'Plots','training-progress', ...
    'ValidationData',dsTest);

Interestingly, when I use a smaller image with the command:

C = semanticseg(I, network);

to segment the image I, I have no problems, including with the output type. However, I need to segment a much larger image (21800x38480x3 uint8). So, I tried to use BLOCKPROC to process the image in parts. I understood that the value used [1000 x 1000] in the command:

C = blockproc(I,[1000 1000],fun);

represents the number of rows and columns that will be processed at a time in each execution until the end of the image, right? Am I doing something wrong?

Could you please help me? I really need to process the segmentation of the entire image.

Best regards,

Answer 1

[Joss Knight]

In the documentation for semanticseg it says that the output is a categorical array.

Converting to uint8 would normally work, but not if the input image isn't uint8. To be sure try cast(...,'like',...), so:

proto = I([]); % Reduce to simple prototype to avoid memory issues
fun = @(block_struct) cast(semanticseg(block_struct.data, net), 'like', proto);