How to find multiple rectangles in an image

Hi,

I have been going through the demo for normxcorr2 and looking for a way to adapt it to find multiple instances of a similarly shaped rectangle from within a test image. I can see that it requires finding a set of multiple peaks but I'm lost after that. Ideally I would want to see the rectangles all highlighted on the test image and a list of their coordinates. Attached are two simplified images where I want to find every rectangle in example1.png (no matter what the shade of it). Any ideas?

clear variables
close all
clc
%==========================================================================
%  % store the photos into memory

file1=('example1.tif');
file2=('exampleBar.tif');
%==========================================================================
%%read in the images
img=imread(file1);
img=img(:,:,1);
imgBar=imread(file2); 
imgBar=imgBar(:,:,1);
%%check images are what you expect
subplot(1,2,1)
imshow(img)
title(file1);
subplot(1,2,2)
imshow(imgBar)
title(file2);
%==========================================================================
%%how to check how many times the bar is in the original image?

   c = normxcorr2(imgBar,img);
  %  figure 
  %  surf(c) 
  %  shading flat

% Find the peak in cross-correlation.
[ypeak, xpeak] = find(c==max(c(:)));

% Account for the padding that normxcorr2 adds.
yoffSet = ypeak-size(imgBar,1);
xoffSet = xpeak-size(imgBar,2);

% Display the matched area.
figure
imshow(img);
imrect(gca, [xoffSet+1, yoffSet+1, size(imgBar,2), size(imgBar,1)]);

[Malte Herrmann]

2 thoughts:

1. You can check the color value for each pixel and see if it is different from [1, 1, 1] (or whatever your background is) and then change its color to [0, 0, 0]. That way all rectangles become black and are separated from the shades. Maybe this will help the cross correlation.

2. IIRC, there's 2D-Fast Fourier Transformation. This might help in recognizing existing shapes by identifying shapes and sizes? Don't know about the implementation though...

Answer 1

[Image Analyst]

Try finding blobs that are not the background gray level. You can count them with bwlabel. Or you can count and find properties like the bounding box, area, etc. with regionprops(). Full fancy demo below.

clc;    % Clear the command window.
close all;  % Close all figures (except those of imtool.)
clearvars;  % Erase all existing variables. Or clearvars if you want.
workspace;  % Make sure the workspace panel is showing.
format long g;
format compact;
fontSize = 20;

folder = pwd;
baseFileName = 'example1.png';
% Get the full filename, with path prepended.
fullFileName = fullfile(folder, baseFileName);
% Check if file exists.
if ~exist(fullFileName, 'file')
  % The file doesn't exist -- didn't find it there in that folder.
  % Check the entire search path (other folders) for the file by stripping off the folder.
  fullFileNameOnSearchPath = baseFileName; % No path this time.
  if ~exist(fullFileNameOnSearchPath, 'file')
    % Still didn't find it.  Alert user.
    errorMessage = sprintf('Error: %s does not exist in the search path folders.', fullFileName);
    uiwait(warndlg(errorMessage));
    return;
  end
end
grayImage = imread(fullFileName);
% Get the dimensions of the image.
% numberOfColorChannels should be = 1 for a gray scale image, and 3 for an RGB color image.
[rows, columns, numberOfColorChannels] = size(grayImage);
if numberOfColorChannels > 1
  % It's not really gray scale like we expected - it's color.
  % Use weighted sum of ALL channels to create a gray scale image.
%   grayImage = rgb2gray(grayImage);
  % ALTERNATE METHOD: Convert it to gray scale by taking only the green channel,
  % which in a typical snapshot will be the least noisy channel.
  grayImage = grayImage(:, :, 1); % Take red channel.
end
% Display the image.
subplot(2, 2, 1);
imshow(grayImage, []);
title('Original Grayscale Image', 'FontSize', fontSize, 'Interpreter', 'None');
hp = impixelinfo;
axis('on', 'image'); % Make sure image is not artificially stretched because of screen's aspect ratio.

%------------------------------------------------------------------------------
% Set up figure properties:
% Enlarge figure to full screen.
set(gcf, 'Units', 'Normalized', 'OuterPosition', [0, 0.04, 1, 0.96]);
drawnow;

backgroundGrayLevel = mode(grayImage(1,:))
% Turn into a binary image.
binaryImage = grayImage ~= backgroundGrayLevel;

% Find all the blobs.
[labeledImage, numberOfRegions] = bwlabel(binaryImage);
fprintf('Found %d regions.\n',numberOfRegions);

% Display the binary image.
subplot(2, 2, 2);
% imhist(grayImage);
imshow(binaryImage, []);
caption = sprintf('Binary Image with %d Rectangles', numberOfRegions);
title(caption, 'FontSize', fontSize, 'Interpreter', 'None');
hp = impixelinfo;
axis('on', 'image'); % Make sure image is not artificially stretched because of screen's aspect ratio.

% Let's assign each blob a different color to visually show the user the distinct blobs.
coloredLabels = label2rgb (labeledImage, 'hsv', 'k', 'shuffle'); % pseudo random color labels
% coloredLabels is an RGB image.  We could have applied a colormap instead (but only with R2014b and later)
subplot(2, 2, 3);
imshow(coloredLabels);
axis('on', 'image'); % Make sure image is not artificially stretched because of screen's aspect ratio.
caption = sprintf('Pseudo colored labels, from label2rgb().');
title(caption, 'FontSize', fontSize);

% Get the bounding boxes
props = regionprops(binaryImage, 'BoundingBox');
allBB = [props.BoundingBox];
allX = allBB(1:4:end) + 0.5;
allY = allBB(2:4:end) + 0.5;
allWidths = allBB(3:4:end);
allHeights = allBB(4:4:end);
% Print out bounding boxes
for k = 1 : length(props)
  row1 = allY(k);
  row2 = allY(k) + allHeights(k) - 1;
  col1 = allX(k);
  col2 = allX(k) + allWidths(k) - 1;
  fprintf('Box %d goes from (for Y) : row %d to row %d, and (for X) column %d to column %d\n',...
    k, row1, row2, col1, col2);
end

In the command window you'll see:

Found 16 regions.
Box 1 goes from (for Y) : row 244 to row 252, and (for X) column 89 to column 151
Box 2 goes from (for Y) : row 367 to row 376, and (for X) column 89 to column 151
Box 3 goes from (for Y) : row 586 to row 594, and (for X) column 89 to column 151
Box 4 goes from (for Y) : row 709 to row 718, and (for X) column 89 to column 151
Box 5 goes from (for Y) : row 182 to row 190, and (for X) column 164 to column 226
Box 6 goes from (for Y) : row 524 to row 532, and (for X) column 164 to column 226
Box 7 goes from (for Y) : row 305 to row 314, and (for X) column 165 to column 227
Box 8 goes from (for Y) : row 648 to row 656, and (for X) column 165 to column 227
Box 9 goes from (for Y) : row 244 to row 252, and (for X) column 231 to column 293
Box 10 goes from (for Y) : row 367 to row 376, and (for X) column 231 to column 293
Box 11 goes from (for Y) : row 586 to row 594, and (for X) column 231 to column 293
Box 12 goes from (for Y) : row 709 to row 718, and (for X) column 231 to column 293
Box 13 goes from (for Y) : row 182 to row 190, and (for X) column 306 to column 368
Box 14 goes from (for Y) : row 524 to row 532, and (for X) column 306 to column 368
Box 15 goes from (for Y) : row 305 to row 314, and (for X) column 307 to column 369
Box 16 goes from (for Y) : row 648 to row 656, and (for X) column 307 to column 369

[Stephen Devlin]

Thank you Image Analyst. Side question, I saw a book called Image Processing Handbook by Russ, is that one you'd recommend or is there a better book?