Localization defect real coordinate

Ive been doing some reseach on localizing defect and actually to provide information about the real coordianate of the defect such as in x and y axis or meters. im detecting red colour, filter it and i segmentized it. But i couldnt find the algorithm to locate the actual location of the defect in the thermal image.

Answer 1

[Image Analyst]

What do you mean by "actual location"? Do you want a list of all the (row, column) coordinates of all the pixels in your blob? Or just the centroid? See my Image Segmentation Tutorial in my File Exchange

Do you want in real world units, like meters? See attached spatial calibration demo.

Do you need to calibrate for non-planar scenes? If so see https://www.mathworks.com/products/computer-vision.html#camera-calibration

[Faiz Yusoff]

Basically i want to display the coordinate real world unit (mm/cm) like in the picture above, which is im detecting red colour or dots.

So i think what i need to is to filter and segmentized

then get the centroid of the object, which i cannot do it yet.

What is the best method to achieve something like in the picture?

find the cenroid and real world coordinate

[Image Analyst]

Use the Color Thresholder app on the Apps tab of the tool ribbon to export a function that will give you the red pixels. Use HSV color space. Then you need to know the pixel coordinates x and y of the origin, (x0, y0), wherever that is, like the upper left of the upper leftmost black square or of the white playing board. Then use regionprops on the red mask to get the centroid:

redMask = bwareafilt(redMask, 1); % Take largest blob only.
props = regionprops(redMask, 'Centroid');
xr = props.Centroid(1);
yr = props.Centroid(2);
% Now find the delta x and delta y
deltaXInPixels = xr - x0
deltaYInPixels = yr - y0
% Convert to mm by multiplying by the spatial calibration factor.
xInMm = deltaXInPixels / pixelsPerMm; % Or * mmPerPixel, depending on what your calibration factor has in the denominator.
yInMm = deltaYInPixels / pixelsPerMm; % Or * mmPerPixel, depending on what your calibration factor has in the denominator.

You can get pixelsPerMm or mmPerPixel by following the demo, which basically just divides the pixel distance by the real world distance. For example if the black square is 120 pixels and it's 40 mm, then

pixelsPerMm = 120/40;
mmPerPixel = 40/120;

[Faiz Yusoff]

% read the original image
I = imread('Thermo engine.jpg');
% call createMask function to get the mask and the filtered image
[BW,maskedRGBImage] = createMask(I);
% plot the original image, mask and filtered image all in one figure
subplot(1,3,1);imshow(I);title('Original Image');
subplot(1,3,2);imshow(BW);title('Mask');
subplot(1,3,3);imshow(maskedRGBImage);title('Filtered Image');

function [BW,maskedRGBImage] = createMask(RGB) 
% Convert RGB image to HSV image
I = rgb2hsv(RGB);
% Define thresholds for 'Hue'. Modify these values to filter out different range of colors.
channel1Min = 0.965;
channel1Max = 0.188;
% Define thresholds for 'Saturation'
channel2Min = 0.000;
channel2Max = 1.000;
% Define thresholds for 'Value'
channel3Min = 0.000;
channel3Max = 1.000;
% Create mask based on chosen histogram thresholds
BW = ( (I(:,:,1) >= channel1Min) | (I(:,:,1) <= channel1Max) ) & ...
    (I(:,:,2) >= channel2Min ) & (I(:,:,2) <= channel2Max) & ...
    (I(:,:,3) >= channel3Min ) & (I(:,:,3) <= channel3Max);
% Initialize output masked image based on input image.
maskedRGBImage = RGB;
% Set background pixels where BW is false to zero.
maskedRGBImage(repmat(~BW,[1 1 3])) = 0;
end

This is my current coding and result. I took the code somewhere else.

Now i need to get the centroid at the filtered image right? and get the x, y axis

sorry but im new at this.

[Image Analyst]

Try using the code I gave you with slight modifications. If you want just the 2 largest blobs....

redMask = bwareafilt(redMask, 2); % Take largest 2 blobs only.
props = regionprops(redMask, 'Centroid');
centroidsXY = vertcat(props.Centroid) % In pixels.

In the case of a moving camera, your origin will be changing with respect to the jet. You might want to choose an origin on the jet somehow. Are you using a jet with a moving camera or video? Or a checkerboard that is still and the camera is still (not moving)?

[Faiz Yusoff]

Its not a moving camera, just a thermal picture. It would be more simple such as just a plate like this. for example now i want to locate the coordinate of blue dot in the middle, just like that.

[Faiz Yusoff]

Is it possible if you help me with full coding? I kinda lost here :(

[Image Analyst]

Do you have the temperature image? If so, it's easy, just threshold for cold stuff and call regionprops.

coldRegions = temperatureImage < 140;
% Fill holes
coldRegions = imfill(coldRegions, 'holes');
% Get rid of blobs touching the border
coldRegions = imclearborder(coldRegions);
% Make measurements of area and location.
props = regionprops(coldRegions, 'Area', 'Centroid')

See my Image Segmentation Tutorial.

If you don't have the temperature image and all you have is this pseudocolored image, you'll have to convert it to a temperature image using the colorbar. I'm attaching a demo where I do that. You can adapt that, like tell it where your color bar is in your image.

Or you can simply use ginput(1) to point and click on the spot you want to locate.

[Faiz Yusoff]

My supervisor will give me the thermal image after i get the coding.

Ive tried your coding, but it seems have a bit of error. I mean maybe the way i put the folder, how to solve this?

Transforming image "C:\Users\faizy\Desktop\thermal_image.png" to a thermal image.
Error using imread>get_full_filename (line 567)
File "C:\Users\faizy\Desktop\thermal_image.png" does not exist.
Error in imread (line 374)
    fullname = get_full_filename(filename);
Error in thermal_image_color_to_grayscale (line 20)
originalRGBImage = imread(fullFileName);

[Image Analyst]

Your image does not exist on your desktop, which is where you put the m-file. Make sure the image is really where you specified the folder to be.

[Faiz Yusoff]

Okay done with it, but i think thats not really information that i want. would you explain more about getting the centroid and the x y axis as per mentioned in your first reply. I really appreciate your help.

[Image Analyst]

You vaguely said "the real coordianate of the defect such as in x and y axis". Well, if you threshold your image, you will have one or more regions ("blobs" as we say in image processing) of pixels that are above or below your threshold. Now, each blob may consist of one or more pixels. Now when you said that, I didn't know if you wanted ALL (x,y) locations - the (x,y) coordinate of each and every pixel in the blob? OR, if you just wanted one (x,y) coordinate - that of the centroid of the blob. Or both? It's possible to get both

props = regionprops(mask, 'PixelList', 'Centroid');

but I'm not sure what would really be helpful in your next step of your analysis. What do you want to do after that, if you could get what you think you want?

And maybe you also want the mean temperature in the thresholded regions:

props = regionprops(mask, temperatureImage, 'PixelList', 'Centroid', 'MeanIntensity');

[Faiz Yusoff]

I just need the coordinate just like the first picture i sent to you. I dont think i need the intensity and all

Where should i put this props code?

props = regionprops(mask, 'PixelList', 'Centroid');

So i segmentize using the picture you sent, but this is not properly red filter yet.

i want to get the centroid of filtered image and the coordinate real world

[Faiz Yusoff]

Maybe you can spare me youre email, i really need help here.

[Image Analyst]

Call regionprops() after you get the mask (binary image).

I still don't know if you're using a chessboard, a thermal image of a jet, or some other thermal image.

If you need private consulting, you can go here. They'd be happy to help you and spend all the time you need.

If you want to create a red mask, use the Color Thresholder app on the Apps tab of the tool ribbon. Use HSV color space, then export the function with the Export button.

[Faiz Yusoff]

ok now already get the thresholding, omg so lame. I just know about it. but now I couldnt manage to find the centroid, it says its not in any function.

[Faiz Yusoff]

My image gonna be just like the one you sent, the plane just for example

[Image Analyst]

OK, it looks like you're really struggling a lot, so I guess I'll have to do a full blown demo for you. Just tell me do you want the centroid of every red region in the jet image, or just the largest region?

[Faiz Yusoff]

ok sounds great. Lets use the thermal image you sent me. So if you threshold that image it would be like as i shown you before. So it would be 2 red region. I want to get the centroid for every red region, and also the coordinate in real time if you could help me. thanks alot

[Image Analyst]

Try this:

% Demo to find centroids of red blobs.  By Image Analyst, Oct 19, 2020.
clc;    % Clear the command window.
close all;  % Close all figures (except those of imtool.)
clearvars;
workspace;  % Make sure the workspace panel is showing.
format long g;
format compact;
fontSize = 16;
fprintf('Beginning to run %s.m ...\n', mfilename);

%-----------------------------------------------------------------------------------------------------------------------------------
% Read in image.
folder = [];
baseFileName = 'thermal_image.png';
fullFileName = fullfile(folder, baseFileName);
% Check if file exists.
if ~exist(fullFileName, 'file')
	% The file doesn't exist -- didn't find it there in that folder.
	% Check the entire search path (other folders) for the file by stripping off the folder.
	fullFileNameOnSearchPath = baseFileName; % No path this time.
	if ~exist(fullFileNameOnSearchPath, 'file')
		% Still didn't find it.  Alert user.
		errorMessage = sprintf('Error: %s does not exist in the search path folders.', fullFileName);
		uiwait(warndlg(errorMessage));
		return;
	end
end
% It's not an RGB image!  It's an indexed image, so read in the indexed image...
rgbImage = imread(fullFileName);
[rows, columns, numberOfColorChannels] = size(rgbImage)
% Display the test image.
subplot(2, 2, 1);
imshow(rgbImage, []);
axis('on', 'image');
caption = sprintf('Image : "%s"', baseFileName);
title(caption, 'FontSize', fontSize, 'Interpreter', 'None');
drawnow;
hp = impixelinfo(); % Set up status line to see values when you mouse over the image.

% Set up figure properties:
% Enlarge figure to full screen.
hFig1 = gcf;
hFig1.Units = 'Normalized';
hFig1.WindowState = 'maximized';
% Get rid of tool bar and pulldown menus that are along top of figure.
% set(gcf, 'Toolbar', 'none', 'Menu', 'none');
% Give a name to the title bar.
hFig1.Name = 'Demo by Image Analyst';

[mask, maskedRGBImage] = createMask(rgbImage);
% Get rid of blobs touching the border.
mask = imclearborder(mask);
% Take just the two largest regions:
mask = bwareafilt(mask, 2);
% Display the initial mask image.
subplot(2, 2, 2);
imshow(mask, []);
hp = impixelinfo(); % Set up status line to see values when you mouse over the image.
axis('on', 'image');
title('Mask', 'FontSize', fontSize, 'Interpreter', 'None');
drawnow;

% Mask the image using bsxfun() function to multiply the mask by each channel individually.  Works for gray scale as well as RGB Color images.
maskedRgbImage = bsxfun(@times, rgbImage, cast(mask, 'like', rgbImage));

% Display the final masked image.
subplot(2, 2, 3);
imshow(maskedRgbImage, []);
axis('on', 'image');
title('Masked Image', 'FontSize', fontSize, 'Interpreter', 'None');
drawnow;
hp = impixelinfo(); % Set up status line to see values when you mouse over the image.

% Display the final masked image of the background by inverting the mask.
% Mask the image using bsxfun() function to multiply the mask by each channel individually.  Works for gray scale as well as RGB Color images.
backgroundImage = bsxfun(@times, rgbImage, cast(~mask, 'like', rgbImage));
subplot(2, 2, 4);
imshow(backgroundImage, []);
axis('on', 'image');
title('Background Image', 'FontSize', fontSize, 'Interpreter', 'None');
drawnow;
hp = impixelinfo(); % Set up status line to see values when you mouse over the image.

%-------------------------------------------------------------------------------------------------------------
% Make measurements
props = regionprops(mask, 'Area', 'Centroid')
allAreas = [props.Area];
xyCentroids = vertcat(props.Centroid);
subplot(2, 2, 2);
hold on;
for k = 1 : length(props)
	x = xyCentroids(k, 1);
	y = xyCentroids(k, 2);
	txt = sprintf('  (x, y) = (%.1f, %.1f).  Area = %d', ...
	x, y, allAreas(k));
	text(x, y, txt, 'Color', 'r', 'FontWeight', 'bold');
	plot(x, y, 'r+', 'MarkerSize', 25, 'LineWidth', 2);
end
fprintf('Done running %s.m ...\n', mfilename);
msgbox('Done!');


function [BW,maskedRGBImage] = createMask(RGB)
%createMask  Threshold RGB image using auto-generated code from colorThresholder app.
%  [BW,MASKEDRGBIMAGE] = createMask(RGB) thresholds image RGB using
%  auto-generated code from the colorThresholder app. The colorspace and
%  range for each channel of the colorspace were set within the app. The
%  segmentation mask is returned in BW, and a composite of the mask and
%  original RGB images is returned in maskedRGBImage.

% Auto-generated by colorThresholder app on 19-Oct-2020
%------------------------------------------------------


% Convert RGB image to chosen color space
I = rgb2hsv(RGB);

% Define thresholds for channel 1 based on histogram settings
channel1Min = 0.901;
channel1Max = 0.110;

% Define thresholds for channel 2 based on histogram settings
channel2Min = 0.000;
channel2Max = 1.000;

% Define thresholds for channel 3 based on histogram settings
channel3Min = 0.000;
channel3Max = 1.000;

% Create mask based on chosen histogram thresholds
sliderBW = ( (I(:,:,1) >= channel1Min) | (I(:,:,1) <= channel1Max) ) & ...
	(I(:,:,2) >= channel2Min ) & (I(:,:,2) <= channel2Max) & ...
	(I(:,:,3) >= channel3Min ) & (I(:,:,3) <= channel3Max);
BW = sliderBW;

% Initialize output masked image based on input image.
maskedRGBImage = RGB;

% Set background pixels where BW is false to zero.
maskedRGBImage(repmat(~BW,[1 1 3])) = 0;

end

[Faiz Yusoff]

I just tried your code and its absolutely amazing, thank god! Im peechless!

I just want to know like is the coordinate in pixel or already in mm?

And if i change the picture or use another thermal image, do i need to modify anything?

[Image Analyst]

You can modify anything you want. It just depends on what colors you want to find. You can change the hue angles to adjust the colors selected. You can change the image if you want.

[Faiz Yusoff]

Thank you so much, you made my whole year better! Let me know if you come by Malaysia xD

The (x,y) coordinates is in pixel or mm sir?

[Faiz Yusoff]

sorry, but can you explain breifly what moethod do you use, trying to learn from it. I get some of it but some i just lost it. I saw the measurement you made for the coordinate but im comfuse that i think you display it in pixel right, cause i cant see the scale to real time coordinate here. Can you help me?

[Image Analyst]

Coordinates are first made in pixels. You then can convert into real world units such as meters by multiplying by the spatial calibration factor which has units of unitsPerPixel. For example if you're looking at the moon and get a distance that's 5 pixels, and you know that the spatial calibration factor is 8 km per pixel, then you'd multiply by 8 to get 5 * 8 = 40 km.

[Faiz Yusoff]

I see. So you haven't put the coordinate/ the calcution in the code right. It just display the pixel coordinate right?

[Faiz Yusoff]

I have received the image from my supervisor as attached. So i would be using those image to find the real coordinates x and y. How would i know the scale or size?

[Image Analyst]

Not sure what this means: "So you haven't put the coordinate/ the calcution in the code right. It just display the pixel coordinate right?"

What my demo does is have you draw a distance along the image and ask you for the real world distance and units, like it's 10 meters or whatever. Then it computes a spatial calibration factor. Yes, that's in the code. Then it asks you to draw a line or region and it tells you the distance or area in pixels and in real world units. And the code for that is also in there.

To run it on your image, you would have to know the size of your field of view of your entire image, or else the size of some object in the image in real world units.

[Faiz Yusoff]

What do you mean by "it ask me for the real world distance and units, like it's 10 meters or whatever"?

Does it mean automactically or i need to insert something? cause when i run, the result comes out immediately.

And i want to display in centimeter/ meter. Still cant get it where to change. Plus i realized the scale is different with some images i tried. So what actually the units of x and y coordinates shows in the result? because i need it in centimeter/ meter.

[Image Analyst]

When you run the demo you will see this:

Are you saying that you don't see that dialog box with the question asking you for the real world units??? If not, please attach a screenshot.

[Faiz Yusoff]

Yes i dont see hat dialog box. When i run it automatically give the result as i attached.

and the dialog box says "done"

[Image Analyst]

That's a totally different demo (it's my thermal demo). But the concept is the same. You measure a known pixel distance and say that it's a known real world distance. Get the spatial calibration factor from the ratio

mmPerPixel = distanceInMm / distanceInPixels;

Then just multiply distances or coordinates by mmPerPixel, and areas by mmPerPixel^2.

[Faiz Yusoff]

I dont get it how to know the distancein Mm to divide with the pixel.

[Image Analyst]

Again, if you run my demo, it will ask to to draw a rubber band line across some object in your image that you do absolutely 100% KNOW the length of. Attach your picture and tell me the object that you know the width of. Like is it a picture of a ruler or something? Or a standard sheet of printer paper, or what????? You must have something in there that you know the size of, right?? Well, show me that.

[Faiz Yusoff]

Still, if i run the demo, it come out the result as shown.

It did not ask me to draw rubber band.

Ok now i use ruler image 35 cm for example.

The circle coordinate is in pixel isnt?

[Faiz Yusoff]

Hi sir, i need help with the latest project, i need to get the size and the distance between each of the defects/ circles in the thermal image and it must be the same as the original real measurement

this the image i need to get done

this is the real measurement of the thing with me

[Image Analyst]

Use thresholding followed by regionprops followed by pdist2()

mask = grayImage > someValue;
mask = bwareafilt(mask, 13); % Take 13 largest blobs.
imshow(mask);
hold on;
props = regionprops(mask, 'Centroid');
xy = vertcat(props.Centroid);
for k = 1 : size(xy, 1)
    caption = sprintf('  %d', k);
    plot(xy(k, 1), xy(k, 2), 'r+', 'MarkerSize', 20);
    text(xy(k, 1), xy(k, 2), caption, 'Color', 'r', 'FontWeight', 'bold', 'FontSize', 20);
end
distances = pdist2(xy, xy)

[Faiz Yusoff]

Do i need to continue from previous code or new one?

[Image Analyst]

You need to use the spatial calibration program to determine the number of cm per pixel, or whatever units you want to use. Then start a new program where you get a temperature image somehow, like take the green channel or do PCA on the image to get a gray scale image that is correlated to temperature. Then run the code to threshold it and get the distances. Then multiply the distances by your calibration factor to convert the distances from pixels to cm:

distancesInPixels = pdist2(xy, xy);
distancesInCm = distancesInPixels * cmPerPixel;

[Faiz Yusoff]

i got some errors, my code just messed up. Could you help me?

[Image Analyst]

Sure. Start a new question since it doesn't involve the same images as the original at the top of the page. Then, after reading this link,

TUTORIAL: How to ask a question (on Answers) and get a fast answer

attach your image and code that messed up.

[Faiz Yusoff]

i already start a new question with the code, wish you could help, thank you.