Grain boundary analysis of images

I have three TEM images taken at different tilt angles, each showing grain boundaries. However, some grain boundaries are missing in each image. I need a MATLAB code to overlay or merge these three images into a single image where the missing boundaries are filled in, creating a more complete representation of the grain structure.

Attached are the images

[Image Analyst]

The grain/cell sizes and boundary locations are very different in all three images. I mena, just look at the lower left of the three images. If you merged them, like say by taking the min value of all 3 images, then you might have three grain boundaries instead of just one solid one because their boundaries don't overlap well.

Answer 1

[Image Analyst]

You might try taking the darkest pixel in each image to combine them. Try this code:

% Demo by Image Analyst
% Initialization steps:
clc;    % Clear the command window.
close all;  % Close all figures (except those of imtool.)
clear;  % Erase all existing variables. Or clearvars if you want.
workspace;  % Make sure the workspace panel is showing.
format long g;
format compact;
fontSize = 12;

%--------------------------------------------------------------------------------------------------------
%    READ IN FIRST IMAGE
folder = pwd;
baseFileName = "predict_al-324-200C_15min_aligned_fov1_-1_testing_saving_75.png";
fullFileName = fullfile(folder, baseFileName);
% Check if file exists.
if ~isfile(fullFileName)
    % The file doesn't exist -- didn't find it there in that folder.
    % Check the entire search path (other folders) for the file by stripping off the folder.
    fullFileNameOnSearchPath = baseFileName; % No path this time.
    if ~exist(fullFileNameOnSearchPath, 'file')
        % Still didn't find it.  Alert user.
        errorMessage = sprintf('Error: %s does not exist in the search path folders.', fullFileName);
        uiwait(warndlg(errorMessage));
        return;
    end
end
% Read in image file.
grayImage1 = imread(fullFileName);
% Get size
[rows, columns, numberOfColorChannels] = size(grayImage1);
% Get gray scale version of it.
if numberOfColorChannels == 3
    grayImage1 = grayImage1(:, :, 1); % Take red channel.
end
% Display the image.
subplot(2, 2, 1);
imshow(grayImage1);
axis('on', 'image');
impixelinfo;
caption = sprintf('Original image : %s', baseFileName);
title(caption, 'FontSize', fontSize, 'Interpreter', 'None');
% Maximize window.
g = gcf;
g.WindowState = 'maximized';
g.Name = 'Demo by Image Analyst';
g.NumberTitle = 'off';
drawnow;

%--------------------------------------------------------------------------------------------------------
%    READ IN SECOND IMAGE
folder = pwd;
baseFileName = "predict_al-324-200C_15min_aligned_fov1_0_testing_saving_75.png";
fullFileName = fullfile(folder, baseFileName);
% Check if file exists.
if ~isfile(fullFileName)
    % The file doesn't exist -- didn't find it there in that folder.
    % Check the entire search path (other folders) for the file by stripping off the folder.
    fullFileNameOnSearchPath = baseFileName; % No path this time.
    if ~exist(fullFileNameOnSearchPath, 'file')
        % Still didn't find it.  Alert user.
        errorMessage = sprintf('Error: %s does not exist in the search path folders.', fullFileName);
        uiwait(warndlg(errorMessage));
        return;
    end
end
% Read in image file.
grayImage2 = imread(fullFileName);
% Get size
[rows, columns, numberOfColorChannels] = size(grayImage2);
% Get gray scale version of it.
if numberOfColorChannels == 3
    grayImage2 = grayImage2(:, :, 1); % Take red channel.
end
% Display the image.
subplot(2, 2, 2);
imshow(grayImage2);
axis('on', 'image');
impixelinfo;
caption = sprintf('Original image : %s', baseFileName);
title(caption, 'FontSize', fontSize, 'Interpreter', 'None');

%--------------------------------------------------------------------------------------------------------
%    READ IN THIRD IMAGE
folder = pwd;
baseFileName = "predict_al-324-200C_15min_aligned_fov1_+1_testing_saving_75.png";
fullFileName = fullfile(folder, baseFileName);
% Check if file exists.
if ~isfile(fullFileName)
    % The file doesn't exist -- didn't find it there in that folder.
    % Check the entire search path (other folders) for the file by stripping off the folder.
    fullFileNameOnSearchPath = baseFileName; % No path this time.
    if ~exist(fullFileNameOnSearchPath, 'file')
        % Still didn't find it.  Alert user.
        errorMessage = sprintf('Error: %s does not exist in the search path folders.', fullFileName);
        uiwait(warndlg(errorMessage));
        return;
    end
end
% Read in image file.
grayImage3 = imread(fullFileName);
% Get size
[rows, columns, numberOfColorChannels] = size(grayImage3);
% Get gray scale version of it.
if numberOfColorChannels == 3
    grayImage3 = grayImage3(:, :, 1); % Take red channel.
end
% Display the image.
subplot(2, 2, 3);
imshow(grayImage3);
axis('on', 'image');
impixelinfo;
caption = sprintf('Original image : %s', baseFileName);
title(caption, 'FontSize', fontSize, 'Interpreter', 'None');

%--------------------------------------------------------------------------------------------------------
% COMBINE THE IMAGES BY TAKING THE DARKEST PIXEL IN EACH
image3D = cat(3, grayImage1, grayImage2, grayImage3);
finalImage = min(image3D, [], 3);
% Display the image.
% subplot(2, 2, 4);
figure
imshow(finalImage, []);
axis('on', 'image');
impixelinfo;
caption = sprintf('Final image');
title(caption, 'FontSize', fontSize, 'Interpreter', 'None');

[Prince]

With regard to the images, can you please produce a single image in which there are black pixels at locations where at least one black pixel exists in one of the images?

[Image Analyst]

@Prince yes, you can threshold it and set the pixel values below the threshold to zero while maintaining the values above the threshold like this

mask = finalImage < someThresholdValue;  % This is now a binary image - just black and white.
% Set pixels below the threshold equal to zero in finalImage
% while keeping values above the threshold the same.
finalImage(mask) = 0; 

Since final image was built using the darkest pixel from each image, it will do what you asked.

Answer 2

[Kanishk]

Hi @Prince

You can use "imadjust" and "min" function in MATLAB to improve the contrast and merge the images. Here is a simple script using the functions.

% Read the images
img1 = imread('1.png');
img2 = imread('0.png');
img3 = imread('-1.png');

% Convert to grayscale if necessary
if size(img1, 3) == 3
    img1 = rgb2gray(img1);
end
if size(img2, 3) == 3
    img2 = rgb2gray(img2);
end
if size(img3, 3) == 3
    img3 = rgb2gray(img3);
end
imshow(img1)
% Enhance contrast (optional)
img1 = imadjust(img1);
img2 = imadjust(img2);
img3 = imadjust(img3);
imshow(img1)
% Overlay images by taking the maximum pixel intensity
mergedImage = min(img1, min(img2, img3));

% Display the result
imshow(mergedImage);
title('Merged TEM Image with Complete Grain Boundaries');

The code produces the below image.

You can further improve the merged image by aligning the individual images. To connect the gray line in the image, select a threshold with suits your needs and convert everything below that threshold to black(0).

mergedImage((mergedImage <= 200)) = 0;
figure, imshow(mergedImage)

You can also look into edge linking. Here is a link to external documentation for same.

https://www.peterkovesi.com/matlabfns/#edgelink

You can learn more about "imadjust" by using the following command in MATLAB.

web(fullfile(docroot, '/images/ref/imadjust.html'))

Edit: Corrected the typo of using "max" function while calculating the "mergedImage". Also, updated the resulting images.

[Image Analyst]

Can you explain this:

% Overlay images by taking the maximum pixel intensity
mergedImage = min(img1, max(img2, img3));

Why would that be better than just taking the darkest/minimum pixel value of the 3 images like I did in my answer? How does taking the max/brightest pixels in images 2 and 3 help to enhance or delineate the dark lines in the final image?

[DGM]

In other words, it's not taking the maximum intensity of the three images as the comment states. It's taking the minimum of img1 and the maximum of img2 and img3.

Should we be using max() at all? Using max() can only remove edge information, so I don't see an apparent reason to assume img1 defines the presence of dark edges while img2 and img3 define the absence of dark edges. Is there a reason?

[Kanishk]

Thank you for pointing out the discrepancy in my previous response. I was earlier experimenting with "max" function for merging the image and realised "min" would be better here. I think I somehow forgot to change both functions and only corrected one of them. I have updated the answer addressing the issue and also updated the resulting images.

Thanks again for the feedback and patience.

I think the core strength of MATLAB community lies here in helping each other out like this.

[Prince]

With regard to the images, can you please produce a single image in which there are black pixels at locations where at least one black pixel exists in one of the images?

[Kanishk]

The 'min' function here takes the minimum of pixels from the three images which will be the blackest pixel. To improve the blacks of the image you can use a threshold to set pixels below from certain threshold to zero.

Answer 3

[Leepakshi]

Hey Prince,

You can also perform this task using the “imfuse” command in MATLAB. Note that, you must use “method” as “blend” to overlay all the three images, resulting into a more refined image.

This is how final image will look like:

This is how the code will look like:

>> A= imread("A.png"); 
>> B=imread('B.png'); 
>> D = imfuse(A,B,"blend"); 
>> C = imread('C.png'); 
>> E = imfuse(D,C,"blend"); 
>> imshow(E); 

Here is a link for documentation of imfuse:

https://in.mathworks.com/help/images/ref/imfuse.html

Hope that added value to your task!

[DGM]

Imfuse() is used for combination of two images. Sequential use of imfuse() is not appropriate.

For 'blend' mode, the blend weight is always 50%, so sequential use with three images means layer weights are 25%-25%-50% instead of 33%-33%-33%

https://www.mathworks.com/matlabcentral/answers/433228-how-to-overlap-more-than-two-images-into-a-one-image-with-matlab

For 'falsecolor' mode, the results will be differently wrong:

https://www.mathworks.com/matlabcentral/answers/303851-how-to-superimpose-many-images-using-loop

For 'checkerboard' mode, sequential use simply replaces one of the prior images:

https://www.mathworks.com/matlabcentral/answers/1439139-how-do-i-fuse-together-4-images-of-the-same-dimensions-together-in-a-checkerboard-like-fashion