Optical character recognition for seven-segment display digit.[MATLAB]

Hi. recently i am doing on a project of Optical character recognition for seven-segment display digit. I am asked to crop out [attach photo] the A,B,C,D,E,F,G accordingly using AUTO -CROPPING.

http://i1149.photobucket.com/albums/o582/Pillowgerr/24bd59a8.jpg

and i had a hard time figuring the codes & ways succesfully do it. I hope any of you can assist me with coding and explanations. thank you so much! :)

[Manpreet Singh]

sir how to recognize seven segment characters in this image by ocr plz help

[Image Analyst]

I gave code for this below. Adapt it as needed.

Answer 1

[Image Analyst]

You don't need to crop, unless you want to, or unless your segments wander all over from image to image. You segments should be in the same place in each image unless for some reason your camera is not mounted properly, or your display is bouncing around.

To check whether segments are on or off is pretty easy. Just look at the color channels to extract the segments, and look in boxes located at certain locations (the middle of where each segment is expected to be) for whether the pixels there are white or black. See my demo to get you started:

% Demo to check a 7 segment display.
% By ImageAnalyst
clc;    % Clear the command window.
close all;  % Close all figures (except those of imtool.)
imtool close all;  % Close all imtool figures.
clear;  % Erase all existing variables.
workspace;  % Make sure the workspace panel is showing.
fontSize = 14;

% Read in a standard MATLAB gray scale demo image.
folder = 'C:\Users\Esther\Documents\Temporary';
baseFileName = '7segmentdisplay.jpg';
% Get the full filename, with path prepended.
fullFileName = fullfile(folder, baseFileName);
if ~exist(fullFileName, 'file')
  % Didn't find it there.  Check the search path for it.
  fullFileName = baseFileName; % No path this time.
  if ~exist(fullFileName, 'file')
    % Still didn't find it.  Alert user.
    errorMessage = sprintf('Error: %s does not exist.', fullFileName);
    uiwait(warndlg(errorMessage));
    return;
  end
end
rgbImage = imread(fullFileName);
% Get the dimensions of the image.  numberOfColorBands should be = 3.
[rows columns numberOfColorBands] = size(rgbImage);
% Display the original color image.
subplot(2, 2, 1);
imshow(rgbImage, []);
title('Original Color Image', 'FontSize', fontSize);
% Enlarge figure to full screen.
set(gcf, 'units','normalized','outerposition',[0 0 1 1]);

% Extract the individual red, green, and blue color channels.
redChannel = rgbImage(:, :, 1);
greenChannel = rgbImage(:, :, 2);
blueChannel = rgbImage(:, :, 3);

% Display the original color image.
subplot(2, 2, 2);
imshow(redChannel, []);
title('Red Channel', 'FontSize', fontSize);
subplot(2, 2, 3);
imshow(greenChannel, []);
title('Green Channel', 'FontSize', fontSize);
subplot(2, 2, 4);
imshow(blueChannel, []);
title('Blue Channel', 'FontSize', fontSize);

% Extract just the segments and not the numbers.
figure;
binaryImage = ~(redChannel < 200) & (blueChannel < 200);
% subplot(2, 2, 1);
imshow(binaryImage, []);
axis on;
drawnow;
hold on;
% Enlarge figure to full screen.
set(gcf, 'units','normalized','outerposition',[0 0 1 1]);

% Now define locations to check.
row1 = 35;
row2 = 110;
row3 = 175;
row4 = 260;
row5 = 330;
col1 = 70;
col2 = 175;
col3 = 275;

% Plot boxes around there to check.
title('Checking Image Inside Red Boxes', 'FontSize', fontSize);
boxWidth = 30;
% First check top segment.
row = row1;
col = col2;
boxX = [col col+boxWidth col+boxWidth col col];
boxY = [row row row + boxWidth, row + boxWidth, row];
plot(boxX, boxY, 'ro-');
imageInsideBox = binaryImage(row:row + boxWidth, col:col+boxWidth)
% Determine if there are any pixels set inside that box.
segmentState(1) = any(imageInsideBox(:))

% Now check upper left segment.
row = row2;
col = col1;
boxX = [col col+boxWidth col+boxWidth col col];
boxY = [row row row + boxWidth, row + boxWidth, row];
plot(boxX, boxY, 'ro-');
imageInsideBox = binaryImage(row:row + boxWidth, col:col+boxWidth)
% Determine if there are any pixels set inside that box.
segmentState(2) = any(imageInsideBox(:))

% Now check lower right segment.
row = row4;
col = col3;
boxX = [col col+boxWidth col+boxWidth col col];
boxY = [row row row + boxWidth, row + boxWidth, row];
plot(boxX, boxY, 'ro-');
imageInsideBox = binaryImage(row:row + boxWidth, col:col+boxWidth)
% Determine if there are any pixels set inside that box.
segmentState(6) = any(imageInsideBox(:))

[Esther]

thank you for your answer! i'll try your demo tomorrow when i get to school. but my professor wanted me to get it cropped. so what should i do if i really need it to be cropped? [ auto cropping ]

[Image Analyst]

Use the any() function on the binary image to find which rows and columns have white pixels in them. Then use find() to get the actual row and column indexes. Then use imcrop(). It should be very easy for you to figure out once you know to use those three functions.

[Esther]

is there any tutorial that i can do/try out to figure out how to use these functions? because i do not have any fundamentals on using matlab program and all my professor does was giving me this project to do on by myself. but i need more understanding on matlab functions that i will JUST need to use on this project because i only left 6 weeks to finish the whole project.

[Image Analyst]

I have a few demos in my File Exchange: http://www.mathworks.com/matlabcentral/fileexchange/?term=authorid%3A31862. BlobsDemo does cropping.

[Esther]

was that manual cropping?

[Esther]

is it possible to just convert orignal images to grayscale to binaryscale, and from binaryscale we crop automatically each A,B,C,D,E,F,G out?

[Image Analyst]

What does "manual" mean to you? It was cropping via the imcrop() function. The coordinates were determined from the bounding box of the object.

Why would you want to crop out each segment individually? I see absolutely no reason or need to do that. It's totally not necessary to see if the segment is red. Of course if it's not red, and it's the same color as the background, you can't find it automatically and would have to use coordinates known in advance. If you still want to crop, for some unknown reason, then just use imcrop() - again I refer you to my demo.

[Esther]

my professor suggest cropping each out because previously there is a student who tried to crop the whole each digit out but doesn't work. 50% failed to read the digits. example: 38.7°C [reading from thermemoter took from camera] she did more testing but only a few readings was succesfully detected.

[Image Analyst]

You need to examine the pixels in a region near each segment, but you do not need to crop the regions into a new, separate smaller subimages. Like I said it's totally unnecessary and you can tell him I said so. Again, you can do it if you want by taking known locations (row1, row2, column1, column2) of where the segment is supposed to be if you want to crop. But you're still going to have to examine the color (intensity) of each segment like I said, regardless of whether or not it was cropped. Is there always a decimal point? If so, you can use that to register your images and crop known locations with respect to that decimal point.

Do you have actual images snapped by a camera, or just graphics hand-drawn in a paint program, like what you've already posted?

[Esther]

i think my professor mind thinking was that, by cropping each out it may detect the image digit more efficiently and the test will be more accurate. Nobody have tried it before and therefore, giving it a try now. Or is there any alternate ways to do that can give accurate results?

[Esther]

Right now, he asked me to take images of a weighing machines with digits 0 to 9. he wants me to convert it to a grayscale to a binaryscale and filter, remove noise. but there are several ways to filter? what are the ways?

[Esther]

could you explain how does H = fspecial('gaussian', 50, 10); work? thank you! 50, 10 meant?

[ASWATHY SUDHAKARAN]

when i run the program,there was an error ,saying

Index exceeds matrix dimensions.

imageInsideBox = binaryImage(row:row + boxWidth, col:col+boxWidth)

how can I correct it ????

[Walter Roberson]

imageInsideBox = binaryImage(row:row +boxWidth-1, col:col+boxWidth-1);

[Image Analyst]

You need to make sure the second index does not leave the image, so clip it to the number of rows or columns with min

row2 = min([size(binaryImage, 1), row+boxWidth]);
col2 = min([size(binaryImage, 2), col+boxWidth]);
imageInsideBox = binaryImage(row:row2, col:col2);

[ASWATHY SUDHAKARAN]

Why we are finding red,green,and blue channels???? also why don't we consider green channel while computing binary image????

[Walter Roberson]

ASWATHY SUDHAKARAN, notice in what Image Analyst wrote,

"To check whether segments are on or off is pretty easy. Just look at the color channels to extract the segments, "

The determination of where the segments are can be done by examining the different color channels in different ways. It happened that it was not necessary to consult the green channel to tell the difference between the various cases for the image that was being tested.

Which channels to look at and what cutoff to use would depend upon what the colors of the instrument segments. For example there is a famous report of a ship in which "the console is black, the switches are black, the labels are little black letters printed on a black background, and when you press anything, a black light lights up in black to tell you you've done it."

[ASWATHY SUDHAKARAN]

Thank you for your valuable help...I would like to ask one more doubt..Can we recognize the values from the display by comparing the pixels with on/off states of the seven segments a,b,c,......g as in fig.below by avoiding the estimation of different colour channels???

[Image Analyst]

I'm not sure what you mean. We're not estimating different color channels. We're measuring them. And you cannot determine if a segment is present or lit if you don't measure/examine the pixel values in the color channels. Please clarify what you mean.

[Walter Roberson]

Perhaps segment on /off could be handled by grayscale brightness? We are hampered by not having a sample image with some of the segments dimmed for comparison.

[ASWATHY SUDHAKARAN]

Actually,this is my image.

after segmentation and binarisation i got

From the image i need to extract the correct value 12345.6. so,while comparing with the below image

for 1:only segments b and c willbe active and a,d,e,f,g will be inactive. Also for 2:segments a,b,g,e,d will be active and so on. Can i recognize the values by the comparison btwn the binarized img and the on/off states of a,b,c,d,e,f,g????

[Walter Roberson]

I suggest you dilate to join the small gaps and then skeletonize. That should make it easier to analyze the shape.

[Image Analyst]

You don't need to do that. You can just have 7 skinny rectangles and get the mean gray level in each. If it's less than 128, there is a black stick in it. If it's more than that, it's all white and the segment is not turned on. Then you just have a look up table of 128 elements that gives you the number based on what elements are on. For example, if a "1" is elements "f" and "e" on, and all others off, and if the bits are abcdefg, then the pattern for a 1 would be 0000110. Well, that's a 6, so in your look up table, you'd have a 1 at element 6. The other 8 digits would be located at 8 other numbers because they have different patterns. As another example, 5 would have segments afgcd on, so the index would be 1011011, which is 91, so if you get 91, that means the pattern is a 5.

lookUpTable = zeros(128, 1);
lookUpTable(6) = 1;
lookUpTable(91) = 5;
% And so on for the other 8 digits.

So just read the intensities in the segments, convert to a binary number then a decimal number for the index and get the number.

theNumber = lookUpTable(index);

[Brett Shoelson]

I would suggest that instead of trying to do this with image morphology or segment detection, you install a 7-segment font (freely available--Google), and train the font using the OCR Training App. Probably easier, almost certainly more accurate.

Cheers,

Brett

Answer 2

[Rubina Easmin]

How do I crop every single character automatically in matlab by using segmentation?

please suggest me, how can I crop every single alphabet from this image automatically?