How do you segment the image horizontally?

Hi guys... I have an image containing a handwritten sentence of four lines... i want to segment the image into 4 segmented parts horizontally, such that each segmented part contain each of the four line... can any one help???... any help is highly appreciated!!!

PS: the image to be segmented is binary image with black background and white handwritten characters on it

Answer 1

[Image Analyst]

You could try summing the image horizontally:

verticalProfile = sum(binaryImage, 2);

then look for bright and non-bright rows. Or you could use a long horizontal structuring element and do a morphological closing with imclose() and then look for the big blobs.

I'm sure there are published algorithms that are much better that you can find here: http://www.visionbib.com/bibliography/contentschar.html#OCR,%20Document%20Analysis%20and%20Character%20Recognition%20Systems

[peyush]

OK I got a vertical line with dark and bright pixels...now how to use this to segment the original image...please help and thanks for your reply

[Image Analyst]

Well the lines with some value are the handwriting lines, right? So you can do this

% Create logical vector of what lines have text and which do not
rowsWithText = verticalProfile > someThreshold;

Set someThreshold high enough so that you don't get any noise, like a line with just one pixel lit up in it.

[peyush]

thanks for quick reply...yes the lines are handwritten lines...I set the threshold value = 20...now I got the vertical line with binary pixel values(0 or 1) with noise removed...pls help me how to use this to segment the original image.... thank you so much for your reply

[Image Analyst]

I don't see that you attached your original image. Why are you making us work "blind"? If you make me somehow get or create my own binary image of text, well, then I probably just won't bother. On that other hand, if you made it easy for me to help you (by attaching your binary image) then I probably would.

[peyush]

sorry I forgot to upload...the binary image I'm using is very distorted...please help me....PS: it is written"Light rays too must follow geodesics in space-time"...thanks for your time

[Image Analyst]

Try using diff() to find the top lines and bottom lines:

topLines = find(diff(rowsWithText) == 1);
bottomLines = find(diff(rowsWithText) == -1);

Then loop over those extracting bands of the image

for k = 1 : length(topLines)
    topRow = topLines(k);
    bottomRow = bottomLines(k;
    thisLine = binaryImage(topRow:bottomRow, :);
    figure;
    imshow(thisLine, []);
end

If that doesn't work, attach your entire script.

[peyush]

thanks it worked...can I use the same method to segment each segmented line vertically to separate each character?...sorry for asking too many questions

[Image Analyst]

Yes you can, just get the horizontal profile by summing vertically along rows:

horizontalProfile = sum(thisLine, 1);

thisLine is the line of handwriting extracted in my prior code snippet.

[peyush]

thanks..i used this

for k = 1 : length(topLines)
   topRow = topLines(k);
   bottomRow = bottomLines(k);
   thisLine = binaryImage(topRow:bottomRow, :);
   figure;
   imshow(thisLine, []);
   horizontalProfile = sum(thisLine, 1);
   columnWithText = horizontalProfile > 0;
   leftCharacter = find(diff(columnWithText) == 1);
   rightCharacter = find(diff(columnWithText) == -1);
   for m = 1 : length(leftCharacter)
       leftColumn = leftCharacter(m);
       rightColumn = rightCharacter(m);
       thisCharacter = thisLine(leftColumn:rightColumn, :);
       figure;
       imshow(thisCharacter, []);
   end
end

but, it shows an error " Index exceeds matrix dimensions ". ....what does that mean, please help & thanks once again

[Image Analyst]

Please give the COMPLETE script, and the COMPLETE error message (all the red text). I'll look at it later today if I get that.

[peyush]

ok thanks...

 %%_summing the image horizontally_
 verticalProfile = sum(binaryImage, 2);

 %%_create logical vector of what lines have text and which do not based on
 %%threshold_
 rowsWithText = verticalProfile > 20;

 %%_finding top and bottom lins_
 topLines = find(diff(rowsWithText) == 1);
 bottomLines = find(diff(rowsWithText) == -1);

 %%_looping for extracting characters in the image_
 for k = 1 : length(topLines)  %%_loop for segmenting horizontal lines_
     topRow = topLines(k);
     bottomRow = bottomLines(k);
     thisLine = binaryImage(topRow:bottomRow, :);
     figure;
     imshow(thisLine, []);

     %%_summing the segmented image vertically_
     horizontalProfile = sum(thisLine, 1);
     columnWithText = horizontalProfile > 0;
     leftCharacter = find(diff(columnWithText) == 1);
     rightCharacter = find(diff(columnWithText) == -1);
     for m = 1 : length(leftCharacter)  %%_loop for segmenting each individual character_
         leftColumn = leftCharacter(m);
         rightColumn = rightCharacter(m);
         thisCharacter = thisLine(leftColumn:rightColumn, :);
         figure;
         imshow(thisCharacter, []);
     end
 end

and, " Index exceeds matrix dimensions " is the only error message it is showing in red

[Image Analyst]

No, it's not. There is more red text. I know there is. It never gives just that. There will be the line of code, line numbers, etc. Also, binary image is not defined, so you did not give the complete script. Your script cannot run as is. You should take chances I give you, because I don't sit at my computer in this forum all day.

[Image Analyst]

In

thisCharacter = thisLine(leftColumn:rightColumn, :);

columns is index #2, not the first index.

[Image Analyst]

This is what I was expecting you to give me. Anyway, try it. It appears that your original binary image has a lot of breaks in the characters.

clc;    % Clear the command window.
close all;  % Close all figures (except those of imtool.)
clear;  % Erase all existing variables. Or clearvars if you want.
workspace;  % Make sure the workspace panel is showing.
format long g;
format compact;
fontSize = 20;
tic;
fullFileName = fullfile(pwd, 'SampleImage.png');

grayImage = imread(fullFileName);
% Get the dimensions of the image.
% numberOfColorBands should be = 1.
[rows, columns, numberOfColorBands] = size(grayImage);
if numberOfColorBands > 1
  % It's not really gray scale like we expected - it's color.
  % Convert it to gray scale by taking only the green channel.
  grayImage = grayImage(:, :, 2); % Take green channel.
end
% Display the original gray scale image.
subplot(2, 2, 1);
imshow(grayImage, []);
title('Original Grayscale Image', 'FontSize', fontSize, 'Interpreter', 'None');
axis on;

% Set up figure properties:
% Enlarge figure to full screen.
set(gcf, 'Units', 'Normalized', 'OuterPosition', [0 0 1 1]);
% Get rid of tool bar and pulldown menus that are along top of figure.
set(gcf, 'Toolbar', 'none', 'Menu', 'none');
% Give a name to the title bar.
set(gcf, 'Name', 'Demo by ImageAnalyst', 'NumberTitle', 'Off')

binaryImage = grayImage > 128;
subplot(2, 2, 2);
imshow(binaryImage);
title('Binary Image', 'FontSize', fontSize, 'Interpreter', 'None');
axis on;

% _summing the image horizontally_
verticalProfile = sum(binaryImage, 2);

subplot(2, 2, 3);
plot(verticalProfile, 'b-');
grid on;

% _create logical vector of what lines have text and which do not based on
%threshold_
rowsWithText = verticalProfile > 20;

% _finding top and bottom lins_
topLines = find(diff(rowsWithText) == 1);
bottomLines = find(diff(rowsWithText) == -1);

% _looping for extracting characters in the image_
  for k = 1 : length(topLines)  % _loop for segmenting horizontal lines_
  topRow = topLines(k);
  bottomRow = bottomLines(k);
  thisLine = binaryImage(topRow:bottomRow, :);
  subplot(2, 2, 3);
  imshow(thisLine, []);
  axis on;
  
  % _summing the segmented image vertically_
  horizontalProfile = sum(thisLine, 1);
  plot(horizontalProfile, 'b-');
  grid on;
  columnWithText = horizontalProfile > 2;
  leftCharacter = find(diff(columnWithText) == 1);
  rightCharacter = find(diff(columnWithText) == -1);
  if isempty(leftCharacter) || isempty(rightCharacter)
    continue;
  end
  promptMessage = sprintf('Do you want to find letters within this line,\nor Cancel to abort processing?');
  titleBarCaption = 'Continue?';
  button = questdlg(promptMessage, titleBarCaption, 'Continue', 'Cancel', 'Continue');
  if strcmpi(button, 'Cancel')
    return;
  end
  imshow(thisLine, []);
  axis on;

  subplot(2, 2, 4);
  for m = 1 : length(leftCharacter)  % loop for segmenting each individual character
    leftColumn = leftCharacter(m);
    rightColumn = rightCharacter(m);
    thisCharacter = thisLine(:, leftColumn:rightColumn);
    imshow(thisCharacter, []);
    axis on;
    promptMessage = sprintf('Do you want to find the next letter,\nor Cancel to abort processing?');
    titleBarCaption = 'Continue?';
    button = questdlg(promptMessage, titleBarCaption, 'Continue', 'Cancel', 'Continue');
    if strcmpi(button, 'Cancel')
      return;
    end
  end
end

[peyush]

ok...thank you so much for all your help :-)

[Meghashree G]

How do i segment an image like this?i have to get 6 separate sentences segmented from a paragraph. Please help me! Thank you

<<

>>

[Image Analyst]

I only see 2 or 3 at most sentences. Maybe you mean lines instead of sentences? If so, did you try my code? You may need to change the threshold a bit, but the basic concept should work.

[Meghashree G]

Yes sir,lines not sentences..sorry for the that misleading statement.. I tried the script which u posted recently in the above comment,but its displaying nothing..Please help me Thank you

[Meghashree G]

This is the output i'm getting when i tried your code

[Image Analyst]

I worked on it a bit but then it got late. It's attached but you need to do a little work to finish it. It's getting the lines but having trouble splitting up the words within the line because they are so poorly formed.

[Meghashree G]

Thank you very much for the help sir.. What can we do next for splitting up the words?? Thank You

[Image Analyst]

I don't know. Use better penmanship perhaps? Otherwise, look at the link I gave to the papers of people who study this as their main career. I don't do OCR and I'm sure my algorithms just off the top of my head are not as successful as those from someone whose full time job OCR is. Please see their published papers if you want something better.

[Poornima Gokhale]

Hello Sir, I used the above code that is test3.m But i am not getting the segmentation of lines properly. Half of the line is getting segmented

After applying the code I am getting like this.

Please help me.

[Poornima Gokhale]

Sir, please help me.

[Meghashree G]

Hello sir,i have many similar forms like this.But segmentation of lines is not correctly achieved .How to modify the code??Please help me with this.

[Walter Roberson]

As Image Analyst already replied,

"Otherwise, look at the link I gave to the papers of people who study this as their main career. I don't do OCR and I'm sure my algorithms just off the top of my head are not as successful as those from someone whose full time job OCR is. Please see their published papers if you want something better."

OCR is not just a trick that you get the hang of, not just a matter of fixing off-by-one errors in indexing, not just a matter of knowing the right threshold to use. OCR takes a lot of work. There are a lot of different techniques with different strengths and weaknesses. You can find several hundred books and proceedings on Amazon.com on the topic. There are a number of different conferences on the topic. You can find over one million articles about optical character recognition on scholar.google.com . It is a hard topic to do well.

Image Analyst does not have time to read and understand all of those on your behalf. And I know that I have no interest in reading and understanding those on your behalf. Very few of the regular volunteers have expressed interest in OCR. When we need OCR, we use someone else's program instead of trying to design our own. For example we might call http://www.mathworks.com/help/vision/ref/ocr.html

Your situation has exceeded our available resources, so is time for you to start doing your research.

Answer 2

[Meghashree G]

Hello sir, i used your code for line segmentation ,but for the below form its not working!! what changes should i make in order to correctly achieve line segmentation ? i have attached your code also.

Please help me..thank you

[Image Analyst]

You didn't threshold it correctly. You need to look for letters less than 100 gray levels:

subplot(2, 3, 2);
histogram(grayImage);
uiwait(helpdlg('This is the histogram'));
binaryImage = grayImage < 100;
imshow(binaryImage);
title('Binary Image', 'FontSize', fontSize, 'Interpreter', 'None');
axis on;

[Meghashree G]

ok sir...still the lines are not segmenting :( please help me!! Thank you !

[Image Analyst]

See attached m-file.

[Meghashree G]

Thank you very very much sir :) it worked :)

[Divya]

Hi Image Analyst,

your code says,

??? Undefined function or method 'histogram' for input arguments of type 'uint8'.

What modifications need to be done??