How can I separate hand from forearm after skin segmentation?

I'm working on a gesture recognition project. I've carried out skin segmentation and I'm now trying to separate the hand from the forearm before the feature extraction phase. I'm wondering how I would go about achieving this? I've attached an image for reference.

Thanks.

Answer 1

[Cedric]

This could be a start. This is not as solid/stable as what ImageAnalyst proposes though. In French we would call that "bricolage" ;-)

 I = im2bw( imread( 'hand.jpg' )) ;

 % - Crop white frame.
 I = I(~all(I,2),~all(I,1)) ;

 % - Crop 10px border to eliminate glitches at the border.
 I = I(10:end-10,10:end-10) ;
 [nRow, nCol] = size( I ) ;

 % - Find first horizontal transition 0->1 from left.
 D = diff( I, 1, 2 ) == 1 ;

 % - Find first column per row where D is 1.
 [~, colId] = max( D, [], 2 ) ;

 % - Smoothen colId.
 kerSize   = round( nCol /10 ) ;
 colId_smz = conv( colId, ones( 1, kerSize )/kerSize, 'same' ) ;

 figure() ;
 set( gcf, 'Units', 'normalized', 'Position', [0.1,0.1,0.8,0.8] ) ;

 % - Plot cropped image.
 subplot( 2, 2, 1 ) ;
 imshow( I ) ;
 title( 'Cropped image' ) ;

 % - Plot left profile.
 subplot( 2, 2, 2 ) ;
 imshow( D ) ;
 title( 'Left profile' ) ;

 % - Plot left profile + smoothen.
 subplot( 2, 2, 3 ) ;
 y_colId = colId ;  y_colId(colId==1) = NaN ;
 y_smz   = colId_smz ; y_smz(colId==1) = NaN ;
 plot( 1:nRow, y_colId, 'b', 1:nRow, y_smz, 'r' ) ;
 grid on ;  title( 'Profile + smoothen' ) ;

 % - Plot difference + smoothen.
 subplot( 2, 2, 4 ) ;  
 dif     = diff( y_smz ) ;
 dif_smz = conv( dif, ones( 1, kerSize )/kerSize, 'same' ) ;
 plot( 1:nRow-1, dif, 'b', 1:nRow-1, dif_smz, 'r' ) ;
 ylim( prctile( dif_smz, [5, 95] )) ;
 grid on ;  title( 'Diff + smoothen' ) ;

With that you get:

Then you have to detect the from the right the point where the smoothen derivative changes significantly.

[Image Analyst]

FYI to crop away the small blobs, instead of imcrop() you can use the new function bwareafilt()

% Extract only the largest blob:
binaryImage = bwareafilt(binaryImage, 1);

It's a handy new function I'm now using a lot.

[Cedric]

I will really have to dig in this Image Processing Toolbox! I am currently doing a lot of operations using a combination of MATLAB and ArcGIS for GIS operations, and I am pretty sure that a lot of these operations could be done with the Image Proc. Tbx only.

[M]

Thank you for your help. I have decided to make things simple and avoid forearm skin detection.

Answer 2

[Image Analyst]

You can probably find an algorithm here:

http://www.visionbib.com/bibliography/contentspeople.html#Face%20Recognition,%20Detection,%20Tracking,%20Gesture%20Recognition,%20Fingerprints,%20Biometrics

[M]

Hi, thanks for your response. I came across an algorithm (referenced here: http://admin.csie.ntust.edu.tw/IEET/syllabus/course/961_CS5014702_65_aGlnaDIucGRm.pdf), but I'm not sure of it's technical implementation (the paper referenced another source which I could not locate). 2 methods were described, demonstrated in the image below:

The first is based on the wrist length, the 2nd on changes in the contour of the image.

I have a simple idea (for the 1st method) where you would scan through the pixels (starting from the bottom of the image), counting the no. of pixels on each row, until you encounter a row which has more pixels than the previous row (indicating the start of the palm area). However, this would assume that the positioning of the hand is always vertical. Any ideas to improve upon that so that the hand doesn't always have to be vertical or how to implement the contour-based approach (where you would detect a sharp turn in the contour, indicating the wrist)? I'm quite new to Matlab. Thanks.

[Image Analyst]

It's not going to be trivial if it's going to be robust. First of all I'd use regionprops to get the orientation and centroid of the blob. Then use bwboundaries() to get the boundary coordinates. Then use the orientation and centroid to figure out which coordinates are on one side of the arm and which are on the other side of the arm - just simple algebra. Now for every point on one side, I'd scan all the points on the other side and find out which point on the other side is the closest to it. This should be the one directly on the opposite side of the arm. Then I'd calculate distances from each point to its nearest corresponding point on the other side of the arm. Then you can for where the distance increases as the same time that a slope changed dramatically on one or both sides. However you need to be careful about special situations, like where the arm intersects the bottom edge of the image at an angle, forming a pointy triangle. The cross-arm distance will increase as you move along that.

I can't give you code for any of that. I don't have it and it would take too much development time to do it and donate it to you. It's no 5 minute job.

[M]

Thanks for the ideas, but implementing it is probably out of my reach. I have decided to just eliminate the forearm skin detection (by wearing long sleeve shirts).

[Image Analyst]

If you can do that, that is a far simpler solution.