Unable to find SURF features in an RGB and IR image

Hello,

I have used SURF algorithm to compare images and find matching features. It works fine normally. But, recently I started working on IR images and its result wasn't satisfactory. Currently I have two cameras, one RGB and the other one is IR. They are arranged in stereo mode. As both of these cameras are at some distance apart so, naturally there would be some transformation between them. So, I applied Stereo camera calibration to first align the images on the same axis i.e. my rotation problem was solved. Now the only problem remains is of translation. Stereo camera calibration provides a translation vector but it is almost useless because whenever I change the distance of the object being viewed and the cameras, i have to manually change the translation vector. So, to solve this problem I used DetectSURFfeatures. Now it should work fine but as in my case I have a RGB and an IR camera, this algorithm is unable to find enough SURF features to allow the transformation. Is their any solution to the problem ? Plz help Thank You

My code is as under

Note: parameters.mat is used load the variable stereoparams

variable indexpairs is an empty matrix

    load parameters.mat
    J1=imread('IR_Front_45cm_3-IR.bmp');
    J2=imread('RGB_Front_45cm_3-RGB.bmp');
    J1=imread('IR_Back_Block_30cm_1-IR.bmp');
    J2=imread('RGB_Back_30cm_1-RGB.bmp');
    [J1, J2] = rectifyStereoImages(I1, I2, stereoParams, 'OutputView', 'Full');

    ptsOriginal  = detectSURFFeatures(rgb2gray(J1));
    ptsDistorted = detectSURFFeatures(rgb2gray(J2));
    [featuresOriginal,  validPtsOriginal]  = extractFeatures(rgb2gray(J1),  ptsOriginal);
    [featuresDistorted, validPtsDistorted] = extractFeatures(rgb2gray(J2), ptsDistorted);
    indexPairs = matchFeatures(featuresOriginal, featuresDistorted);
    matchedOriginal  = validPtsOriginal(indexPairs(:,1));
    matchedDistorted = validPtsDistorted(indexPairs(:,2));
    figure;
    showMatchedFeatures(rgb2gray(J1),rgb2gray(J2),matchedOriginal,matchedDistorted);
    title('Putatively matched points (including outliers)');
    [tform, inlierDistorted, inlierOriginal] = estimateGeometricTransform(...
        matchedDistorted, matchedOriginal, 'similarity');
    figure;
    showMatchedFeatures(rgb2gray(J1),rgb2gray(J2),inlierOriginal,inlierDistorted);
    title('Matching points (inliers only)');
    legend('ptsOriginal','ptsDistorted');
    Tinv  = tform.invert.T;
    ss = Tinv(2,1);
    sc = Tinv(1,1);
    scaleRecovered = sqrt(ss*ss + sc*sc)
    thetaRecovered = atan2(ss,sc)*180/pi
    outputView = imref2d(size(rgb2gray(J1)));
    recovered  = imwarp(rgb2gray(J2),tform,'OutputView',outputView);
    figure, imshowpair(rgb2gray(J1),recovered)

Answer 1

[Dima Lisin]

Hi Faseeh,

It would help if you could post a sample pair of images.

This is not really surprising, because an IR and and an RGB image of the same scene may well look very different. You can try tweaking the parameters of the matchFeatures function, such as increasing the 'MaxRatio', which would make it easier to find matches. You can also tweak the parameters of detectSURFFeatures. Try reducing 'MetricThreshold' to 500 to get more points, or try increasing 'NumOctaves' to capture larger-scale features.

You can also try using other feature detectors, such as detectHarrisFeatures or detectMSERFeatures.

[Faseeh]

<<

>>Here are some IR images.

Answer 2

[Faseeh]

Thank You I will try the above options. I have attached some images taken from RGB camera

Answer 3

[Faseeh]

I found another solution. Instead of using one checkerboard. I used a cluster of checkerboard images and the result was quite good. Now it detects enough features for applying transformation. Their is some disparity in after transformation. I am now working on that. If you have some tips regarding that please guide me. Thank You

Answer 4

[elnaz akbari]

Hi , I have same problem, I have two pictures , RGB and IR that I want to match them in Matlab but it doesn't work correct. could you please guide me ? Thank you