Incorrect result from camera calibrator app

I used the camera calibrator app to calibrate lens distortion and I specified correct checkerboard size of 8mm. The calibration goes fine and i'm able to undistort images and points. However, the plot from showextrinsics() shows camera at 8mm distance from object but in reality it is 140mm. Why is this wrong?

My setup has a camera looking downwards and pattern is placed at the same working distance in all 30+ images where the lens is focused.

Calibration settings: All default settings - Standard lens, 2 coeffs, no tangential distortion, no skew, etc.

[Qu Cao]

It looks to me that all the images are almost placed in the same plane. For accurate calibration results, we recommend taking images from different orientations and view angles.

[cr]

Hi Qu

Thanks for quick response. Images were taken with a telecentric lens -the application involves taking image at the specified distance which doesnt change. The images were indeeded taken with positions and rotation angle changed. Another requirement of the calibration would be to compute pixels to mm conversion factor which cannot be done if distance to camera changes. This got verified correctly.

Is constant distance to camera the reason for incorrect distance-to-camera estimation?

Thanks

[Giridharan Kumaravelu]

Would you mind providing us with the calibration images to help troubleshoot this behavior?

[cr]

The folder is a little too big to be attached as a zipped file. Here is a link to google drive.

https://drive.google.com/file/d/1364FOSwJ5gDIyzWeSDh6U-pxcqwT7P9W/view?usp=share_link

I do not recall if this is the exact same set of images yielding the results I shared above, but all 3 sets I have produced similar results in all aspects.

Thanks & Regards

Answer 1

[Giridharan Kumaravelu]

Thanks for providing the calibration images. This is a limitation of the technique (Zhang's algorithm) that camera calibrator app uses, that it cannot calibrate a camera with calibration images that are coplanar. To accurately calibrate a camera with the app, you need some variance in the calibration images in the z-direction. Even a little wiggle in the z direction could avoid this degenerate configuration.

[cr]

Hi Giridharan

Thanks for the response. That's interesting. Does it mean only this particular aspect of the algorithm is degenerate? Are other results valid? We can see that undistorting the images seems to work fine with the calibration output, but are they valid? What are other caveats of not changing distance to object in calibration images? The problem i have with changing distance in Z is pixels-to-mm information from calibration that is needed for metrology. Is there a way to vary distance and somehow arrive at correct pixel-to-mm figure?

I'm also intrigued by what is done in this example here. On a cursory look, I would think this works using the pixels-to-mm information from the checkerboard in the pic. How accurate will the estimations be if checkerboard is removed from the image and distance to camera changes? Or, in other words, how good the estimates be in some arbitrary picture of pennies using the same camera?

What's also intriguing to me in this example is the estimation of distance of penny-1 from camera. Although an estimate is arrived at there is no mention of what the real distance was. So here is the question: If I calibrate a camera using suggested method, will I be able to guage distances to objects? I presume not, but a clarification on the example would be good.

Thank you.

p.s. I know that's a lot of questions, really appreciate your patience answering them.

[Giridharan Kumaravelu]

The camera calibration algorithm in MATLAB jointly optimizes for the intrinsics (including distortion parameters) and extrinsics. So, all the results are degenerate and invalid although the undistortion may look normal.

If you are in a position to vary the z-distances, you might as well capture the calibration images by varying the z-distance and you can arrive at the correct pixel-to-mm.

With respect to your questions related to the example, here is a typical workflow of measuring planar objects:

1. Calibrate the camera for intrinsics in any location.
2. Calibrate the camera for extrinsics in the workplace with the known intrinsics from step 1 and with the help of calibration pattern. Do not move the camera after this step to avoid invalidating the extrinsics. (Extrinsics is the relative position of the camera w.r.t the calibration pattern).
3. You can remove the calibration pattern and start measuring planar objects that are in the same plane as the calibration pattern.

To answer your question, the estimations will still be accurate even if the checkerboard is removed after step 2 mentioned above. But if the camera is moved after step 2, then you need repeat step 2 to accurately measure objects.

It is true that you can gauge distances to "planar" objects, after calibration step but only with the help of a calibration target like a checkerboard or an apriltag.

[cr]

Giridharan

Thanks for the detailed answers.

[Alexander Groeger]

How exactly is the use of the camera calibrator intended in the case of a telecentric camera?

Thanks in advance.

[cr]

I think this is a great question for MW. In my applications I need calibration only for (un)distortion. Since the telecentrics have very less distortion I never considered calibration when using them. If your concern is shallow DoFs of telecentrics for changing distance to cam, for lenses with low magnification the DoF should be ok with changing the depth by a small fraction of a millimetre -I’m presuming Giridharan meant that will be fine, but remains to be seen.

Despite MW’s answer here I have a feeling distortion part is working fine. Perhaps we should look into the method deeper to understand what can be done to remove the limitation at least for distortion part of it.

[Alexander Groeger]

I'm not an expert in this field but I think the telecentric (aka orthographic) cameras requiere a different camera model for calibration. Since in the case of orthographic projection the Z-dependancy of magnification is eliminated (constant magnification instead).

For my application I'm currently working with a model based on Chen et al's work:

Telecentric stereo micro-vision system: Calibration method and experiments

But I still think it would make MW's camera calibrator app more complete if an orthographic model would be integrated as well.