Skip to content
MathWorks - Mobile View
  • Sign In to Your MathWorks AccountSe connecter
  • Access your MathWorks Account
    • Mon compte
    • Mon profil
    • Mes licences
    • Se déconnecter
  • Produits
  • Solutions
  • Le monde académique
  • Support
  • Communauté
  • Événements
  • Obtenir MATLAB
MathWorks
  • Produits
  • Solutions
  • Le monde académique
  • Support
  • Communauté
  • Événements
  • Obtenir MATLAB
  • Sign In to Your MathWorks AccountSe connecter
  • Access your MathWorks Account
    • Mon compte
    • Mon profil
    • Mes licences
    • Se déconnecter

Vidéos et webinars

  • MathWorks
  • Vidéos
  • Vidéos
  • Recherche
  • Vidéos
  • Recherche
  • Contacter l'équipe commerciale
  • Version d'essai
5:01 Video length is 5:01.
  • Description
  • Full Transcript
  • Related Resources

Lidar Camera Calibration with MATLAB

Use the Lidar Camera Calibrator app in Lidar Toolbox™ to cross-calibrate lidars and cameras in a system—an essential step to combine data. Cameras provide rich color information, while lidar sensors provide accurate 3D structural and locational information of objects. When fused, you can enhance the performance of perception and mapping algorithms for autonomous driving and robotics applications.

Lidar camera calibration helps to estimate relative position and orientation between a lidar and a camera in a system

Cameras provide rich color information, while lidar sensors provide an accurate 3-D structural and locational information of objects. When fused together, we can enhance the performance of perception and mapping algorithms for autonomous driving and robotics applications.

Lidar camera calibration involves calculation of extrinsic parameters of the lidar-camera system in the form of a rigid transformation matrix.

Extrinsic parameters define the location and orientation of the sensors with respect to the world frame and with each other. These parameters are essential to fuse data from these sensors. Lidar Toolbox provides both MATLAB functions and an interactive app for performing lidar- camera calibration. The Lidar Camera Calibrator app is an easy and interactive interface to cross calibrate cameras and lidars in a system.

Here is the workflow we will be following. We will load and then extract checkerboard features from images and corresponding point clouds. We will then use these features to estimate transformation matrix between camera and lidar.

Let’s now see how to use Lidar Camera Calibrator app to implement this workflow.

We can open the app using command line or you find the app from add on menu. 

Once the app is open, we will first add images and the corresponding point clouds of a checkerboard calibration pattern. A checkerboard is used because its regular pattern makes it easy to detect automatically. It is recommended to use between 10 and 20 checkerboard data for accurate calibration results.

Next, enter the size of the checkerboard square in world units in millimeters, centimeters, on in meters. Here for this data, we are using a checkerboard square of 200 mm in size.

This step is necessary to find the mapping between world units with the image-point cloud pairs. 

While we load the data, the app automatically calibrate the camera intrinsics and detects the checkerboard calibration pattern in the loaded image-point cloud pairs.

Alternatively, if you have camera intrinsic already available, you can load them by choosing use fixed intrinsics in camera intrinsics section.

In the left panel you can see accepted data and rejected data. Every image-point cloud pair from which checkerboard pattern was detected can be seen in the accepted data panel. These data will be used to calibrate the transformation between the sensors.  

You can tweak the values on ‘Feature Detection’ section on toolstrip to help with detection in rejected data also

You can also use the edit ROI functionality to improve your detection.  You can change/ move ROI to capture the region of point cloud that consist of the checkerboard. 

Then click ‘Detect’ to detect checkerboard patterns on the entire data again. You can see here that 8 out of 9 pairs are now accepted for calibrating the transformation.

After checkerboard detection, we can calibrate the transformation by clicking ‘calibrate’ button.

You can now see lidar checkerboard points projected on image, and image colors fused into lidar point cloud.

Once calibration is done, you can evaluate calibration results by visualizing translation, rotation, and reprojection errors.

This is helpful to identify bad data that you can remove and recalibrate for better results. 

We can do this by dragging the threshold line in any of the error plots and then deleting the highlighted data pairs in accepted data browser by right clicking and selecting ‘remove and recalibrate’ option.

Now we can export the estimated transformation and errors to the MATLAB workspace, or to a file.

You can also generate a MATLAB script out of it.

A calibrated lidar camera system can be further used for applications like

1- estimating coordinates of 3D bounding boxes in lidar data from 2D bounding boxes in the corresponding image. 

2- Fusing color information from camera on point cloud.

Refer to MathWorks documentation and Lidar Toolbox product pages to learn more.

If you have any questions or comments, please let us know.

Related Products

  • Lidar Toolbox

3 Ways to Speed Up Model Predictive Controllers

Read white paper

A Practical Guide to Deep Learning: From Data to Deployment

Read ebook

Bridging Wireless Communications Design and Testing with MATLAB

Read white paper

Deep Learning and Traditional Machine Learning: Choosing the Right Approach

Read ebook

Hardware-in-the-Loop Testing for Power Electronics Control Design

Read white paper

Predictive Maintenance with MATLAB

Read ebook

Electric Vehicle Modeling and Simulation - Architecture to Deployment : Webinar Series

Register for Free

How much do you know about power conversion control?

Start quiz
Related Information
Related Information
Learn more about the Lidar Camera Calibrator app

Feedback

Featured Product

Lidar Toolbox

  • Request Trial
  • Get Pricing

Up Next:

5:51
Camera Calibration with MATLAB

Related Videos:

30:17
Designing a Digital Camera Pipeline with MATLAB
6:33
Using MATLAB with a Raspberry Pi Camera Board
34:46
Using MATLAB for Real-Measure Calibration of Stochastic...
13:46
Virtual Engine Calibration: Making Engine Calibration Part...

View more related videos

MathWorks - Domain Selector

Select a Web Site

Choose a web site to get translated content where available and see local events and offers. Based on your location, we recommend that you select: .

  • Switzerland (English)
  • Switzerland (Deutsch)
  • Switzerland (Français)
  • 中国 (简体中文)
  • 中国 (English)

You can also select a web site from the following list:

How to Get Best Site Performance

Select the China site (in Chinese or English) for best site performance. Other MathWorks country sites are not optimized for visits from your location.

Americas

  • América Latina (Español)
  • Canada (English)
  • United States (English)

Europe

  • Belgium (English)
  • Denmark (English)
  • Deutschland (Deutsch)
  • España (Español)
  • Finland (English)
  • France (Français)
  • Ireland (English)
  • Italia (Italiano)
  • Luxembourg (English)
  • Netherlands (English)
  • Norway (English)
  • Österreich (Deutsch)
  • Portugal (English)
  • Sweden (English)
  • Switzerland
    • Deutsch
    • English
    • Français
  • United Kingdom (English)

Asia Pacific

  • Australia (English)
  • India (English)
  • New Zealand (English)
  • 中国
    • 简体中文Chinese
    • English
  • 日本Japanese (日本語)
  • 한국Korean (한국어)

Contact your local office

  • Contacter l'équipe commerciale
  • Version d'essai

MathWorks

Accelerating the pace of engineering and science

MathWorks est le leader mondial des logiciels de calcul mathématique pour les ingénieurs et les scientifiques.

Découvrir…

Découvrir les produits

  • MATLAB
  • Simulink
  • Version étudiante
  • Support Hardware
  • File Exchange

Essayer ou acheter

  • Téléchargements
  • Version d'essai
  • Contacter un commercial
  • Tarifs et licences
  • Comment acheter

Se former

  • Documentation
  • Tutoriels
  • Exemples
  • Vidéos et webinars
  • Formation

Obtenir de l'aide

  • Aide à l'installation
  • MATLAB Answers
  • Services de consulting
  • Centre de gestion des licences
  • Contacter le support technique

La société

  • Offres d'emploi
  • Actualités
  • Mission sociale
  • Témoignages clients
  • La société
  • Select a Web Site United States
  • Trust Center
  • Marques déposées
  • Charte de confidentialité
  • Lutte anti-piratage
  • État des applications

© 1994-2022 The MathWorks, Inc.

  • Facebook
  • Twitter
  • Instagram
  • YouTube
  • LinkedIn
  • RSS

Rejoignez la conversation