Kinect Infrared (IR) and Depth Image Simulator

Simulation tool that generates noisy Kinect IR and depth images from a user inputted CAD model
1.3K Downloads
Updated 22 Dec 2017

View License

This zip-file contains functions to simulate noisy Kinect infrared (IR) and depth images from a user inputted CAD model.
This work was motivated by an extensive study of the sensor's underlying mechanisms and performance characteristics, as well as newly constructed empirical models for the intensity, speckle, and detector noise of the received IR dot pattern. The simulator models the physics of the transmitter/receiver system, unique IR dot pattern, disparity/depth processing technology, and random intensity speckle and IR noise in the detectors. Our model accounts for important characteristics of Kinect's stereo triangulation system, including depth shadowing, IR dot splitting, spreading, and occlusions, correlation-based disparity estimation between windows of measured and reference IR images, and sub-pixel refinement.
The Kinect simulator functions require the CAD model to include vertex, facet, and facet normal arrays. The structure of the arrays must be:
vertex - 3xn, n vertices of each 3D coordinate that defines the CAD model.
face - 3xm, m facets, each represented by 3 vertices that defines the CAD model.
normalf - 3xm, m facets, representing the normal direction of each facet.
Note, the single CAD model input could be an aggregate collection of multiple CAD models of interesting objects and background structures. I also provide the option to include a CAD model of a flat wall to function as the scene’s background.

This package requires the Matlab wrapper for OPCODE, which is a collision detection/ray casting library for triangular 3D meshes. The wrapper was written by Vipin Vijayan, and can be found here:
http://www.mathworks.com/matlabcentral/fileexchange/41504-ray-casting-for-deformable-triangular-3d-meshes/content/opcodemesh/matlab/opcodemesh.m

This code was used to simulate line-of-sight vectors emanating from the transmitter coordinate system, which in effect represent the IR laser system that transmits the dot pattern onto the given scene. An idealized binary representation of the Kinect dot pattern is used as default for the simulator, which was constructed by Andreas Reichinger, and can be found here:
https://azttm.wordpress.com/2011/04/03/kinect-pattern-uncovered/

The downloadable package includes a demo to exhibit the functionality of the IR and depth image simulator options, which generates images of numerous CAD models (also included).

For a detailed description of how this simulator was developed, please refer to

M. J. Landau, B. Y. Choo, P. A. Beling, “Simulating Kinect Infrared and Depth Images,” IEEE Transactions on Cybernetics, vol. 46, no. 12, pp. 3018 - 3031, Dec. 2016.

This paper can be found here:

http://ieeexplore.ieee.org/xpl/articleDetails.jsp?arnumber=7328728

or

https://www.researchgate.net/publication/283734301_Simulating_Kinect_Infrared_and_Depth_Images

If this simulator is used for publication, please cite this in your references.

Cite As

Michael Landau (2024). Kinect Infrared (IR) and Depth Image Simulator (https://www.mathworks.com/matlabcentral/fileexchange/50357-kinect-infrared-ir-and-depth-image-simulator), MATLAB Central File Exchange. Retrieved .

MATLAB Release Compatibility
Created with R2014b
Compatible with any release
Platform Compatibility
Windows macOS Linux
Acknowledgements

Inspired by: Ray casting for deformable triangular 3D meshes

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!
Version Published Release Notes
1.5.0.0

Reference citation updated in description
Updated citation for journal paper
Updated IR noise models and routine
Fixed shift from transmitter to receiver coordinate system
Improved ray occlusion detection method for IR simulator
Speed up improvements made for IR and depth image simulator
opcode package needs to be downloaded

1.4.0.0

Acknowledgement updated