Get Started with RandLA-Net

RandLA-Net is a fast and efficient deep learning network designed for the semantic segmentation of point clouds. Semantic segmentation associates each point in a 3-D point cloud with a class label, such as car, truck, ground, or vegetation. To learn more about semantic segmentation with point clouds, see Semantic Segmentation in Point Clouds Using Deep Learning.

The RandLA-Net network combines simple random point sampling with a sophisticated local feature aggregation module. The network avoids computationally expensive operations, but preserves the intricate geometric details essential for accurate segmentation. The network is also capable of handling large-scale 3-D point clouds with millions of points, spanning extensive areas, making it highly suitable for applications in autonomous driving and robotics, as well as urban planning.

RandLA-Net Network

The RandLA-Net network uses an encoder-decoder architecture with skip connections. The network initiates with a fully connected (FC) shared multilayer perceptron (MLP) to extract initial per-point features from the input point cloud. Then, the network has a series of encoding and decoding layers to further refine these features.

Each encoding layer downsamples its input by using random sampling (RS) to decrease the point density. However, random sampling is prone to potential information loss. To compensate for this loss, each encoding layer incorporates a sophisticated local feature aggregation (LFA) module. This module highlights the complex local structures within the point cloud data.

Each decoding layer upsamples (US) the point feature set using nearest-neighbor interpolation. The network concatenates these upsampled maps with feature maps produced by the encoding layers through skip connections, and applies a shared MLP to the concatenated feature maps.

Finally, the architecture of RandLA-Net concludes with three fully connected layers and a dropout (DP) layer. The network outputs a 3-D point cloud by associating each point with a class label, such as car, truck, ground, or vegetation.

Create RandLA-Net Network

To programmatically create a RandLA-Net network, use the randlanet object. You can create an untrained network or a pretained network that is trained on the Dayton Annotated Lidar Earth Scan (DALES) or PandaSet point cloud data set. You can also retrain a pretrained RandLA-Net network to perform transfer learning using the specified set of segmentation classes.

Note

This functionality requires Deep Learning Toolbox™ and the Lidar Toolbox™ Model for RandLA-Net Semantic Segmentation. You can download and install the Lidar Toolbox Model for RandLA-Net Semantic Segmentation from Add-On Explorer. For more information about installing add-ons, see Get and Manage Add-Ons.

Transfer Learning

Transfer learning is a common deep learning technique in which you reuse a pretrained network as a starting point to learn a new task through further training. This process involves fine-tuning the network with new data and apply its previously learned knowledge to the new task. This approach reduces the need for extensive data and training time for the new task.

To perform transfer learning with a pretrained RandLA-Net network, specify new object classes. Then, retrain the network on a new data set.

Train RandLA-Net Network

Use the trainRandlanet function to train a RandLA-Net network. For more information on how to train a RandLA-Net network, see Aerial Lidar Semantic Segmentation Using RandLANet Deep Learning. To segment point cloud data using a trained network, use the segmentObjects function.

References

[1] Hu, Qingyong, Bo Yang, Linhai Xie, Stefano Rosa, Yulan Guo, Zhihua Wang, Niki Trigoni, and Andrew Markham. “Learning Semantic Segmentation of Large-Scale Point Clouds With Random Sampling.” IEEE Transactions on Pattern Analysis and Machine Intelligence 44, no. 11 (November 2022): 8338–54. https://doi.org/10.1109/TPAMI.2021.3083288.