Get Started with PointNet++

PointNet++ is a popular neural network architecture for learning from unorganized point clouds. You can use PointNet++ for semantic segmentation to assign a class label to each point in a point cloud. You can also use PointNet++ for object classification to assign a single class label to a point cloud that contains one object.

PointNet++ network hierarchically processes individual points in a point cloud. The network uses PointNet to extract local features from point clusters and groups those features to produce higher level features from the entire point cloud.

Applications of PointNet++ include:

  • Tree segmentation for digital forestry applications.

  • Extracting a digital terrain model from aerial lidar data.

  • Perception for indoor navigation in robotics.

  • 3-D city modelling from aerial lidar data.

  • Object classification.

PointNet++ Network for Semantic Segmentation

The PointNet++ network contains an encoder with set abstraction modules and a decoder with feature propagation modules.

The set abstraction module processes and extracts a set of points to produce a new set with fewer elements. Each set abstraction module contains a sampling and grouping layer followed by a PointNet layer.

  • The sampling and grouping layer performs sampling by identifying the centroids of local regions. It then performs grouping by constructing local region sets of the neighboring points around the centroids.

  • The PointNet layer contains a mini-PointNet network with a series of convolution, normalization, relu layers followed by a max pooling layer. This layer encodes the local region patterns into feature vectors.

The feature propagation module interpolates the subsampled points and then concatenates them with the point features from the set abstraction modules. The network then passes these features through a unit PointNet network where they are further processed to produce higher level features.

In other words, PointNet++ applies PointNet recursively on the nested, partitioned inputs to extract multi-scale features for accurate semantic segmentation.

Create PointNet++ Network

Use the pointnetplusNetwork function to create a PointNet++ network for segmenting point cloud data.

Train PointNet++ Network

To learn how to train a PointNet++ network for segmenting point cloud data, see Aerial Lidar Semantic Segmentation Using PointNet++ Deep Learning.

Code Generation

To learn how to generate CUDA® code for a PointNet++ network, see Code Generation for Aerial Lidar Semantic Segmentation Using PointNet++ Deep Learning.

PointNet++ Network for Object Classification

For object classification, PointNet++ learns a global feature representation of the input point cloud and predicts a single class label for the object. Similar to the segmentation network encoder, the classification network hierarchically processes points using set abstraction modules to learn features from local point neighborhood at multiple scales.

After the final set abstraction module, the network aggregates the learned features into a global feature representation for the entire point cloud. The network then uses fully connected layers to compute class scores and output a single class label for the input point cloud. Note that feature propagation modules (decoder) are not required because the output is a single label for the entire point cloud rather than point-wise labels.

Create PointNet++ Classifier

Use the pointNetPlusClassifier function to create a pretrained or untrained PointNet++ classifier for classifying point clouds. The pretrained classification network has been trained on using only the location information of the point cloud data from the Sydney Urban Objects data set. Note the input point cloud must contain only one object.

Train PointNet++ Classifier

To train a PointNet++ classifier, use the trainPointNetPlusClassifier function.

Classify Point Clouds

To predict a class label for an input point cloud, use the classify object function of the pointNetPlusClassifier object. The input point cloud must contain only one object. For more information on classifying point clouds, see the Point Cloud Classification Using PointNet++ Deep Learning example.

References

[1] Qi, Charles R., Li Yi, Hao Su, and Leonidas J. Guibas. ‘PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space’. ArXiv:1706.02413 [Cs], 7 June 2017. https://arxiv.org/abs/1706.02413.

[2] Varney, Nina, Vijayan K. Asari, and Quinn Graehling. ‘DALES: A Large-Scale Aerial LiDAR Data Set for Semantic Segmentation’. ArXiv:2004.11985 [Cs, Stat], 14 April 2020. https://arxiv.org/abs/2004.11985.

[3] De Deuge, Mark, Alastair Quadras, Calvin Hung, and Bertrand Douillard. "Unsupervised Feature Learning for Classification of Outdoor 3D Scans." In Australasian Conference on Robotics and Automation 2013 (ACRA 13). Sydney, Australia: ACRA, 2013.

See Also

Apps

Functions

Topics