Lidar 3-D Object Detection Using PointPillars Deep Learning
This example shows how to detect objects in lidar using PointPillars deep learning network [1]. In this example, you
Configure a dataset for training and testing of PointPillars object detection network. You also perform data augmentation on the training dataset to improve the network efficiency.
Compute anchor boxes from the training data to train the PointPillars object detection network.
Create a PointPillars object detector using the
pointPillarsObjectDetectorfunction and train the detector usingtrainPointPillarsObjectDetectorfunction.
This example also provides a pretrained PointPillars object detector to use for detecting objects in a point cloud. The pretrained model is trained on Pandaset dataset. For information on pointpillars object detection network, see Get Started with PointPillars (Lidar Toolbox).
Download Lidar Data Set
Download a ZIP file (approximately 5.2GB in size) that contains a subset of sensor data from the PandaSet data set [2] . After downloading, unzip the file. This file includes two main folders: Lidar and Cuboids, which contain the following data:
The PointCloud folder contains 2560 preprocessed organized point clouds in PCD format, arranged so that the ego vehicle moves along the positive y-axis. If your point cloud deviates from this orientation, you can utilize the
pctransform(Computer Vision Toolbox) function to apply the necessary transformation.The
Cuboidfolder contains the corresponding ground truth data in table format, stored in thePandasetLidarGroundTruth.matfile. This file provides 3-D bounding box information for three categories: car, truck, and pedestrian. Any tansformation applied to the point cloud should be applied to the bounding boxes using thebboxwarp(Computer Vision Toolbox) function.
Download the Pandaset dataset from the given URL using the helperDownloadPandasetData helper function, defined at the end of this example.
outputFolder = fullfile(tempdir,'Pandaset'); lidarURL = ['https://ssd.mathworks.com/supportfiles/lidar/data/' ... 'Pandaset_LidarData.tar.gz']; helperDownloadPandasetData(outputFolder,lidarURL);
Downloading PandaSet Lidar driving data (5.2 GB)...
Depending on your Internet connection, the download process can take some time. The code suspends MATLAB® execution until the download process is complete. Alternatively, you can download the data set to your local disk using your web browser and extract the file. If you do so, change the outputFolder variable in the code to the location of the downloaded file.
Load Data set
Create a file datastore to load the PCD files from the specified path using the pcread (Computer Vision Toolbox) function.
path = fullfile(outputFolder,'Lidar'); lidarData = fileDatastore(path,'ReadFcn',@(x) pcread(x));
Load the 3-D bounding box labels of the car and truck objects.
gtPath = fullfile(outputFolder,'Cuboids','PandaSetLidarGroundTruth.mat'); data = load(gtPath,'lidarGtLabels'); Labels = timetable2table(data.lidarGtLabels); boxLabels = Labels(:,2:3);
Display the full-view point cloud.
figure ptCld = preview(lidarData); ax = pcshow(ptCld.Location); set(ax,'XLim',[-50 50],'YLim',[-40 40]); zoom(ax,2.5); axis off;
Preprocess Data
The PandaSet data consists of full-view point clouds. For this example, crop the full-view point clouds to front-view point clouds using the standard parameters [1]. These parameters determine the size of the input passed to the network. Select a smaller point cloud range along the x, y, and z-axis to detect objects closer to origin. This also decreases the overall training time of the network.
xMin = 0.0; % Minimum value along X-axis. yMin = -39.68; % Minimum value along Y-axis. zMin = -5.0; % Minimum value along Z-axis. xMax = 69.12; % Maximum value along X-axis. yMax = 39.68; % Maximum value along Y-axis. zMax = 5.0; % Maximum value along Z-axis. xStep = 0.16; % Resolution along X-axis. yStep = 0.16; % Resolution along Y-axis. % Define point cloud parameters. pointCloudRange = [xMin xMax yMin yMax zMin zMax]; voxelSize = [xStep yStep];
Use the cropFrontViewFromLidarData helper function, attached to this example as a supporting file, to:
Crop the front view from the input full-view point cloud.
Select the box labels that are inside the ROI specified by
gridParams.
[croppedPointCloudObj,processedLabels] = cropFrontViewFromLidarData(...
lidarData,boxLabels,pointCloudRange);Processing data 100% complete
Display the cropped point cloud and the ground truth box labels.
pc = croppedPointCloudObj{1,1};
bboxes = [processedLabels.Car{1};processedLabels.Truck{1}];
ax = pcshow(pc);
showShape('cuboid',bboxes,'Parent',ax,'Opacity',0.1,...
'Color','green','LineWidth',0.5);
reset(lidarData);
Create Datastore Objects
Split the data set into training and test sets. Select 70% of the data for training the network and the rest for evaluation.
rng(1); shuffledIndices = randperm(size(processedLabels,1)); idx = floor(0.7 * length(shuffledIndices)); trainData = croppedPointCloudObj(shuffledIndices(1:idx),:); testData = croppedPointCloudObj(shuffledIndices(idx+1:end),:); trainLabels = processedLabels(shuffledIndices(1:idx),:); testLabels = processedLabels(shuffledIndices(idx+1:end),:);
So that you can easily access the datastores, save the training data as PCD files by using the saveptCldToPCD helper function, attached to this example as a supporting file. You can set writeFiles to "false" if your training data is saved in a folder and is supported by the pcread (Computer Vision Toolbox) function.
writeFiles = true; dataLocation = fullfile(outputFolder,'InputData'); [trainData,trainLabels] = saveptCldToPCD(trainData,trainLabels,... dataLocation,writeFiles);
Processing data 100% complete
Create a file datastore using fileDatastore to load PCD files using the pcread (Computer Vision Toolbox) function.
lds = fileDatastore(dataLocation,'ReadFcn',@(x) pcread(x));Createa box label datastore using boxLabelDatastore (Computer Vision Toolbox) for loading the 3-D bounding box labels.
bds = boxLabelDatastore(trainLabels);
Use the combine function to combine the point clouds and 3-D bounding box labels into a single datastore for training.
cds = combine(lds,bds);
Perform Data Augmentation
This example uses ground truth data augmentation and several other global data augmentation techniques to add more variety to the training data and corresponding boxes. For more information on typical data augmentation techniques used in 3-D object detection workflows with lidar data, see the Data Augmentations for Lidar Object Detection Using Deep Learning (Lidar Toolbox).
Read and display a point cloud before augmentation using the helperShowPointCloudWith3DBoxes helper function, defined at the end of the example.
augData = preview(cds);
[ptCld,bboxes,labels] = deal(augData{1},augData{2},augData{3});
% Define the classes for object detection.
classNames = {'Car','Truck'};
% Define colors for each class to plot bounding boxes.
colors = {'green','magenta'};
helperShowPointCloudWith3DBoxes(ptCld,bboxes,labels,classNames,colors)
Use the sampleLidarData (Lidar Toolbox) function to sample 3-D bounding boxes and their corresponding points from the training data.
sampleLocation = fullfile(outputFolder,'GTsamples'); [ldsSampled,bdsSampled] = sampleLidarData(cds,classNames,'MinPoints',20,... 'Verbose',false,'WriteLocation',sampleLocation); cdsSampled = combine(ldsSampled,bdsSampled);
Use the pcBboxOversample (Lidar Toolbox) function to randomly add a fixed number of car and truck class objects to every point cloud. Use the transform function to apply the ground truth and custom data augmentations to the training data.
numObjects = [10 10]; cdsAugmented = transform(cds,@(x)pcBboxOversample(x,cdsSampled,classNames,numObjects));
Apply following additional data augmentation techniques to every point cloud using the helperAugmentLidarData helper function, defined at the end of the example:
Random scaling by 5 percent
Random rotation along the z-axis from [-pi/4, pi/4]
Random translation by [0.2, 0.2, 0.1] meters along the x-, y-, and z-axis respectively
cdsAugmented = transform(cdsAugmented,@(x)helperAugmentData(x));
Display an augmented point cloud along with the ground truth augmented boxes using the helperShowPointCloudWith3DBoxes helper function, defined at the end of the example.
augData = preview(cdsAugmented);
[ptCld,bboxes,labels] = deal(augData{1},augData{2},augData{3});
helperShowPointCloudWith3DBoxes(ptCld,bboxes,labels,classNames,colors)
Create PointPillars Object Detector
Use the pointPillarsObjectDetector (Lidar Toolbox) function to create a PointPillars object detection network. For more information on PointPillars network, see Get Started with PointPillars (Lidar Toolbox).
The diagram shows the network architecture of a PointPillars object detector. You can use the Deep Network Designer App to create a PointPillars network.
Estimate the anchor boxes from training data using calculateAnchorsPointPillars helper function, attached to this example as a supporting file.
anchorBoxes = calculateAnchorsPointPillars(trainLabels);
Define the PointPillars detector.
detector = pointPillarsObjectDetector(pointCloudRange,classNames,anchorBoxes,... 'VoxelSize',voxelSize);
Specify Training Options
Specify the network training parameters using the trainingOptions function. If training is interrupted, you can resume training from the saved checkpoint.
Train the detector using a CPU or GPU. Using a GPU requires Parallel Computing Toolbox™ and a CUDA® enabled NVIDIA® GPU. For more information, see GPU Computing Requirements (Parallel Computing Toolbox). To automatically detect if you have a GPU available, set executionEnvironment to "auto". If you do not have a GPU, or do not want to use one for training, set executionEnvironment to "cpu". To ensure the use of a GPU for training, set executionEnvironment to "gpu".
executionEnvironment = "auto"; options = trainingOptions('adam',... Plots = "training-progress",... MaxEpochs = 60,... MiniBatchSize = 3,... GradientDecayFactor = 0.9,... SquaredGradientDecayFactor = 0.999,... LearnRateSchedule = "piecewise",... InitialLearnRate = 0.0002,... LearnRateDropPeriod = 15,... LearnRateDropFactor = 0.8,... ExecutionEnvironment= executionEnvironment, ... PreprocessingEnvironment = 'parallel',... BatchNormalizationStatistics = 'moving',... ResetInputNormalization = false,... CheckpointFrequency = 10, ... CheckpointFrequencyUnit = 'epoch', ... CheckpointPath = userpath);
Train PointPillars Object Detector
Use the trainPointPillarsObjectDetector (Lidar Toolbox) function to train the PointPillars object detector if doTraining is "true". Otherwise, load a pretrained detector.
doTraining = false; if doTraining [detector,info] = trainPointPillarsObjectDetector(cdsAugmented,detector,options); else pretrainedDetector = load('pretrainedPointPillarsDetector.mat','detector'); detector = pretrainedDetector.detector; end
Note: The pretrained network pretrainedPointPillarsDetector.mat is trained on point cloud data captured by a Pandar 64 sensor where the ego vehicle direction is along positive y-axis.
To generate accurate detections using this pretrained network on a custom dataset,
Transform your point cloud data such that the ego vehicle moves along positive y-axis.
Restructure your data with Pandar 64 parameters by using the
pcorganize(Lidar Toolbox) function.
Generate Detections
Use the trained network to detect objects in the test data:
Read the point cloud from the test data.
Run the detector on the test point cloud to get the predicted bounding boxes and confidence scores.
Display the point cloud with bounding boxes using the
helperDisplay3DBoxesOverlaidPointCloudhelper function, defined at the end of the example.
ptCloud = testData{1,1};
% Run the detector on the test point cloud.
[bboxes,score,labels] = detect(detector,ptCloud);
% Display the predictions on the point cloud.
helperShowPointCloudWith3DBoxes(ptCloud,bboxes,labels,classNames,colors)
Evaluate Detector Using Test Set
Evaluate the trained object detector on a large set of point cloud data to measure the performance.
numInputs = 50; % Generate rotated rectangles from the cuboid labels. bds = boxLabelDatastore(testLabels(1:numInputs,:)); groundTruthData = transform(bds,@(x)createRotRect(x)); detectionResults = detect(detector,testData(1:numInputs,:),... 'Threshold',0.25); % Convert the bounding boxes to rotated rectangles format and calculate % the evaluation metrics. for i = 1:height(detectionResults) box = detectionResults.Boxes{i}; detectionResults.Boxes{i} = box(:,[1,2,4,5,9]); detectionResults.Labels{i} = detectionResults.Labels{i}'; end % Evaluate the object detector using average orietation similarity metric metrics = evaluateObjectDetection(detectionResults,groundTruthData,"AdditionalMetrics","AOS"); metrics.ClassMetrics
ans=2×8 table
NumObjects APOverlapAvg AP Precision Recall AOSOverlapAvg AOS OrientationSimilarity
__________ ____________ __________ _______________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________ ____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________ _____________ __________ ____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________
Car 480 0.76503 {[0.7650]} {[1 1 0.5000 0.6667 0.7500 0.8000 0.8333 0.8571 0.8750 0.8889 0.8000 0.8182 0.8333 0.8462 0.8571 0.8667 0.8750 0.8824 0.8889 0.8947 0.9000 0.9048 0.9091 0.9130 0.9167 0.9200 0.9231 0.9259 0.9286 0.9310 0.9333 0.9355 0.9375 0.9394 0.9412 0.9429 0.9444 0.9459 0.9474 0.9487 0.9500 0.9512 0.9524 0.9535 0.9545 0.9556 0.9565 0.9574 0.9375 0.9388 0.9400 0.9412 0.9423 0.9434 0.9444 0.9455 0.9464 0.9474 0.9310 0.9322 0.9333 0.9180 0.9194 0.9206 0.9219 0.9231 0.9242 0.9254 0.9265 0.9275 0.9286 0.9155 0.9167 0.9178 0.9189 0.9200 0.9211 0.9221 0.9231 0.9241 0.9250 0.9259 0.9268 0.9277 0.9286 0.9294 0.9302 0.9310 0.9318 0.9326 0.9333 0.9341 0.9348 0.9355 0.9362 0.9368 0.9375 0.9381 0.9388 0.9394 0.9300 0.9307 0.9314 0.9320 0.9327 0.9333 0.9340 0.9346 0.9352 0.9266 0.9273 0.9279 0.9286 0.9204 0.9211 0.9217 0.9224 0.9231 0.9237 0.9244 0.9250 0.9256 0.9262 0.9187 0.9194 0.9200 0.9127 0.9134 0.9062 0.9070 0.9077 0.9084 0.9091 0.9023 0.9030 0.9037 0.9044 0.9051 0.9058 0.9065 0.9071 0.9078 0.9085 0.9091 0.9097 0.9103 0.9110 0.9116 0.9122 0.9128 0.9133 0.9139 0.9145 0.9085 0.9026 0.9032 0.9038 0.9045 0.9051 0.9057 0.9062 0.9068 0.9074 0.9080 0.9085 0.9030 0.9036 0.9042 0.9048 0.9053 0.9059 0.9064 0.9070 0.9075 0.9080 0.9086 0.9091 0.9096 0.9101 0.9106 0.9111 0.9116 0.9121 0.9126 0.9130 0.9135 0.9086 0.9091 0.9096 0.9048 0.9053 0.9058 0.9062 0.9067 0.9072 0.9077 0.9082 0.9086 0.9091 0.9095 0.9100 0.9104 0.9109 0.9113 0.9118 0.9122 0.9126 0.9082 0.9087 0.9091 0.9095 0.9100 0.9104 0.9108 0.9112 0.9116 0.9120 0.9124 0.9128 0.9132 0.9136 0.9140 0.9144 0.9148 0.9152 0.9156 0.9159 0.9163 0.9167 0.9170 0.9174 0.9177 0.9181 0.9185 0.9188 0.9191 0.9195 0.9198 0.9202 0.9205 0.9208 0.9212 0.9174 0.9177 0.9180 0.9184 0.9146 0.9109 0.9113 0.9116 0.9120 0.9124 0.9127 0.9130 0.9134 0.9137 0.9141 0.9105 0.9109 0.9112 0.9115 0.9119 0.9122 0.9125 0.9129 0.9094 0.9098 0.9101 0.9104 0.9108 0.9111 0.9114 0.9118 0.9121 0.9124 0.9127 0.9130 0.9134 0.9137 0.9140 0.9143 0.9146 0.9149 0.9152 0.9155 0.9158 0.9161 0.9164 0.9167 0.9170 0.9172 0.9175 0.9178 0.9181 0.9150 0.9119 0.9122 0.9125 0.9128 0.9130 0.9133 0.9136 0.9139 0.9142 0.9145 0.9148 0.9150 0.9121 0.9123 0.9126 0.9129 0.9132 0.9135 0.9137 0.9140 0.9143 0.9146 0.9148 0.9151 0.9154 0.9156 0.9159 0.9161 0.9164 0.9167 0.9169 0.9172 0.9174 0.9177 0.9149 0.9152 0.9154 0.9157 0.9159 0.9162 0.9164 0.9167 0.9169 0.9172 0.9174 0.9147 0.9150 0.9152 0.9125 0.9128 0.9130 0.9104 0.9107 0.9109 0.9112 0.9114 0.9117 0.9119 0.9122 0.9096 0.9099 0.9101 0.9104 0.9106 0.9081 0.9083 0.9086 0.9061 0.9036 0.9038 0.9041 0.9044 0.9046 0.9049 0.9051 0.9054 0.9057 0.9059 0.9062 0.9037 0.9040 0.9043 0.9045 0.9048 0.9050 0.9053 0.9055 0.9058 0.9034 0.9010 0.9013 0.9016 0.9018 0.9021 0.9023 0.9026 0.9028 0.9031 0.9008 0.9010 0.9013 0.9015 0.8992 0.8970 0.8947 0.8925 0.8928 0.8930 0.8908 0.8911 0.8914 0.8892 0.8894 0.8897 0.8900 0.8902 0.8881 0.8883 0.8862 0.8865 0.8867 0.8846 0.8849 0.8852 0.8831 0.8810 0.8789 0.8768 0.8771 0.8750 0.8729 0.8709 0.8712 0.8715 0.8695 0.8698 0.8677 0.8657 0.8661 0.8641 0.8644 0.8647 0.8627 0.8630 0.8633 0.8636 0.8617 0.8597 0.8600 0.8581 0.8584 0.8565 0.8568 0.8549 0.8530 0.8511 0.8492 0.8473 0.8455 0.8458 0.8440 0.8443 0.8425 0.8406 0.8388 0.8370 0.8351 0.8355 0.8359 0.8341 0.8344 0.8348 0.8351 0.8333 0.8316 0.8298 0.8280 0.8263 0.8245 0.8228 0.8211 0.8193 0.8197 0.8180 0.8163 0.8146 0.8129 0.8112 0.8095 0.8079 0.8062 0.8045 0.8049 0.8053 0.8057 0.8061 0.8045 0.8028 0.8012 0.7996 0.8000 0.7984 0.7968 0.7952 0.7936 0.7920 0.7904 0.7908 0.7893 0.7877 0.7861 0.7846 0.7830 0.7815 0.7800 0.7784 0.7769 0.7754 0.7739 0.7724 0.7728 0.7733 0.7718 0.7703 0.7688 0.7673 0.7658 0.7644 0.7629 0.7615 0.7619 0.7605 0.7609 0.7595 0.7580 0.7566 0.7552 0.7538 0.7523 0.7509 0.7495 0.7481 0.7467 0.7454 0.7440 0.7426 0.7412 0.7399 0.7403 0.7390 0.7376 0.7363 0.7349 0.7336 0.7322 0.7309 0.7296 0.7301 0.7288 0.7274 0.7261 0.7248 0.7235 0.7222 0.7227 0.7214 0.7201 0.7189 0.7176 0.7163 0.7150 0.7138 0.7125 0.7113 0.7100 0.7088 0.7075 0.7063 0.7051 0.7038 0.7026 0.7014 0.7002 0.6990 0.6978 0.6983 0.6971 0.6959 0.6947 0.6935 0.6923 0.6911 0.6899 0.6888 0.6876 0.6864 0.6853 0.6858 0.6847]} {[0 0.0021 0.0021 0.0042 0.0063 0.0083 0.0104 0.0125 0.0146 0.0167 0.0167 0.0187 0.0208 0.0229 0.0250 0.0271 0.0292 0.0312 0.0333 0.0354 0.0375 0.0396 0.0417 0.0437 0.0458 0.0479 0.0500 0.0521 0.0542 0.0563 0.0583 0.0604 0.0625 0.0646 0.0667 0.0688 0.0708 0.0729 0.0750 0.0771 0.0792 0.0813 0.0833 0.0854 0.0875 0.0896 0.0917 0.0938 0.0938 0.0958 0.0979 0.1000 0.1021 0.1042 0.1062 0.1083 0.1104 0.1125 0.1125 0.1146 0.1167 0.1167 0.1187 0.1208 0.1229 0.1250 0.1271 0.1292 0.1313 0.1333 0.1354 0.1354 0.1375 0.1396 0.1417 0.1437 0.1458 0.1479 0.1500 0.1521 0.1542 0.1562 0.1583 0.1604 0.1625 0.1646 0.1667 0.1688 0.1708 0.1729 0.1750 0.1771 0.1792 0.1812 0.1833 0.1854 0.1875 0.1896 0.1917 0.1938 0.1938 0.1958 0.1979 0.2000 0.2021 0.2042 0.2062 0.2083 0.2104 0.2104 0.2125 0.2146 0.2167 0.2167 0.2188 0.2208 0.2229 0.2250 0.2271 0.2292 0.2313 0.2333 0.2354 0.2354 0.2375 0.2396 0.2396 0.2417 0.2417 0.2437 0.2458 0.2479 0.2500 0.2500 0.2521 0.2542 0.2562 0.2583 0.2604 0.2625 0.2646 0.2667 0.2687 0.2708 0.2729 0.2750 0.2771 0.2792 0.2812 0.2833 0.2854 0.2875 0.2896 0.2896 0.2896 0.2917 0.2938 0.2958 0.2979 0.3000 0.3021 0.3042 0.3063 0.3083 0.3104 0.3104 0.3125 0.3146 0.3167 0.3187 0.3208 0.3229 0.3250 0.3271 0.3292 0.3312 0.3333 0.3354 0.3375 0.3396 0.3417 0.3438 0.3458 0.3479 0.3500 0.3521 0.3521 0.3542 0.3563 0.3563 0.3583 0.3604 0.3625 0.3646 0.3667 0.3688 0.3708 0.3729 0.3750 0.3771 0.3792 0.3812 0.3833 0.3854 0.3875 0.3896 0.3917 0.3917 0.3937 0.3958 0.3979 0.4000 0.4021 0.4042 0.4062 0.4083 0.4104 0.4125 0.4146 0.4167 0.4188 0.4208 0.4229 0.4250 0.4271 0.4292 0.4313 0.4333 0.4354 0.4375 0.4396 0.4417 0.4437 0.4458 0.4479 0.4500 0.4521 0.4542 0.4562 0.4583 0.4604 0.4625 0.4625 0.4646 0.4667 0.4688 0.4688 0.4688 0.4708 0.4729 0.4750 0.4771 0.4792 0.4813 0.4833 0.4854 0.4875 0.4875 0.4896 0.4917 0.4938 0.4958 0.4979 0.5000 0.5021 0.5021 0.5042 0.5062 0.5083 0.5104 0.5125 0.5146 0.5167 0.5188 0.5208 0.5229 0.5250 0.5271 0.5292 0.5312 0.5333 0.5354 0.5375 0.5396 0.5417 0.5437 0.5458 0.5479 0.5500 0.5521 0.5542 0.5563 0.5583 0.5604 0.5604 0.5604 0.5625 0.5646 0.5667 0.5687 0.5708 0.5729 0.5750 0.5771 0.5792 0.5813 0.5833 0.5833 0.5854 0.5875 0.5896 0.5917 0.5938 0.5958 0.5979 0.6000 0.6021 0.6042 0.6062 0.6083 0.6104 0.6125 0.6146 0.6167 0.6188 0.6208 0.6229 0.6250 0.6271 0.6271 0.6292 0.6312 0.6333 0.6354 0.6375 0.6396 0.6417 0.6438 0.6458 0.6479 0.6479 0.6500 0.6521 0.6521 0.6542 0.6562 0.6562 0.6583 0.6604 0.6625 0.6646 0.6667 0.6687 0.6708 0.6708 0.6729 0.6750 0.6771 0.6792 0.6792 0.6813 0.6833 0.6833 0.6833 0.6854 0.6875 0.6896 0.6917 0.6937 0.6958 0.6979 0.7000 0.7021 0.7042 0.7042 0.7063 0.7083 0.7104 0.7125 0.7146 0.7167 0.7188 0.7208 0.7208 0.7208 0.7229 0.7250 0.7271 0.7292 0.7312 0.7333 0.7354 0.7375 0.7375 0.7396 0.7417 0.7438 0.7438 0.7438 0.7438 0.7438 0.7458 0.7479 0.7479 0.7500 0.7521 0.7521 0.7542 0.7562 0.7583 0.7604 0.7604 0.7625 0.7625 0.7646 0.7667 0.7667 0.7688 0.7708 0.7708 0.7708 0.7708 0.7708 0.7729 0.7729 0.7729 0.7729 0.7750 0.7771 0.7771 0.7792 0.7792 0.7792 0.7812 0.7812 0.7833 0.7854 0.7854 0.7875 0.7896 0.7917 0.7917 0.7917 0.7937 0.7937 0.7958 0.7958 0.7979 0.7979 0.7979 0.7979 0.7979 0.7979 0.7979 0.8000 0.8000 0.8021 0.8021 0.8021 0.8021 0.8021 0.8021 0.8042 0.8063 0.8063 0.8083 0.8104 0.8125 0.8125 0.8125 0.8125 0.8125 0.8125 0.8125 0.8125 0.8125 0.8125 0.8146 0.8146 0.8146 0.8146 0.8146 0.8146 0.8146 0.8146 0.8146 0.8146 0.8167 0.8187 0.8208 0.8229 0.8229 0.8229 0.8229 0.8229 0.8250 0.8250 0.8250 0.8250 0.8250 0.8250 0.8250 0.8271 0.8271 0.8271 0.8271 0.8271 0.8271 0.8271 0.8271 0.8271 0.8271 0.8271 0.8271 0.8271 0.8292 0.8313 0.8313 0.8313 0.8313 0.8313 0.8313 0.8313 0.8313 0.8313 0.8333 0.8333 0.8354 0.8354 0.8354 0.8354 0.8354 0.8354 0.8354 0.8354 0.8354 0.8354 0.8354 0.8354 0.8354 0.8354 0.8354 0.8354 0.8375 0.8375 0.8375 0.8375 0.8375 0.8375 0.8375 0.8375 0.8375 0.8396 0.8396 0.8396 0.8396 0.8396 0.8396 0.8396 0.8417 0.8417 0.8417 0.8417 0.8417 0.8417 0.8417 0.8417 0.8417 0.8417 0.8417 0.8417 0.8417 0.8417 0.8417 0.8417 0.8417 0.8417 0.8417 0.8417 0.8417 0.8438 0.8438 0.8438 0.8438 0.8438 0.8438 0.8438 0.8438 0.8438 0.8438 0.8438 0.8438 0.8458 0.8458]} 0.74857 {[0.7486]} {[1 0.9847 0.4924 0.6552 0.7414 0.7931 0.8275 0.8522 0.8705 0.8849 0.7964 0.8142 0.8295 0.8426 0.8538 0.8635 0.8720 0.8795 0.8862 0.8922 0.8975 0.9003 0.9031 0.9072 0.9110 0.9146 0.9178 0.9196 0.9225 0.9250 0.9269 0.9293 0.9315 0.9335 0.9355 0.9373 0.9390 0.9407 0.9420 0.9433 0.9447 0.9461 0.9473 0.9485 0.9497 0.9508 0.9519 0.9529 0.9330 0.9343 0.9347 0.9348 0.9361 0.9373 0.9382 0.9394 0.9404 0.9415 0.9252 0.9265 0.9277 0.9124 0.9138 0.9152 0.9165 0.9178 0.9190 0.9197 0.9208 0.9220 0.9231 0.9101 0.9113 0.9125 0.9137 0.9148 0.9159 0.9170 0.9181 0.9191 0.9201 0.9211 0.9221 0.9230 0.9239 0.9248 0.9257 0.9265 0.9267 0.9275 0.9281 0.9289 0.9296 0.9304 0.9311 0.9319 0.9325 0.9332 0.9339 0.9346 0.9252 0.9260 0.9267 0.9274 0.9281 0.9280 0.9287 0.9294 0.9300 0.9215 0.9222 0.9229 0.9231 0.9149 0.9156 0.9164 0.9171 0.9178 0.9185 0.9191 0.9198 0.9205 0.9211 0.9136 0.9143 0.9150 0.9077 0.9084 0.9013 0.9021 0.9029 0.9036 0.9043 0.8975 0.8983 0.8990 0.8998 0.9005 0.9012 0.9019 0.9025 0.9032 0.9039 0.9046 0.9052 0.9059 0.9065 0.9071 0.9078 0.9084 0.9088 0.9094 0.9100 0.9040 0.8981 0.8988 0.8995 0.9001 0.9007 0.9013 0.9019 0.9026 0.9032 0.9037 0.9043 0.8988 0.8994 0.9000 0.9006 0.9012 0.9017 0.9023 0.9029 0.9034 0.9040 0.9045 0.9050 0.9056 0.9061 0.9066 0.9071 0.9077 0.9081 0.9086 0.9091 0.9096 0.9047 0.9052 0.9057 0.9009 0.9015 0.9020 0.9025 0.9030 0.9035 0.9040 0.9045 0.9050 0.9054 0.9059 0.9064 0.9068 0.9073 0.9078 0.9082 0.9087 0.9091 0.9047 0.9051 0.9056 0.9060 0.9065 0.9069 0.9074 0.9076 0.9081 0.9085 0.9089 0.9093 0.9097 0.9101 0.9105 0.9109 0.9069 0.9073 0.9077 0.9081 0.9085 0.9089 0.9093 0.9097 0.9101 0.9105 0.9108 0.9112 0.9116 0.9120 0.9123 0.9127 0.9131 0.9134 0.9138 0.9100 0.9104 0.9107 0.9111 0.9074 0.9037 0.9041 0.9045 0.9049 0.9053 0.9056 0.9060 0.9064 0.9067 0.9071 0.9036 0.9039 0.9043 0.9047 0.9050 0.9054 0.9058 0.9061 0.9027 0.9028 0.9032 0.9035 0.9039 0.9042 0.9046 0.9049 0.9053 0.9056 0.9060 0.9063 0.9066 0.9070 0.9073 0.9076 0.9079 0.9082 0.9086 0.9089 0.9092 0.9095 0.9098 0.9101 0.9102 0.9105 0.9108 0.9111 0.9114 0.9083 0.9052 0.9055 0.9059 0.9062 0.9065 0.9068 0.9071 0.9074 0.9077 0.9080 0.9083 0.9086 0.9057 0.9060 0.9063 0.9066 0.9069 0.9071 0.9074 0.9077 0.9080 0.9083 0.9083 0.9086 0.9089 0.9091 0.9094 0.9097 0.9100 0.9102 0.9105 0.9108 0.9111 0.9113 0.9086 0.9088 0.9091 0.9094 0.9096 0.9099 0.9100 0.9073 0.9076 0.9078 0.9081 0.9054 0.9056 0.9059 0.9032 0.9035 0.9038 0.9012 0.9015 0.8989 0.8992 0.8994 0.8996 0.8999 0.9002 0.8976 0.8979 0.8982 0.8985 0.8988 0.8962 0.8965 0.8968 0.8943 0.8919 0.8922 0.8925 0.8928 0.8929 0.8931 0.8934 0.8936 0.8939 0.8942 0.8945 0.8921 0.8923 0.8926 0.8929 0.8932 0.8934 0.8937 0.8940 0.8943 0.8919 0.8896 0.8899 0.8902 0.8904 0.8907 0.8910 0.8912 0.8915 0.8918 0.8895 0.8898 0.8901 0.8903 0.8880 0.8858 0.8836 0.8814 0.8817 0.8820 0.8798 0.8801 0.8804 0.8782 0.8784 0.8787 0.8789 0.8792 0.8771 0.8774 0.8753 0.8755 0.8758 0.8737 0.8740 0.8743 0.8722 0.8701 0.8681 0.8660 0.8662 0.8642 0.8621 0.8601 0.8604 0.8608 0.8588 0.8591 0.8571 0.8551 0.8554 0.8535 0.8538 0.8539 0.8520 0.8500 0.8504 0.8507 0.8488 0.8469 0.8472 0.8453 0.8457 0.8438 0.8441 0.8422 0.8404 0.8385 0.8366 0.8348 0.8329 0.8333 0.8315 0.8318 0.8300 0.8282 0.8264 0.8246 0.8228 0.8210 0.8214 0.8197 0.8200 0.8204 0.8186 0.8169 0.8151 0.8134 0.8117 0.8099 0.8082 0.8065 0.8048 0.8031 0.8035 0.8018 0.8001 0.7985 0.7968 0.7952 0.7935 0.7919 0.7902 0.7886 0.7890 0.7895 0.7899 0.7903 0.7887 0.7871 0.7855 0.7839 0.7843 0.7828 0.7812 0.7796 0.7781 0.7765 0.7749 0.7754 0.7739 0.7723 0.7708 0.7693 0.7677 0.7662 0.7647 0.7632 0.7617 0.7602 0.7588 0.7573 0.7578 0.7582 0.7568 0.7553 0.7538 0.7524 0.7510 0.7495 0.7481 0.7467 0.7452 0.7438 0.7443 0.7429 0.7415 0.7401 0.7387 0.7373 0.7359 0.7345 0.7331 0.7318 0.7304 0.7291 0.7277 0.7264 0.7250 0.7237 0.7241 0.7228 0.7215 0.7201 0.7188 0.7175 0.7162 0.7149 0.7136 0.7141 0.7128 0.7115 0.7102 0.7090 0.7077 0.7064 0.7052 0.7039 0.7026 0.7014 0.7001 0.6989 0.6977 0.6964 0.6952 0.6940 0.6928 0.6915 0.6903 0.6891 0.6879 0.6867 0.6855 0.6843 0.6832 0.6820 0.6808 0.6813 0.6802 0.6790 0.6778 0.6767 0.6755 0.6744 0.6732 0.6721 0.6709 0.6698 0.6687 0.6692 0.6681]}
Truck 31 0.63687 {[0.6369]} {[ 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0.9286 0.8667 0.8125 0.8235 0.8333 0.8421 0.8500 0.8571 0.8636 0.8261 0.8333 0.8400 0.8077 0.7778 0.7500 0.7241 0.7000 0.6774 0.6562 0.6364 0.6176 0.6000 0.5833 0.5676 0.5526 0.5385 0.5250 0.5122 0.5000 0.4884 0.4773 0.4667 0.4565 0.4468 0.4375 0.4286 0.4200 0.4118 0.4038 0.3962 0.3889 0.3818 0.3750 0.3684 0.3621 0.3559 0.3500 0.3443 0.3387 0.3333 0.3281 0.3231 0.3182 0.3134 0.3088 0.3043 0.3000 0.2958 0.2917 0.2877 0.2838 0.2800 0.2763 0.2727 0.2692 0.2658 0.2625 0.2593 0.2561 0.2530 0.2500 0.2471 0.2442 0.2414 0.2386 0.2360 0.2333 0.2308 0.2283 0.2258 0.2234 0.2211 0.2188 0.2165 0.2143 0.2121 0.2100 0.2079 0.2059 0.2039 0.2019 0.2000 0.1981 0.1963 0.1944 0.1927 0.1909 0.1892 0.1875 0.1858 0.1842 0.1826 0.1810 0.1795 0.1780 0.1765 0.1750 0.1736 0.1721 0.1707 0.1694 0.1680 0.1667 0.1654 0.1641 0.1628 0.1615 0.1603 0.1591 0.1579 0.1567 0.1556 0.1544 0.1533 0.1522 0.1511 0.1500 0.1489 0.1479 0.1469 0.1458 0.1448 0.1438 0.1429 0.1419 0.1409 0.1400 0.1391 0.1382 0.1373 0.1364 0.1355 0.1346 0.1338 0.1329 0.1321 0.1313 0.1304 0.1296 0.1288 0.1280 0.1273 0.1265 0.1257 0.1250 0.1243 0.1235 0.1228 0.1221 0.1214 0.1207 0.1200 0.1193 0.1186 0.1180 0.1173 0.1167 0.1160 0.1154 0.1148 0.1141 0.1135 0.1129 0.1123]} {[ 0 0.0323 0.0645 0.0968 0.1290 0.1613 0.1935 0.2258 0.2581 0.2903 0.3226 0.3548 0.3871 0.4194 0.4194 0.4194 0.4194 0.4516 0.4839 0.5161 0.5484 0.5806 0.6129 0.6129 0.6452 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774 0.6774]} 0.6105 {[0.6105]} {[ 1 0.9997 0.9996 0.9989 0.9992 0.9993 0.9994 0.9993 0.9992 0.9992 0.9993 0.9993 0.9960 0.9961 0.9249 0.8633 0.8093 0.8204 0.8303 0.8392 0.8470 0.8542 0.8608 0.8234 0.8297 0.8365 0.8043 0.7745 0.7468 0.7211 0.6971 0.6746 0.6535 0.6337 0.6151 0.5975 0.5809 0.5652 0.5503 0.5362 0.5228 0.5100 0.4979 0.4863 0.4753 0.4647 0.4546 0.4449 0.4357 0.4268 0.4182 0.4100 0.4021 0.3946 0.3873 0.3802 0.3734 0.3669 0.3605 0.3544 0.3485 0.3428 0.3373 0.3319 0.3267 0.3217 0.3168 0.3121 0.3075 0.3031 0.2987 0.2945 0.2904 0.2865 0.2826 0.2788 0.2752 0.2716 0.2681 0.2647 0.2614 0.2582 0.2550 0.2519 0.2489 0.2460 0.2432 0.2404 0.2376 0.2350 0.2324 0.2298 0.2273 0.2249 0.2225 0.2201 0.2178 0.2156 0.2134 0.2112 0.2091 0.2070 0.2050 0.2030 0.2011 0.1992 0.1973 0.1954 0.1936 0.1919 0.1901 0.1884 0.1867 0.1851 0.1834 0.1818 0.1803 0.1787 0.1772 0.1757 0.1743 0.1728 0.1714 0.1700 0.1686 0.1673 0.1660 0.1647 0.1634 0.1621 0.1609 0.1596 0.1584 0.1572 0.1561 0.1549 0.1538 0.1526 0.1515 0.1504 0.1494 0.1483 0.1473 0.1462 0.1452 0.1442 0.1432 0.1423 0.1413 0.1403 0.1394 0.1385 0.1376 0.1367 0.1358 0.1349 0.1340 0.1332 0.1324 0.1315 0.1307 0.1299 0.1291 0.1283 0.1275 0.1267 0.1260 0.1252 0.1245 0.1237 0.1230 0.1223 0.1216 0.1209 0.1202 0.1195 0.1188 0.1181 0.1175 0.1168 0.1162 0.1155 0.1149 0.1143 0.1137 0.1130 0.1124 0.1118]}
Supporting Functions
The helperDownloadPandasetData function download the Pandaset data.
function helperDownloadPandasetData(outputFolder,lidarURL) % Download the data set from the given URL to the output folder. lidarDataTarFile = fullfile(outputFolder,'Pandaset_LidarData.tar.gz'); if ~exist(lidarDataTarFile,'file') mkdir(outputFolder); disp('Downloading PandaSet Lidar driving data (5.2 GB)...'); websave(lidarDataTarFile,lidarURL); untar(lidarDataTarFile,outputFolder); end % Extract the file. if (~exist(fullfile(outputFolder,'Lidar'),'dir'))... &&(~exist(fullfile(outputFolder,'Cuboids'),'dir')) untar(lidarDataTarFile,outputFolder); end end
The helperShowPointCloudWith3DBoxes function displays the point cloud alongside its associated bounding boxes, employing distinct colors for different classes to facilitate easy differentiation.
function helperShowPointCloudWith3DBoxes(ptCld,bboxes,labels,classNames,colors) % Validate the length of classNames and colors are the same assert(numel(classNames) == numel(colors), 'ClassNames and Colors must have the same number of elements.'); % Get unique categories from labels uniqueCategories = categories(labels); % Create a mapping from category to color colorMap = containers.Map(uniqueCategories, colors); labelColor = cell(size(labels)); % Populate labelColor based on the mapping for i = 1:length(labels) labelColor{i} = colorMap(char(labels(i))); end figure; ax = pcshow(ptCld); showShape('cuboid', bboxes, 'Parent', ax, 'Opacity', 0.1, ... 'Color', labelColor, 'LineWidth', 0.5); zoom(ax,1.5); end
The helperAugmentData function applies following augmentation to the data:
Random scaling by 5 percent.
Random rotation along the z-axis from [-pi/4, pi/4].
Random translation by [0.2, 0.2, 0.1] meters along the x-, y-, and z-axis respectively.
function data = helperAugmentData(data) % Apply random scaling, rotation and translation. pc = data{1}; minAngle = -45; maxAngle = 45; % Define outputView based on the grid-size and XYZ limits. outView = imref3d([32,32,32],[-100,100],... [-100,100],[-100,100]); theta = minAngle + rand(1,1)*(maxAngle - minAngle); tform = randomAffine3d('Rotation',@() deal([0,0,1],theta),... 'Scale',[0.95,1.05],... 'XTranslation',[0,0.2],... 'YTranslation',[0,0.2],... 'ZTranslation',[0,0.1]); % Apply the transformation to the point cloud. ptCloudTransformed = pctransform(pc,tform); % Apply the same transformation to the boxes. bbox = data{2}; [bbox,indices] = bboxwarp(bbox,tform,outView); if ~isempty(indices) data{1} = ptCloudTransformed; data{2} = bbox; data{3} = data{1,3}(indices,:); end end
References
[1] Lang, Alex H., Sourabh Vora, Holger Caesar, Lubing Zhou, Jiong Yang, and Oscar Beijbom. "PointPillars: Fast Encoders for Object Detection From Point Clouds." In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 12689-12697. Long Beach, CA, USA: IEEE, 2019. https://doi.org/10.1109/CVPR.2019.01298.
[2] Hesai and Scale. PandaSet. https://scale.com/open-datasets/pandaset.
You can also select a web site from the following list:
Americas
- América Latina (Español)
- Canada (English)
- United States (English)
Europe
- Belgium (English)
- Denmark (English)
- Deutschland (Deutsch)
- España (Español)
- Finland (English)
- France (Français)
- Ireland (English)
- Italia (Italiano)
- Luxembourg (English)
- Netherlands (English)
- Norway (English)
- Österreich (Deutsch)
- Portugal (English)
- Sweden (English)
- Switzerland
- United Kingdom (English)