Data signal processing method through real-time connection of lidar sensor in simulink (jetson, simulink)

I am trying to actually compile it on Jetson and Drone using the example in the link below.

https://kr.mathworks.com/help/supportpkg/px4/ref/uvify-ifo-s-nvidia-jetson-hitl.html

In the above example, an obstacle avoidance algorithm is created by receiving image information using a virtual camera.

So, I'm curious what method should be used when actually compiling.

In the case of a general webcamera, I think image information (R, G, B) can be received using the block below.

However, I don't know what to do when using a Lidar sensor.

There is an example code that receives Lidar data using a Matlab function.

https://kr.mathworks.com/help/supportpkg/velodynelidar/ug/log-and-visualize-3D-data-from-velodyne-lidar-sensor.html

https://kr.mathworks.com/help/supportpkg/velodynelidar/lidar-sensors-data-acquisition.html?searchHighlight=velodyne&s_tid=srchtitle_support_results_1_velodyne

However, I would like to implement an avoidance algorithm by receiving point cloud information from Lidar sensors using Simulink.

If you have any good examples or methods, please let me know.

Answer 1

[Garmit Pant]

Hello Ohrum Cha

I understand that you are trying to receive point cloud information from Lidar sensors from an NVIDIA Jetson device using Simulink.

MATLAB has the 'MATLAB Coder Support Package for NVIDIA Jetson and NVIDIA DRIVE Platforms' to automate the deployment of MATLAB algorithms and Simulink models to embedded NVIDIA Platforms. Under this support package, multiple Simulink blocks are available to directly target NVIDIA hardware from Simulink. The 'Camera' block that you have attached in your question description is available under the same support package. As of MATLAB R2023a release, support for LiDAR sensor on an NVIDIA platform is not available in Simulink as a block. You can find all the sensors that have Simulink support in the following link:

https://www.mathworks.com/help/supportpkg/nvidia/modeling.html

MATLAB provides the support to receive data from a Velodyne LiDAR® sensor from the NVIDIA DRIVE® or Jetson™ hardware. It requires the installation of the 'MATLAB Coder Support Package for NVIDIA Jetson and NVIDIA DRIVE Platforms'. The connection to the LiDAR sensor can be made by using the 'velodynelidar' function. You need to create a MATLAB entry-point function to capture the data from the LiDAR sensor and then deploy it on your NVIDIA Jetson. Refer to the following code snippet to make the function:

function outStruct = readPointCloudFrame(mdlName,calibFile,port) 
%#codegen 

% Create hwobj 
hwobj = jetson(); 

% % Create Velodyne Lidar Object for 'VLP16' make 
obj = velodynelidar(hwobj,mdlName,calibFile,'Port',port); 

% Start Receiving LiDAR Packets 
start(obj); 
if strcmp(mdlName,'VLP16') 
    lenOut = 120; 
    outStructLen = 28000; 
else 
    lenOut = 70; 
    outStructLen = 48000; 
end 
% timePause = uint8(1); 
outStruct = cell(2,lenOut); 

for i=1:lenOut 
    outStruct{1,i} = zeros(outStructLen,3); 
    outStruct{2,i} = zeros(outStructLen,1,'uint8'); 
end 

for i=1:lenOut 
    
    % Read a Point Cloud frame 
    pcFrame = read(obj); 
    
    outStruct{1,i} = pcFrame.Location(1:min(pcFrame.Count,outStructLen),:); 
    outStruct{2,i} = pcFrame.Intensity(1:min(pcFrame.Count,outStructLen)); 
    
    pause(0.1); 
    
end 


% Stop receiving LiDAR Packets 
stop(obj); 

% xyz = pcFrame.Location; 
% Intensity = pcFrame.Intensity; 

end 

For further understanding, please refer to the following MATLAB Documentation:

1. https://www.mathworks.com/help/supportpkg/nvidia/index.html?s_tid=CRUX_lftnav – Refer to get installation guide and documentation of the 'MATLAB Coder Support Package for NVIDIA Jetson and NVIDIA DRIVE Platforms'.
2. https://www.mathworks.com/help/coder/nvidia/ref/velodynelidar.html - Follow the given example to capture point cloud data from LiDAR sensor on a Jetson Platform.

I hope this helps!

Best Regards

Garmit

Answer 2

[Kilsu Kim]

From the question, I am little confused, whether you want to read Velodyne Lidar sensor data from Simulink and test your avoidance algorithm OR read from virtual Lidar sensor and process it?

There is no Simulink block to read the Velodyne Lidar sensor data into MATLAB however, we can create a MATLAB Function block using velodynelidar API and read the data into MATLAB.

For the virtual case we do have a MATLAB API and Simulink block to generate/simulate the Lidar sensor data into MATLAB. Refer: Generate Lidar Point Cloud Data for Driving Scenario with Multiple Actors - MATLAB & Simulink - MathWorks India

This example works with DrivingScenario. In R2023b, we do support lidarSensor interfacing with RoadRunner Scenario tools using MATLAB API.