Unable to get the camera frame and at the same time perform serial communication with the bot without excessive delay.

Hi,

I needed to perform a task where using the frames from the intel camera, I need to navigate the robot to perform collision avoidance. However, my code is unable to perform the task simultaneously. Both the .m files for serial communication of the bot (bot_serial.m) and for acquiring camera frames (values1.m) are attached.

I tried creating a function too for the camera frames and then called it in the bot_serial files but with this logic I am getting a single frame each time a condition is triggered in the bot_serial files and thus the camera cannot stream continuously and neither is the bot able to run continuously.

Please, help me with any possibility to continuously stream the camera and also run the bot at the same time. The values v1,v2 and v3 in values1.m file should be continuously updated and fed to the bot for it navigate accordingly.

Thanks.

[Ahad Maqsood]

I have been waiting for the answer as I am in the middle of a project and need to solve this problem as soon as I can, therefore, any help would be highly appreciated. Thanks

[Guillaume]

Have you calculated what bandwidth you need in order to transmit an image and send your commands at the same time over the serial port? How does it compare to your transmission rate?

[Ahad Maqsood]

It takes around 100ms from each frame to be first processed and then transmitted to the software and thus, the serial command should also be send at the same rate in order to react to the accquired frame and perform the task.

But the problem is, how to interconnect the two functions so that the variables i.e the area values of the frame (subdivided into left, middle and right) be send to the other function ( bot_serial) and using those values, the bot can move accordingly.

[Guillaume]

What I'm asking is the theoretical bandwidth required to transmit your image.

At a baud rate of 57600 bit/s with no parity and a stop bit of 1 (= 2 extra bits with the start bit), you can transmit at most 576 bytes of data in 100 ms. That's not a very large image. Assuming 1 byte per pixel that's a 24x24 image before you saturate your serial bus.

[Ahad Maqsood]

Each frame is a 720x1280 image which means it will require large number of bytes of data to be transmitted. However, since I am performing serial communication at a baud rate of 57600 which can only transfer 576 bytes per 100ms, so should I change the resolution of the image?

[Guillaume]

Well, as I said in 100 ms, the maximum you could transmit at that rate would be a 24x24 image. That's really small, downright useless for any image recognition. Your 720x1280 would take 2 and half minutes to transfer.

Serial is not suitable at all for transmitting images in real time. You need a different way to connect to your camera.

[Ahad Maqsood]

What can be the other way to connect to it and perform the operation?

[Ahad Maqsood]

I have downsized the image to 72x148 pixels. Greater pixels doesn't really matter as it is a threshold image. The frames can now be obtained much faster. The only think I need to do now is to use the values v1, v2 and v3 obtained from the frame in the other function i.e. bot_serial where I am performing serial communication.