Code generation of a trained reinforcement learning agent

Hi,

i want to integrate a trained reinforcement learning agent to a microcontroller.

The training takes place in simulink.

For the codegeneration I am using this workaround: https://de.mathworks.com/matlabcentral/answers/757154-how-to-implement-reinforcement-learning-using-code-generation.

The microcontroller only has a single precision FPU.

How do I setup the agent, that the agent uses single precision and not double presicion, especially if the observations are also of the single data type?

Answer 1

[Sayan Saha]

Hi Allmo,

Since you are targeting a micro-controller to deploy code you can generate plain C/C++ code from the trained RL agent following the approaches in https://www.mathworks.com/help/reinforcement-learning/ug/deploy-trained-reinforcement-learning-agents.html. You'll have to use the library-free deep learning configuration object to generate plain C/C++ code as mentioned in https://www.mathworks.com/help/coder/ug/generate-generic-cc-code-for-deep-learning-networks.html :

dlconfig = coder.DeepLearningConfig(TargetLibrary='none');

While the input data to the entry-point function can be of non-single type, the data will always be casted to single precision before invoking the inference method on the network. You can specify the input data to be of single precision so that generated code will only have single precision data without any typecasting.

https://www.mathworks.com/support/search.html/videos/generate-generic-c-c-code-for-deep-learning-networks-in-simulink-1622706081351.html?fq[]=asset_type_name:video&fq[]=category:simulink/index&page=1 will be helpful if you are planning to deploy from Simulink directly.

~Sayan

[Allmo]

Hi Sayan,

thank you for your answer.

I created a small model with the observations and the generated policy function and the code generation for single precision worked.

Is there a reason, why the network is default generated with single precision?

BR Allmo

[Sayan Saha]

Great that it worked for you.

Single precision is the one common datatype across all different deep learning libraries like CuDNN/TensorRT/MKLDNN/... Many embedded targets also comes with only single precision FPUs. Even in MATLAB all the deep learning processing happens only in single precision.

We do support generating code for quantized deep learning networks as well where the datatype is INT8. You can find examples for both CPU and GPU target in https://www.mathworks.com/help/deeplearning/quantization.html however this is not yet supported for generic C/C++ codegen with TargetLibrary='none'.

~Sayan

[Allmo]

Thank you for the information!