What Is Reinforcement Learning Toolbox?

Emmanouil Tzorakoleftherakis, MathWorks

Reinforcement Learning Toolbox™ provides an app, functions, and a Simulink^® block for training policies using reinforcement learning algorithms, including DQN, PPO, SAC, and DDPG. You can use these policies to implement controllers and decision-making algorithms for complex applications such as resource allocation, robotics, and autonomous systems.

The toolbox lets you represent policies and value functions using deep neural networks or look-up tables and train them through interactions with environments modeled in MATLAB^® or Simulink. You can evaluate the single- or multi-agent reinforcement learning algorithms provided in the toolbox or develop your own. You can experiment with hyperparameter settings, monitor training progress, and simulate trained agents either interactively through the app or programmatically. To improve training performance, simulations can be run in parallel on multiple CPUs, GPUs, computer clusters, and the cloud (with Parallel Computing Toolbox™ and MATLAB Parallel Server™).

Through the ONNX™ model format, existing policies can be imported from deep learning frameworks such as TensorFlow™ Keras and PyTorch (with Deep Learning Toolbox™). You can generate optimized C, C++, and CUDA^® code to deploy trained policies on microcontrollers and GPUs. The toolbox includes reference examples to help you get started.

Reinforcement Learning Toolbox™ provides an app, functions, and blocks that let you implement controllers and decision-making algorithms for complex applications such as robotics and automated driving.

The toolbox enables you to work through all steps of the reinforcement learning workflow, from creating the environment and the agent, to policy training and deployment, with MATLAB^® & Simulink^®.

Define an agent by selecting from built-in algorithms such as DQN, PPO, and SAC, or develop your own custom algorithm.

Create the agent object interactively in Reinforcement Learning Designer, or programmatically.

For deep neural network policies, Reinforcement Learning Toolbox can automatically generate a default network architecture.

Alternatively, create deep neural network policies and value functions with Deep Network Designer…

or programmatically, with built-in functions.

In addition to neural networks, polynomials and look-up tables are also supported.

You can create environments in both MATLAB and Simulink.

In Simulink, create a model that describes the environment dynamics, observations, and reward signal.

To interface the environment model with the created agent objects use one or multiple instances of the Agent block, for single or multi-agent training respectively.

For MATLAB environments, you may start with provided templates and make modifications as needed.

You can also select from several predefined MATLAB & Simulink environments.

To begin training, specify training options such as stopping criteria and train the agent in the app or programmatically.

Parallel Computing Toolbox™ and MATLAB Parallel Server™ let you accelerate training by parallelizing simulations and calculations.

During training, the Episode manager helps you visually monitor the training progress and provides summary statistics.

After training is complete, you can verify the trained agent with the simulation environment and make modifications as needed.

You can then generate CUDA and C/C++ code to deploy the trained policy.

For more information on Reinforcement Learning Toolbox, please refer to the documentation and provided examples.