GPU Training RL Toolbox on R2022a

Hello everyone,

I am trying to train my agent with using Reinforcement Learning Toolbox Matlab2022a. Unfortunately, I couldn't run the training due to some packages were updated. rlRepresentationOptions was contain 'UseDevice','gpu' option, but the package has been changed to rlOptimizerOptions without any 'UseDevice' option. How can I run my training with using gpu? Thank you in advance.

[Kaustubh Khedkar]

Hi, I am too facing same issue. Trying to find a solution to this and will post it once I figure it out.

[Berk Agin]

Hi Kaustubh,

Thank you for your answer. I'll be looking forward to the solution. Have a nice day.

[Stefan]

This is still broken...

Answer 1

[Aashita Dutta]

Hello Berk,

According to my understanding, you are trying to train agent using Reinforcement Learning Toolbox MATLAB 2022a. Due to package upgrades, rlRepresentationOptions is not recommended, instead rlOptimizerOptions is used to train agent.

Could you please try the following workaround to train using GPU:

criticOptions = rlOptimizerOptions('LearnRate',1e-03,'GradientThreshold',1); 
criticOptions.UseDevice = 'gpu';

Please follow the instructions given in this documentation- How to training agent using GPU example to get more information.

[Berk Agin]

Hello Aashita,

Yes you are right, I am studying on RL Toolbox MATLAB 2022a. I saw and applied that package upgrades in order to use function rlOptimizerOptions instead of rlRepresentationOptions. The problem was that point, when I try to use :

criticOptions.UseDevice = 'gpu';

command, I got error about there is no any feauture in this function. I can only use Parallel Computing with using CPUs. Thank you for your response.

[FATAO ZHOU]

And I have the same situation with you, did you find out how to solve it

[Abolfazl Nejatian]

but as i check there is no property 'UseDevice' for the rlOptimizerOptions!

could you please help me on this matter.

[Bradley Fourie]

Hi everyone,

I would like to add my 2 cents since the Matlab R2022a reinforcement learning toolbox documentation is a complete mess.

I think I have figured it out:

• Step 1: figure out if you have a supported GPU with

availableGPUs = gpuDeviceCount("available")
gpuDevice(1)

• Step 2: When creating your actor and critic use the following to select the GPU (yours might differ but the UseDevice is what is important here)

actor = rlDiscreteCategoricalActor(actorNetWork,oinfo,ainfo,'UseDevice','gpu');
critic = rlValueFunction(criticNetwork,oinfo,'UseDevice','gpu');

• Step 3: Create your optimizer as usual

actorOpts = rlOptimizerOptions('LearnRate',3e-4,'GradientThreshold',1);
criticOpts = rlOptimizerOptions('LearnRate',3e-4,'GradientThreshold',1);

From what I can gather (with my last two braincells), the function call to use a GPU has moved to the newer actor and critic constructors. However, some staff guidance would be HIGHLY appreciated.

Answer 2

[Valerio]

Hi guys, I solved in this way:

% define the agent
myagent = rlPPOAgent(obsInfo, actInfo, initopts, agent_opt);
    
% try to change it into gpu
myactor = getActor(myagent);
mycritic = getCritic(myagent);
myactor.UseDevice = 'gpu';
mycritic.UseDevice = 'gpu';

in this way I could use the rlOptimizerOptions function.

hoping this might be useful, have a good day.