MATLAB Answers

DDPG - Noise Model - sample time step - definition

13 views (last 30 days)
Hello!
At the moment i am tuning the Parameters of my DDPG Algorithm and i don´t fully understand the Updating process of the Ornstein-Uhlenbeck Noise Model.
The Matlab Documentation describes the Process of Updating the Noise Model of a DDPG-algorithm, consisting of a Formula, which is used in every "sample time step".
But how is the "sample time step" defined? Does this correlate with the episode or step count of the RL trainingprocess?
Thanks
Niklas

  0 Comments

Sign in to comment.

Accepted Answer

Emmanouil Tzorakoleftherakis
Hi Niklas,
This post should be helpful. By "sample time step" the documentation refers to the "step count of the RL trainingprocess", i.e. each episode consists of a number of time steps, and noise is applied to the selected action at the begining of each time step. See this link for a description of DDPG (and specifically step #1). The time step value can be specified in the agent options here.

  2 Comments

Niklas Reinisch
Niklas Reinisch on 5 Aug 2019
Hello Emmanouil,
Thank you for your fast answer! It was helpful to understand the noise generation process.
But I got a consequential question, because the NoiseDecay(-Rate) is not explaned in the Link you mentioned. Am i right when saying:
?
I hope you can help me with that one as well.
Thanks
Niklas
Emmanouil Tzorakoleftherakis
Correct. This is why you want to keep the decay rate small or zero if you want to promote exploration.

Sign in to comment.

More Answers (0)

Sign in to answer this question.