Terminate RL DDPG episode if ode solver do not have a solution

Hello Everyone,

I am building an LQR type controller using RL DDPG in MATLAB. I had question.

I am steping the eviroment using ode15s solver (my evriormen is system of DAEs which I am solving using ode15s). I want to terminate the episode if for any action in the begining the ode15s is not able to solve the enviroment. How can I do it. Any help will highly be apprciated.

Thanks,

Answer 1

[Shivam Lahoti]

Hi Muhammad Nadeem,

As per my understanding, you are stepping your environment (system of DAEs) using the ode15s solver and wish to terminate the environment if ode15s is unable to solve the environment.

The simplest solution could be to use a timestamp and a threshold time that you want the solver to run and find an answer in.

At the start of your code:

StartTime=clock;

At the end of each loop/the point you might want to exit

TimeElapsed=clock-StartTime;
if TimeElapsed(end)>10          %Set it to a value that you want (I chose 10 seconds)
return
end

If the above approach doesn’t work well then refer to the answer below.

https://in.mathworks.com/matlabcentral/answers/69677-how-to-stop-an-ode-solver-if-integration-takes-too-long

I hope this helps.

Regards,

Shivam Lahoti.

[Muhammad Nadeem]

Perfect. Thank you so much.