Does integrator blocks get reset automatically at the end of a reinforcement learning episode?

So in my simulink reinforcement learning model, I'm using integrator blocks to integrate velocity to get displacement. I was wondering if during the end of a episode the states reset, does the integrate block also reset back to the initial condition or do I have to manually do that myself?

Thanks

Answer 1

[Spruha]

Hi Vincent,

I noticed you're working with an "integrator" block in a reinforcement learning model using Simulink. It's important to note that the "integrator" block does not automatically reset at the end of each episode. The "integrator" block has the “external reset” set to "None" by default. You need to implement a reset mechanism to ensure proper functionality.

Here are a few methods to achieve this:

• External Reset Signal: You can use an external signal to reset the "integrator block". This involves connecting a signal to the "reset" port of the integrator block that triggers the reset at the appropriate time. For detailed instructions on setting this up, refer to this documentation: https://www.mathworks.com/help/simulink/slref/integrator.html#f7-890704_sep_f7-1513475
• Custom Environment Reset Function: Another approach is to configure a custom reset function for your reinforcement learning environment. This method involves writing a function that resets the environment's state, including the integrator, at the beginning of each episode. You can find more information and examples on how to implement this in this documentation: https://www.mathworks.com/help/reinforcement-learning/ug/create-matlab-environments-using-custom-functions.html
• MATLAB Function Blocks: You can also implement the reset logic using "MATLAB Function" Blocks within Simulink. This involves writing a MATLAB function that includes the reset logic and using the function block to integrate it into your model. For guidance on using MATLAB Function Blocks, refer to this documentation: https://www.mathworks.com/help/simulink/ug/what-is-a-matlab-function-block.html

Hope this helps.