How do I simulate a microgrid to which is applied reinforcement learning using Simulink and Simscape?

I have to simuate using Simulink and Simscape a microgrid to which is applied reinforcment learning. I am having troubles on finding a starting point to this since it is a new field and I don't find some examples on this topic.

I am trying to take inspiration from the following paper: https://www.researchgate.net/publication/326808045_Reinforcement_Learning_Based_Energy_Management_Algorithm_for_Smart_Energy_Buildings but I don't have to simulate this, I am just using it as a guideline to follow.

I have found some videos made by Mathworks on how to build a microgrid, but they use some blocks the have build and that are not on the blockset, and moreover I don't know hot to apply reinforcment learning to the microgrid.

Can somebody help me understand how to operate and/or link me some material that can help me understand how to do this simulation using Simulink and Simscape?

Thanks in advance.

[Karim Darwich]

@Flavio Clarizia Were you able to slove your problem ? I have exactly the same problem.

Answer 1

[Tony Lennon]

I'll pass this along to some people who work in reinforcment learning. In the meantime, may I suggest that you look at these resources?

EBook and Videos

Reinforcement Learning Toolbox

Use Simulink and Simscape™ models to represent an environment. Specify the observation, action, and reward signals within the model.

Create Simulink Environments for Reinforcement Learning

Load Predefined Simulink Environments

[Emmanouil Tzorakoleftherakis]

Some more links that should be helpful:

Reinforcement Learning Toolbox webinar

RL Video series from Brian Douglas

Documentation examples

Ebooks

Walking robot demo video

Inverted pendulum demo video

[Flavio Clarizia]

Thanks for answering. Isn't there any practical tutorial or example of application of reinforcement learning for energy management?

To understand better how to start my work I am looking at this tutorial by Mathworks:

https://www.mathworks.com/videos/microgrid-system-development-and-analysis-part-3-using-simscape-power-systems-to-simulate-microgrids-1522853406434.html

but it does not explain a lot on how to design a microgrid, since it focuses more on how to use Simscape, which probably is the reason for the video.

And then, even if for starting I use a similar model of this, I have no idea on how to apply to it reinforcement learning.

Do you have any good source or suggestion to give me?

Thanks again.

[Emmanouil Tzorakoleftherakis]

Hi Flavio,

We do not have an example specific to microgrid as of now. Reinforcement Learning Toolbox is still a new product and there is a lot of active work happening, including new example development. This is why I suggested the resources above, because there is a standard high-level recipe you follow every time you apply RL to a new problem. Sure some specifics like reward signal design and hyperparameters change across problems, but the template you follow is the same (environment->reward->policy+agent->training). The examples link above would be a more hands-on way to see how to set up a problem for training.

[mary]

Have you find anything new i am stuck the same place?!

[Flavio Clarizia]

Hi, I don't think there is something, but i was looking more for a guide to follow for starting.

Answer 2

[Zeyue Sun]

Hi Flavio,

I have the same question as you do. Do you konw how to apply DQN on battery energy management system?

[Flavio Clarizia]

Hi, sorry I don't know how to do it. I haven't been working on this topic for a while.