DQN Implementation for Capacitor Bank Control in MATLAB and OpenDSS

Hello everyone,

I hope you are doing well. I am seeking guidance and assistance for a project I am currently working on. The purpose of the project is to control the switching of capacitor banks in a distribution network using a DQN agent, with the implementation planned in MATLAB. Additionally, I intend to simulate the electrical network using OpenDSS, considering it is a smart grid with a high photovoltaic penetration. The foundation of my project is based on this paper: https://doi.org/10.1016/j.segan.2023.101086.

I was wondering if any of you could provide guidance on how to approach the development of this project, especially concerning MATLAB and DQN. Also, I need assistance regarding the communication or interconnection of OpenDSS with MATLAB.

I appreciate any advice, resources, or direction you can offer. I will be attentive to your responses.

Thank you in advance.

Best regards.

[Ganesh]

Hi @Bryan,

I hope you have gone through the documentation of Implementing Deep Q-Network Agents in MATLAB, this should be a starting point for your implementation. Please find the documentation for the same below.

Deep Q-Network (DQN) Agents - MATLAB & Simulink - MathWorks India

If you are stuck with specific parts of implementation or are running into errors, kindly respond back to me.

Hope this helps!

[Bryan]

Hi @Ganesh

I want to express my gratitude for your prompt response. After reviewing the document and completing the MATLAB reinforcement learning course, I have established a solid foundation to commence my project.

Currently, I am building my environment based on "Create MATLAB Environment Using Custom Functions." At this stage, I have simplified my problem, which involves controlling a capacitor (1 or 0). The initial state comprises 99 voltages, and after taking an action, the system progresses to the next state.

ObservationInfo = rlNumericSpec([99 1]); % Voltages

ActionInfo = rlFiniteSetSpec([0 1]); % Off/On

So far, I haven't encountered any issues. However, doubts arise when defining the "set" function. I am uncertain about the decision MATLAB makes or which variable it assigns. According to my understanding, in the case of the "cart-pole," the "Action" variable is assigned -10 or 10. In my case, it would be 1 or 0. I would appreciate guidance on this part.

DSSText.command = sprintf('New Capacitor.c_1 bus1=bus_3 kVAR=600 State = %d', Action);

In a second step to approach my problem, I am attempting to control 5 capacitors (on/off of 5 capacitors). The specifications are as follows:

ObservationInfo = rlNumericSpec([99 1]); % Voltages

ActionInfo = rlFiniteSetSpec(0:32); % Off/On

When defining "ActionInfo," I based it on the MATLAB course example. Given that there are 5 capacitors and 2 possible states for each one (2^5=32, 32 possible actions), the idea is to convert "Action" into a 5-digit binary number and then assign each digit to a variable (for example, a1, a2, ..., a5).

DSSText.command = sprintf('New Capacitor.c_1 bus1=bus_3 kVAR=600 State = %d', a1);

DSSText.command = sprintf('New Capacitor.c_2 bus1=bus_23 kVAR=600 State = %d', a2);

DSSText.command = sprintf('New Capacitor.c_5 bus1=bus_12 kVAR=600 State = %d', a5);

I would appreciate guidance on whether there is a simpler way to implement this. Another idea I considered was creating a matrix with the 32 possibilities, but I didn't develop it as I didn't know how to implement it.

I consulted an AI, and it suggested the following code:

ActionInfo = rlFiniteSetSpec(5, {1, 0});

However, I am also unsure of how to implement it. Are the actions assigned to variables Action(1), ..., Action(5), and are these variables assigned to the capacitors?

In summary, the question preventing my progress is how to assign the state (0 or 1) to the capacitors (in State). I don't understand how MATLAB assigns the decision variable and how to load that decision into OpenDSS (capacitors).

To complete my problem, my goal is to control 5 capacitors at 4 times during the day. At each moment, there is a state, and an action is executed (one of the 32). Since I have not resolved the previous issues, I find it challenging to complete this last part of the problem, and my doubts also center around the actions section.

ObservationInfo = rlNumericSpec([99 4]); % Voltages

ActionInfo = rlFiniteSetSpec(); % Off/On

I appreciate any guidance you can provide, and I hope I have expressed myself clearly, ensuring that my message is not lost in translation. I will be attentive to your comments. Thank you very much.