Using environment reset function: Access Simulink workspace for if then tests
Afficher commentaires plus anciens
I'm attempting to code a PPO RL agent in which a glider is released and attempts to navigate to a target within "R997" distance. I've generated the model within Simulink including outputing the last distance value to the target ("D2Tgt") with a workspace block. For navigating to the target within a R997 value, it gets a reward. It gets other reward and penalties along the way as well. Some publications have mentioned such training cases can be speed up by holding the target location constant instead of randomizing it each training episode until the it starts getting close where you start randomizing it. This is done by starting with a large acceptable distance and decreasing it over the training time. This is what I've attempted to code below. The issue I'm having is pulling the "ans.D2Tgt" from the workspace during RL training even though I have a simulink workspace out block for this value. Any advice???
ans.D2Tgt=random initalization for now
env = rlSimulinkEnv('SimulinkModel','SimulinkModel/RL Agent',obsInfo,actInfo);
env.ResetFcn = @(in) localResetFcn(in, R997, ans);
function in = localResetFcn(in, R997, ans)
persistent n
if isempty(n)
n=0
end
ans.D2Tgt=ans.D2Tgt
DropHeight = randi([deleted values were here],1,1);
blk = 'SimulinkModel/6DOF Wind (Wind Angles)';
in = setBlockParameter(in,blk,'xme_0',mat2str([0 0 -DropHeight]));
ReleaseVelocity=randi([deleted values were here],1,1);
blk = 'SimulinkModel/6DOF Wind (Wind Angles)';
in = setBlockParameter(in,blk,'Vm_0',mat2str([ReleaseVelocity 0 0]));
if (ans.D2Tgt <= R997 * 5) && (n==0)
n=n+1
Target_X = randi([deleted values were here],1,1)
Target_Y = randi([deleted values were here],1,1)
blk = 'SimulinkModel/Constant5';
in = setBlockParameter(in,blk,'Value',mat2str([Target_X;Target_Y;0]));
blk = 'SimulinkModel/Calculate Reward/R997RewardDistance';
in = setBlockParameter(in,blk,'const',mat2str(R997 * 4));
elseif (ans.D2Tgt <= R997 * 4) && (n==1)
n=n+1
Target_X = randi([deleted values were here],1,1)
Target_Y = randi([deleted values were here],1,1)
blk = 'SimulinkModel/Constant5';
in = setBlockParameter(in,blk,'Value',mat2str([Target_X;Target_Y;0]));
blk = 'SimulinkModel/Calculate Reward/R997RewardDistance';
in = setBlockParameter(in,blk,'const',mat2str(R997 * 3));
elseif (ans.D2Tgt <= R997 * 3) && n==2
n=n+1
Target_X = randi([deleted values were here],1,1)
Target_Y = randi([deleted values were here],1,1)
blk = 'SimulinkModel/Constant5';
in = setBlockParameter(in,blk,'Value',mat2str([Target_X;Target_Y;0]));
blk = 'SimulinkModel/Calculate Reward/R997RewardDistance';
in = setBlockParameter(in,blk,'const',mat2str(R997 * 2));
elseif (ans.D2Tgt <= R997 * 2) && n==3
n=n+1
Target_X = randi([deleted values were here],1,1)
Target_Y = randi([deleted values were here],1,1)
blk = 'SimulinkModel/Constant5';
in = setBlockParameter(in,blk,'Value',mat2str([Target_X;Target_Y;0]));
blk = 'SimulinkModel/Calculate Reward/R997RewardDistance';
in = setBlockParameter(in,blk,'const',mat2str(R997 * 1));
elseif (ans.D2Tgt <= R997 * 1) && (n>=4)
n=n+1
Target_X = randi([deleted values were here],1,1)
Target_Y = randi([deleted values were here],1,1)
blk = 'SimulinkModel/Constant5';
in = setBlockParameter(in,blk,'Value',mat2str([Target_X;Target_Y;0]));
else
end
end
1 commentaire
Emmanouil Tzorakoleftherakis
le 21 Déc 2023
Apologies, I am not following the question. Are you able to use D2Tgt inside the reset function? If yes, where is the problem exactly?
Réponses (0)
Catégories
En savoir plus sur Reinforcement Learning Toolbox dans Centre d'aide et File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!