Create policy evaluation function for RL agent
3 vues (au cours des 30 derniers jours)
Afficher commentaires plus anciens
Juan Nunez
le 23 Sep 2019
Commenté : Sviatoslav Klos
le 25 Mai 2020
Hello,
I'm using the Reinforcement Learning Toolbox to train an agent for a particular task. Once the training is complete, I want to generate a policy evaluation function so I can use it in a simulink file. I can create the policy function using "generatePolicyFuncion" command. The matlab script that is generated is the following:
function action1 = evaluatePolicy(observation1)
%#codegen
% Reinforcement Learning Toolbox
% Generated on: 23-Sep-2019 09:27:39
action1 = localEvaluate(observation1);
end
%% Local Functions
function action1 = localEvaluate(observation1)
persistent policy
if isempty(policy)
policy = coder.loadDeepLearningNetwork('agentData.mat','policy');
end
action1 = predict(policy,observation1);
end
When I run the simulink file, the following errors pop up:
Simulink does not have enough information to determine output sizes for this block. If you think the errors below are inaccurate, try specifying types for the block inputs and/or sizes for the block outputs.
Component:MATLAB Function | Category:Coder error
Dot indexing is not supported for variables of this type.
Function 'MATLAB Function' (#171.286.341), line 13, column 11:
"coder.loadDeepLearningNetwork('agentData.mat','policy')"
Launch diagnostic report.
Component:MATLAB Function | Category:Coder error
Persistent variable 'policy' must be assigned before it is used. The only exception is a check using 'isempty(policy)' that can be performed prior to assignment.
Function 'MATLAB Function' (#171.365.371), line 15, column 19:
"policy"
Launch diagnostic report.
Component:MATLAB Function | Category:Coder error
Function call failed.
Function 'MATLAB Function' (#171.140.167), line 7, column 11:
"localEvaluate(observation1)"
Launch diagnostic report.
Component:MATLAB Function | Category:Coder error
Errors occurred during parsing of MATLAB function 'test_RL/MATLAB Function'
Component:MATLAB Function | Category:Coder error
Simulink cannot determine sizes and/or types of the outputs for block 'test_RL/MATLAB Function' due to errors in the block body, or limitations of the underlying analysis. The errors might be inaccurate. Fix the indicated errors, or explicitly specify sizes and/or types for all block outputs.
Component:MATLAB Function | Category:Coder error
Dot indexing is not supported for variables of this type.
Function 'MATLAB Function' (#171.286.341), line 13, column 11:
"coder.loadDeepLearningNetwork('agentData.mat','policy')"
Launch diagnostic report.
Component:MATLAB Function | Category:Coder error
Persistent variable 'policy' must be assigned before it is used. The only exception is a check using 'isempty(policy)' that can be performed prior to assignment.
Function 'MATLAB Function' (#171.365.371), line 15, column 19:
"policy"
Launch diagnostic report.
Component:MATLAB Function | Category:Coder error
Function call failed.
Function 'MATLAB Function' (#171.140.167), line 7, column 11:
"localEvaluate(observation1)"
Launch diagnostic report.
Component:MATLAB Function | Category:Coder error
Errors occurred during parsing of MATLAB function 'test_RL/MATLAB Function'
How can I solve this issue?
Thanks
0 commentaires
Réponse acceptée
Emmanouil Tzorakoleftherakis
le 25 Sep 2019
Modifié(e) : Emmanouil Tzorakoleftherakis
le 25 Sep 2019
Can you try defining the size of inputs and outputs in the MATLAB Function block? This seems to be coming up a lot in the error message
3 commentaires
J
le 17 Oct 2019
Hi there,
Were you able to resolve this issuse? I am facing the same problem as you and haven't been able to figure it out...
SomeMatlabUser
le 20 Fév 2020
Modifié(e) : SomeMatlabUser
le 20 Fév 2020
Any Update? I am having the same problem. Also there is no documentation by Mathworks itself how to setup an Simulinkmodel executing the policy for code generation.
Plus de réponses (1)
Sviatoslav Klos
le 3 Mar 2020
Hello.
I experienced the same issue. If you do not need code generation (only normal mode simulation is requested) I might suggest to substitute your two functions with the following code and insert it as a MATLAB User-defined fcn. Please, respond if it suits you.
function action1 = evaluatePolicy(observation1)%, policy) %#codegen
coder.extrinsic('evalin', 'predict');
action1 = single(0);
policy = evalin('base', 'policy') ;
action1 = predict(policy,observation1);
end
2 commentaires
Sviatoslav Klos
le 25 Mai 2020
I think this is your project specific error. You might check the action of the RL agent. I might suggest to test this function with some RL example from MATLAB and the try to go into your application with obtained conclusions.
Voir également
Catégories
En savoir plus sur Block Libraries dans Help Center et File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!