I would like to use a DQN agent where I have multiple continuous states (or observations) and two action signals, each with three possible values for a total of 9 combinations. For example, see the next lines to understand what I mean:
a = [-2,0,2];
b = [-3,0,3];
[A,B] = meshgrid(a,b);
actions = reshape(cat(2,A',B'),,2);
If I want to create discrete actions, I need to convert the matrix into a cell and run the command:
actionInfo = rlFiniteSetSpec(num2cell(actions,2));
actionInfo.Name = 'actions';
Additionally, in DQN, you have a critic, which comprises of a deep neural network. I have created the critic as follows:
hiddenLayerSize = 48;
observationPath = [
imageInputLayer([numObs 1 1],'Normalization','none',...
actionPath = [
imageInputLayer([value 1 1],'Normalization','none','Name','action')
criticNetwork = layerGraph(observationPath);
criticNetwork = addLayers(criticNetwork,actionPath);
criticNetwork = connectLayers(criticNetwork,'CriticActionFC1','add/in2');
criticOpts = rlRepresentationOptions('LearnRate',1e-03,...
critic = rlRepresentation(criticNetwork,observationInfo,actionInfo,...
agentOptions = rlDQNAgentOptions(...
My problem is the first image input layer to the action path imageInputLayer([value 1 1],'Normalization','none','Name','action'). I have tried values of 1, 2, 9 and 18 for value, but all results in an error when I run
agent = rlDQNAgent(critic,agentOptions);
This is because actionInfo has a cell of 9 elements, each with a double vector of dimensions [1,2], whereas the imageInputLayer is expecting dimensions [value,1,1].
So, how can I set up a DQN agent in MATLAB with two main discrete action signals, each with three possible values?
Many thanks in advance for the help!