Matalb Academy - Reinforcement Learning Onramp: submission failed

I am taking the course titled Reinforcement Learning Onramp on MATLAB Academy and everything works fine until section 3.6 (Creating Default Agent Representations).

When I submit my answer, the system showed the given answer is wrong.

Then I tried the code from the solution, but the result was the same.

So, I cannot proceed from here.

Screenshot has been attached.

Answer 1

[Matt Tearle]

There was a change in R2021a that caused an incompatibility. We have a fix ready that will go out with the next update to the training course content. In the meantime, use this workaround to enable you to keep working through the Onramp:

layers = [
    imageInputLayer([28 28 1],'Name','input','Normalization',"none")
    averagePooling2dLayer(2,'Stride',2,'Name','avpool1')
    averagePooling2dLayer(2,'Stride',2,'Name','avpool2')
    averagePooling2dLayer(2,'Stride',2,'Name','avpool3')
    fullyConnectedLayer(2,'Name','fc','Weights',zeros(2,9),'Bias',zeros(2,1))
    softmaxLayer('Name','softmax')
    classificationLayer('Name','classOutput',"Classes",["a" "b"])];
lgraph = layerGraph(layers)
actnet = assembleNetwork(lgraph)

[Isuru Jayarathne]

Thanks for the answer. It worked.

[Mukesh]

but where to apply this code??

[Matt Tearle]

Hi Mukesh, it's probably worth clarifying...

The solution file is what you should do in practice to get the network from the agent (actnet = getModel(actor)). As you can see from Isuru's screenshot, it does get the network. However, there were some changes in how the networks were represented (from 20b to 21a), which messes with what the assessment code expected.

So, to get past this task, use the code given above instead of the correct solution code (actnet = getModel(actor)). This just manually creates a network that will pass the assessment code.

[mayur waghchoure]

Thanks

[Pradyumna Joshi]

Thanks.

[Pradyumna Joshi]

But can you just explain what the different functons in your code do?...since those used above have not appeared anywhere in the Onramp training

It would be really helpful..

Regards.

[Pradyumna Joshi]

I am talking about the "averagePooling2dLayer" and the "classificationLayer" functions..just to clarify my above comment.

[Matt Tearle]

Those are just different kinds of neural network layers. It's probably worth clarifying that this is a temporary hack to fool the grading code until the incompatibility is resolved in the next release - these are not layers you'd actually use in practice (in this application). They are layers more commonly used in deep learning for a task like image classification.

As an aside, there are cool RL applications that combine deep learning for images with RL, such as playing video games or controlling a robot/vehicle from video input (eg self-driving cars).

Answer 2

[Pavankumar Patil]

Use the sim function to run multiple simulations of the model, using the simulation options in opts.The environment and RL agent are stored in the variables env and agent, respectively.save the result of the simulation to a variable called simout.