MATLAB Answers

0

How to apply Reinforcement Learning techniques using the Neural Network Toolbox R2018a?

How to apply Reinforcement Learning techniques using the Neural Network Toolbox R2018a?

3 Answers

 채택된 답변

Reinforcement Learning (RL) is an area of machine learning concerned with how software agents ought to act in an environment so as to maximize reward. We currently do not have any documentation examples for RL, but there are several ways to use it with the Neural Network Toolbox R2018a.
A) The attached code example ("nn_reinforcemen.m") uses two neural networks: one for predicting the rewards of different actions, the other for choosing actions that produce the best reward. This approach is a little unusual but valid, and is more limited than other reinforcement learning algorithms: # In reinforcement learning, you usually learn by continually interacting with an “environment” which gives you “states” and “rewards”. But in this script, the “states” (x1) and the “rewards” (t) have been precomputed. # The script starts by training a neural network to learn the reward function. “x1” seems to represent the states, and “x2” represents the actions. Most of the reinforcement learning algorithms I am familiar with do not learn the reward function. Instead, they normally interact with the environment to get rewards, and use these rewards in the loss function. # Because of the approach of learning the reward function, and training on precomputed states “x1”, the algorithm does not learn “online” like most reinforcement learning algorithms. While there some approaches to Reinforcement Learning that work offline (like imitation learning), they are usually inferior to approaches that can “explore” the space of states by repeatedly interacting with the environment.
B) The adapt function can be used to perform incremental training of a neural network as new inputs are presented. The documentation page for adapt can be found at this link: https://mathworks.com/help/releases/R2018a/nnet/ref/adapt.html
C) Furthermore, illustrations of Reinforcement Learning or the variant of Q-Learning can be found on File Exchange:
Please note that MathWorks does not control the content posted by visitors to MATLAB Central and, does not guarantee the accuracy, integrity, or quality of such content.

  0 Comments

Sign in to comment.



Starting in R2019a, you can do deep reinforcement learning directly in MATLAB and Simulink with Reinforcement Learning Toolbox, using layers available from Deep Learning Toolbox.
Please see this link for a list of MATLAB/Simulink examples.
For more information, have a look at this video, to see how to use the toolbox to control a walking robot, and this one, which shows how to invert a pendulum system based on image data.

  0 Comments

Sign in to comment.


christttttttophe 님의 답변 26 Sep 2019 21:38

Do you have any tutorials or examples with regards to time series data and reinforcement learning? It seems very promising but I can't find any examples at all and I am at a loss as to how to implement this in Matlab, especially when creating a new environment.
A simple example would just be a simulation of stock with 1 year data of let's say 1 minute historic open, close, low, high, volume values. The actions would be buy (1), hold (0) and sell (-1). The environment output would assumably be the open, low, volume of that particular time (t). There would also be some indication that the stock has been previously purchased, or sold in there. Seems relatively simple but I think I'm just missing how to get the data into the environment and then setting it up.
Thanks for checking this out.

  0 Comments

Sign in to comment.