rlNormalizer

Configure normalization for input of function approximator object

Since R2024a

Description

Use a normalizer object to configure normalization of an input channel for an actor or critic. You can also use a normalizer object to specify normalization options for transition, reward, and is-done function approximator inputs.

Note

For function approximators, use only rlNormalizer objects to normalize the input channels. Using the Normalization property of an input layer (such as featureInputLayer, imageInputLayer, sequenceInputLayer, image3dInputLayer, or pointCloudInputLayer) is not recommended.

Creation

Syntax

nrz = rlNormalizer(dim)

nrz = rlNormalizer(chanInfo)

nrz = rlNormalizer(___,Name=Value)

Description

nrz = rlNormalizer(dim) creates an rlNormalizer object with default options for an input of dimensions dim.

nrz = rlNormalizer(chanInfo) creates an rlNormalizer object with default options for an input of dimensions specified in the specification object chanInfo.

nrz = rlNormalizer(___,Name=Value) sets the properties of nrz using one or more name-value arguments. For example, rlNormalizer([1 1],Normalization="rescale-symmetric") creates a normalizer object that uses "rescale-symmetric" normalization. You can specify multiple name-value arguments.

Input Arguments

dim — Channel dimensions

vector

Channel dimensions, specified as a vector. Use this argument to specify the size of the signal that the channel carries from the environment to the agent.

Example: [3 1]

chanInfo — Channel specifications

rlNumericSpec | rlFiniteSetSpec

Channel specifications, specified as an rlFiniteSetSpec or rlNumericSpec object. The function uses only the Dimension property of chanInfo to create the normalizer object.

Example: rlNumericSpec([2 1])

Properties

Normalization — Normalization method

"none" (default) | string | character vector

Normalization method, specified as one of the following values:

• "none" — Do not normalize the input of the function approximator object.

• "zerocenter" — Normalize the input using zero-center normalization. The normalized input Y is U–Mean, where U is the nonnormalized input.

• "zscore" — Normalize the input using z-score normalization, also known as standardization. The normalized input Y is (U–Mean)./StandardDeviation, where U is the nonnormalized input.

• "rescale-zero-one" — Normalize the input by rescaling it to the interval between 0 and 1. The normalized input Y is (U–Min)./(Max–Min), where U is the nonnormalized input. Note that nonnormalized input values lower than Min result in normalized values lower than 0. Similarly, nonnormalized input values higher than Max result in normalized values higher than 1.

• "rescale-symmetric" — Normalize the input by rescaling it to the interval between –1 and 1. The normalized input Y is 2(U–Min)./(Max–Min) – 1, where U is the nonnormalized input. Note that nonnormalized input values lower than Min result in normalized values lower than –1. Similarly, nonnormalized input values higher than Max result in normalized values higher than 1.

Example: "rescale-symmetric"

Mean — Mean value

0 (default) | scalar | vector | matrix

Mean value, specified as a matrix with the same dimensions specified in dim, and with the data type double. Alternatively, specify Mean as a scalar to apply the same value to all elements of the channel. The normalizer object uses this property when Normalization is "zerocenter" or "zscore".

Example: 2

StandardDeviation — Standard deviation value

1 (default) | positive scalar | positive vector | positive matrix

Standard deviation value, specified as a matrix of positive numbers with the same dimensions specified in dim, and with the data type double. Alternatively, specify StandardDeviation as a scalar to apply the same value to all elements of the channel. The normalizer object uses this property when Normalization is "zerocenter" or "zscore".

Example: 2

Min — Minimum value

-1 (default) | scalar | vector | matrix

Minimum value, specified as a matrix with the same dimensions specified in dim, and with the data type double. Alternatively, specify Min as a scalar to apply the same value to all elements of the channel. The normalizer object uses this property when Normalization is "rescale-symmetric" or "rescale-zero-one".

Example: -10

Max — Maximum value

10 (default) | scalar | vector | matrix

Maximum value, specified as a matrix with the same dimensions specified in dim, and with the data type double. Alternatively, specify Max as a scalar to apply the same value to all elements of the channel. The normalizer object uses this property when Normalization is "rescale-symmetric" or "rescale-zero-one".

Example: Max=10

Object Functions

normalize

Normalize input data using method defined in normalizer object

Examples

Assign Normalizers to Actor and Critic Inputs

This example shows how to assign normalizer objects to the actor and critics of a DDPG agent.

Create DDPG Agent and Extract Actor and Critic

Create specification objects to define observation and action channels. For this example, the agent has two observation channels. The first channel has a vector with three elements and the second channel has a vector with four elements.

The action channel carries a two-dimensional vector.

obsInfo = [rlNumericSpec([3,1]) rlNumericSpec([4,1])];
actInfo = rlNumericSpec([2,1]);

Create a default DDPG agent.

agent = rlDDPGAgent(obsInfo, actInfo);

Extract the approximator objects.

actor = getActor(agent);
critic = getCritic(agent);

Create Normalizer Objects

Create one normalizer object for each input channel. DDPG agents use a Q-value function critic, which requires both the observation and the action as inputs. The Mean, StandardDeviation, Min, and Max properties apply to each element of the channel.

Create the normalizer for the first observation channel.

obs1Nrz = rlNormalizer(obsInfo(1).Dimension, ...
    Normalization="zscore", Mean=2, StandardDeviation=3)

obs1Nrz = 
  rlNormalizer with properties:

            Dimension: [3 1]
        Normalization: "zscore"
                 Mean: 2
    StandardDeviation: 3

Create the normalizer for the second observation channel.

obs2Nrz = rlNormalizer(obsInfo(2).Dimension, ...
    Normalization="zerocenter", Mean=4)

obs2Nrz = 
  rlNormalizer with properties:

        Dimension: [4 1]
    Normalization: "zerocenter"
             Mean: 4

Create the normalizer for the action input channel of the Q-value function critic.

actInNrz = rlNormalizer(actInfo.Dimension, ...
    Normalization="rescale-symmetric", Min=-2, Max=2)

actInNrz = 
  rlNormalizer with properties:

        Dimension: [2 1]
    Normalization: "rescale-symmetric"
              Min: -2
              Max: 2

To check how the normalizer works on an input, use normalize.

normalize(obs2Nrz,6)

ans = 
2

Assign Normalizer Objects to Actor and Critic

To assign new normalizers to the actor and critic, use setNormalizer.

actor = setNormalizer(actor, [obs1Nrz, obs2Nrz]);
critic = setNormalizer(critic, [obs1Nrz, obs2Nrz, actInNrz]);

You can also assign normalizers to selected channels only. For example, to assign normalizers only to the first observation channel (in the order specified by obsInfo) and the action channel, use an indices vector.

critic = setNormalizer(critic, [obs1Nrz actInNrz], [1 3]);

Display the normalization properties of the actor and critic.

actor.Normalization

ans = 1x2 string
    "zscore"    "zerocenter"

critic.Normalization

ans = 1x3 string
    "zscore"    "zerocenter"    "rescale-symmetric"

Assign Actor and Critic to Agent

To assign the new actor and critic to the agent, use setActor and setCritic.

setCritic(agent, critic);
setActor(agent, actor);

To check that agent works, use getAction.

a = getAction(agent, { ...
    rand(obsInfo(1).Dimension) ...
    rand(obsInfo(2).Dimension)});
a{1}

ans = 2×1

   -0.1039
    0.5166

Create DQN Agent and Extract Critic

Define a mixed observation space and a discrete action channel.

obsInfo = [rlNumericSpec([3,1]), rlFiniteSetSpec([5,3,4,2])];
actInfo = rlFiniteSetSpec([-1,0,1]);

Create a default DQN agent.

agent = rlDQNAgent(obsInfo, actInfo);

Extract the agent critic.

critic = getCritic(agent);

Extract Normalizer Objects from Actor or Critic

Use getNormalizer to extract an array of normalizer objects from the agent critic.

crtNrz = getNormalizer(critic)

crtNrz = 
  1x2 rlNormalizer array with properties:

    Dimension
    Normalization
    Mean
    StandardDeviation
    Min
    Max

Modify Normalizer Objects Using Dot Notation

Use dot notation to access and change the property of the second normalizer object (associated with the second observation channel).

crtNrz(2).Normalization="rescale-zero-one";
crtNrz(2).Min=-5;
crtNrz(2).Max=15;

Assign the modified normalizer to the critic and assign the critic to the agent.

critic = setNormalizer(critic, crtNrz);
setCritic(agent, critic);

To check that the agent works, use getAction.

a = getAction(agent, { ...
    rand(obsInfo(1).Dimension) ...
    rand(obsInfo(2).Dimension)});
a{1}

ans = 
0

Version History

Introduced in R2024a