RegressionNeuralNetwork
Description
A RegressionNeuralNetwork
object is a trained, feedforward, and
fully connected neural network for regression. The first fully connected layer of the neural
network has a connection from the network input (predictor data X
), and each
subsequent layer has a connection from the previous layer. Each fully connected layer
multiplies the input by a weight matrix (LayerWeights
) and
then adds a bias vector (LayerBiases
). An
activation function follows each fully connected layer, excluding the last (Activations
and
OutputLayerActivation
). The final fully connected layer produces the network's
output, namely predicted response values. For more information, see Neural Network Structure.
Creation
Create a RegressionNeuralNetwork
object by using fitrnet
.
Properties
Neural Network Properties
LayerSizes
— Sizes of fully connected layers
positive integer vector
This property is read-only.
Sizes of the fully connected layers in the neural network model, returned as a positive integer vector. The ith element of LayerSizes
is the number of outputs in the ith fully connected layer of the neural network model.
LayerSizes
does not include the size of the final fully connected layer.
This layer always has one output for each response variable.
Data Types: single
| double
LayerWeights
— Learned layer weights
cell array
This property is read-only.
Learned layer weights for the fully connected layers, returned as a cell array. Entry
i in the cell array corresponds to the layer weights for the
fully connected layer i. For example,
Mdl.LayerWeights{1}
returns the weights for the first fully
connected layer of the model Mdl
.
LayerWeights
includes the weights for the final fully connected layer.
Data Types: cell
LayerBiases
— Learned layer biases
cell array
This property is read-only.
Learned layer biases for the fully connected layers, returned as a cell array. Entry
i in the cell array corresponds to the layer biases for the fully
connected layer i. For example, Mdl.LayerBiases{1}
returns the biases for the first fully connected layer of the model
Mdl
.
LayerBiases
includes the biases for the final fully connected layer.
Data Types: cell
Activations
— Activation functions for fully connected layers
'relu'
| 'tanh'
| 'sigmoid'
| 'none'
| cell array of character vectors
This property is read-only.
Activation functions for the fully connected layers of the neural network model, returned as a character vector or cell array of character vectors with values from this table.
Value | Description |
---|---|
"relu" | Rectified linear unit (ReLU) function — Performs a threshold operation on each element of the input, where any value less than zero is set to zero, that is, |
"tanh" | Hyperbolic tangent (tanh) function — Applies the |
"sigmoid" | Sigmoid function — Performs the following operation on each input element: |
"none" | Identity function — Returns each input element without performing any transformation, that is, f(x) = x |
If
Activations
contains only one activation function, then it is the activation function for every fully connected layer of the neural network model, excluding the final fully connected layer, which does not have an activation function (OutputLayerActivation
).If
Activations
is an array of activation functions, then the ith element is the activation function for the ith layer of the neural network model.
Data Types: char
| cell
OutputLayerActivation
— Activation function for final fully connected layer
'none'
This property is read-only.
Activation function for final fully connected layer, returned as
'none'
.
ModelParameters
— Parameter values used to train model
NeuralNetworkParams
object
This property is read-only.
Parameter values used to train the RegressionNeuralNetwork
model,
returned as a NeuralNetworkParams
object.
ModelParameters
contains parameter values such as the
name-value arguments used to train the regression neural network model.
Access the properties of ModelParameters
by using dot
notation. For example, access the function used to initialize the fully connected
layer weights of a model Mdl
by using
Mdl.ModelParameters.LayerWeightsInitializer
.
Convergence Control Properties
ConvergenceInfo
— Convergence information
structure array
This property is read-only.
Convergence information, returned as a structure array.
Field | Description |
---|---|
Iterations | Number of training iterations used to train the neural network model |
TrainingLoss | Training mean squared error (MSE) for the returned model, or
resubLoss(Mdl) for model Mdl |
Gradient | Gradient of the loss function with respect to the weights and biases at the iteration corresponding to the returned model |
Step | Step size at the iteration corresponding to the returned model |
Time | Total time spent across all iterations (in seconds) |
ValidationLoss | Validation MSE for the returned model |
ValidationChecks | Maximum number of times in a row that the validation loss was greater than or equal to the minimum validation loss |
ConvergenceCriterion | Criterion for convergence |
History | See TrainingHistory |
Data Types: struct
TrainingHistory
— Training history
table
This property is read-only.
Training history, returned as a table.
Column | Description |
---|---|
Iteration | Training iteration |
TrainingLoss | Training mean squared error (MSE) for the model at this iteration |
Gradient | Gradient of the loss function with respect to the weights and biases at this iteration |
Step | Step size at this iteration |
Time | Time spent during this iteration (in seconds) |
ValidationLoss | Validation MSE for the model at this iteration |
ValidationChecks | Running total of times that the validation loss is greater than or equal to the minimum validation loss |
Data Types: table
Solver
— Solver used to train neural network model
'LBFGS'
This property is read-only.
Solver used to train the neural network model, returned as
'LBFGS'
. To create a RegressionNeuralNetwork
model, fitrnet
uses a limited-memory
Broyden-Fletcher-Goldfarb-Shanno quasi-Newton algorithm (LBFGS) as its loss function
minimization technique, where the software minimizes the mean squared error
(MSE).
Predictor Properties
PredictorNames
— Predictor variable names
cell array of character vectors
This property is read-only.
Predictor variable names, returned as a cell array of character vectors. The order of the elements of PredictorNames
corresponds to the order in which the predictor names appear in the training data.
Data Types: cell
CategoricalPredictors
— Categorical predictor indices
vector of positive integers | []
This property is read-only.
Categorical predictor indices, returned as a vector of positive integers. Assuming that the predictor data contains observations in rows, CategoricalPredictors
contains index values corresponding to the columns of the predictor data that contain categorical predictors. If none of the predictors are categorical, then this property is empty ([]
).
Data Types: double
ExpandedPredictorNames
— Expanded predictor names
cell array of character vectors
This property is read-only.
Expanded predictor names, returned as a cell array of character vectors. If the model uses encoding for categorical variables, then ExpandedPredictorNames
includes the names that describe the expanded variables. Otherwise, ExpandedPredictorNames
is the same as PredictorNames
.
Data Types: cell
Mu
— Predictor means
numeric vector | []
Since R2023b
This property is read-only.
Predictor means, returned as a numeric vector. If you set Standardize
to
1
or true
when
you train the neural network model, then the length of the
Mu
vector is equal to the
number of expanded predictors (see
ExpandedPredictorNames
). The
vector contains 0
values for dummy variables
corresponding to expanded categorical predictors.
If you set Standardize
to 0
or false
when you train the neural network model, then the Mu
value is an empty vector ([]
).
Data Types: double
Sigma
— Predictor standard deviations
numeric vector | []
Since R2023b
This property is read-only.
Predictor standard deviations, returned as a numeric vector. If you set
Standardize
to 1
or true
when you train the neural network model, then the length of the
Sigma
vector is equal to the number of expanded predictors (see
ExpandedPredictorNames
). The vector contains
1
values for dummy variables corresponding to expanded
categorical predictors.
If you set Standardize
to 0
or false
when you train the neural network model, then the Sigma
value is an empty vector ([]
).
Data Types: double
X
— Unstandardized predictors
numeric matrix | table
This property is read-only.
Unstandardized predictors used to train the neural network model, returned as a
numeric matrix or table. X
retains its original orientation, with
observations in rows or columns depending on the value of the
ObservationsIn
name-value argument in the call to
fitrnet
.
Data Types: single
| double
| table
Response Properties
ResponseName
— Names of response variables
character vector | cell array of character vectors
This property is read-only.
Names of the response variables, returned as a character vector or cell array of character vectors.
Data Types: char
| cell
Y
— Response data
numeric vector | numeric matrix | numeric table
This property is read-only.
Response data used to train the model, returned as a numeric vector, matrix, or
table. Each row of Y
represents the response values of the
corresponding observation in X
.
Data Types: single
| double
ResponseTransform
— Response transformation function
'none'
| function handle
Response transformation function, specified as 'none'
or a
function handle. ResponseTransform
describes how the software
transforms raw response values.
For a MATLAB® function or a function that you define, enter its function handle. For
example, you can enter Mdl.ResponseTransform =
@function
, where
function
accepts the original response values
and returns an output of the same size containing the transformed responses.
Data Types: char
| function_handle
Other Data Properties
HyperparameterOptimizationResults
— Cross-validation optimization of hyperparameters
BayesianOptimization
object | table
This property is read-only.
Cross-validation optimization of hyperparameters, specified as a BayesianOptimization
object or a table of hyperparameters and associated
values. This property is nonempty if the 'OptimizeHyperparameters'
name-value pair argument is nonempty when you create the model. The value of
HyperparameterOptimizationResults
depends on the setting of the
Optimizer
field in the
HyperparameterOptimizationOptions
structure when you create the
model.
Value of Optimizer Option | Value of HyperparameterOptimizationResults |
---|---|
"bayesopt" (default) | Object of class BayesianOptimization |
"gridsearch" or "randomsearch" | Table of hyperparameters used, observed objective function values (cross-validation loss), and rank of observations from lowest (best) to highest (worst) |
NumObservations
— Number of observations
positive numeric scalar
This property is read-only.
Number of observations in the training data stored in X
and
Y
, returned as a positive numeric scalar.
Data Types: double
RowsUsed
— Observations of original training data stored
logical vector | []
This property is read-only.
Observations of the original training data stored in the model, returned as a
logical vector. This property is empty if all observations are stored in
X
and Y
.
Data Types: logical
W
— Observation weights
numeric vector
This property is read-only.
Observation weights used to train the model, returned as an
n-by-1 numeric vector. n is the number of
observations (NumObservations
).
The software normalizes the observation weights specified in the
Weights
name-value argument so that the elements of
W
sum up to 1.
Data Types: single
| double
Object Functions
Create CompactRegressionNeuralNetwork
compact | Reduce size of machine learning model |
Create RegressionPartitionedNeuralNetwork
crossval | Cross-validate machine learning model |
Create dlnetwork
dlnetwork (Deep Learning Toolbox) | Deep learning neural network |
Interpret Prediction
lime | Local interpretable model-agnostic explanations (LIME) |
partialDependence | Compute partial dependence |
plotPartialDependence | Create partial dependence plot (PDP) and individual conditional expectation (ICE) plots |
shapley | Shapley values |
Assess Predictive Performance on New Observations
Assess Predictive Performance on Training Data
resubLoss | Resubstitution regression loss |
resubPredict | Predict responses for training data using trained regression model |
Gather Properties of Regression Neural Network Model
gather | Gather properties of Statistics and Machine Learning Toolbox object from GPU |
Examples
Train Neural Network Regression Model
Train a neural network regression model, and assess the performance of the model on a test set.
Load the carbig
data set, which contains measurements of cars made in the 1970s and early 1980s. Create a table containing the predictor variables Acceleration
, Displacement
, and so on, as well as the response variable MPG
.
load carbig cars = table(Acceleration,Displacement,Horsepower, ... Model_Year,Origin,Weight,MPG);
Remove rows of cars
where the table has missing values.
cars = rmmissing(cars);
Categorize the cars based on whether they were made in the USA.
cars.Origin = categorical(cellstr(cars.Origin)); cars.Origin = mergecats(cars.Origin,["France","Japan",... "Germany","Sweden","Italy","England"],"NotUSA");
Partition the data into training and test sets. Use approximately 80% of the observations to train a neural network model, and 20% of the observations to test the performance of the trained model on new data. Use cvpartition
to partition the data.
rng("default") % For reproducibility of the data partition c = cvpartition(height(cars),"Holdout",0.20); trainingIdx = training(c); % Training set indices carsTrain = cars(trainingIdx,:); testIdx = test(c); % Test set indices carsTest = cars(testIdx,:);
Train a neural network regression model by passing the carsTrain
training data to the fitrnet
function. For better results, specify to standardize the predictor data.
Mdl = fitrnet(carsTrain,"MPG","Standardize",true)
Mdl = RegressionNeuralNetwork PredictorNames: {'Acceleration' 'Displacement' 'Horsepower' 'Model_Year' 'Origin' 'Weight'} ResponseName: 'MPG' CategoricalPredictors: 5 ResponseTransform: 'none' NumObservations: 314 LayerSizes: 10 Activations: 'relu' OutputLayerActivation: 'none' Solver: 'LBFGS' ConvergenceInfo: [1x1 struct] TrainingHistory: [708x7 table]
Mdl
is a trained RegressionNeuralNetwork
model. You can use dot notation to access the properties of Mdl
. For example, you can specify Mdl.TrainingHistory
to get more information about the training history of the neural network model.
Evaluate the performance of the regression model on the test set by computing the test mean squared error (MSE). Smaller MSE values indicate better performance.
testMSE = loss(Mdl,carsTest,"MPG")
testMSE = 7.1092
Specify Neural Network Regression Model Architecture
Specify the structure of the neural network regression model, including the size of the fully connected layers.
Load the carbig
data set, which contains measurements of cars made in the 1970s and early 1980s. Create a matrix X
containing the predictor variables Acceleration
, Cylinders
, and so on. Store the response variable MPG
in the variable Y
.
load carbig
X = [Acceleration Cylinders Displacement Weight];
Y = MPG;
Delete rows of X
and Y
where either array has missing values.
R = rmmissing([X Y]); X = R(:,1:end-1); Y = R(:,end);
Partition the data into training data (XTrain
and YTrain
) and test data (XTest
and YTest
). Reserve approximately 20% of the observations for testing, and use the rest of the observations for training.
rng("default") % For reproducibility of the partition c = cvpartition(length(Y),"Holdout",0.20); trainingIdx = training(c); % Indices for the training set XTrain = X(trainingIdx,:); YTrain = Y(trainingIdx); testIdx = test(c); % Indices for the test set XTest = X(testIdx,:); YTest = Y(testIdx);
Train a neural network regression model. Specify to standardize the predictor data, and to have 30 outputs in the first fully connected layer and 10 outputs in the second fully connected layer. By default, both layers use a rectified linear unit (ReLU) activation function. You can change the activation functions for the fully connected layers by using the Activations
name-value argument.
Mdl = fitrnet(XTrain,YTrain,"Standardize",true, ... "LayerSizes",[30 10])
Mdl = RegressionNeuralNetwork ResponseName: 'Y' CategoricalPredictors: [] ResponseTransform: 'none' NumObservations: 319 LayerSizes: [30 10] Activations: 'relu' OutputLayerActivation: 'none' Solver: 'LBFGS' ConvergenceInfo: [1x1 struct] TrainingHistory: [1000x7 table]
Access the weights and biases for the fully connected layers of the trained model by using the LayerWeights
and LayerBiases
properties of Mdl
. The first two elements of each property correspond to the values for the first two fully connected layers, and the third element corresponds to the values for the final fully connected layer for regression. For example, display the weights and biases for the first fully connected layer.
Mdl.LayerWeights{1}
ans = 30×4
0.0123 0.0117 -0.0094 0.1175
-0.4081 -0.7849 -0.7201 -2.1720
0.6041 0.1680 -2.3952 0.0934
-3.2332 -2.8360 -1.8264 -1.5723
0.5851 1.5370 1.4623 0.6742
-0.2106 1.2830 -1.7489 -1.5556
0.4800 0.1012 -1.0044 -0.7959
1.8015 -0.5272 -0.7670 0.7496
-1.1428 -0.9902 0.2436 1.2288
-0.0833 -2.4265 0.8388 1.8597
⋮
Mdl.LayerBiases{1}
ans = 30×1
-0.4450
-0.8751
-0.3872
-1.1345
0.4499
-2.1555
2.2111
1.2040
-1.4595
0.4639
⋮
The final fully connected layer has one output. The number of layer outputs corresponds to the first dimension of the layer weights and layer biases.
size(Mdl.LayerWeights{end})
ans = 1×2
1 10
size(Mdl.LayerBiases{end})
ans = 1×2
1 1
To estimate the performance of the trained model, compute the test set mean squared error (MSE) for Mdl
. Smaller MSE values indicate better performance.
testMSE = loss(Mdl,XTest,YTest)
testMSE = 18.3681
Compare the predicted test set response values to the true response values. Plot the predicted miles per gallon (MPG) along the vertical axis and the true MPG along the horizontal axis. Points on the reference line indicate correct predictions. A good model produces predictions that are scattered near the line.
testPredictions = predict(Mdl,XTest); plot(YTest,testPredictions,".") hold on plot(YTest,YTest) hold off xlabel("True MPG") ylabel("Predicted MPG")
Extended Capabilities
C/C++ Code Generation
Generate C and C++ code using MATLAB® Coder™.
Usage notes and limitations:
The
predict
object function supports code generation.
For more information, see Introduction to Code Generation.
GPU Arrays
Accelerate code by running on a graphics processing unit (GPU) using Parallel Computing Toolbox™. (since R2024b)
Usage notes and limitations:
The following object functions fully support GPU arrays:
The object functions execute on a GPU if at least one of the following applies:
The model was fitted with GPU arrays.
The predictor data that you pass to the object function is a GPU array.
The response data that you pass to the object function is a GPU array.
For more information, see Run MATLAB Functions on a GPU (Parallel Computing Toolbox).
Version History
Introduced in R2021aR2024b: Specify GPU arrays (requires Parallel Computing Toolbox)
R2023b: Model stores observations with missing predictor values
Starting in R2023b, training observations with missing predictor values are included in the X
, Y
, and W
data properties. The RowsUsed
property indicates the training observations stored in the model, rather than those used for training. Observations with missing predictor values continue to be omitted from the model training process.
In previous releases, the software omitted training observations that contained missing predictor values from the data properties of the model.
R2023b: Neural network models include standardization properties
Neural network models include Mu
and Sigma
properties that contain the means and standard deviations, respectively, used to standardize the predictors before training. The properties are empty when the fitting function does not perform any standardization.
Commande MATLAB
Vous avez cliqué sur un lien qui correspond à cette commande MATLAB :
Pour exécuter la commande, saisissez-la dans la fenêtre de commande de MATLAB. Les navigateurs web ne supportent pas les commandes MATLAB.
Select a Web Site
Choose a web site to get translated content where available and see local events and offers. Based on your location, we recommend that you select: .
You can also select a web site from the following list:
How to Get Best Site Performance
Select the China site (in Chinese or English) for best site performance. Other MathWorks country sites are not optimized for visits from your location.
Americas
- América Latina (Español)
- Canada (English)
- United States (English)
Europe
- Belgium (English)
- Denmark (English)
- Deutschland (Deutsch)
- España (Español)
- Finland (English)
- France (Français)
- Ireland (English)
- Italia (Italiano)
- Luxembourg (English)
- Netherlands (English)
- Norway (English)
- Österreich (Deutsch)
- Portugal (English)
- Sweden (English)
- Switzerland
- United Kingdom (English)