Data pre-processing function in ANN model

9 vues (au cours des 30 derniers jours)
Shing Mei Chiew
Shing Mei Chiew le 21 Oct 2022
I need to use ANN model to formulate explicit equations for predicting my data output. I preprocessed my raw data where I normalised my input variables using min-max normalization and logarithmic transformation for my output variable. After that, only I fed the data into the neural network tool in MATLAB. However, after finished training the network, only I realised that the neural network model has built-in preprocessing function which preprocess the input data and post process the output data automatically within the network. Nevertheless, the results from the training and testing is desirable and satisfactory.
Since the ANN has its built-in data preprocessing function, is the first data preprocessing step unneccessary or is it okay to have data preprocessing (manually) before feeding the data into the neural network?

Réponse acceptée

Ben
Ben le 1 Déc 2022
Are you using the Deep Learning Toolbox tools such as DAGNetwork, dlnetwork, trainNetwork and/or custom training loops? In that case the preprocessing is defined by the input layer and can be turned off, for example imageInputLayer(inputSize,Normalization="none")
If that isn't the case could you let us know which network and training functions you are using?
Whether or not it is OK to manually preprocess and let the network also do preprocessing will depend on the particular preprocessing being done. For example if you manually scale the data into the interval [0,1] then it makes no difference if the network training also does this (since the data will already be in [0,1] if you manually preprocessed).
  1 commentaire
Shing Mei Chiew
Shing Mei Chiew le 2 Déc 2022
Modifié(e) : Shing Mei Chiew le 2 Déc 2022
Thank you for your response. I did not use deep learning but only fitnet for estimation. I use tansig transfer function in the hidden layer, and purelin for the output layer.
I scale my input data manually to [0,1] initially with the following equation:
(x-xmin)/(xmax-xmin)
and output data with logarithmic function because of the skewness of data:
log(y+1) with range [1,8]
After I feeded in my "Scaled data" to the network, the preprocessed function in ANN scale my data to [-1,1] with mapminmax function.
Is it considered double job and it is no need to carry out preprocessing manually?
Yet I get a quite good accuracy and estimation.

Connectez-vous pour commenter.

Plus de réponses (0)

Catégories

En savoir plus sur AI for Signals dans Help Center et File Exchange

Produits


Version

R2022b

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by