Will the deep learning toolbox train convert double-precision training data into single-precision during training in default?

9 vues (au cours des 30 derniers jours)
Hello,
I am using the MATLAB deep learning toolbox to build and train a neural network. I trained the network using the provided function 'trainNetwork' using my double-precision data. But the obtained trained weights are in single-precision. My question is that does MATLAB deep learning tool automatically convert the double-precision training data into single-precision first and then do the consequent training in single-precision?
If so, does the two following cases will produce the same results (ignoring the uncertainty brought by SGD)?
1. Training data in double-precision and single-precision network weights.
2. Training data in single-precision and single-precision network weights.
The MATLAB doc says that 'When you train a network using the trainNetwork function, or when you use prediction or validation functions with DAGNetwork and SeriesNetwork objects, the software performs these computations using single-precision, floating-point arithmetic.' I checked the trainNetwork function and found that the network weights were initialized in single-precision but did not see any command on changing the training data precision.
Thank you! Any feedback and suggestion will be much appreciated.

Réponses (1)

yanqi liu
yanqi liu le 17 Jan 2022
yes,sir,in trainNetwork,the precision make to single,such as
  2 commentaires
Hengfa Lu
Hengfa Lu le 17 Jan 2022
Thank you, Yanqi.
I did see the script code for setting the desired precision before. I want to make sure the setting actually works during the training, i.e., the training data did be converted to single precision before training.
I observed that for the same network architecture and hyperparameters, the training using double-precision data is obviously slower than that using single-precision data. This observation is contradicted to the setting mentioned above. Based on my understanding, if the training data did be converted to single-precision first before the training, then the training speed should not depend on the precision of my training data.
yanqi liu
yanqi liu le 18 Jan 2022
yes,sir,i think the data may be converted to single-precision first before the training

Connectez-vous pour commenter.

Catégories

En savoir plus sur Image Data Workflows dans Help Center et File Exchange

Produits


Version

R2020a

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by