Double precision in deep learning

12 vues (au cours des 30 derniers jours)
bah le 28 Nov 2022
Commenté : Walter Roberson le 12 Nov 2023
Hello guys,
i want to train my LSTM using double precision but the deep learning toolbox only allows single precision. Can i change that to double precision or how do i train my neural network using double precision ?
  5 commentaires
bah le 29 Nov 2022
Thanks a lot for the information.
Walter Roberson
Walter Roberson le 12 Nov 2023
A lot of Nvidia models have double precision at 1/32 of single precision rates. Some improve to 1/24. And a few uncommon models more intended for server improve to 1/8: you have to examine the models carefully to determine the best for your purposes.

Connectez-vous pour commenter.

Réponse acceptée

Joss Knight
Joss Knight le 18 Déc 2022
This is possible with a dlnetwork but it is (currently) more of a workaround than anything else. Once your dlnetwork is ready to go, convert it to double precision:
net = dlupdate(@double, net);
net.State = dlupdate(@double, net.State);
Then make sure your input data to the network is always in double precision. Everything should then remain in double as you train and use the network.
Let me know if this doesn't work for you.
  2 commentaires
Rohan Mahala
Rohan Mahala le 12 Nov 2023
Hi Joss, can you please tell how to do this without a dlnetwork
Joss Knight
Joss Knight le 12 Nov 2023
To do this without a dlnetwork you will need to implement your network in functional form. Either way, to train it will require use of automatic differentiation and a custom training loop.

Connectez-vous pour commenter.

Plus de réponses (0)


En savoir plus sur Parallel and Cloud dans Help Center et File Exchange

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by