Feed data into Neural Networks file-by-file
2 vues (au cours des 30 derniers jours)
Afficher commentaires plus anciens
Currently for training my neural network, I need to load a huge dataset and it costs almost available RAM in the computer. That's why the training step (using GPU) is very slow. The pseudo-code is following:
Loading and pre-processing dataset (300 files)
Training Neural Network
So how can I don't have to load all at one time, and will be loaded in time the neural network is being trained, like this:
For file in Dataset
Loading and pre-processing file
Training Neural Network
or
Training Neural network
At some epoch:
Loading and Pre-processing a file in dataset
Continue to train.
Does MATLAB currently support this feature?
0 commentaires
Réponses (1)
Joss Knight
le 1 Nov 2019
Datastores are designed for precisely this purpose. It may be that you're after an imageDatastore processed by a transform.
0 commentaires
Voir également
Catégories
En savoir plus sur Image Data Workflows dans Help Center et File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!