Feed data into Neural Networks file-by-file

2 vues (au cours des 30 derniers jours)
Le Anh
Le Anh le 31 Oct 2019
Currently for training my neural network, I need to load a huge dataset and it costs almost available RAM in the computer. That's why the training step (using GPU) is very slow. The pseudo-code is following:
Loading and pre-processing dataset (300 files)
Training Neural Network
So how can I don't have to load all at one time, and will be loaded in time the neural network is being trained, like this:
For file in Dataset
Loading and pre-processing file
Training Neural Network
or
Training Neural network
At some epoch:
Loading and Pre-processing a file in dataset
Continue to train.
Does MATLAB currently support this feature?

Réponses (1)

Joss Knight
Joss Knight le 1 Nov 2019
Datastores are designed for precisely this purpose. It may be that you're after an imageDatastore processed by a transform.

Catégories

En savoir plus sur Image Data Workflows dans Help Center et File Exchange

Produits


Version

R2019b

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by