Using datastore for a matrix

9 vues (au cours des 30 derniers jours)
Chuck Noise
Chuck Noise le 9 Sep 2023
Commenté : Catalytic le 9 Sep 2023
Hi
I am trying to set up a datastore in order to train a feedforward neural network with a trainingset that is too large for my workspace.
I have a training set, X, that is 80002x10000 which I stored in a .mat file called ¨TrainingData'. I am now trying to set up a datastore so I can load mini batches and train on those, but I can't get it working.
I have tried datastore and arrayDatastore.
datastore cannot determine the datastore type for the specified location and arrayDatastore just output the name of the .mat-file whenever i read, i.e.
>> read(ds)
ans =
1×1 cell array
{'X.mat'}
What am I doing wrong here?
EDIT
I just read that the arrayDatastore is only for in-memory data which makes it useless in my case
Thanks

Réponse acceptée

Catalytic
Catalytic le 9 Sep 2023
If you're not going to store your data in memory, then the training samples have to be split up into separate .mat files. You could then use a fileDatastore to read from and manage them.
  2 commentaires
Chuck Noise
Chuck Noise le 9 Sep 2023
Thanks @Catalytic. I'll try that :)
Catalytic
Catalytic le 9 Sep 2023
A TallDatastore might also be appropriate.

Connectez-vous pour commenter.

Plus de réponses (0)

Catégories

En savoir plus sur Statistics and Machine Learning Toolbox dans Help Center et File Exchange

Tags

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by