Problem with processing large data set
1 vue (au cours des 30 derniers jours)
Afficher commentaires plus anciens
Andrei-Ieronim Constantinescu
le 9 Déc 2020
Commenté : Andrei-Ieronim Constantinescu
le 17 Déc 2020
I want to train a BiLSTM-CNN (Bidirectional Long Short-Term Memory Convolutional Neural Network) with 203457 timeseries. I am trying to store them in a table with one line and 203457 columns, each cell containing an array with 100,000 values, but my server freezes whenever I run my code. I don’t get any error messages so I am not sure what the problem is and how to fix it. Do you have any idea why this happens and how to solve this?
Thank you!
0 commentaires
Réponses (1)
Eric Sofen
le 14 Déc 2020
I'm not sure of the BiLSTM-CNN input requirements, but for starters you're probably better off storing those timeseries as rows in a table rather than variables (columns). You've got some 20 billion total values, so you may be hitting memory limits. You may want to look into using a tall variable for out-of-memory computation.
Voir également
Catégories
En savoir plus sur Sequence and Numeric Feature Data Workflows dans Help Center et File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!