Effacer les filtres
Effacer les filtres

Read a datastore by logical groups

1 vue (au cours des 30 derniers jours)
Stepp Gyogi
Stepp Gyogi le 19 Août 2020
Commenté : Mohammad Sami le 3 Sep 2020
Currently, I have been able to load a tabularTextDatastore which contains about 4 year's worth of daily data for a bunch of uniquely identified products. These products have unique keys, and the export I did from the database sorts the data by key, date. Therefore, the first (say) 1000 lines will be for product 1, the next (say) 800 for product 2, etc. As you notice in my example, products don't necessarily all have data for all days, therefore I cannot read this datastore by constant chunks of N rows (which I have found in the documentation is possible).
Question: is there an easy or better way to read the data by chunks which correspond to the key which is one of the variables in the datastore?
PS: my DIY way around this would be to read by N rows, such that N is greater than the number of days for any product, and then only select the lines corresponding to a given product. This is awkward at best as it requires bookkeeping where your pointer to the last relevant row is, so I'm hoping this is not the answer.
  1 commentaire
Mohammad Sami
Mohammad Sami le 3 Sep 2020
If you are trying to calculate group statistics, you may convert the datastore into tall array and then use the groupsummary function.
(see limitation of group summary for tall arrays at the end of the documentation)

Connectez-vous pour commenter.

Réponses (0)

Catégories

En savoir plus sur Big Data Processing dans Help Center et File Exchange

Tags

Produits


Version

R2018a

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by