How to run a program on an csv file data in batches

1 vue (au cours des 30 derniers jours)
Tino
Tino le 17 Juin 2019
Commenté : Walter Roberson le 20 Juin 2019
Hello
I am trying to import data from csv file with size (1046520 1). Using the importdata function.
K = load('caamel.csv');
It is taking over an hour to compute my program.
How do I run my code on the excel file in batches instead of one whole data which is taking forever to see result
I will appreciate your help
thanks
Tino
  12 commentaires
Tino
Tino le 20 Juin 2019
Hi Walter, How do I then implement the computation
Thanks in advance
Tino
Walter Roberson
Walter Roberson le 20 Juin 2019
You implied that the reason that your code is taking so long is that import (but really load) is slow. I showed timing on a file 50 times larger than yours could take about 20 minutes to load() so your file would take about half a minute to load(). If you were to switch to textscan then it should take roughly 250/50 = 5 seconds. At that point you might as well read in all the data and loop over portions of it.
textscan also permits you to to specify a count to read, if you find that easier than loading everything and looping indexing.

Connectez-vous pour commenter.

Réponses (0)

Catégories

En savoir plus sur Large Files and Big Data dans Help Center et File Exchange

Tags

Produits


Version

R2019a

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by