using datastore does not generate exact data files (extra rows or rows missing)

1 vue (au cours des 30 derniers jours)
minomi
minomi le 14 Jan 2019
Hi there,
I have been trying to use datastore and apply it to the file attached (the file is an extracted section of one of our data files). The script I use to set up the datastore setting is as follows:
ds = datastore(data_file_loc_ds);
ds.ReadSize = 1000000;
ds.NumHeaderLines = 0;
ds.Delimiter = {'\t',' '};
% ds.Whitespace = {' \b\t'};
ds.MultipleDelimitersAsOne = 1;
ds.CommentStyle = {'%'}
ds.TreatAsMissing = {'-1.#IO'};
ds.TextscanFormats = {'%{dd/MM/yyyy}D','%{HH:mm:ss}D',...
'%f','%f','%f','%f','%f','%f','%f','%f','%f','%f',...
'%f','%f','%f','%f','%f','%f','%f','%f','%f','%f',...
'%f','%f','%f','%f','%f','%f','%f','%f','%f','%f',...
'%f','%f','%f','%f','%f','%f','%f','%f','%f','%f',...
'%f','%f','%f','%f','%f','%f','%f','%f','%f','%f',...
'%f','%f','%f','%f',' %f','%f','%f','%f','%f','%f',...
'%f','%f','%f','%f','%f','%f','%f','%f','%f','%f',...
'%f','%f','%f','%f','%f','%f','%f','%f','%f','%f',...
'%f','%f','%f','%f','%f','%f','%f','%f','%f','%f',...
'%f','%f','%f','%f','%f','%f','%f','%f','%f','%f',...
'%f','%f','%f','%f'};
ds.ReadVariableNames = 1;
ds.VariableNames(1:2) = {'date','time'};
preview(ds)
The created datastore seems to do different things for slightly different files. It generally tends to skip the first row (or a few) of the files. At times it repeats a section of the data file and hence for instance I end up with several extra files and sometimes it skips rows. I have been trying different combinations of the "TabularTextDatastore" properties to get the rows exactly as they are but it seems to be impossible.

Réponses (0)

Catégories

En savoir plus sur Large Files and Big Data dans Help Center et File Exchange

Produits


Version

R2017b

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by