readData() function error message when using mapreduce

I have a very large file (100's of millions of data points) and I am using mapreduce and datastore to get access to specifc chunks of data.

I get the following error (below) when I run it and I do not know how to go around it. lAny suggestions on how to deal with this?

Error using matlab.io.datastore.TabularTextDatastore/readData (line 81)
Mismatch between file and format character vector.
Trouble reading 'Numeric' field from file (row number 1136, field number 15) ==>
AAC,A,0,116,COM,7.63,25,24597\n

Error in matlab.io.datastore.TabularDatastore/read (line 120)
            [t, info] = readData(ds);

Error in matlab.io.datastore.internal.shim.read (line 7)
    [varargout{1 : nargout}] = read(ds);

Error in mapreduce (line 116)
    outds = execMapReduce(mrcer, ds, mapfun, reducefun, parsedStruct);

Caused by:
    Reading the variable name 'ticker' using format '%f' from file

Answer 1

[Jeremy Hughes]

ds.TextscanFormats{15} = '%q'

That will at least get you past the one line that's causing the problem here. For a more robust answer, I'd have to look at the file. It could be that there's an error in a line somewhere that's throwing off the parsing.

[Daniel Pinto]

thank you! does this imply any loss of information ?

[Jeremy Hughes]

Nope, %f is trying to read numbers, and if there isn't a number, it errors. %q reads (possibly double quoted) strings--it cannot really error.

[Daniel Pinto]

thanks!