Reading csv file and formating timestamp

I am starting with a csv file with 3 columns and over 3500 lines. It looks like the following:

[2011/111/23:25:42.720108,CDTime_Time_Text,-000:09:00:00.000]

[2011/111/23:25:45.167073,GN2 Initial State, 0]

[2011/111/23:25:45.184703,LN2 DCVNC-3009 State, 0]

This is: [timestamp, identifier, timestamp -or- data]

I want to read the file into Matlab and: a) timestamp - remove the year and day and keep the hr:min:sec.sec. eventually convert this to seconds b) keep the identifier as a string c) keep the timestamp or data as integer

I have tried the textread but am having problems getting the timestamp converted into seconds. here is something that I tried: [main.utc, main.fd, main.data]= textread(rawdatafile, '%s %s %s', 'delimiter', ',')

Any help would be appreciated!

Thanks, Aaron

Answer 1

[Teja Muppirala]

Part a) timestamp - remove the year and day and keep the hr:min:sec.sec. eventually convert this to seconds

Once you run this command:

[main.utc, main.fd, main.data]= textread(rawdatafile, '%s %s %s', 'delimiter', ',')

Your main.utc =

    '2011/111/23:25:42.720108'
    '2011/111/23:25:45.167073'
    '2011/111/23:25:45.184703'

Correct?

If so, then:

cellfun(@(x) [0 0 3600 60 1]*sscanf(x,'%f/%f/%f:%f:%f'),main.utc)

will strip away the year/month/day part and convert everything into seconds.

ans =

1.0e+004 *

   8.434272010799999
   8.434516707300000
   8.434518470300001

That looks about right.

Part b) keep the identifier as a string

I think that is already be done.

main.fd =

   'CDTime_Time_Text'
   'GN2 Initial State'
   'LN2 DCVNC-3009 State'

Part c) keep the timestamp or data as integer

I'm not really sure what you mean by "keep the timestamp as an integer"

[Aaron]

This solution works great. Thank you. It was late when I was working on this and Part C was written in error so please disregard it because it doesn't make sense.

[Teja Muppirala]

Your question: How would you modify this solution if the time format changes to 2011-03-10_09:48:41.560

Answer:

cellfun(@(x) [0 0 0 3600 60 1]*sscanf(x,'%f-%f-%f_%f:%f:%f'),main.utc)

Basically just mimic the way it is input.

Oleg's answer is also a good idea, because it reads the information using the proper format from the beginning (In this case since your file isn't fantastically large, it won't make too much of a difference. Either way is fine, whatever you're comfortable with.)

Answer 2

[Oleg Komarov]

Textscan solution: import skipping the non relevant parts

fid  = fopen('C:\Users\Oleg\Desktop\test.txt');
fmt  = '%*f/%*f/%f:%f:%f%s%s';
opt  = {'Delimiter',',','CollectOutput',1};
data = textscan(fid,fmt,opt{:});
fid  = fclose(fid);

Point a)

secs = data{1}*[3600; 60; 1];

Point b)

data{2}(:,1)

Point c) - not clear

[Aaron]

Thank you for the solution. This is a different approach that I was thinking but it works except for it cuts off the last two decimal places on the seconds. Should be 6 decimal places (:xx.xxxxxx) and I am only getting 4(:xx.xxxx. It isn't clear to me why though? I am missing something?

[Aaron]

The script works perfect. I figured it out from another post that I was only seeing the DISPALY precision.

Here is what I found:

"You are merely seeing the DISPLAY precision, not the actual double precision value stored in matlab. Instead, try this:

format long g"

It works. Thank you for the accurate solution.