how to speed up datestr conversion?

I want to know how to speed up the conversion from date string to date number. My code takes around 1 min to finish.

I have a large files with the first column of time strings, as below:

'17:11:17.02000'
'17:11:17.03000'
'17:11:17.04000'
'17:11:17.05000'

My conversion code:

tnum=datenum(M1.Var1)-datenum(2017,1,1)+datenum(2016,5,20);

Answer 1

[Guillaume]

Specifying the format would greatly speed up the conversion:

tnum = datenum(M1.Var1, 'HH.MM.SS.FFF') - datenum(2017,1,1) + datenum(2016,5,20); %only works this year!

However note that it will only keep the first three digits of the milliseconds. I don't know a way of telling datenum to accept 5 digits of millisecond. Note that even your original code did not keep the 5 digit correctly (e.g. .12345 gets rounded to .12339)

As fast as datenum conversion, more flexible and a lot more modern is to convert to datetime:

td = datetime(M1.Var, 'ConvertFrom', 'HH:mm:ss.SSSSS');  %keeps all digits of the millisecond correctly
[td.Year, td.Month, td.Day] = deal(2016, 5, 20); %Override date works regardless of the year it's executed in
td.Format = 'dd-MMM-uuuu HH:mm:ss.SSSSS';  %for display only. Set to what you prefer

Answer 2

[Fan Mei]

Thank you very much for your quick reply.

Answer 3

[dpb]

Two alternatives--

dn=datenum(c,'HH:MM:SS.FFF')

if stick with outmoded datenum or switch to the newer datetime class...

dt=datetime(c,'inputformat','HH:mm:ss.SSSSS')

Either way, specify the specific input format string. You can't quite hit the precise string with datenum; it won't accept other than three-digit fractional seconds indicator. You could try

dn=datenum(c(:,1:12),'HH:MM:SS.FFF')

to see if the precise string length would make any improvement in speed (I'd guess probably not, but "ya' never knows".

I've never compared the two for speed so don't know if the newer class has more overhead owing to it being a class and all or whether there have been sufficient "tweaks" to make it perform as well or better than its predecessor. Would be interesting to know.

Oh...another possibility would be to do the conversion on input if you're using (or could use) textscan --

dt=textscan(fid,'%{HH:mm:ss.SSSSS}'D);

I dunno on performance with textscan, either, altho while it might be more overhead in the initial read, you might make up for it overall in eliminating the second specific conversion call. This will, of course, be a datetime array this way, so that is a difference in subsequent code.

[Guillaume]

There's hardly any difference in conversion time between datenum and datetime, so I would prefer the latter:

>>c = repmat({'17:11:17.02000'}, 1e5, 1);  

>> tic; t1 = datenum(c, 'HH:MM:SS.FFF'); toc
Elapsed time is 1.076548 seconds.

>> tic; t2 = datetime(c, 'InputFormat', 'HH:mm:ss.SSSSS'); toc
Elapsed time is 1.102092 seconds.

Not providing a conversion string is the real slowdown:

>> tic; t3 = datenum(c); toc
Elapsed time is 9.821827 seconds.

[Fan Mei]

Thank you very much

[dpb]

If solved the problem, please Accept Answer to at least indicate closure of the topic if nothing else.