How to use datetime with leap second strings?

Suppose we have the following date time string:

ds1 = "30-Jun-1972 23:59:59.5";

You can convert that to a datetime easily:

datetime(ds1)

ans = datetime
   30-Jun-1972 23:59:59

But now suppose you have a string that is in the middle of a leap second:

ds2 = "30-Jun-1972 23:59:60.5";

MATLAB won't convert this:

try
    datetime(ds2)
catch
    disp('Error')
end

Error

So apparently the seconds value >= 60 is causing this. But what if I tell it to use leap seconds? Still doesn't work:

try
    datetime(ds2,'TimeZone','UTCLeapSeconds')
catch
    disp('Error')
end

Error

This is disappointing, and quite frankly inconsistent with other ways of creating a datetime within a leap second. E.g., this works fine:

datetime(1972,6,30,23,59,60.5,'TimeZone','UTCLeapSeconds')

ans = datetime
   1972-06-30T23:59:60.500Z

So, is there a way to coax datetime( ) into reading a date time string in the middle of a leap second? And at the same time retain full precision of the string? I would like to avoid manually parsing this myself if possible, but currently see no other way. E.g., the precision loss issue with string input illustrated:

format longg
ds3 = "01-Jan-2024";
dt3 = "23:59:01.2345678912345";
d3 = datetime(ds3+" "+dt3)

d3 = datetime
   01-Jan-2024 23:59:01

d3.Second

ans = 
               1.234567891

d3.Second == 1.2345678912345  % LOSES PRECISION!

ans = logical
   0

d4 = datetime(ds3) + duration(dt3)

d4 = datetime
   01-Jan-2024 23:59:01

d4.Second

ans = 
          1.23456789124012

d4.Second == 1.2345678912345  % LOSES PRECISION!

ans = logical
   0

You can see the loss of precision datetime has when reading strings in general. The duration class has a similar issue. So even if datetime could handle seconds >= 60, there would be a precision loss issue. I would like to avoid that.

At the moment, the only solution I see is to manually parse out the seconds and then piece things together. E.g.,

dt3a = "23:59:00";
dt3b = "00:00:01.2345678912345";
d5 = datetime(ds3) + duration(dt3a) + duration(dt3b)

d5 = datetime
   01-Jan-2024 23:59:01

d5.Second

ans = 
           1.2345678912345

d5.Second == 1.2345678912345

ans = logical
   1

Even this approach has problems with duration in the middle of a leap second because it can't handle string seconds >=60 for some reason:

seconds(60.5) % this is OK

ans = duration
   60.5 sec

duration("00:00:60.5") % but not this!

Error using duration (line 126)
Could not recognize the format of '00:00:60.5'.
Expected data of the form 'hh:mm:ss' or 'dd:hh:mm:ss'.

So to handle the leap second issue I would be forced to read the seconds manually instead of using duration. (Why???)

I am hoping for a simpler way.

Answer 1

[Shivam Gothi]

Hello @James Tursa,

As per my understanding, you are trying to save leap-seconds in "datetime" object, in such a way that it does not looses precision. You want to pass the date-time string as an input argument to the "datetime" function.

I also faced similiar issue, but tried to resolve it by using the following work-around.

The documentation of "datetime" function (https://www.mathworks.com/help/matlab/ref/datetime.html#d126e365305) say that:

While passing character string as an input argument to the "datetime" function, we can specify the precision upto 9 decimal places by specifing an additional input argument "infmt", as demonstrated below:

DT_string= "2024-01-01T23:59:01.2345678912345Z"; %Same date and time as used in the above question
t = datetime(DT_string,'InputFormat','uuuu-MM-dd''T''HH:mm:ss.SSSSSSSSS''Z')

t = datetime
   01-Jan-2024 23:59:01

format longg;
t.Second

ans = 
               1.234567891

This says that the maximum precision available is "9". Therefore, "1.2345678912345" is getting truncated to "1.234567891" in your case. Even if we do not explicitely specify the "inputFormat", it will automatically truncate to the maximum 9 digits after decimal.

Therefore, if you want to retain the precision, the below given command works perfectly.

t=datetime(1972,1,1,23,59,01.2345678912345,'TimeZone','UTCLeapSeconds') % same date and time entered manually

t = datetime
   1972-01-01T23:59:01.234Z

t.Second %This is not truncated

ans = 
           1.2345678912345

In this case, instead of passing the date-time string, you have to manually enter the the values of year, months, days and time. This is not favourable as mentioned by you.

I am suggesting one of the possible work-around which I implemented in order to pass date-time string and store it in a "datetime" object which supports leap seconds as well as retains precision.

I have made a user defined function "DateTime" which takes the date-time string and returns a "datetime" object. It is demonstrated below:

format longg
ds3 = "01-Jan-2024";
dt3 = "23:59:01.2345678912345";
stringg=(ds3+" "+dt3);
t=DateTime(stringg)

t = datetime
   2024-01-01T23:59:01.234Z

t.Second %this does not losses precision

ans = 
           1.2345678912345

function out=DateTime(DT_string)
arguments
    DT_string string
end
    C = textscan(DT_string,'%s %s'); %gives (1 x 2) cell array consisting of date and time
    date_string=string(C{1});
    time_string=string(C{2});
    Date=textscan(date_string,'%f %s %f','delimiter','-'); %save the day, month, year in cell array
    Time=textscan(time_string,'%f %f %f','delimiter',':'); %Save hour, minutes and seconds in celll array
    
    DT_object=datetime(date_string);
    
    DT_object.Hour = Time{1}; %extract the hour, minutes and seconds and save them in variable
    DT_object.Minute = Time{2};
    DT_object.Second = Time{3};
    DT_object.TimeZone='UTCLeapSeconds';
    
    out=DT_object;
end

I hope you find this information useful !!

Answer 2

[Peter Perkins]

Parsing timestamps with leap seconds

At the moment, con9verting from text to the UTCLeapSeconds "time zone" requires a specifc (ISO) format. Your try/catch was hiding the err msg that isn't exactly spot on but probably would have helped figure out what was happening:

ds2 = "30-Jun-1972 23:59:60.5"

ds2 = "30-Jun-1972 23:59:60.5"

try, datetime(ds2), catch, disp(lasterr); end

Error using datetime (line 257)
Could not recognize the date/time format of '30-Jun-1972 23:59:60.5'. You can specify a format using the 'InputFormat' parameter. If the date/time text contains day, month, or time zone names in a language foreign to the 'en_US' locale, those might not be recognized. You can specify a different locale using the 'Locale' parameter.

ds3 = "1972-06-30T23:59:60.5Z"

ds3 = "1972-06-30T23:59:60.5Z"

datetime(ds3,TimeZone="UTCLeapSeconds")

ans = datetime
   1972-06-30T23:59:60.500Z

There is work to relax that restriction, but for now that's what you have to do. I'd be curious to hear where you are getting timestamps that honor leap seconds but are in something other than an ISO format.

Parsing duration timestamps

try, duration("00:00:60.5"), catch, disp(lasterr); end

Error using duration (line 126)
Could not recognize the format of '00:00:60.5'.
Expected data of the form 'hh:mm:ss' or 'dd:hh:mm:ss'.

This does not work because duration is not a time of day. It is an elapsed time. The concept of leap seconds doesn't apply here. A duration can certainly hold 60.5 seconds, but to parse a "timer format" string for 60.5 seonds, it'd have to be

duration("00:01:00.5",Format="hh:mm:ss.SSS")

ans = duration
   00:01:00.500

Not sure but I suspect you are trying to use duration to parse the time of day portion of a datetime timestamp. That isn't gonna work for a leap second. Not sure what you are using this for, but I bet there's another way to do what you need.

Precision issues

I think the doc is pretty clear that only 9 fractional second digits are honored when converting from text, and only 9 can be displayed. So

ds3 = "01-Jan-2024";
dt3 = "23:59:01.2345678912345";
d3 = datetime(ds3+" "+dt3,Format="dd-MMM-uuuu HH:mm:ss.SSSSSSSSS")

d3 = datetime
   01-Jan-2024 23:59:01.234567891

is gonna throw away the last "2345". Perhaps it should warn, I will make a note about that. I'd be curious to hear the use case for text timestamps with precisions that are that high. I would have thought that sub-picoseconds would more typically be measured using elapsed time, not absolute time.

So I think you are going to need to parse the seconds field separately, but as you say, duration won't work because of leap seconds. So I recommend splitting the timestamps at the ms as two strings, using datetime to convert the leading portion, stick a decimal in front of the second piece and use double to convert the other string to numeric, and add that to the datetime:

str1 = "21-Nov-2024 16:15:14.123";
str2 = "4567890123";
msFrac = double("." + str2);
dt = datetime(str1,Format="dd-MMM-uuuu HH:mm:ss.SSS") + milliseconds(msFrac)

dt = datetime
   21-Nov-2024 16:15:14.123

sprintf("%.13f",dt.Second)

ans = "14.1234567890123"

Internally, datetime has the precision to handle sub-nanoseconds, but I don't think internal precision is the problem you are flagging.

[James Tursa]

@Peter Perkins Thanks for the reply. My comments ...

"... I'd be curious to hear where you are getting timestamps that honor leap seconds but are in something other than an ISO format ..."

I'm creating my own class to do time scale conversions and I am trying to make my code robust for various user inputs. Forcing the user to reformat their input into strict ISO format is too overbearing IMO when the intent of the user is obvious.

"... Not sure but I suspect you are trying to use duration to parse the time of day portion of a datetime timestamp ..."

Yes, that is one thing my class does. As well as simply allowing the user to enter strings that get parsed the same way as individual entries to create durations or my own custom duration objects that maintain input precision. E.g., I see no compelling reason why these two don't do the same thing:

try, duration("00:00:60.5"), catch, disp(lasterr); end

Error using duration (line 126)
Could not recognize the format of '00:00:60.5'.
Expected data of the form 'hh:mm:ss' or 'dd:hh:mm:ss'.

duration(00,00,60.5)

ans = duration
   00:01:00

My code will make it easy on the user and do the same thing in both cases.

As a side issue, I also see no compelling reason why these two do different things:

duration("1:2:3:4")

ans = duration
   26:03:04

duration(1,2,3,4)

ans = duration
   01:02:03

But to maintain consistency with MATLAB, I suppose my code will follow this inconsistency.

"... I think the doc is pretty clear that only 9 fractional second digits are honored when converting from text, and only 9 can be displayed ..."

Yes, it is. I was hoping to learn something to avoid it, but it looks like I will have to manually parse the string to get the data loaded into the datetime accurately. Again, I don't have a specific use case to cite to you. I am creating a class and I want certain accurate behavior from the class without presuming what the user is using the class for.