Convenient handling of datetime function arguments and class properties

Hi folks,

At work, we're in the process of adopting Matlab's new datetime type. We're on R2019b currently. Our code previously used primarily datenums and datestr cellstrs or charvecs to represent dates. And we're running in to a couple hitches.

My first thought was to take class properties and function arguments that used to take datenums or datestrs as inputs, and slap a datetime validator on them:

classdef Foo
    properties
        aDate datetime
    end
    
    methods
        function fcn(this, dt)
            arguments
                this
                dt datetime
            end
        end
    end
end

But that didn't work out, because Matlab's one-arg datetime(x) constructor interprets numeric inputs as datevecs, not datetimes. This surprised me, because I thought datetimes were more common than datevecs, and there's a more natural correspondence between datenum and datetime than datevec and datetime, because in a datenum array, each element represents a single date value, the same as in a datetime array.

The other hitch we're running in to is that the datetime(str) constructor that takes string inputs hits a try/catch in the ordinary "happy path" course of parsing dates. So if your code commonly uses datetime() to convert strings to dates, dbstop if all error becomes basically unusable in your programs.

>> dbstop if all error
>> datetime('1/1/2020')

Caught-error breakpoint was hit in @datetime/private/guessFormat>tryOneFmt at line 154. The error was:

Error using matlab.internal.datetime.createFromString
Unable to convert the text to datetime using the format 'dd-MMM-uuuu'.

154             t = createFromString(tryStr,fmt,2,tz,locale,pivot); % error, don't return NaT
K>> dbquit
>> datetime('1/1/2020 01:23:45')

Caught-error breakpoint was hit in @datetime/private/guessFormat>tryOneFmt at line 154. The error was:

Error using matlab.internal.datetime.createFromString
Unable to convert the text to datetime using the format 'dd-MMM-uuuu HH:mm:ss'.

154             t = createFromString(tryStr,fmt,2,tz,locale,pivot); % error, don't return NaT
K>> dbstack
> In guessFormat/tryOneFmt (line 154)
  In guessFormat (line 43)
  In datetime (line 632)
K>> version
ans =
    '9.7.0.1319299 (R2019b) Update 5'
K>> 

I ended up writing my own todatetime() function to use as a substitue for the datetime() constructor, and use that instead of argument validators. todatetime in MatlabProjectTemplate

classdef foo
    properties
        aDate datetime
    end
    methods
        function fcn(this, dt)
            dt = todatetime(dt);
        end
        function this = set.aDate(this, aDate)
            this.aDate = todatetime(aDate);
        end
    end
end

How are you all handling function signatures and class property constraints in codebases that have a mix of datetimes and datenums? What are you doing about that try/catch issue?

In your code, prior to datetime, were your date values predominantly datenums or datevecs?

[per isakson]

"were your date values predominantly datenums or datevecs?"

BUMP. Datenums and the variable name sdn. The resolution of most of the experimental time series I work with is one second. To simplify the code I use 3600*sdn, stored as integer, and called ssn. Then I don't need to worry about rounding errors.

[Steven Lord]

per isakson, I'm not sure what the context is for this comment. Is it referring to a variable in the original post that was renamed or removed on edit?

If you're working with experimental time series and you want the data stored with a time basis of one second, consider using a timetable.

n = datetime('now');
n.Format = n.Format + ".SSSS";
x = (1:5).';
v = sort(n + seconds(5*randn(5, 1)));
tt = timetable(v, x)

tt = 5x1 timetable
                v                x
    _________________________    _

    26-Jan-2021 15:19:06.1229    1
    26-Jan-2021 15:19:14.7319    2
    26-Jan-2021 15:19:15.0385    3
    26-Jan-2021 15:19:18.0733    4
    26-Jan-2021 15:19:21.6234    5

tt2 = retime(tt, 'secondly', 'linear')

tt2 = 17x1 timetable
                v                   x   
    _________________________    _______

    26-Jan-2021 15:19:06.0000    0.98572
    26-Jan-2021 15:19:07.0000     1.1019
    26-Jan-2021 15:19:08.0000      1.218
    26-Jan-2021 15:19:09.0000     1.3342
    26-Jan-2021 15:19:10.0000     1.4504
    26-Jan-2021 15:19:11.0000     1.5665
    26-Jan-2021 15:19:12.0000     1.6827
    26-Jan-2021 15:19:13.0000     1.7988
    26-Jan-2021 15:19:14.0000      1.915
    26-Jan-2021 15:19:15.0000     2.8744
    26-Jan-2021 15:19:16.0000     3.3168
    26-Jan-2021 15:19:17.0000     3.6463
    26-Jan-2021 15:19:18.0000     3.9758
    26-Jan-2021 15:19:19.0000      4.261
    26-Jan-2021 15:19:20.0000     4.5427
    26-Jan-2021 15:19:21.0000     4.8244

[per isakson]

Steven Lord, My main intent was to bump an interesting question. Second, I answered the question of the last line of the question.

Answer 1

[Steven Lord]

But that didn't work out, because Matlab's one-arg datetime(x) constructor interprets numeric inputs as datevecs, not datetimes. This surprised me, because I thought datetimes were more common than datevecs, and there's a more natural correspondence between datenum and datetime than datevec and datetime, because in a datenum array, each element represents a single date value, the same as in a datetime array.

Prior to the introduction of datetime users stored dates and times as serial date numbers that I believe were most often constructed using datenum. If I remember correctly the reasoning was that the most common ways to call datenum to create the serial date numbers were with a date vector or with a text representation S of the date and time (usually along with a format F) so the 1-input constructor of datetime matches the 1-input constructor of datenum. The 2-input (S and F) datenum call becomes a call to datetime with the 'InputFormat' option.

If you have a serial date number, you can call datetime with 'ConvertFrom' to tell MATLAB to convert the serial date number to a datetime.

The other hitch we're running in to is that the datetime(str) constructor that takes string inputs hits a try/catch in the ordinary "happy path" course of parsing dates. So if your code commonly uses datetime() to convert strings to dates, dbstop if all error becomes basically unusable in your programs.

This assumes you call datetime in such a way that its internal helpers need to throw an error (which datetime will effectively rethrow) because they can't determine what you're trying to do.

try % Using this so Answers can evaluate all lines of code in this message
    dt = datetime('02-01-2021') % February 1st or January 2nd? Could be either.
catch ME
    disp(ME.message)
end

Could not recognize the date/time format of '02-01-2021'. You can specify a format using the 'InputFormat' parameter. If the date/time text contains day, month, or time zone names in a language foreign to the 'en_US' locale, those might not be recognized. You can specify a different locale using the 'Locale' parameter.

If you know or suspect that the text representation of your datetime is ambiguous, help datetime out by specifying information to make it unambiguous.

dt = datetime('02-01-2021', 'InputFormat', 'dd-MM-yyyy') % unambiguously January 2nd

dt = datetime
   02-Jan-2021

I ran that last call with dbstop if all error enabled in my desktop MATLAB (I'm not sure if MATLAB Answers evaluation of code will accept debugging commands) and it ran without stopping. No ambiguity means no internal error which means no stopping.

[Andrew Janke]

> ...so the 1-input constructor of datetime matches the 1-input constructor of datenum

I don't think that's quite how the 1-argument behavior of datenum works. It only treats its 1-arg input as a datevec if the input has exactly 3 or 6 columns. Otherwise, numeric inputs are returned unmodified.

>> datenum(now)
ans =
   7.3818e+05
>> datenum([2000 1])
ans =
        2000           1
>> datenum([2000 1 1])
ans =
      730486
>> datenum([2000 1 1 1])
ans =
        2000           1           1           1
>> 

In addition, at least in recent versions of Matlab, it seems to do some heuristics on the range of the inputs, so that even 3-column or 6-column inputs are not converted if they look like they're already datenums.

>> datenum([2000 1 1 1 1 1])
ans =
   7.3049e+05
>> datenum([now now now now now now])
ans =
   1.0e+05 *
    7.3818    7.3818    7.3818    7.3818    7.3818    7.3818
>> datestr(ans(1))
ans =
    '26-Jan-2021 10:14:25'
>> 

The behaviors don't quite match up.

>> datenum(now)
ans =
   7.3818e+05
>> datetime(now)
Error using datetime (line 589)
Numeric input data must be a matrix with three or six columns, or else three, six, or seven separate numeric arrays. You can also create datetimes
from a single numeric array using the 'ConvertFrom' parameter. 
>> datenum([now now now now now now])
ans =
   1.0e+05 *
    7.3818    7.3818    7.3818    7.3818    7.3818    7.3818
>> datetime([now now now now now now])
Error using datetime (line 601)
Year, month, day, hour, and minute components must be integer values. 
>> 

With respect to the 'ConvertFrom' and 'InputFormat' calling forms, none of those work with function arguments blocks or property validators, do they, since validators only use single-argument conversion functions?

Your internal helper functions don't need to throw an error in the ambiguous cases. That logic could have just as well been coded with non-try/catch control flow mechanisms, like regular if/else statements and error return codes, which wouldn't have the dbstop if all error side effects.

It's unlikely that I'll be able to get my whole programming team to switch over to explicitly specifying InputFormat everywhere. Especially because that's not the behavior most of my coders want: in many cases, we want to have the flexibility to accept dates in multiple formats, and in the case of ambiguity, just go with the U.S. locale interpretation. The sad fact of the matter is, in the U.S. business world, most users and many data sources are going to use the ambiguous US "mm/dd/yyyy" format, so our code better support that.

We'll probably just end up writing our own date parsing logic and adding that to todatetime and end up using that everywhere instead of property and argument validators for datetimes. It's just a bit of a bummer because property and argument validators are so convenient.

Like, this is nice:

function myfcn(dt)
arguments
    dt datetime
end
% ... do work ...
end

This is less nice:

function myfcn(dt)
if isa(dt, 'datetime')
    % Leave it alone
elseif isnumeric(dt)
    dt = datetime(dt, 'ConvertFrom','datenum');
elseif isstring(dt) || ischar(dt) || iscellstr(dt)
    dt = datetime(dt, 'InputFormat','yyyy-mm-dd HH:MM:SS');
    % I hope that's the only format callers will be passing dates in
else
    error('myprogram:InvalidInput', 'Cannot convert %s to datetime', class(dt))
end
% ... do work ...
end

Hence, wrapping all that up in a custom todatetime().