verifyError in a script based unit test

Hi, my question is regarding the script-based unit testing framework. I would like to use the verifyError functionality and am struggling to generate a suitable testCase object that I can use in my script-based unit test.

For example, the run suite is called with:

import matlab.unittest.TestSuite;
import matlab.unittest.TestCase;
import matlab.unittest.TestRunner;
import matlab.unittest.plugins.TAPPlugin;
import matlab.unittest.plugins.ToFile;


suite = testsuite('unitTests','IncludingSubpackages',true);
runner = TestRunner.withTextOutput();
tapFile = fullfile(getenv('WORKSPACE'), sprintf('%s_results.tap', 'test'));
runner.addPlugin(TAPPlugin.producingOriginalFormat(ToFile(tapFile), 'LoggingLevel', 3, 'OutputDetail', 3));
results = runner.run(suite);

In the folder unitTests I would have the following script based unit test:

% Test to evaluate foo

% Initial Set-Up
a = 1;

%% Test 1. Verify Error Condition
% How can I get a test case here to call either:
% verifyError(testCase, @()foo('badArguement'), 'SomeError');
% or 
% testCase.verifyError(@()foo('badArguement'), 'SomeError');

It seems that the following is not valid, as this instantiation is only for interactive use.

% Test to evaluate foo

% Initial Set-Up
a = 1;
testCase = matlab.unittest.TestCase.forInteractiveUse;

%% Test 1. Verify Error Condition
% How can I get a test case here to call either:
% verifyError(testCase, @()foo('badArguement'), 'SomeError');
% or 
% testCase.verifyError(@()foo('badArguement'), 'SomeError');

or

% Test to evaluate foo

% Initial Set-Up
a = 1;

%% Test 1. Verify Error Condition
testCase = matlab.unittest.TestCase.forInteractiveUse;
% How can I get a test case here to call either:
% verifyError(testCase, @()foo('badArguement'), 'SomeError');
% or 
% testCase.verifyError(@()foo('badArguement'), 'SomeError');

Answer 1

[Andy Campbell]

Hello,

Since there is no testCase provided for script based tests, they don't support the Qualification API, and that definitely is one reason someone might want to use function based tests instead. Can you describe some of your reasoning for using script based tests? Do you prefer them over functions or classes? This may be helpful to determine whether we should look into providing the qualification api to script based tests or if you are happy to use functions.

Thanks,

Andy

[Siva Movva]

Hi Andy,

Thank you for your prompt response to clarify the question that I had. I prefer to use the scripts since they are less verbose then the function based test scripts, but this is more personal preference rather than technical merit. I quite like the scripts ability to divide itself into seperate tests just by comment headers.

However, if the Qualifications API doesn't exist for script based testing, I can re-write particular tests with the function based testing framework to make use of more of the API's features.

Thanks again!

[Andy Campbell]

Sounds great. Thank you for your feedback. This is helpful to see your usage and preference, and we may certainly want to consider making script based tests at least a little more powerful to enable these workflows.

Until then, you could define a function like this (and maybe the other qualification api methods you like) and perhaps at least get some of the way there:

function assertError(fh, id, varargin)

import matlab.unittest.diagnostics.Diagnostic;
import matlab.unittest.constraints.Throws;

throws = Throws(id);
passed = throws.satisfiedBy(fh);
diagText = "";
if ~passed
    diag = Diagnostic.join(varargin{:}, throws.getDiagnosticFor(fh));
    arrayfun(@diagnose, diag);
    diagText = strjoin({diag.DiagnosticText},[newline newline]);
end
assert(passed, diagText); 

Then you can call this in a script and get the following

>> assertError(@() true, 'some:id','This is Siva''s diag')
Error using assertError (line 11)
This is Siva's diag

Throws failed.
--> The function did not throw any exception.

    Expected Exception:
        'some:id'

Evaluated Function:
  function_handle with value:
    @()true