verifyExecutionMatchesMATLAB

Class: matlabtest.coder.TestCase
Namespace: matlabtest.coder

Verify that generated C/C++ code execution results match MATLAB results

Since R2023a

Syntax

verifyExecutionMatchesMATLAB(testCase,executionResults)

verifyExecutionMatchesMATLAB(testCase,executionResults,diagnostic)

verifyExecutionMatchesMATLAB(___,Name=Value)

Description

verifyExecutionMatchesMATLAB(testCase,executionResults) verifies that the execution results specified by executionResults for the C/C++ code generated by MATLAB^® Coder™ match the execution of the MATLAB source code in the equivalence test case testCase.

verifyExecutionMatchesMATLAB(testCase,executionResults,diagnostic) returns diagnostic information specified by diagnostic.

verifyExecutionMatchesMATLAB(___,Name=Value) specifies options using one or more name-value arguments in addition to the input arguments in previous syntaxes.

Input Arguments

testCase — Test case

matlabtest.coder.TestCase object

Test case, specified as a matlabtest.coder.TestCase object.

executionResults — Execution results for generated C/C++ code in equivalence tests

matlabtest.coder.results.ExecutionResults object

Execution results for the generated C/C++ code in the equivalence tests, specified as a matlabtest.coder.results.ExecutionResults object.

diagnostic — Failure diagnostic information

string scalar | character vector | function handle | matlab.unittest.diagnostics.Diagnostic object

Failure diagnostic information, specified as a:

• String scalar or character vector

• Function handle

• matlab.automation.diagnostics.Diagnostic object or object of one of its subclasses

Example: verifyExecutionMatchesMATLAB(testCase,executionResults,"Equivalence test failed")

Name-Value Arguments

Specify optional pairs of arguments as Name1=Value1,...,NameN=ValueN, where Name is the argument name and Value is the corresponding value. Name-value arguments must appear after other arguments, but the order of the pairs does not matter.

Example: AbsTol=0.01

AbsTol — Absolute tolerance

numeric array

Absolute tolerance, specified as a numeric array. The sizes of AbsTol and expected, where expected is the output of the MATLAB execution of the function, must be the same or compatible. See Compatible Array Sizes for Basic Operations for more information about compatible arrays.

The tolerance is applied only to values of the same data type. For an absolute tolerance to be satisfied, abs(expected-actual) <= AbsTol must be true, where actual is ExecutableOutput.

RelTol — Relative tolerance

numeric array

Relative tolerance, specified as a numeric array. The sizes of RelTol and expected, where expected is the output of the MATLAB execution of the function, must be the same or compatible. See Compatible Array Sizes for Basic Operations for more information about compatible arrays.

The tolerance is applied only to values of the same data type. For a relative tolerance to be satisfied, abs(expected-actual) <= RelTol.*abs(expected) must be true, where actual is ExecutableOutput.

Examples

Generate C Code from Function with No Inputs and Test for Equivalence

This example shows how to generate C code from a MATLAB function that has no inputs and test for equivalence by using MATLAB Coder.

The function helloWorld displays a string of text.

function y = helloWorld 
y = "Hello World!";
end

This class definition file defines an equivalence test case that inherits from matlabtest.coder.TestCase. The test case in the methods block defines a test case that:

1. Builds C code from the helloWorld function

2. Executes the C code with no inputs

3. Verifies the execution of the C code against the execution of the MATLAB function helloWorld

classdef tEquivalence < matlabtest.coder.TestCase
    methods (Test)
        function tHelloWorld(testCase)
            buildResults = build(testCase,"helloWorld");         
            executionResults = execute(testCase,buildResults);
            verifyExecutionMatchesMATLAB(testCase,executionResults)
        end
    end
end

Run the tHelloWorld test.

runtests("tEquivalence", ...
    procedureName="tHelloWorld")

Running tHelloWorld
..
Done tHelloWorld
__________


ans = 
  TestResult with properties:

          Name: 'tEquivalence/tHelloWorld'
        Passed: 1
        Failed: 0
    Incomplete: 0
      Duration: 2.8921
       Details: [1×1 struct]

Totals:
   1 Passed, 0 Failed, 0 Incomplete.
   2.8921 seconds testing time.

Generate C Code, Verify Equivalence, and Return Custom Diagnostic Result

This example shows how to generate C code from a MATLAB function, test for equivalence, and return a custom diagnostic result.

The function myAdd takes two numbers as inputs, adds them together, and outputs the result.

function y = myAdd(a,b) %#codegen
y = a+b;
end

This class definition file defines an equivalence test case that inherits from matlabtest.coder.TestCase. The test case in the methods block defines a test case that:

1. Builds C code from the myAdd function with inputs set to (0,0)

2. Executes the C code with inputs set to (1,2)

3. Verifies the execution of the C code against the execution of the MATLAB function myAdd and returns a string diagnostic result if the test fails

classdef tEquivalence < matlabtest.coder.TestCase
    methods (Test)
        function tMyAdd(testCase)
            buildResults = build(testCase,"myAdd", ...
                Inputs={1,2});          
            executionResults = execute(testCase, ...
                buildResults);            
            d = "Equivalence test failed.";
            verifyExecutionMatchesMATLAB(testCase,executionResults,d)
        end
    end
end

Run the tMyAdd test.

runtests("tEquivalence", ...
    procedureName="tMyAdd")

Running tMyAdd
..
Done tMyAdd
__________


ans = 
  TestResult with properties:

          Name: 'tEquivalence/tMyAdd'
        Passed: 1
        Failed: 0
    Incomplete: 0
      Duration: 2.6670
       Details: [1×1 struct]

Totals:
   1 Passed, 0 Failed, 0 Incomplete.
   2.667 seconds testing time.

Generate and Test C Code for Equivalence

This example shows how to generate C code from a MATLAB function and test for equivalence by using MATLAB Coder.

The function myAdd takes two numbers as inputs, adds them together, and outputs the result.

function y = myAdd(a,b) %#codegen
y = a+b;
end

This class definition file defines an equivalence test case that inherits from matlabtest.coder.TestCase. The test case in the methods block defines a test case that:

1. Builds C code from the myAdd function with inputs set to (0,0)

2. Executes the C code with inputs set to (1,2)

3. Verifies the execution of the C code against the execution of the MATLAB function myAdd with the same inputs and an absolute tolerance of 0.01

classdef tEquivalence< matlabtest.coder.TestCase
    methods (Test)
        function tMyAdd(testCase)
            buildResults = build(testCase,"myAdd", ...
                Inputs={0,0});         
            executionResults = execute(testCase,buildResults, ...
                Inputs={1,2});
            verifyExecutionMatchesMATLAB(testCase,executionResults, ...
                AbsTol=0.01)
        end
    end
end

Run the tMyAdd test.

runtests("tEquivalence", ...
    procedureName="tMyAdd")

Running tMyAdd
..
Done tMyAdd
__________


ans = 
  TestResult with properties:

          Name: 'tEquivalence/tMyAdd'
        Passed: 1
        Failed: 0
    Incomplete: 0
      Duration: 2.6670
       Details: [1×1 struct]

Totals:
   1 Passed, 0 Failed, 0 Incomplete.
   2.667 seconds testing time.

Tips

• To specify multiple tolerances in the verification method, use the verifyThat method and pass an matlabtest.constraints.ExecutionMatchesMATLAB object as an input. Specify the tolerances by using the Within name-value argument.

  This example test class contains a test that generates C code for a MATLAB function called myAdd, executes the generated code using the specified inputs, and compares the result to the execution of the MATLAB source code with absolute tolerances that have two different data types.

  classdef tDemoTolerance < matlabtest.coder.TestCase
      methods(Test)
          function specifyMultipleTolerances(testCase)
              import matlab.unittest.constraints.AbsoluteTolerance
              import matlabtest.constraints.ExecutionMatchesMATLAB
  
              buildResults = build(testCase,"myAdd", ...
                  Inputs={int16(1),single(1)});
              actual = execute(testCase,buildResults);
  
              absTol = AbsoluteTolerance(int16(0.5), ...
                  single(0.5));            
              verifyThat(testCase,actual, ...
                  ExecutionMatchesMATLAB("Within",absTol));
          end
      end
  end

Version History

Introduced in R2023a