Collect Coverage for Generated C/C++ Code in Equivalence Tests

When you run C/C++ equivalence tests using software-in-the-loop (SIL) or processor-in-the-loop (PIL) verification with Embedded Coder^®, you can collect code coverage information for the generated C/C++ code. Collect coverage by adding an instance of the matlabtest.coder.plugins.GeneratedCodeCoveragePlugin class to the test runner. As the tests run, the plugin collects information about the parts of the source code that the tests executed. You can access coverage information as a code coverage report or at the MATLAB^® command line. For more information about generated C/C++ equivalence tests, see Generate C/C++ Code and Test for Equivalence.

You can collect these types of coverage:

• Statement

• Function

• Decision

• Condition

• Modified condition/decision (MC/DC)

For more information, see Types of Coverage for Generated C/C++ Code in Equivalence Tests.

Note

Collecting coverage during generated C/C++ equivalence tests requires Embedded Coder. You can only collect coverage on Windows^® and Linux^® platforms.

Write SIL and PIL Equivalence Tests

You can only collect coverage for C/C++ code generated in an equivalence test if the test uses SIL or PIL verification and builds a static library. For more information, see Execute and Verify in SIL or PIL Mode in Generate C/C++ Code and Test for Equivalence.

Suppose that you want to generate code for a function called myMath, which allows a user to provide two numeric inputs and an operation string to indicate if the inputs are added or subtracted:

function y = myMath(a,b,operation) %#codegen
if operation == "add"
    y = a+b;
elseif operation == "subtract"
    y = b-a;
else
    y = [];
end
end

This SIL equivalence test uses class properties and class-level setup to generate a static library once and specifies that the operation argument can be a variable-sized string. It uses parameterization to execute and verify the generated C code twice, with different inputs each time.

classdef tMyMathSIL < matlabtest.coder.TestCase
    properties
        buildResults;
    end

    methods (TestClassSetup)        
        function generateCode(testCase)           
            operationSize = coder.typeof("add");
            operationSize.StringLength = inf;
            buildInputs = {1,2,operationSize};
            testCase.buildResults = build(testCase,"myMath", ...
                Inputs=buildInputs,Configuration="lib");
        end        
    end

    properties (TestParameter)
        runInputs = {{1,2,"add"},{1,2,"subtract"}};
    end
    
    methods(Test)
        function testSILvsMATLAB(testCase,runInputs)                        
            executionResults = execute(testCase,testCase.buildResults, ...
                Inputs=runInputs);
            verifyExecutionMatchesMATLAB(testCase,executionResults);
        end        
    end
end

This PIL equivalence test uses the same format as the SIL equivalence test and defines a code generation configuration parameter object to generate code for an ARM^® Cortex^®-M3 board and execute and verify in PIL mode.

classdef tMyMathPIL < matlabtest.coder.TestCase
    properties
        buildResults;
    end

    methods (TestClassSetup)
        function generateCode(testCase)
            operationSize = coder.typeof("add");
            operationSize.StringLength = inf;
            buildInputs = {1,2,operationSize};

            cfg = coder.config("lib","ecoder",true);
            cfg.Hardware = coder.hardware("ARM Cortex-M3 (QEMU)");
            cfg.VerificationMode = "PIL";
            
            testCase.buildResults = build(testCase,"myMath", ...
                Inputs=buildInputs,Configuration=cfg);
        end
    end

    properties (TestParameter)
        runInputs = {{1,2,"add"},{1,2,"subtract"}};
    end

    methods(Test)
        function testPILvsMATLAB(testCase,runInputs)
            executionResults = execute(testCase,testCase.buildResults, ...
                Inputs=runInputs);
            verifyExecutionMatchesMATLAB(testCase,executionResults);
        end
    end
end

Collect Coverage

To run equivalence tests with SIL or PIL verification and collect coverage:

1. Define the coverage format by creating an instance of matlab.unittest.plugins.codecoverage.CoverageResult.

2. Create a plugin for the generated C/C++ code by creating an instance of matlabtest.coder.plugins.GeneratedCodeCoveragePlugin. Specify the desired coverage type for the plugin or use the default settings, which specify statement and function coverage.

3. Create a test runner by using testrunner.

4. Add the code coverage plugin to the test runner by using addPlugin.

5. Create a test suite for the equivalence tests by using testsuite.

6. Run the tests by using run.

This example code creates a test runner with a plugin that programmatically accesses the coverage information for all types of code coverage.

import matlab.unittest.plugins.codecoverage.CoverageResult
import matlabtest.coder.plugins.GeneratedCodeCoveragePlugin

format = CoverageResult;
plugin = GeneratedCodeCoveragePlugin(Producing=format,MetricLevel="mcdc");
runner = testrunner("textoutput");
addPlugin(runner,plugin);

This example code creates a test suite for the tMyMathSIL equivalence test class and runs the tests. All tests pass.

suite = testsuite("tMyMathSIL.m");
run(runner,suite);

Running tMyMathSIL
..
Done tMyMathSIL
__________

View Coverage Results

You can view coverage results in the code coverage report or programmatically access the results at the MATLAB command line.

Generate Coverage Report

To generate a code coverage report, pass the Result property of the matlab.unittest.plugins.codecoverage.CoverageResult object to the generateHTMLReport method. For more information about the code coverage report, see Generate Code Coverage Report in Collect Code Coverage Metrics for MATLAB Source Code.

This example code generates an HTML code coverage report with name myCoverageReport for the coverage results collected from the test in tMyMathSIL.m. The code stores the report in a folder called myArtifacts in the current folder.

covResult = format.Result;
generateHTMLReport(covResult,"myArtifacts",MainFile="myCoverageReport.html");

View Results Programmatically

To view the coverage results programmatically, pass the Result property of the matlab.unittest.plugins.codecoverage.CoverageResult object to the coverageSummary method.

This example code returns the statement coverage results collected from the test in tMyMathSIL.m.

covResult = format.Result;
covSummary = coverageSummary(covResult,"statement")

covSummary = 5×2    
    33    37
     0    49
    38    45
     0     0
     0     0

Each row in the matrix indicates the coverage results for a file in the static library. The first column indicates the number of executed statements and the second column indicates the number of total statements.