How to generate code coverage and cobertura test reports for the same source code in R2022b

I have used the following code to execute test prior to R2022b (R2018b-R2021b).

import matlab.unittest.TestRunner;
import matlab.unittest.TestSuite;
import matlab.unittest.plugins.CodeCoveragePlugin;
import matlab.unittest.plugins.XMLPlugin;
import matlab.unittest.plugins.codecoverage.CoberturaFormat;
import matlab.unittest.plugins.codecoverage.CoverageReport
import matlab.unittest.selectors.HasBaseFolder;


suite = testsuite(fullfile(pwd, 'tests'), 'IncludeSubfolders', true);
fprintf("Running %d tests.\n", length([{suite.ProcedureName}]));

runner = TestRunner.withTextOutput();
runner.addPlugin(XMLPlugin.producingJUnitFormat('results.xml'));
runner.addPlugin( ...
    CodeCoveragePlugin.forFolder({'.'}, ...
    'IncludingSubfolders', true, ...
    'Producing', CoberturaFormat('coverage.xml') ...
    ) ...
);
runner.addPlugin( ...
    CodeCoveragePlugin.forFolder({'.'}, ...
    'IncludingSubfolders', true, ...
    'Producing', CoverageReport('coverageReport') ...
    ) ...
);

results = runner.run(suite);

display(results.table);
assert( ...
    ~any([results.Failed]), ...
    'At least one test failed in the test session.' ...
);

% Optional, opens the test results in the matlab browser
% open(fullfile("coverageReport","index.html"))

As of R2022b this code fails with the following error:

MATLAB:unittest:CoverageReport:CoverageSessionsWithOverlappingSourcesNotAllowed
Error using matlab.unittest.internal.coverage.CodeCoverageCollector/initialize
Simultaneously generating coverage reports for the same source files is not
supported.

I noticed in the R2022b docs a new section about generating coverage with two plugins that isn't in the R2018b-R2021b CodeCoveragePlugin document, but didn't notice any breaking change in the R2022b release notes to the CodeCoveragePlugin. Based on the R2022b documents it seems like we can't generate two reports for the same source code. Did the functionality change?

Thanks for any insights or help.

Answer 1

[David Hruska]

Hi Alexander,

In releases prior to R2022a, running tests with more than one code coverage plugin added to the TestRunner could sometimes produce incorrect results for one or both of the reports.

As of R2022a, you can only use multiple CodeCoveragePlugin instances in a single test run if they report on different source files. I see that this is documented only on the CodeCoveragePlugin page (see the "Version History" section), but not in the release notes. I'm not sure if we can retroactively adjust the release notes for R2022a, but I'll check with our documentation team on that.

It's still possible to produce multiple reports at the same time by providing multiple formats to a single CodeCoveragePlugin. For example:

runner.addPlugin( ...

CodeCoveragePlugin.forFolder({'.'}, ...

'IncludingSubfolders', true, ...

'Producing', [CoberturaFormat('coverage.xml'), CoverageReport('coverageReport')] ...

) ...

);

Hope this helps,

David

[Alexander]

Hi David,

Thanks for the fast response and pointing out that producing would accept an array out outputs. Sorry I missed that, appreciate your help.

-Alexander

[David Hruska]

Happy to help! Following up on the release notes question: I confirmed with our documentation team that we're no longer making any updates to the R2022a release notes at this point and the best source of information is the Version History on the CodeCoveragePlugin documentation page.