matlab.buildtool.tasks.TestTask Class

Namespace: matlab.buildtool.tasks
Superclasses: matlab.buildtool.Task

Task for running tests

Since R2023b

Description

The matlab.buildtool.tasks.TestTask class provides a task for running a suite of tests using the MATLAB^® unit testing framework.

Tasks that you create with the TestTask class support incremental builds. For more information, see Up-To-Date Check.

Creation

Description

task = matlab.buildtool.tasks.TestTask creates a task for running the tests in the current folder and its subfolders. The task fails if any of the tests fail.

task = matlab.buildtool.tasks.TestTask(tests) sets the Tests property. The task runs the tests in the specified files and folders, including their subfolders.

task = matlab.buildtool.tasks.TestTask(___,Name=Value) sets properties using one or more name-value arguments in addition to any of the input argument combinations in previous syntaxes. You can also set the Description and Dependencies properties, which the class inherits from the Task class, using name-value arguments. For example, task = matlab.buildtool.tasks.TestTask(TestResults="results.xml") creates a task that produces test results in JUnit-style XML format.

Properties

This section lists the properties of the TestTask class. In addition to these properties, the TestTask class inherits properties from the Task class. Of the inherited properties, you can set the Description and Dependencies properties using name-value arguments during task creation.

Tests — Test files and folders to run

string vector | character vector | cell vector of character vectors | matlab.buildtool.io.FileCollection vector

Test files and folders to run, specified as a string vector, character vector, cell vector of character vectors, or matlab.buildtool.io.FileCollection vector, and returned as a matlab.buildtool.io.FileCollection row vector. For example, you can specify a test file, a folder that contains test files, or a project folder that contains test files. By default, the task runs the tests in the current folder and its subfolders.

If Tests contains tests created using Simulink^® Test™, the task provides access to their results. You must have a Simulink Test license to run tests created using Simulink Test. (since R2024a)

Attributes:

GetAccess	public
SetAccess	public

SourceFiles — Source files and folders

string vector | character vector | cell vector of character vectors | matlab.buildtool.io.FileCollection vector

Source files and folders under test, specified as a string vector, character vector, cell vector of character vectors, or matlab.buildtool.io.FileCollection vector, and returned as a matlab.buildtool.io.FileCollection row vector. Set this property if you use the CodeCoverageResults property or addCodeCoverage method to collect code coverage information. The specified source code must be on the MATLAB search path when the task runs the tests and collects code coverage information.

If you do not set the SourceFiles property, the build tool does not skip the task. For more information on how the build tool treats the task in incremental builds, see Up-To-Date Check.

Attributes:

GetAccess	public
SetAccess	public

IncludeSubfolders — Option to run tests in subfolders

true or 1 (default) | false or 0

Option to run the tests in subfolders, specified as a numeric or logical 1 (true) or 0 (false). By default, the task runs the tests in the specified folders and their subfolders.

Attributes:

GetAccess	public
SetAccess	public

Tag — Test tag

string vector | character vector | cell vector of character vectors

Test tag, specified as a string vector, character vector, or cell vector of character vectors. If you specify test tags, the task filters the test suite and runs only the tests that have at least one of the specified tags. For more information about test tags, see Tag Unit Tests.

Attributes:

GetAccess	public
SetAccess	public

OutputDetail — Display level of event details

0 | 1 | 2 | 3 | 4 | matlab.automation.Verbosity enumeration object | text representation of enumeration

Display level of event details, specified as an integer scalar from 0 through 4, a matlab.automation.Verbosity enumeration object, or a text representation of the enumeration.

Numeric Representation	Enumeration Member Name	Verbosity Description
0	None	No information
1	Terse	Minimal information
2	Concise	Moderate amount of information
3	Detailed	Some supplemental information
4	Verbose	Lots of supplemental information

By default, the task displays failing and logged events at the matlab.automation.Verbosity.Detailed level (level 3) and test run progress at the matlab.automation.Verbosity.Terse level (level 1).

Example: task = matlab.buildtool.tasks.TestTask(OutputDetail="verbose") creates a task that displays event details at the matlab.automation.Verbosity.Verbose level.

Attributes:

GetAccess	public
SetAccess	public

LoggingLevel — Verbosity level of logged diagnostics

1 (default) | 0 | 2 | 3 | 4 | matlab.automation.Verbosity enumeration object | text representation of enumeration

Verbosity level of logged diagnostics, specified as an integer scalar from 0 through 4, a matlab.automation.Verbosity enumeration object, or a text representation of the enumeration. The task includes diagnostics logged at the specified level and below.

Numeric Representation	Enumeration Member Name	Verbosity Description
0	None	No information
1	Terse	Minimal information
2	Concise	Moderate amount of information
3	Detailed	Some supplemental information
4	Verbose	Lots of supplemental information

By default, the task includes diagnostics logged at the matlab.automation.Verbosity.Terse level (level 1). To exclude logged diagnostics, specify LoggingLevel as matlab.automation.Verbosity.None (level 0).

Logged diagnostics are diagnostics that you supply to the unit testing framework with the log (TestCase) and log (Fixture) methods.

Example: task = matlab.buildtool.tasks.TestTask(LoggingLevel="detailed") creates a task that includes diagnostics logged at the matlab.automation.Verbosity.Detailed level and below.

Attributes:

GetAccess	public
SetAccess	public

Strict — Option to apply strict checks

false or 0 (default) | true or 1

Option to apply strict checks when running the tests, specified as a numeric or logical 0 (false) or 1 (true). If the value is true, the task produces a qualification failure when a test issues a warning. By default, the task does not apply strict checks when running the tests.

Attributes:

GetAccess	public
SetAccess	public

TestResults — Test results

matlab.buildtool.io.File row vector

Since R2024a

Test results, returned as a matlab.buildtool.io.File row vector. You can set this property during creation of the task using a string vector, character vector, cell vector of character vectors, or matlab.buildtool.io.File vector.

The task can produce test results in various formats. Specify formats by using file extensions:

• .html — Produce test results as a single-file HTML test report.

• .pdf — Produce test results as a PDF test report.

• .xml — Produce test results in JUnit-style XML format.

• .mat — Save the matlab.unittest.TestResult array to a MAT-file.

• .mldatx — Export Simulink Test Manager results in MLDATX format (requires Simulink Test). (since R2024a)

Example: task = matlab.buildtool.tasks.TestTask(TestResults="test-results/report.html") creates a task that produces a single-file HTML test report.

Example: task = matlab.buildtool.tasks.TestTask(TestResults=["test-results/results.xml" "test-results/results.mat"]) creates a task that produces test results in both JUnit-style XML and MAT-file formats.

Attributes:

GetAccess	public
SetAccess	private

CodeCoverageResults — Code coverage results

matlab.buildtool.io.File row vector

Since R2024a

Code coverage results, returned as a matlab.buildtool.io.File row vector. You can set this property during creation of the task by assigning a string vector, character vector, cell vector of character vectors, or matlab.buildtool.io.File vector. Alternatively, you can set this property by using the addCodeCoverage method, which gives you more flexibility in producing code coverage results. For example, the method provides access to additional coverage metrics (requires MATLAB Test).

The task can produce code coverage results in various formats. Specify formats by using file extensions when you assign a value to the property:

• .html — Produce an interactive HTML code coverage report.

• .xml — Produce code coverage results in Cobertura XML format.

• .mat — Save the matlab.coverage.Result array to a MAT-file.

Note

To collect code coverage information, you must specify your MATLAB source code by setting the SourceFiles property.

Example: task = matlab.buildtool.tasks.TestTask(SourceFiles="mySource",CodeCoverageResults="code-coverage/report.html") creates a task that produces an interactive HTML code coverage report for the specified source code.

Example: task = matlab.buildtool.tasks.TestTask(SourceFiles="mySource",CodeCoverageResults=["code-coverage/results.xml" "code-coverage/results.mat"]) creates a task that produces code coverage results in both Cobertura XML and MAT-file formats for the specified source code.

Attributes:

GetAccess	public
SetAccess	private

ModelCoverageResults — Model coverage results

matlab.buildtool.io.File row vector

Since R2024a

Model coverage results for the models in Simulink Test and MATLAB-based Simulink tests of the Tests property, returned as a matlab.buildtool.io.File row vector (requires Simulink Test and Simulink Coverage™). You can set this property during creation of the task by assigning a string vector, character vector, cell vector of character vectors, or matlab.buildtool.io.File vector. Alternatively, you can set this property by using the addModelCoverage method, which gives you more flexibility in producing model coverage results. For example, the method lets you customize the model coverage options instead of relying on the options already applied in Simulink Test.

The task can produce model coverage results in various formats. Specify formats by using file extensions:

• .html — Produce an HTML model coverage report.

• .xml — Produce model coverage results in Cobertura XML format.

Example: task = matlab.buildtool.tasks.TestTask(ModelCoverageResults="model-coverage/report.html") creates a task that produces an HTML model coverage report.

Example: task = matlab.buildtool.tasks.TestTask(ModelCoverageResults=["model-coverage/report.html" "model-coverage/results.xml"]) creates a task that produces an HTML model coverage report as well as model coverage results in Cobertura XML format.

Attributes:

GetAccess	public
SetAccess	private

Methods

Public Methods

addCodeCoverage	Enable code coverage collection for TestTask instance
addModelCoverage (Simulink Test)	Enable model coverage collection for TestTask instance

Examples

Run Tests in Project

Run the tests in your project by using the TestTask class.

Open the example and then navigate to the test_task_example folder, which contains a build file, a source file named quadraticSolver.m, and a test file named SolverTest.m. For more information about the source and test files used in this example, see Write Simple Test Case Using Classes.

cd test_task_example

This code shows the contents of the build file. The build file contains one task that runs the tests in the current folder and its subfolders and fails the build if any of the tests fail.

function plan = buildfile
import matlab.buildtool.tasks.TestTask

% Create a plan with no tasks
plan = buildplan;

% Add a task to run the tests in the project
plan("test") = TestTask;
end

Run the "test" task. In this example, all the tests pass, and the task runs successfully.

buildtool test

** Starting test
...

Test Summary:
    Total Tests: 3
         Passed: 3
         Failed: 0
     Incomplete: 0
       Duration: 0.04164 seconds testing time.
                 
** Finished test

Generate Test and Coverage Artifacts

Run your tests and generate test and coverage results by using the TestTask class.

Open the example and then navigate to the test_task_example1 folder, which contains a build file, a source file named quadraticSolver.m, and a test file named SolverTest.m. For more information about the source and test files used in this example, see Write Simple Test Case Using Classes.

cd test_task_example1

This code shows the contents of the build file. The build file contains one task that:

• Runs the tests in the current folder and its subfolders and fails the build if any of the tests fail

• Produces test results in JUnit-style XML format

• Produces code coverage results in Cobertura XML format for the source code in quadraticSolver.m

function plan = buildfile
import matlab.buildtool.tasks.TestTask

% Create a plan with no tasks
plan = buildplan;

% Add a task to run tests and generate test and coverage results
plan("test") = TestTask(SourceFiles="quadraticSolver.m", ...
    TestResults="test-results/results.xml", ...
    CodeCoverageResults="code-coverage/results.xml");
end

Run the "test" task. The task runs the tests and saves the test and coverage results to the specified locations in your current folder. In this example, all the tests pass, and the task runs successfully.

buildtool test

** Starting test
...

Test Summary:
    Total Tests: 3
         Passed: 3
         Failed: 0
     Incomplete: 0
       Duration: 0.023641 seconds testing time.
                 

Test Results:
    JUnit: test-results\results.xml

Code Coverage:
    Cobertura: code-coverage\results.xml
** Finished test

More About

Up-To-Date Check

To avoid redundant work, the build tool does not run a TestTask instance if it is up to date and its DisableIncremental property is false. The build tool runs the task if the tests, source code, artifacts, or certain properties have changed since the last run:

• Tests — If you modify a test file, test class folder, or superclass of a test or if you change the value of the Tests property, the build tool runs the task.

• Source code — If you have not specified any source files and folders or if you change the value of the SourceFiles property, the build tool runs the task.

• Test and coverage artifacts — If you generate a new test or coverage artifact or if you modify or remove an existing artifact, the build tool runs the task.

• Test outcome — If it is possible to have a different test outcome, the build tool runs the task. Therefore, if you change the value of the IncludeSubfolders, Tag, or Strict property, the build tool runs the task.

Tips

• A TestTask instance respects the build output verbosity level specified at run time using the -verbosity option of the buildtool command. The run-time verbosity level takes precedence over any values specified in the OutputDetail and LoggingLevel properties. (since R2024b)

• You cannot overwrite or remove the actions of a built-in task, but you can specify additional task actions. For example, append an action to the Actions property of a built-in task.

  plan("myTask").Actions(end+1) = @myAction;

Version History

Introduced in R2023b

R2024a: Programmatically access test or coverage results

The matlab.buildtool.tasks.TestTask class has three new properties: TestResults, CodeCoverageResults, and ModelCoverageResults. You can use these properties to programmatically access the test or coverage results. For example, you can return the paths to the test results by using p = plan("test").TestResults.paths'. You can also specify the test or coverage results as inputs of other tasks, for instance, plan("archive").Inputs = plan("test").CodeCoverageResults.

R2024a: Collect code coverage information using addCodeCoverage method

You can use the addCodeCoverage method to enable a task to collect code coverage information for your MATLAB source code. If you have a MATLAB Test license, you can also use the method to produce decision coverage, condition coverage, and modified condition/decision coverage (MC/DC) metrics.

R2024a: Produce Simulink Test Manager results and model coverage results

If you have a Simulink Test license, you can export Simulink Test Manager results in MLDATX format by setting the TestResults property during task creation. If you also have a Simulink Coverage license, you can produce model coverage results by setting the ModelCoverageResults property during task creation or by using the addModelCoverage method.