Main Content

Model Testing Metrics

The Model Testing Dashboard collects metric data from the model design and testing artifacts in a project, such as requirements, models, and test results. Use the metric data to assess the status and quality of your model testing. Each metric in the dashboard measures a different aspect of the quality of the testing of your model and reflects guidelines in industry-recognized software development standards, such as ISO 26262 and DO-178. Use the widgets in the Model Testing Dashboard to see high-level metric results and testing gaps, as described in Explore Status and Quality of Testing Activities Using Model Testing Dashboard.

Alternatively, you can use the API functions to collect metric results programmatically. When using the API, use the metric IDs to refer to each metric. See Collect Metrics on Model Testing Artifacts Programmatically for an example of how to collect these metrics programmatically. You can use the function getAvailableMetricIds to return a list of available metric identifiers.

There are model testing metrics that can analyze:

Requirements Linked to Tests

The metrics associated with the Requirements Linked to Tests section of the dashboard include:

MetricDescription
Requirement with Test Case

Determine whether a requirement is linked to tests.

Requirement with Test Case Distribution

Distribution of the number of requirements linked to tests compared to the number of requirements that are missing tests.

Test Cases per Requirement

Count the number of tests linked to each requirement.

Test Cases per Requirement Distribution

Distribution of the number of tests linked to each requirement.

Tests Linked to Requirements

The metrics associated with the Tests Linked to Requirements section of the dashboard include:

MetricDescription
Test Case with Requirement

Determine whether a test is linked to requirements.

Test Case with Requirement Distribution

Distribution of the number of tests linked to requirements compared to the number of tests that are missing links to requirements.

Requirements per Test Case

Count the number of requirements linked to each test.

Requirements per Test Case Distribution

Distribution of the number of requirements linked to each test.

Test Case Types and Tags

The metrics associated with the Test Case Breakdown section of the dashboard include:

MetricDescription
Test Case Type

Return the type of the test case.

Test Case Type Distribution

Distribution of the types of the test cases for the unit.

Test Case Tag

Return the tags for a test case.

Test Case Tag Distribution

Distribution of the tags of the test cases for the unit.

Status of Model Tests

The metrics associated with the Model Test Status section of the dashboard include:

MetricDescription
Model Test Status

Return the status of the model test results.

Model Test Status Distribution

Distribution of the statuses of the model test results for the unit.

Test Case Verification Status

Determine whether a model test has pass/fail criteria such as verify statements, verification blocks, custom criteria, and logical or temporal assessments.

Test Case Verification Status Distribution

Distribution of the number of model tests that do not have pass/fail criteria compared to the number of tests that do have pass/fail criteria.

Model Coverage

The widgets in the Model Coverage section of the dashboard shows the aggregated coverage for the unit.

MetricDescription
Model Coverage BreakdownOverall execution, decision, condition, and modified condition and decision coverage (MC/DC) achieved, justified, or missed by the model tests in the unit.

When you click on a bar in the Model Coverage bar chart, the Metric Details show the coverage for each model in the unit.

MetricDescription
Model Coverage FragmentOverall execution, decision, condition, and modified condition and decision coverage (MC/DC) achieved, justified, or missed by the model tests for each model in the unit.

Requirements-Based Tests

The Achieved Coverage Ratio section of the dashboard shows the sources of achieved coverage for the unit. The Requirements-Based Tests section shows how much of the overall achieved coverage comes from requirements-based tests.

The metrics associated with the widgets in the Requirements-Based Tests section of the dashboard include:

MetricDescription
Requirements Execution Coverage Breakdown

Fraction of the overall achieved execution coverage that comes from requirements-based tests in the unit.

Requirements Decision Coverage Breakdown

Fraction of the overall achieved decision coverage that comes from requirements-based tests in the unit.

Requirements Condition Coverage Breakdown

Fraction of the overall achieved condition coverage that comes from requirements-based tests in the unit.

Requirements MCDC Coverage Breakdown

Fraction of the overall achieved MC/DC coverage that comes from requirements-based tests in the unit.

When you click on a bar in the Requirements-Based Tests section, the Metric Details show the coverage ratio for each model in the unit. For requirements-based tests, the coverage ratio is the percentage of the overall achieved coverage that comes from requirements-based tests.

The metrics associated with requirements-based coverage for each model in the unit include:

MetricDescription
Requirements Execution Coverage Fragment

Fraction of the overall achieved execution coverage that comes from requirements-based tests in each model in the unit.

Requirements Decision Coverage Fragment

Fraction of the overall achieved decision coverage that comes from requirements-based tests in each model in the unit.

Requirements Condition Coverage Fragment

Fraction of the overall achieved condition coverage that comes from requirements-based tests in each model in the unit.

Requirements MCDC Coverage Fragment

Fraction of the overall achieved MC/DC coverage that comes from requirements-based tests in each model in the unit.

Unit-Boundary Tests

The Achieved Coverage Ratio section of the dashboard shows the sources of achieved coverage for the unit. The Unit-Boundary Tests section shows how much of the overall achieved coverage comes from unit-boundary tests.

The metrics associated with the widgets in the Unit-Boundary Tests section of the dashboard include:

MetricDescription
Unit Boundary Execution Coverage Breakdown

Fraction of the overall achieved execution coverage that comes from unit-boundary tests in the unit.

Unit Boundary Decision Coverage Breakdown

Fraction of the overall achieved decision coverage that comes from unit-boundary tests in the unit.

Unit Boundary Condition Coverage Breakdown

Fraction of the overall achieved condition coverage that comes from unit-boundary tests in the unit.

Unit Boundary MCDC Coverage Breakdown

Fraction of the overall achieved MC/DC coverage that comes from unit-boundary tests in the unit.

When you click on a bar in the Unit-Boundary Tests section, the Metric Details show the coverage ratio for each model in the unit. For unit-boundary tests, the coverage ratio is the percentage of the overall achieved coverage that comes from unit-boundary tests.

The metrics associated with unit-boundary coverage for each model in the unit include:

MetricDescription
Unit Boundary Execution Coverage Fragment

Fraction of the overall achieved execution coverage that comes from unit-boundary tests in each model in the unit.

Unit Boundary Decision Coverage Fragment

Fraction of the overall achieved decision coverage that comes from unit-boundary tests in each model in the unit.

Unit Boundary Condition Coverage Fragment

Fraction of the overall achieved condition coverage that comes from unit-boundary tests in each model in the unit.

Unit Boundary MCDC Coverage Fragment

Fraction of the overall achieved MC/DC coverage that comes from unit-boundary tests in each model in the unit.

See Also

Related Topics