Simulink.sdi.compareRuns

Compare data in two simulation runs

Syntax

diffResult = Simulink.sdi.compareRuns(runID1,runID2)

diffResult = Simulink.sdi.compareRuns(runID1,runID2,Name=Value)

Description

diffResult = Simulink.sdi.compareRuns(runID1,runID2) compares the data in the runs that correspond to runID1 and runID2 and returns the result in the Simulink.sdi.DiffRunResult object diffResult. For more information about the comparison algorithm, see How the Simulation Data Inspector Compares Data.

diffResult = Simulink.sdi.compareRuns(runID1,runID2,Name=Value) compares the simulation runs that correspond to runID1 and runID2 using the options specified by one or more name-value arguments. For more information about comparison options, see How the Simulation Data Inspector Compares Data.

Examples

Compare Runs with Global Tolerance

You can specify global tolerance values to use when comparing two simulation runs. Global tolerance values are applied to all signals within the run. This example shows how to specify global tolerance values for a run comparison and how to analyze and save the comparison results.

Load the Simulation Data Inspector session file that contains the data to compare. The session file contains data for four simulations of an aircraft longitudinal flight controller. This example compares data from two runs that use different input filter time constants.

Simulink.sdi.load("AircraftExample.mldatx");

To access the run data to compare, use the Simulink.sdi.getAllRunIDs function to get the run IDs that correspond to the last two simulation runs.

runIDs = Simulink.sdi.getAllRunIDs;
runID1 = runIDs(end - 1);
runID2 = runIDs(end);

Use the Simulink.sdi.compareRuns function to compare the runs. Specify a global relative tolerance value of 0.2 and a global time tolerance value of 0.5.

runResult = Simulink.sdi.compareRuns(runID1,runID2,"reltol",0.2,"timetol",0.5);

Check the Summary property of the returned Simulink.sdi.DiffRunResult object to see whether signals are within the tolerance values or out of tolerance.

runResult.Summary

ans = struct with fields:
       OutOfTolerance: 0
      WithinTolerance: 3
            Unaligned: 0
        UnitsMismatch: 0
                Empty: 0
             Canceled: 0
          EmptySynced: 0
     DataTypeMismatch: 0
         TimeMismatch: 0
    StartStopMismatch: 0
          Unsupported: 0

All three signal comparison results fall within the specified global tolerance.

You can save the comparison results to an MLDATX file using the saveResult function.

saveResult(runResult,"InputFilterComparison");

Analyze Simulation Data Using Signal Tolerances

You can programmatically specify signal tolerance values to use in comparisons performed using the Simulation Data Inspector. In this example, you compare data collected by simulating a model of an aircraft longitudinal flight control system. Each simulation uses a different value for the input filter time constant and logs the input and output signals. You analyze the effect of the time constant change by comparing results using the Simulation Data Inspector and signal tolerances.

First, load the session file that contains the simulation data.

Simulink.sdi.load('AircraftExample.mldatx');

The session file contains four runs. In this example, you compare data from the first two runs in the file. Access the Simulink.sdi.Run objects for the first two runs loaded from the file.

runIDs = Simulink.sdi.getAllRunIDs;
runIDTs1 = runIDs(end-3);
runIDTs2 = runIDs(end-2);

Compare the two runs without specifying any tolerances.

noTolDiffResult = Simulink.sdi.compareRuns(runIDTs1,runIDTs2);

Use the getResultByIndex function to access the comparison results for the q and alpha signals.

qResult = getResultByIndex(noTolDiffResult,1);
alphaResult = getResultByIndex(noTolDiffResult,2);

Check the Status property of each signal result to see whether the comparison result falls within or out of tolerance.

qResult.Status

ans = 
  ComparisonSignalStatus enumeration

    OutOfTolerance

alphaResult.Status

ans = 
  ComparisonSignalStatus enumeration

    OutOfTolerance

The comparison uses a value of 0 for all tolerances, so the OutOfTolerance result means the signals are not identical.

You can further analyze the effect of the time constant by specifying tolerance values for the signals. Specify the tolerances by setting the properties for the Simulink.sdi.Signal objects that correspond to the signals being compared. Comparisons use tolerances specified for the baseline signals. This example specifies a time tolerance and an absolute tolerance.

To specify a tolerance, first access the Signal objects from the baseline run.

runTs1 = Simulink.sdi.getRun(runIDTs1);
qSig = getSignalsByName(runTs1,'q, rad/sec');
alphaSig = getSignalsByName(runTs1,'alpha, rad');

For the q signal, specify an absolute tolerance of 0.1 and a time tolerance of 0.6 using the AbsTol and TimeTol properties, respectively.

qSig.AbsTol = 0.1;
qSig.TimeTol = 0.6;

For the alpha signal, specify an absolute tolerance of 0.2 and a time tolerance of 0.8.

alphaSig.AbsTol = 0.2;
alphaSig.TimeTol = 0.8;

Compare the results again. Access the results from the comparison and check the Status property for each signal.

tolDiffResult = Simulink.sdi.compareRuns(runIDTs1,runIDTs2);
qResult2 = getResultByIndex(tolDiffResult,1);
alphaResult2 = getResultByIndex(tolDiffResult,2);

qResult2.Status

ans = 
  ComparisonSignalStatus enumeration

    WithinTolerance

alphaResult2.Status

ans = 
  ComparisonSignalStatus enumeration

    WithinTolerance

Configure Comparisons to Check Metadata

You can use the Simulink.sdi.compareRuns function to compare signal data and metadata, including data type and start and stop times. A single comparison may check for mismatches in one or more pieces of metadata. When you check for mismatches in signal metadata, the Summary property of the Simulink.sdi.DiffRunResult object may differ from a basic comparison because the Status property for a Simulink.sdi.DiffSignalResult object can indicate the metadata mismatch. You can configure comparisons using the Simulink.sdi.compareRuns function for imported data and for data logged from a simulation.

This example configures a comparison of runs created from workspace data three ways to show how the Summary of the DiffSignalResult object can provide specific information about signal mismatches.

Create Workspace Data

The Simulink.sdi.compareRuns function compares time series data. Create data for a sine wave to use as the baseline signal, using the timeseries format. Give the timeseries the name Wave Data.

time = 0:0.1:20;
sig1vals = sin(2*pi/5*time);
sig1_ts = timeseries(sig1vals,time);
sig1_ts.Name = 'Wave Data';

Create a second sine wave to compare against the baseline signal. Use a slightly different time vector and attenuate the signal so the two signals are not identical. Cast the signal data to the single data type. Also name this timeseries object Wave Data. The Simulation Data Inspector comparison algorithm will align these signals for comparison using the name.

time2 = 0:0.1:22;
sig2vals = single(0.98*sin(2*pi/5*time2));
sig2_ts = timeseries(sig2vals,time2);
sig2_ts.Name = 'Wave Data';

Create and Compare Runs in the Simulation Data Inspector

The Simulink.sdi.compareRuns function compares data contained in Simulink.sdi.Run objects. Use the Simulink.sdi.createRun function to create runs in the Simulation Data Inspector for the data. The Simulink.sdi.createRun function returns the run ID for each created run.

runID1 = Simulink.sdi.createRun('Baseline Run','vars',sig1_ts);
runID2 = Simulink.sdi.createRun('Compare to Run','vars',sig2_ts);

You can use the Simulink.sdi.compareRuns function to compare the runs. The comparison algorithm converts the signal data to the double data type and synchronizes the signal data before computing the difference signal.

basic_DRR = Simulink.sdi.compareRuns(runID1,runID2);

Check the Summary property of the returned Simulink.sdi.DiffRunResult object to see the result of the comparison.

basic_DRR.Summary

ans = struct with fields:
       OutOfTolerance: 1
      WithinTolerance: 0
            Unaligned: 0
        UnitsMismatch: 0
                Empty: 0
             Canceled: 0
          EmptySynced: 0
     DataTypeMismatch: 0
         TimeMismatch: 0
    StartStopMismatch: 0
          Unsupported: 0

The difference between the signals is out of tolerance.

Compare Runs and Check for Data Type Match

Depending on your system requirements, you may want the data types for signals you compare to match. You can use the Simulink.sdi.compareRuns function to configure the comparison algorithm to check for and report data type mismatches.

dataType_DRR = Simulink.sdi.compareRuns(runID1,runID2,'DataType','MustMatch');
dataType_DRR.Summary

ans = struct with fields:
       OutOfTolerance: 0
      WithinTolerance: 0
            Unaligned: 0
        UnitsMismatch: 0
                Empty: 0
             Canceled: 0
          EmptySynced: 0
     DataTypeMismatch: 1
         TimeMismatch: 0
    StartStopMismatch: 0
          Unsupported: 0

The result of the signal comparison is now DataTypeMismatch because the data for the baseline signal is double data type, while the data for the signal compared to the baseline is single data type.

Compare Runs and Check for Start and Stop Time Match

You can use the Simulink.sdi.compareRuns function to configure the comparison algorithm to check whether the aligned signals have the same start and stop times.

startStop_DRR = Simulink.sdi.compareRuns(runID1,runID2,'StartStop','MustMatch');
startStop_DRR.Summary

ans = struct with fields:
       OutOfTolerance: 0
      WithinTolerance: 0
            Unaligned: 0
        UnitsMismatch: 0
                Empty: 0
             Canceled: 0
          EmptySynced: 0
     DataTypeMismatch: 0
         TimeMismatch: 0
    StartStopMismatch: 1
          Unsupported: 0

The signal comparison result is now StartStopMismatch because the signals created in the workspace have different stop times.

Compare Runs with Alignment Criteria

When you compare runs using the Simulation Data Inspector, you can specify alignment criteria that determine how signals are paired with each other for comparison. This example compares data from simulations of a model of an aircraft longitudinal control system. The simulations used a square wave input. The first simulation used an input filter time constant of 0.1s and the second simulation used an input filter time constant of 0.5s.

First, load the simulation data from the session file that contains the data for this example.

Simulink.sdi.load('AircraftExample.mldatx');

The session file contains data for four simulations. This example compares data from the first two runs. Access the run IDs for the first two runs loaded from the session file.

runIDs = Simulink.sdi.getAllRunIDs;
runIDTs1 = runIDs(end-3);
runIDTs2 = runIDs(end-2);

Before running the comparison, define how you want the Simulation Data Inspector to align the signals between the runs. This example aligns signals by their name, then by their block path, and then by their Simulink identifier.

alignMethods = [Simulink.sdi.AlignType.SignalName
               Simulink.sdi.AlignType.BlockPath
               Simulink.sdi.AlignType.SID];

Compare the simulation data in your two runs, using the alignment criteria you specified. The comparison uses a small time tolerance to account for the effect of differences in the step size used by the solver on the transition of the square wave input.

diffResults = Simulink.sdi.compareRuns(runIDTs1,runIDTs2,'align',alignMethods,...
    'timetol',0.005);

You can use the getResultByIndex function to access the comparison results for the aligned signals in the runs you compared. You can use the Count property of the Simulink.sdi.DiffRunResult object to set up a for loop to check the Status property for each Simulink.sdi.DiffSignalResult object.

numComparisons = diffResults.count;

for k = 1:numComparisons
    resultAtIdx = getResultByIndex(diffResults,k);
    
    sigID1 = resultAtIdx.signalID1;
    sigID2 = resultAtIdx.signalID2;
    
    sig1 = Simulink.sdi.getSignal(sigID1);
    sig2 = Simulink.sdi.getSignal(sigID2);
    
    displayStr = 'Signals %s and %s: %s \n';
    fprintf(displayStr,sig1.Name,sig2.Name,resultAtIdx.Status);
end

Signals q, rad/sec and q, rad/sec: OutOfTolerance 
Signals alpha, rad and alpha, rad: OutOfTolerance 
Signals Stick and Stick: WithinTolerance 

Input Arguments

runID1 — Baseline run identifier

integer

Numeric identifier for the baseline run in the comparison, specified as a run ID that corresponds to a run in the Simulation Data Inspector. The Simulation Data Inspector assigns run IDs when runs are created. You can get the run ID for a run by using the ID property of the Simulink.sdi.Run object, the Simulink.sdi.getAllRunIDs function, or the Simulink.sdi.getRunIDByIndex function.

runID2 — Identifier for run to compare

integer

Numeric identifier for the run to compare, specified as a run ID that corresponds to a run in the Simulation Data Inspector. The Simulation Data Inspector assigns run IDs when runs are created. You can get the run ID for a run by using the ID property of the Simulink.sdi.Run object, the Simulink.sdi.getAllRunIDs function, or the Simulink.sdi.getRunIDByIndex function.

Name-Value Arguments

Specify optional pairs of arguments as Name1=Value1,...,NameN=ValueN, where Name is the argument name and Value is the corresponding value. Name-value arguments must appear after other arguments, but the order of the pairs does not matter.

Before R2021a, use commas to separate each name and value, and enclose Name in quotes.

Example: AbsTol=x,Align=alignOpts

Align — Signal alignment options

Simulink.sdi.AlignType scalar | Simulink.sdi.AlignType vector

Signal alignment options, specified as a Simulink.sdi.AlignType scalar or vector. The Simulink.sdi.AlignType enumeration includes a value for each option available for pairing each signal in the baseline run with a signal in the comparison run. You can specify one or more alignment options for the comparison. To use more than one alignment option, specify an array. When you specify multiple alignment options, the Simulation Data Inspector aligns signals first by the option in the first element of the array, then by the option in the second element array, and so on. For more information, see Signal Alignment.

Value	Aligns By
Simulink.sdi.AlignType.BlockPath	Path to the source block for the signal
Simulink.sdi.AlignType.SID	Automatically assigned Simulink® identifier
Simulink.sdi.AlignType.SignalName	Signal name
Simulink.sdi.AlignType.DataSource	Path of the variable in the MATLAB® workspace

Example: [Simulink.sdi.AlignType.SignalName,Simulink.sdi.AlignType.BlockPath] specifies signal alignment by signal name and then by block path.

AbsTol — Global absolute tolerance for comparison

0 (default) | positive-valued scalar

Global absolute tolerance for comparison, specified as a positive-valued scalar.

Global tolerances apply to all signals in the run comparison. To use a different tolerance value for a signal in the comparison, specify the tolerance you want to use on the Simulink.sdi.Signal object in the baseline run and set the OverrideGlobalTol property for that signal to true.

The Simulation Data Inspector applies the absolute tolerance to only numeric signals in the comparison and ignores the absolute tolerance for any nonnumeric signals, such as those that contain enumerated or Boolean data.

For more information about how tolerances are used in comparisons, see Tolerance Specification.

Example: 0.5

Data Types: double

RelTol — Global relative tolerance for comparison

0 (default) | positive-valued scalar

Global relative tolerance for comparison, specified as a positive-valued scalar. The relative tolerance is expressed as a fractional multiplier. For example, 0.1 specifies a 10 percent tolerance.

Global tolerances apply to all signals in the run comparison. To use a different tolerance value for a signal in the comparison, specify the tolerance you want to use on the Simulink.sdi.Signal object in the baseline run and set the OverrideGlobalTol property for that signal to true.

The Simulation Data Inspector applies the relative tolerance to only numeric signals in the comparison and ignores the relative tolerance for any nonnumeric signals, such as those that contain enumerated or Boolean data.

For more information about how tolerances are used in comparisons, see Tolerance Specification.

Example: 0.1

Data Types: double

TimeTol — Global time tolerance for comparison

0 (default) | positive-valued scalar

Global time tolerance for comparison, specified as a positive-valued scalar, using units of seconds.

Global tolerances apply to all signals in the run comparison. To use a different tolerance value for a signal in the comparison, specify the tolerance you want to use on the Simulink.sdi.Signal object in the baseline run and set the OverrideGlobalTol property for that signal to true.

For more information about tolerances in the Simulation Data Inspector, see Tolerance Specification.

Example: 0.2

Data Types: double

DataType — Comparison sensitivity to signal data types

"MustMatch"

Comparison sensitivity to signal data types, specified as "MustMatch". Specify DataType="MustMatch" when you want the comparison to be sensitive to numeric data type mismatches in compared signals.

When signal data types do not match, the Status property of the Simulink.sdi.DiffSignalResult object for the result is set to DataTypeMismatch.

The Simulink.sdi.compareRuns function compares the data types for aligned signals before synchronizing and comparing the signal data. When you do not specify this name-value argument, the comparison checks data types only to detect a comparison between string and numeric data. For a comparison between string and numeric data, results are not computed, and the status for the result is DataTypeMismatch. For aligned signals that have different numeric data types, the comparison computes results.

When you configure the comparison to stop on the first mismatch, a data type mismatch stops the comparison. A stopped comparison may not compute results for all signals.

Time — Comparison sensitivity to signal time vectors

"MustMatch"

Comparison sensitivity to signal time vectors, specified as "MustMatch". Specify Time="MustMatch" when you want the comparison to be sensitive to mismatches in the time vectors of compared signals. When you specify this name-value argument, the algorithm compares the time vectors of aligned signals before synchronizing and comparing the signal data.

When the time vectors for signals do not match, the Status property of the Simulink.sdi.DiffSignalResult object for the result is set to TimeMismatch.

Comparisons are not sensitive to differences in signal time vectors unless you specify this name-value argument. For comparisons that are not sensitive to differences in the time vectors, the comparison algorithm synchronizes the signals prior to the comparison. For more information about how synchronization works, see How the Simulation Data Inspector Compares Data.

When you specify that time vectors must match and configure the comparison to stop on the first mismatch, a time vector mismatch stops the comparison. A stopped comparison may not compute results for all signals.

StartStop — Comparison sensitivity to signal start and stop times

"MustMatch"

Comparison sensitivity to signal start and stop times, specified as "MustMatch". Specify StartStop="MustMatch" when you want the comparison to be sensitive to mismatches in signal start and stop times. When you specify this name-value argument, the algorithm compares the start and stop times for aligned signals before synchronizing and comparing the signal data.

When the start times and stop times do not match, the Status property of the Simulink.sdi.DiffSignalResult object for the result is set to StartStopMismatch.

When you specify that start and stop times must match and configure the comparison to stop on the first mismatch, a start or stop time mismatch stops the comparison. A stopped comparison may not compute results for all signals.

StopOnFirstMismatch — Whether comparison stops on first detected mismatch

"Metadata" | "Any"

Whether comparison stops on first detected mismatch without comparing remaining signals, specified as "Metadata" or "Any". A stopped comparison may not compute results for all signals, and can return a mismatched result more quickly.

• "Metadata" — A mismatch in metadata for aligned signals stops the comparison. Metadata comparisons happen before comparing signal data.

  The Simulation Data Inspector always aligns signals and compares signal units. When you configure the comparison to stop on the first mismatch, an unaligned signal or mismatched units always stop the comparison. You can specify additional name-value arguments to configure the comparison to check and stop on the first mismatch for additional metadata, such as signal data type, start and stop times, and time vectors.

• "Any" — A mismatch in metadata or signal data for aligned signals stops the comparison.

ExpandChannels — Whether to compute comparison results for each channel in multidimensional signals

true or 1 (default) | false or 0

Whether to compute comparison results for each channel in multidimensional signals, specified as logical true (1) or false (0).

• true or 1 — Comparison expands multidimensional signals represented as a single signal with nonscalar sample values to a set of signals with scalar sample values and computes a comparison result for each signal.

  The representation of the multidimensional signal in the Simulation Data Inspector as a single signal with nonscalar sample values does not change.

• false or 0 — Comparison does not compute results for multidimensional signals represented as a single signal with nonscalar sample values.

Output Arguments

diffResult — Comparison results

Simulink.sdi.DiffRunResult object

Comparison results, returned as a Simulink.sdi.DiffRunResult object.

Limitations

The Simulation Data Inspector does not support comparing:

• Before R2020a: Signals of data types int64 or uint64.

• Variable-size signals.

Version History

Introduced in R2011b