trackErrorMetrics

Track error and NEES

Description

The trackErrorMetrics System object™ provides quantitative comparisons between tracks and known truth trajectories.

To generate track assignment metrics:

1. Create the trackErrorMetrics object and set its properties.

2. Call the object with arguments, as if it were a function.

To learn more about how System objects work, see What Are System Objects?

Creation

Syntax

errorMetrics = trackErrorMetrics

errorMetrics = trackErrorMetrics(Name,Value)

Description

errorMetrics = trackErrorMetrics creates a trackErrorMetrics System object with default property values.

errorMetrics = trackErrorMetrics(Name,Value) sets properties for the trackErrorMetrics object using one or more name-value pairs. For example, metrics = trackErrorMetrics('MotionModel','constvel') creates a trackErrorMetrics object with a constant velocity motion model. Enclose property names in quotes.

Properties

Unless otherwise indicated, properties are nontunable, which means you cannot change their values after calling the object. Objects lock when you call them, and the release function unlocks them.

If a property is tunable, you can change its value at any time.

For more information on changing property values, see System Design in MATLAB Using System Objects.

ErrorFunctionFormat — Error function format

'built-in' (default) | 'custom'

Error function format specified as 'built-in' or 'custom'.

• 'built-in' – Enable the MotionModel property.

  This property is a convenient interface when using tracks reported by any built-in multi-object tracker, and truths reported by the platformPoses object function of a trackingScenario object. The default estimation error function assumes tracks and truths are arrays of structures or arrays of objects.

• 'custom' – Enable custom properties: EstimationErrorLabels, EstimationErrorFcn, TruthIdentifierFcn, and TrackIdentifierFcns. These properties can be used to construct error functions for arbitrary tracks and truths input arrays.

MotionModel — Desired platform motion model

'constvel' (default) | 'constacc' | 'constturn' | 'singer'

Desired platform motion model, specified as 'constvel', 'constacc', 'constturn', or 'singer'. This property selects the motion model used by the tracks input.

The motion models expect the 'State' field of the tracks to have a column vector containing these values:

• 'constvel' — Position is in elements [1 3 5], and velocity is in elements [2 4 6].

• 'constacc' — Position is in elements [1 4 7], velocity is in elements [2 5 8], and acceleration is in elements [3 6 9].

• 'constturn' — Position is in elements [1 3 6], velocity is in elements [2 4 7], and yaw rate is in element 5.

• 'singer' — Position is in elements [1 4 7], velocity is in elements [2 5 8], and acceleration is in elements [3 6 9].

The 'StateCovariance' field of the tracks input must have position, velocity, and turn-rate covariances in the rows and columns corresponding to the position, velocity, and turn-rate of the 'State' field of the tracks input.

EstimationErrorLabels — Labels for outputs of error estimation function

'posMSE' (default) | array of strings | cell array of character vectors

Labels for outputs of error estimation function, specified as an array of strings or cell array of character vectors. The number of labels must correspond to the number of outputs of the error estimation function. Specify the error estimation functions using the EstimationErrorFcn property.

Example: {'posMSE','velMSE'}

Dependencies

To enable this property, set the ErrorFunctionFormat property to 'custom'.

Data Types: char | string

EstimationErrorFcn — Error estimation function

function handle

Error estimation function, specified as a function handle. The function determines estimation errors of truths to tracks.

The error estimation function can have multiple scalar outputs and must have the following syntax.

 [out1,out2, ...,outN] = estimationerror(onetrack,onetruth)

The number of outputs must match the number of entries in the labels array specified in the EstimationErrorLabels property.

onetrack is an element of the tracks array passed in as input trackErrorMetric at object updates. onetruth is an element of the truths array passed in at object updates. The trackErrorMetrics object averages each output arithmetically when reporting across tracks or truths.

Example: @errorFunction

Dependencies

To enable this property, set the ErrorFunctionFormat property to 'custom'.

Data Types: function_handle

TrackIdentifierFcn — Track identifier function

@trackIDFunction (default) | function handle

Track identifier function, specified as a function handle. Specifies the track identifiers for the tracks input at object update. The track identifiers are unique string or numeric values.

The track identifier function must have the following syntax:

 trackID = trackIDentifier(tracks)

tracks is the same as the tracks array passed as input for trackErrorMetric at object update. trackID is the same size as tracks. The default identification function handle, @defaultTrackIdentifier, assumes tracks is an array of structures or objects with a 'TrackID' field name or property.

Dependencies

To enable this property, set the ErrorFunctionFormat property to 'custom'.

Data Types: function_handle

TruthIdentifierFcn — Truth identifier function

@truthIDFunction (default) | function handle

Truth identifier function, specified as a function handle. Specifies the truth identifiers for the truths input at object update. The truth identifiers are unique string or numeric values.

The truth identifier function must have the following syntax:

 truthID = truthIDentifier(truths)

truths is the same as the truths array passed as input for trackErrorMetric updates. truthID must have the same size as truths. The default identification function handle, @defaultTruthIdentifier, assumes truths is an array or structures or objects with a 'PlatformID' field name or property.

Dependencies

To enable this property, set the ErrorFunctionFormat property to 'custom'.

Data Types: function_handle

Usage

To estimate errors, call the track error metrics object with arguments, as if it were a function (described here).

Syntax

[posRMSE,velRMSE,posANEES,velANEES] = errorMetrics(tracks,trackIDs,truths,truthIDs)

[posRMSE,velRMSE,accRMSE,posANEES,velANEES,accANEES] = errorMetrics(tracks,trackIDs,truths,truthIDs)

[posRMSE,velRMSE,yawRateRMSE,posANEES,velANEES,yawRateANEES] = errorMetrics(tracks,trackIDs,truths,truthIDs)

[out1,out2, ... ,outN] = errorMetrics(tracks,trackIDs,truths,truthIDs)

Description

[posRMSE,velRMSE,posANEES,velANEES] = errorMetrics(tracks,trackIDs,truths,truthIDs) returns the metrics

• posRMSE – Position root mean squared error

• velRMSE – Velocity root mean squared error

• posANEES – Position average normalized-estimation error squared

• velANEES – Velocity average normalized-estimation error squared

for constant velocity motion at the current time step. trackIDs is the set of track identifiers for all tracks. truthIDs is the set of truth identifiers. tracks are the set of tracks, and truths are the set of truths. trackIDs and truthIDs are each a vector whose corresponding elements match the track and truth identifiers found in tracks and truths, respectively.

The RMSE and ANEES values for different states are calculated by averaging the errors of all tracks at the current time step. For example, the position RMSE value, posRMSE, is defined as:

$$\text{posRMSE}=\sqrt{\frac{1}{M}{\displaystyle \sum _{i=1}^M\Vert \Delta p_i{\Vert }^2}}$$

where M is the total number of tracks with associated truth trajectories in the current time step, and

$$\Delta p_i=p_{track,i}-p_{truth,i}$$

is the position difference between the position of track i, p_track,i, and the position of the corresponding truth, p_truth,i, at the current time step. The RMSE values for other states (vel, pos, acc, and yawRate) are defined similarly.

The position ANEES value, posANEES, is defined as:

$$\text{posANEES}=\frac{1}{M}{\displaystyle \sum _{i=1}^M\Delta p_i{}^T{C}_{p,i}^{-1}\Delta p_i}$$

where C_p,i is the covariance matrix corresponding to the position of track i at the current time step. The ANEES values for other states (vel, pos, acc, and yawRate) are defined similarly.

To enable this syntax, set the ErrorFunctionFormat property to 'built-in' and the MotionModel property to 'constvel'.

[posRMSE,velRMSE,accRMSE,posANEES,velANEES,accANEES] = errorMetrics(tracks,trackIDs,truths,truthIDs) also returns the metrics

• accRMS – Acceleration root mean squared error

• accANEES – acceleration average normalized-estimation error squared

for constant acceleration motion at the current time step.

To enable this syntax, set the ErrorFunctionFormat property to 'built-in' and the MotionModel property to 'constacc'.

[posRMSE,velRMSE,yawRateRMSE,posANEES,velANEES,yawRateANEES] = errorMetrics(tracks,trackIDs,truths,truthIDs) also returns the metrics

• yawRateRMSE – yaw rate root mean squared error

• yawRateANEES – yaw rate average normalized-estimation error squared

for constant turn-rate motion at the current time step.

To enable this syntax, set the ErrorFunctionFormat property to 'built-in' and the MotionModel property to 'constturn'.

[out1,out2, ... ,outN] = errorMetrics(tracks,trackIDs,truths,truthIDs) returns the user-defined metrics out1, out2, ... , outN.

To enable this syntax, set the ErrorFunctionFormat property to 'custom'. The number of outputs corresponds to the number of elements listed in the EstimationErrorLabels property, and must match the number of outputs in the EstimationErrorFcn. The results of the estimation errors are averaged arithmetically over all track-to-truth assignments.

Tip

These usage syntaxes only calculate the RMSE and ANEES values of all tracks with associated truths at the current time step. To obtain the cumulative RMSE and ANEES values for each track and truth, use the cumulativeTrackMetrics and cumulativeTruthMetrics object functions, respectively. To obtain the current RMSE and ANEES values for each track and truth, use the currentTrackMetrics and currentTruthMetrics object functions, respectively.

Input Arguments

tracks — Track information

array of structures | array of objects

Track information, specified as an array of structures or objects. For built-in trackers such as trackerGNN or trackerTOMHT, the objectTrack output contains 'State', 'StateCovariance', and 'TrackID' information.

Data Types: struct

trackIDs — Track identifiers

real-valued vector

Track identifiers, specified as a real-valued vector. trackIDs elements match the tracks found in tracks.

truths — Truth information

array of structures | array of objects

Truth information, specified as an array of structures or objects. When using a trackingScenario, truth information can be obtained from the platformPoses object function.

Data Types: struct

truthIDs — Truth identifiers

real-valued vector

Truth identifiers, specified as a real-valued vector. truthIDs elements match the truths found in truths.

Output Arguments

posRMSE — Position root mean squared error

scalar

Position root mean squared error for all tracks associated with truths, returned as a scalar.

Dependencies

To enable this argument, set the ErrorFunctionFormat property to 'built-in'.

velRMSE — Velocity root mean squared error

scalar

Velocity root mean squared error for all tracks associated with truths, returned as a scalar.

Dependencies

To enable this argument, set the ErrorFunctionFormat property to 'built-in'.

accRMSE — Acceleration root mean squared error

scalar

Acceleration root mean squared error for all tracks associated with truths, returned as a scalar.

Dependencies

To enable this argument, set the ErrorFunctionFormat property to 'built-in'.

yawRateRMSE — Yaw rate root mean squared error

scalar

Yaw rate root mean squared error for all tracks associated with truths, returned as a scalar.

Dependencies

To enable this argument, set the ErrorFunctionFormat property to 'built-in'.

posANEES — Position average normalized estimation error squared

scalar

Position average normalized estimation error squared for all tracks associated with truths, returned as a scalar.

Dependencies

To enable this argument, set the ErrorFunctionFormat property to 'built-in'.

velANEES — Velocity average normalized estimation error squared

scalar

Velocity average normalized estimation error squared for all tracks associated with truths, returned as a scalar.

Dependencies

To enable this argument, set the ErrorFunctionFormat property to 'built-in'.

accANEES — Acceleration average normalized estimation error squared

scalar

Acceleration average normalized estimation error squared for all tracks associated with truths, returned as a scalar.

Dependencies

To enable this argument, set the ErrorFunctionFormat property to 'built-in'.

yawRateANEES — Yaw rate average normalized estimation error squared

scalar

Yaw rate average normalized estimation error squared for all tracks associated with truths, returned as a scalar.

Dependencies

To enable this argument, set the ErrorFunctionFormat property to 'built-in'.

out1, out2, outN — Custom error metric outputs

scalar

Custom error metric outputs, returned as scalars. These errors are the output of the error estimation function specified in the EstimationErrorFcn property.

Dependencies

To enable these arguments, set the ErrorFunctionFormat property to 'custom'.

Object Functions

To use an object function, specify the System object as the first input argument. For example, to release system resources of a System object named obj, use this syntax:

release(obj)

Specific to trackErrorMetrics

cumulativeTrackMetrics	Cumulative metrics for recent tracks
cumulativeTruthMetrics	Cumulative metrics for recent truths
currentTrackMetrics	Metrics for recent tracks
currentTruthMetrics	Metrics for recent truths
evaluateBatch	Compute track error metrics for recorded data

Common to All System Objects

release	Release resources and allow changes to System object property values and input characteristics
reset	Reset internal states of System object
isLocked	Determine if System object is in use
clone	Create duplicate System object

Examples

Assignment and Error Metrics for Two Tracked Targets

Examine the assignments and errors for a system tracking two targets.

First, load the stored track data.

load trackmetricex tracklog truthlog

Create objects to analyze assignment and error metrics.

tam = trackAssignmentMetrics;
tem = trackErrorMetrics;

Create the output variables.

posRMSE = zeros(numel(tracklog),1);
velRMSE = zeros(numel(tracklog),1);
posANEES = zeros(numel(tracklog),1);
velANEES = zeros(numel(tracklog),1);

Loop over all tracks to:

• Extract the tracks and ground truth at the i th tracker update.

• Analyze and retrieve the current track-to-truth assignment.

• Analyze instantaneous error metrics over all tracks and truths.

for i=1:numel(tracklog)
    tracks = tracklog{i};
    truths = truthlog{i};
    [trackAM,truthAM] = tam(tracks, truths);
    [trackIDs,truthIDs] = currentAssignment(tam);
    [posRMSE(i),velRMSE(i),posANEES(i),velANEES(i)] = ...
        tem(tracks,trackIDs,truths,truthIDs);
end

Show the track metrics table.

trackMetricsTable(tam)

ans=4×16 table
    TrackID    AssignedTruthID    Surviving    TotalLength    UnreportedStatus    DeletionStatus    DeletionLength    DivergenceStatus    DivergenceCount    DivergenceLength    RedundancyStatus    RedundancyCount    RedundancyLength    FalseTrackStatus    FalseTrackLength    SwapCount
    _______    _______________    _________    ___________    ________________    ______________    ______________    ________________    _______________    ________________    ________________    _______________    ________________    ________________    ________________    _________

       1             NaN            false         1120             false              false               0                false                 3                   3                false                 0                   0                false                  0               0    
       2             NaN            false         1736             false              false               0                false                 8                  88                false                 0                   0                false                 28               3    
       6               3            true          1138             false              false               0                false                 4                 314                false                 1                  28                false                  0               2    
       8               2            true           662             false              false               0                false                 2                  29                false                 1                 169                false                 28               0    

Show the truth metrics table.

truthMetricsTable(tam)

ans=2×11 table
    TruthID    AssociatedTrackID    DeletionStatus    TotalLength    UnreportedStatus    BreakStatus    BreakCount    BreakLength    InCoverageArea    EstablishmentStatus    EstablishmentLength
    _______    _________________    ______________    ___________    ________________    ___________    __________    ___________    ______________    ___________________    ___________________

       2               8                false            2678             false             false           4             168            true                 true                    56         
       3               6                false            2678             false             false           3             645            true                 true                    84         

Plot the RMSE and ANEES error metrics.

subplot(2,2,1)
plot(posRMSE)
title('Position Error')
xlabel('tracker update')
ylabel('RMSE (m)')

subplot(2,2,2)
plot(velRMSE)
title('Velocity Error')
xlabel('tracker update')
ylabel('RMSE (m/s)')

subplot(2,2,3)
plot(posANEES)
title('Position Error')
xlabel('tracker update')
ylabel('ANEES')

subplot(2,2,4)
plot(velANEES)
title('Velocity Error')
xlabel('tracker update')
ylabel('ANEES')

Show the current error metrics for each individual recorded track.

currentTrackMetrics(tem)

ans=2×5 table
    TrackID    posRMS    velRMS    posANEES    velANEES
    _______    ______    ______    ________    ________

       6       44.712    20.988    0.05974     0.31325 
       8       129.26    12.739     1.6745      0.2453 

Show the current error metrics for each individual recorded truth object.

currentTruthMetrics(tem)

ans=2×5 table
    TruthID    posRMS    velRMS    posANEES    velANEES
    _______    ______    ______    ________    ________

       2       129.26    12.739     1.6745      0.2453 
       3       44.712    20.988    0.05974     0.31325 

Show the cumulative error metrics for each individual recorded track.

cumulativeTrackMetrics(tem)

ans=4×5 table
    TrackID    posRMS    velRMS    posANEES    velANEES
    _______    ______    ______    ________    ________

       1       117.69    43.951    0.58338     0.44127 
       2        129.7      42.8    0.81094     0.42509 
       6       371.35    87.083     4.5208      1.6952 
       8       130.45    53.914     1.0448     0.44813 

Show the cumulative error metrics for each individual recorded truth object.

cumulativeTruthMetrics(tem)

ans=2×5 table
    TruthID    posRMS    velRMS    posANEES    velANEES
    _______    ______    ______    ________    ________

       2       258.21    65.078     2.2514     0.93359 
       3       134.41    48.253    0.96314     0.49183 

Version History

Introduced in R2018b

R2023b: Evaluate track error metrics for recorded track and truth data

You can use the new evaluateBatch object function of the trackErrorMetrics object to evaluate track error metrics between a log of tracks and a log of truths based on their predetermined assignments.