evaluateBatch

Compute assignment metrics for recorded data

Since R2023b

Syntax

[trackSummary,truthSummary,assignHistory] = evaluateBatch(assignmentMetric,tracklog,truthlog)

Description

[trackSummary,truthSummary,assignHistory] = evaluateBatch(assignmentMetric,tracklog,truthlog) returns the track summary, truth summary, and the assignment history based on a log of tracks and a log of truths using the trackingAssignmentMetric object. The function places tracks and truths of the same time in a group and performs assignment between the tracks and truths in each group.

Tip

This function ignores the MaxUnreportedPeriod property of the trackAssignmentMetrics object. The function computes a maximum unreported period value based on the tracklog and truthlog inputs and uses the calculated value to delete tracks and truths if they are not updated within the maximum period.

Examples

Evaluate Track Metrics for Recorded Data

First, load the stored track data.

load trackmetricex tracklog truthlog

Create a trackAssignmentMetric object.

tam = trackAssignmentMetrics;

Use the evaluateBatch object function to obtain the track summary, the truth summary, and the assignment history.

[trackSummary,truthSummary,assignHistory] = evaluateBatch(tam,tracklog,truthlog)

trackSummary = struct with fields:
            TotalNumTracks: 4
            NumFalseTracks: 0
              MaxSwapCount: 3
            TotalSwapCount: 5
        MaxDivergenceCount: 8
      TotalDivergenceCount: 17
       MaxDivergenceLength: 314
     TotalDivergenceLength: 434
        MaxRedundancyCount: 1
      TotalRedundancyCount: 3
       MaxRedundancyLength: 169
     TotalRedundancyLength: 199
     MaxTimeBetweenReports: 0.0187
    MeanTimeBetweenReports: 0.0187

truthSummary = struct with fields:
              TotalNumTruths: 2
            NumMissingTruths: 0
      MaxEstablishmentLength: 84
    TotalEstablishmentLength: 140
               MaxBreakCount: 4
             TotalBreakCount: 7
              MaxBreakLength: 645
            TotalBreakLength: 813
       MaxTimeBetweenReports: 0.0187
      MeanTimeBetweenReports: 0.0187

assignHistory=2678×1 struct array with fields:
    Time
    TrackIDs
    TruthIDs

Next, create a trackErrorMetrics object.

tem = trackErrorMetrics;

Use the evaluateBatch object function to obtain the cumulative track and truth error metric tables as well as the track and truth error history tables.

[cumTrackError,cumTruthError,trackErrorHistory,truthErrorHistory] = evaluateBatch(tem,tracklog,truthlog,assignHistory)

cumTrackError=4×5 table
    TrackID    posRMS    velRMS    posANEES    velANEES
    _______    ______    ______    ________    ________

       1       117.69    43.951    0.58338     0.44127 
       2        129.7      42.8    0.81094     0.42509 
       6       371.35    87.083     4.5208      1.6952 
       8       130.45    53.914     1.0448     0.44813 

cumTruthError=2×5 table
    TruthID    posRMS    velRMS    posANEES    velANEES
    _______    ______    ______    ________    ________

       2       258.21    65.078     2.2514     0.93359 
       3       134.41    48.253    0.96314     0.49183 

trackErrorHistory=4600×6 table
     Time     TrackID    posRMS    velRMS    posANEES    velANEES
    ______    _______    ______    ______    ________    ________

     1.064       1       30.104    12.092    0.049206    0.024983
    1.0827       1       29.996    12.092    0.035699    0.022217
    1.1013       1        29.89    12.092    0.035276    0.022217
      1.12       1       29.785    12.092    0.034862    0.022217
    1.1387       1       29.681    12.092    0.034458    0.022217
    1.1573       1       29.579    12.092    0.034065    0.022217
     1.176       1       29.478    12.092    0.033682    0.022217
    1.1947       1       29.379    12.092     0.03331    0.022217
    1.2133       1       29.281    12.092    0.032948    0.022217
     1.232       1       29.184    12.092    0.032597    0.022217
    1.2507       1       29.089    12.092    0.032257    0.022217
    1.2693       1       28.995    12.092    0.031927    0.022217
     1.288       1       28.902    12.092    0.031608    0.022217
    1.3067       1       28.812    12.092      0.0313    0.022217
    1.3253       1       28.722    12.092    0.031003    0.022217
     1.344       1       28.634    12.092    0.030715    0.022217
      ⋮

truthErrorHistory=4403×6 table
     Time     TruthID    posRMS    velRMS    posANEES    velANEES
    ______    _______    ______    ______    ________    ________

     1.064       2       30.104    12.092    0.049206    0.024983
    1.0827       2       29.996    12.092    0.035699    0.022217
    1.1013       2        29.89    12.092    0.035276    0.022217
      1.12       2       29.785    12.092    0.034862    0.022217
    1.1387       2       29.681    12.092    0.034458    0.022217
    1.1573       2       29.579    12.092    0.034065    0.022217
     1.176       2       29.478    12.092    0.033682    0.022217
    1.1947       2       29.379    12.092     0.03331    0.022217
    1.2133       2       29.281    12.092    0.032948    0.022217
     1.232       2       29.184    12.092    0.032597    0.022217
    1.2507       2       29.089    12.092    0.032257    0.022217
    1.2693       2       28.995    12.092    0.031927    0.022217
     1.288       2       28.902    12.092    0.031608    0.022217
    1.3067       2       28.812    12.092      0.0313    0.022217
    1.3253       2       28.722    12.092    0.031003    0.022217
     1.344       2       28.634    12.092    0.030715    0.022217
      ⋮

Input Arguments

assignmentMetric — Track assignment metrics object

trackAssignmentMetrics System object

Track assignment metrics object, specified as a trackAssignmentMetrics System object™.

tracklog — Recorded track log

array of objects or structures | cell array of objects or structures

Recorded track log, specified as an array of objects or structures. Alternately, you can specify it as a cell array of objects or structures. Each object or structure must have an UpdateTime property or field, respectively.

If the DistanceFunctionFormat property is specified as 'built-in', each object must be an objectTrack object and each structure must be a track structure. The fields of the track structure should be the same as the properties of the objectTrack object. The track structure must at least have the State, StateCovariance, TrackID, and UpdateTime fields.

truthlog — Recorded truth log

array of structures | cell array of structures | trackScenarioRecording object

Recorded truth log, specified as an array of structures, a cell array of structures, or a trackingScenarioRecording object. Each structure must have a Time field.

If the DistanceFunctionFormat property is specified as 'built-in', each structure must have at least these fields.

Field	Description
Time	Time of the truth, specified as a nonnegative scalar.
PlatformID	Unique identifier for the platform, specified as a positive integer. This is a required field with no default value.
Position	Position of the platform in scenario coordinates, specified as a real-valued 1-by-3 row vector. If the coordinateSystem argument is specified as 'Cartesian', Position is the 3-element vector of Cartesian position coordinates in meters. If the coordinateSystem argument is specified as 'Geodetic', Position is the 3-element vector of geodetic coordinates: latitude in degrees, longitude in degrees, and altitude in meters.
Velocity	Velocity of the platform in scenario coordinates, specified as a real-valued 1-by-3 row vector. Units are meters per second. The default value is [0 0 0].

You can use the platformPoses function of the trackingScenario object to generate this structure except the Time field, which you can obtain via the SimulationTime property of the trackingScenario object.

Output Arguments

trackSummary — Track assignment summary

structure

Track assignment summary, returned as a structure. The structure has these fields.

Field	Description
TotalNumTracks	The total number of unique track identifiers encountered
NumFalseTracks	The number of tracks never assigned to any truth
MaxSwapCount	Maximum number of track swaps of each track. A track swap occurs whenever a track is assigned to a different truth.
TotalSwapCount	Total number of track swaps of each track. A track swap occurs whenever a track is assigned to a different truth.
MaxDivergenceCount	Maximum number of divergences. A track is divergent when the result of the DivergenceDistanceFcn is greater than the divergence threshold.
TotalDivergenceCount	Total number of divergences. A track is divergent when the result of the divergence distance function is greater than the divergence threshold.
MaxDivergenceLength	Maximum number of updates during which each track was in a divergent state
TotalDivergenceLength	Total number of updates during which each track was in a divergent state
MaxRedundancyCount	The maximum number of additional tracks assigned to the same truth
TotalRedundancyCount	The total number of additional tracks assigned to the same truth
MaxRedundancyLength	Maximum number of updates during which each track was in a redundant state
TotalRedundancyLength	Total number of updates during which each track was in a redundant state
MaxTimeBetweenReports	The maximum time interval between adjacent track updates to the metric across all tracks. The field is returned as NaN if there has been only one update to the metric.
MeanTimeBetweenReports	The average time interval between adjacent track updates to the metric across all tracks. The field is returned as NaN if there has been only one update to the metric.

truthSummary — Truth assignment summary

structure

Truth assignment summary, returned as a structure. The structure has these fields.

Field	Description
TotalNumTruths	The total number of unique truth identifiers encountered
NumMissingTruths	The number of truths never established with any track
MaxEstablishmentLength	Maximum number of updates before a truth was associated with any track while inside the coverage area. The lengths of missing truths do not count toward this summary metric.
TotalEstablishmentLength	Total number of updates before a truth was associated with any track while inside the coverage area. The lengths of missing truths do not count toward this summary metric.
MaxBreakCount	Maximum number of times each truth was unassociated with any track after being established.
TotalBreakCount	Total number of times each truth was unassociated with any track after being established.
MaxBreakLength	Maximum number of updates during which each truth was in a broken state
TotalBreakLength	Total number of updates during which each truth was in a broken state
MaxTimeBetweenReports	The maximum time interval between adjacent truth updates to the metric across all truths. The field is returned as NaN if there has been only one update to the metric.
MeanTimeBetweenReports	The average time interval between adjacent truth updates to the metric across all truths. The field is returned as NaN if there has been only one update to the metric.

assignHistory — Assignment history

array of structures

Assignment history, returned as an array of structures. Each structure has these fields.

Field	Description
Time	Time of assignment, returned as a nonnegative scalar.
TrackIDs	IDs of tracks, returned as a vector of track IDs.
TruthIDs	IDs of truths, returned as a vector of truth IDs.

The corresponding tracks and truths from the TrackIDs and TruthIDs fields are assigned to each other.

Data Types: struct

Version History

Introduced in R2023b

R2024a: Performance improvement

evaluateBatch shows improved performance for longer input logs with at least 1000 elements. For example, this code is about 5x faster than in the previous release:

function t = timingTest
[truthlog,tracklog] = createlonglogs(1000);
tam = trackAssignmentMetrics();
tic; evaluateBatch(tam,tracklog,truthlog); toc
end

function [truths,tracks]=createlonglogs(numSteps)
v = ones(1,3);
t1 = [struct("PlatformID",1,"Position",v,"Velocity",v,"Time",1);...
    struct("PlatformID",2,"Position",10*v,"Velocity",10*v,"Time",1)];
t2 = [objectTrack(TrackID=1,state=ones(6,1));...
    objectTrack(TrackID=2,state=10*ones(6,1))];
truths = cell(numSteps,1);
tracks = cell(numSteps,1);
for i=1:numSteps
    truths{i} = t1;
    truths{i}(1).Time = i;
    truths{i}(2).Time = i;
    tracks{i} = t2;
    tracks{i}(1).UpdateTime = i;
    tracks{i}(2).UpdateTime = i;
end
end

The approximate execution times are:

• R2023b: 9.6 s

• R2024a: 1.8 s

The code was timed on a Windows 11, AMD(R) EPYC 74F3 24-Core Processor @ 3.19 GHz test system by calling the timingTest function.