AEB Euro NCAP Testing with RoadRunner Scenario

Since R2024a

This example uses:

This example shows how to assess the functionality of an autonomous emergency braking (AEB) application using scenarios defined per European New Car Assessment Programme (Euro NCAP®) testing protocols.

Introduction

In the realm of automotive safety, the Euro NCAP sets important safety standards for vehicles, including tests for AEB systems. AEB is an advanced active safety system that helps drivers avoid or mitigate collisions with other vehicles. The Euro NCAP has established a comprehensive set of testing protocols to rigorously assess the effectiveness of AEB systems. This example enables you to assess the functionality of an AEB application using AEB Car-to-Car tests per Euro NCAP testing protocols.

This example shows how to generate Euro NCAP scenarios for Safety Assist (SA) AEB Car-to-Car scenarios and their variants with RoadRunner Scenario and simulate them using the test bench model. The example also shows how to use the iterative testing methodology in Simulink® Test Manager to test the generated variants. In this example, you:

Configure Euro NCAP Test Environment — Specify a Euro NCAP test name, ego asset, and ego behavior information.
Generate Euro NCAP Scenarios — Generate scenarios for the specified test name, compatible with RoadRunner Scenario, per Euro NCAP Test Protocol – AEB Car-to-Car systems version 4.2.
Review and Simulate AEB Test Bench — The test bench model contains controller, vehicle dynamics, and scenario and environment components. The scenario and environment component contains RoadRunner Scenario interfaces. Simulate the test bench model for an SA AEB Car-to-Car Front turn-across-path (SA AEB CCFtap) test scenario, and plot the simulation results.
Perform Iterative Testing — Generate a test manager file for SA AEB CCFtap test scenarios to enable testing using table iterations. Run scenarios programmatically, or by using Test Manager.
Review Euro NCAP Test Report — Analyze the Euro NCAP test report, generated from Test Manager, for the SA AEB CCFtap scenarios.
Explore Further — Explore how to perform iterative testing for other SA AEB Car-to-Car test scenarios. You can also modify the ego asset for test scenarios before performing iterative testing.

This example generates, simulates, and analyzes Safety Assist AEB Car-to-Car Front turn-across-path scenario (SA AEB CCFtap) scenarios for demonstration purposes.

Configure Euro NCAP Test Environment

This section shows how to set up the test environment and specify the necessary inputs to generate SA AEB Car-to-Car test scenarios per Euro NCAP protocol.

Set Up Environment

Start RoadRunner interactively using the roadrunnerSetup function. In the RoadRunner Setup dialog box, specify the RoadRunner Installation Folder and RoadRunner Project Folder locations. You must use a RoadRunner Project created with Base + Add On Assets because this example uses assets from RoadRunner Asset Library.

rrApp = roadrunnerSetup;

The function returns a roadrunner object, rrApp, that provides functions for performing basic workflow tasks such as opening, closing, and saving scenes and projects. After setting up the required paths, close RoadRunner. The subsequent steps in this example assume the RoadRunner application is closed.

After setting up the required paths, close RoadRunner. The subsequent steps in this example use rrApp, and assume the RoadRunner application is closed.

close(rrApp)

Specify Inputs

testName — Names of Euro NCAP tests, specified as a cell array of string scalars. This example uses the "SA AEB CCFtap" test. You can also specify any of these tests: "SA AEB CCRs", "SA AEB CCRm", "SA AEB CCRb", "SA AEB CCFtap", "SA AEB CCCscp", "SA AEB CCFhol", and "SA AEB CCFhos".
egoAssetFile — File path of the RoadRunner asset for the ego vehicle, specified as a string scalar. This example specifies the Sedan vehicle as the ego asset. For more information on the dimensions of the Sedan vehicle asset, see Sedan.
egoBehaviorFile — Behavior file to associate with the ego vehicle in RoadRunner Scenario, specified as a string scalar. The example implements the behavior of the ego vehicle using the Simulink model.

Specify the Euro NCAP test scenario.

testName = {"SA AEB CCFtap"};

Specify the ego asset file path.

egoAssetFile = "Vehicles/ADT Vehicles/SK_Sedan.fbx_rrx";

Specify the ego behavior file.

egoBehaviorFile = "Behaviors/DrivingTestBench.rrbehavior.rrmeta";

Set the simulation step size for RoadRunner Scenario simulation.

stepSize = 0.03;

Set the vehicle fidelity model to either 3DOF or 14DOF.

vehicleModel = "3DOF";

Set the required paths for the RoadRunner application AEB test bench project by using the helperInitializeEuroNCAPTestSession helper function.

sessionTestConfig = helperInitializeEuroNCAPTestSession( ...
    testName,egoAssetFile,egoBehaviorFile,stepSize,vehicleModel);

Session test folder exists for the current configuration.
Loading available session test configuration...

Examine the sessionTestConfig structure.

display(sessionTestConfig)

sessionTestConfig = 

  struct with fields:

                      Id: "24A9643B127C2C49EA1966DE1BD48CAC"
              WorkFolder: "C:\Work\EuroNCAP\DrivingTest\Work\24A9643B127C2C49EA1966DE1BD48CAC"
            TestTypeList: {["SA AEB CCFtap"]}
                TestSpec: {[9×11 table]}
                TestInfo: [9×6 table]
          PathToEgoAsset: "Vehicles/ADT Vehicles/SK_Sedan.fbx_rrx"
       PathToEgoBehavior: "Behaviors/DrivingTestBench.rrbehavior.rrmeta"
    PathToTargetBehavior: "Behaviors/Target.rrbehavior.rrmeta"
        DrivingConfigObj: [1×1 configureDrivingTestBench]
               TestBench: "DrivingTestBench.slx"
        GenerateVariants: 0
                StepSize: 0.0300
                 Vehicle: [1×1 struct]

The helperInitializeEuroNCAPTestSession helper function creates the required folders for the current test session and opens the DrivingTest project that contains the AEB test bench model for cosimulation with RoadRunner Scenario. The function also creates a session folder within the working folder WorkFolder. This session folder contains the Scene, Scenarios, and Simulation logs folders for the current session. This function outputs the sessionTestConfig structure, which contains the information required to generate test scenarios and to set up the test bench. The GenerateVariants field of the sessionTestConfig strucure specifies whether to generate scenarios for the specified test or to reuse scenarios from existing session based on their availability.

Generate Euro NCAP Scenarios

In this section, you generate Euro NCAP test scenarios for the SA AEB CCFtap test.

Review SA AEB CCFtap Scenario

The Euro NCAP specification designates the ego vehicle as the vehicle under test (VUT) and the target vehicle as a global vehicle target (GVT). In the SA AEB CCFtap scenario, a VUT turns across the path of an oncoming GVT that is traveling at a constant speed, causing a collision in which the frontal structure of the GVT strikes the frontal structure of the VUT. The VUT and GVT follow predefined paths specified in the scenario. The GVT travels on a straight-line path. The VUT travels straight, makes a left turn, and then travels straight again per Euro NCAP Test Protocol – AEB Car-to-Car systems version 4.2. You can generate nine different scenario variations by using these speed values for the VUT and GVT:

VUT — 10, 15, or 20 km/h
GVT — 30, 45, or 60 km/h

Generate SA AEB CCFtap Scenarios

This example uses the helperGenerateEuroNCAPScenarios helper function to generate the Euro NCAP scenarios. For more information on generating AEB Car-to-Car scenarios, see the Generate Variants for Testing AEB Car-to-Car Scenarios example. The helper function does not regenerate the scenarios if they are already available and you do not modify any input parameters. Depending on the number of tests and the number of variants for each test, the helper function can take some time to generate variants.

The helper function also copies the generated scenarios to the RoadRunner project and updates the TestInfo field of the sessionTestConfig structure with scenario descriptions, to help you understand the scenario parameters.

sessionTestConfig = helperGenerateEuroNCAPScenarios( ...
    sessionTestConfig,egoAssetFile);

To verify the generated scenario information, examine the TestInfo field of the sessionTestConfig structure.

disp(sessionTestConfig.TestInfo)

    Test ID       Test Type       RRScenario File Name    Test Start Time    Test End Time               Scenario Description           
    _______    _______________    ____________________    _______________    _____________    __________________________________________

       1       "SA AEB CCFtap"     "SA_AEB_CCFtap_1"           27.63             35.62        "VUT Speed = 10 km/h; GVT Speed = 30 km/h"
       2       "SA AEB CCFtap"     "SA_AEB_CCFtap_2"           27.66             35.65        "VUT Speed = 10 km/h; GVT Speed = 45 km/h"
       3       "SA AEB CCFtap"     "SA_AEB_CCFtap_3"           27.66             35.65        "VUT Speed = 10 km/h; GVT Speed = 60 km/h"
       4       "SA AEB CCFtap"     "SA_AEB_CCFtap_4"           17.88             25.87        "VUT Speed = 15 km/h; GVT Speed = 30 km/h"
       5       "SA AEB CCFtap"     "SA_AEB_CCFtap_5"           17.19             25.18        "VUT Speed = 15 km/h; GVT Speed = 45 km/h"
       6       "SA AEB CCFtap"     "SA_AEB_CCFtap_6"           17.19             25.18        "VUT Speed = 15 km/h; GVT Speed = 60 km/h"
       7       "SA AEB CCFtap"     "SA_AEB_CCFtap_7"           18.18             26.17        "VUT Speed = 20 km/h; GVT Speed = 30 km/h"
       8       "SA AEB CCFtap"     "SA_AEB_CCFtap_8"           13.77             21.76        "VUT Speed = 20 km/h; GVT Speed = 45 km/h"
       9       "SA AEB CCFtap"     "SA_AEB_CCFtap_9"           12.09             20.08        "VUT Speed = 20 km/h; GVT Speed = 60 km/h"

Review One SA AEB CCFtap Scenario

In this example, you use the AEB test bench model to simulate the first generated scenario, SA_AEB_CCFtap_1.rrscenario, with RoadRunner Scenario. Open the scenario.

rrApp = roadrunner;
testID = 1;
scenarioFileName = fullfile(sessionTestConfig.Id,sessionTestConfig.TestInfo(testID,:).("RRScenario File Name"));
openScenario(rrApp,scenarioFileName)

Note that the example saves scenarios that include behaviors for the ego and target vehicles.

The ego vehicle has the DrivingTestBench.rrbehavior.rrmeta behavior, which contains the Simulink behavior DrivingTestBench.slx. In this example, the ego vehicle must start from rest and attain a constant speed to meet the test conditions per Euro NCAP protocol. The target vehicle has the MATLAB behavior Target.rrbehavior, which contains the TargetBehavior.m MATLAB System object™. The target vehicle continuously monitors the ego vehicle speed to adjust its timing to ensure a collision with the ego vehicle.

Review and Simulate AEB Test Bench

In this example, you use a DrivingTestBench.slx model to simulate and test the performance of the AEB controller.

Open the AEB test bench model.

open_system(sessionTestConfig.TestBench)

The test bench model contains these modules:

Controller — Algorithm model that specifies the steering angle and acceleration controls of the ego vehicle.
Vehicle Dynamics — Specifies ego vehicle dynamics. The Vehicle Dynamics subsystem consists of a variant subsystem that has two vehicle variants, 3DOF (default) and 14DOF.
Scenario and Environment — Specifies vehicle paths, sensor blocks, and RoadRunner Scenario blocks, which configure, read from, and write to RoadRunner Scenario.

For more information on the AEB test bench model, see the AEB Test Bench for Euro NCAP Scenarios example.

Simulate Test Bench Model

This example uses one of the generated variants, SA_AEB_CCFtap_1.rrscenario, to simulate the AEB test bench model. You can use the configureDrivingTestBench object to configure the AEB test bench. Simulate the test bench model and plot the results.

Connect to the RoadRunner Scenario server for cosimulation by using the createSimulation function, and enable data logging.

rrSim = createSimulation(rrApp);
set(rrSim,PacerStatus="Off")
set(rrSim,Logging="on")
set(rrSim,StepSize=sessionTestConfig.StepSize)
set(rrSim,MaxSimulationTime=sessionTestConfig.TestInfo(testID,:).("Test End Time"));

Configure the AEB test bench model with AEB enabled.

drivingConfigObj  = sessionTestConfig.DrivingConfigObj;
setup(drivingConfigObj,AEBEnable=1, ...
        TestStartTime=sessionTestConfig.TestInfo(testID,:).("Test Start Time"),...
    Vehicle=sessionTestConfig.Vehicle);
fprintf("\nSetup of DrivingTestBench complete.\n")

Setup of DrivingTestBench complete.

Simulate the AEB test bench model.

set(rrSim,SimulationCommand="Start")
while strcmp(get(rrSim,"SimulationStatus"),"Running")
    pause(1)
end

Plot the simulation results by using the helperPlotAEBResults helper function.

helperPlotAEBResults(out.logsout, ...
    sessionTestConfig.TestInfo(testID,:).("RRScenario File Name"));

The simulation results contain these charts:

AEB Test Enable — Displays the test enable signal that indicates the start of the test.
FCW and AEB Status — Displays the forward collision warning (FCW) and AEB activation status.
TTC vs. Stopping Time — Compares the time-to-collision (TTC) and the stopping times for the FCW, first stage partial brake, second stage partial brake, and full brake.
Ego Car Acceleration — Shows the acceleration of the ego vehicle.
Ego Car Velocity — Shows the velocity of the ego vehicle.
Headway — Shows the headway between the ego vehicle and the target vehicle.
Ego Car Yaw and Yaw Rate — Shows the yaw and yaw rate of the ego vehicle.

When the ego vehicle attains the steady-state speed, this example sets the AEB Test Enable parameter to true, indicating the start of the test. Once the test starts, the ego vehicle applies brakes based on the TTC value to avoid collision with the target vehicle.

Perform Iterative Testing

This example enables you to perform automatic testing using table iterations from the test manager file. Use the helperGenerateTestManagerFile function to generate the test suite to perform automatic testing.

Generate a test manager file.

testManagerFileName = helperGenerateTestManagerFile(sessionTestConfig,rrApp);

Close the active RoadRunner Scenario instance.

close(rrApp)

Review the generated test suites and test cases by opening Simulink Test Manager and running them programmatically.

Open and observe the test cases.

sltestmgr
testFileObj = sltest.testmanager.load(testManagerFileName);

Run the test file and get the results.

run(testFileObj);

Get the latest result object.

resultSet = sltest.testmanager.getResultSets();
resultsObj = resultSet(end);

You also can run the tests for all the variants of the SA AEB CCFtap scenario by clicking Run in the Test Manager.

You can select or deselect scenarios in the Table Iterations section to simulate one or many scenarios.

You can observe the completion status of the SA AEB CCFtap scenario simulations in the Results and Artifacts tab.

Review Euro NCAP Test Report

In this section, you learn how to programmatically create a consolidated report after successfully simulating the AEB test bench model with all the SA AEB CCFtap scenarios, and how to observe this information in the Test Manager UI.

Collate the test results from the simulation runs by using the helperCollateTestResults helper function.

testResults = helperCollateTestResults(resultsObj);

Display the collated collision status results for the SA AEB CCFtap scenarios.

testType = testName{1};
disp(testType)
fieldName = strrep(testType,' ','_');
disp(testResults.(fieldName))

SA AEB CCFtap
    CollisionAvoidanceStatus: [3×3 double]

Get the Euro NCAP Car-to-Car assessment score using the helperGetEuroNCAPCarToCarScore helper function.

ncapScore = helperGetEuroNCAPCarToCarScore(testResults,testType);

Plot the consolidated results.

helperAEBCarToCarReport(testResults,ncapScore,testType)

Observe the consolidated report generated from the Test Manager after the successful completion of AEB testing with all the variants. Note the table in the report that shows the test status for all variants, in this case titled Car-to-Car Front turn across path (CCFtap) scenarios: Collision Avoidance Status. A green field signifies that AEB has triggered as expected, in that variant scenario, and full points for avoiding the collision have been awarded. If AEB fails to trigger, and if a collision occurs, the field for that variant scenario appears in red.

You can click a field in this table, to get information about the variant name, as well as the Replay Simulation and AEB Simulation Result options.

Replay Simulation — Replay the simulation using simulation data logged from the RoadRunner application.

AEB Simulation Result — Display the AEB simulation results from the test bench model.

Alternatively, you can find this consolidated report and AEB Simulation result in Test Manager after the successful completion of all SA AEB CCFtap scenarios.

To find the consolidated report, on the Results and Artifacts tab, select the folder containing the SA AEB CCFtap test and, in the MATLAB Figures section, under Cleanup, select EURO NCAP SA AEB CCFtap scenario variation results & scoring.

If you select an individual test case, the Results and Artifacts tab instead contains the simulation results for only that test case.

Explore Further

In this example, you have generated scenarios for SA AEB CCFtap test scenarios for the Sedan vehicle. You have simulated the test bench model and analyzed the performance using Test Manager.

To use the same process for other Euro NCAP AEB Car-to-Car scenarios, change the testName value. You can specify any test scenario from this list.

"SA AEB CCRs" — Safety Assist Autonomous Emergency Braking Car-to-Car Rear Stationary
"SA AEB CCRm" — Safety Assist Autonomous Emergency Braking Car-to-Car Rear Moving
"SA AEB CCRb" — Safety Assist Autonomous Emergency Braking Car-to-Car Rear Braking
"SA AEB CCFtap" — Safety Assist Autonomous Emergency Braking Car-to-Car Front Turn-Across-Path
"SA AEB CCCscp" — Safety Assist Autonomous Emergency Braking Car-to-Car Crossing Straight Crossing Path
"SA AEB CCFhol" — Safety Assist Autonomous Emergency Braking Car-to-Car Front Head-On Lane change
"SA AEB CCFhos" — Safety Assist Autonomous Emergency Braking Car-to-Car Front Head-On Straight

You can also specify sequence of test scenarios as well.

testName = {"SA AEB CCRs"; "SA AEB CCRm"; "SA AEB CCRb"; "SA AEB CCFtap"; "SA AEB CCCscp"; "SA AEB CCFhol"; "SA AEB CCFhos"};

If you specify multiple tests, you can get the status of each test from Test Manager after successful simulation completion by selecting that test folder on the Results and Artifacts tab.

You can also observe the consolidated Euro NCAP report for each specified test. For example, a report for variants of the SA AEB CCRm scenario might look like this.

You can also modify the RoadRunner asset for the ego vehicle by changing the egoAssetFile value.

egoAssetFile = "Vehicles/ADT Vehicles/SK_MuscleCar.fbx_rrx";

For more information on how to generate variants by modifying the ego asset information, see the Generate RoadRunner Scenario Variants by Modifying Actor Assets example.

For more information on how to configure, simulate, and analyze the performance of the AEB test bench for one Euro NCAP scenario, see the AEB Test Bench for Euro NCAP Scenarios example.

Supporting Functions

helperGenerateEuroNCAPScenarios — Generates variants using the ncapScenario and generateVariants functions for specified tests when sessionTestConfig.GenerateVariants is True. If it is False, the function reuses the existing scenarios and copies them to the RoadRunner project path.

function sessionTestConfig = helperGenerateEuroNCAPScenarios(sessionTestConfig,egoAssetFile)

numTests = length(sessionTestConfig.TestTypeList);

if(sessionTestConfig.GenerateVariants)
    fprintf("Started generating Euro NCAP scenarios...\n")
    egoAssetDim = helperGetActorDimensions(egoAssetFile);
    for testIdx = 1:numTests
        testName = string(sessionTestConfig.TestTypeList(testIdx));

        % Get seed scenario descriptor
        fprintf("Generating seed scenario for %s (%d of %d)...\n",testName,testIdx,numTests)
        seedScenarioDescriptor = ncapScenario(testName);

        % Populate ego asset information
        actorIDs = identifyActorOfInterest(seedScenarioDescriptor);

        % Ego asset information
        egoAssetInfo = struct("ID",actorIDs.EgoID,"AssetDimension",egoAssetDim,"AssetPath", egoAssetFile);

        % Get variation properties
        variationProperties = helperEuroNCAPVariationProperties(testName, egoAssetInfo);

        % Generate scenario descriptors for the selected variations
        fprintf("Generating Euro NCAP scenario descriptors for %s (%d of %d)...\n",testName,testIdx,numTests)
        [ncapScenarioDescriptor,variantInfo] = generateVariants(seedScenarioDescriptor,variationProperties);

        % Save descriptor and variant info to sessionTestConfig
        sessionTestConfig.Variant(testIdx).Descriptor = ncapScenarioDescriptor;
        sessionTestConfig.Variant(testIdx).Info = variantInfo;
    end
    % Save the variant descriptors to sessionTestConfig before generating
    % RR scenario files.
    save(fullfile(sessionTestConfig.WorkFolder,"sessionTestConfig.mat"), ...
        "sessionTestConfig")

    fprintf("Completed generating Euro NCAP variation descriptors.\n")
end
% Generate or copy RoadRunner Scenario files based on
% sessionTestConfig.GenerateVariants. Populate test information like
% scenario file name, test start time, test end time and description
% required for testing, analysis, and visualization.
sessionTestConfig = helperPrepareTestScenarios(sessionTestConfig);
fprintf("Completed preparing Euro NCAP test scenarios for RoadRunner Scenario.\n\n")
end

Session test folder exists.
Copying existing scenarios from session test folder to RR project...
Completed preparing Euro NCAP test scenarios for RoadRunner Scenario.