Safety Assurance Concepts for Automated Driving Systems

<div class="section abstract"> <div class="htmlview paragraph">This Recommended Practice provides a taxonomy for motor vehicle <i>driving automation systems</i> that perform part or all of the <i>dynamic driving task</i> (<i>DDT</i>) on a <i>sustained</i> basis and that range in level from <i>no driving automation</i> (level 0) to <i>full driving automation</i> (level 5). It provides detailed definitions for these six levels of <i>driving automation</i> in the context of <i>motor vehicles</i> (hereafter also referred to as “<i>vehicle</i>” or “<i>vehicles</i>”) and their <i>operation</i> on roadways. These level definitions, along with additional supporting terms and definitions provided herein, can be used to describe the full range of <i>driving automation features</i> equipped on <i>motor vehicles</i> in a functionally consistent and coherent manner. “On-road” refers to publicly accessible roadways (including parking areas and private campuses that permit public access) that collectively serve users of <i>vehicles</i> of all classes and <i>driving automation</i> levels (including no <i>driving automation</i>), as well as motorcyclists, pedal cyclists, and pedestrians.</div> <div class="htmlview paragraph">The levels apply to the <i>driving automation feature(s)</i> that are engaged in any given instance of on-road <i>operation</i> of an equipped <i>vehicle</i>. As such, although a given <i>vehicle</i> may be equipped with a <i>driving automation system</i> that is capable of delivering multiple <i>driving automation features</i> that perform at different levels, the level of <i>driving automation</i> exhibited in any given instance is determined by the <i>feature(s)</i> that are engaged.</div> <div class="htmlview paragraph">This document also refers to three primary actors in <i>driving</i>: the (human) <i>driver</i>, the <i>driving automation system</i>, and other <i>vehicle</i> systems and components. These other <i>vehicle</i> systems (or the <i>vehicle</i> in general terms) do not include the <i>driving automation system</i> in this model, even though as a practical matter a <i>driving automation system</i> may actually share hardware and software components with other <i>vehicle</i> systems, such as a processing module(s) or operating code.</div> <div class="htmlview paragraph">The levels of <i>driving automation</i> are defined by reference to the specific role played by each of the three primary actors in performance of the <i>DDT</i>. “Role” in this context refers to the expected role of a given primary actor, based on the design of the <i>driving automation system</i> in question and not necessarily to the actual performance of a given primary actor. For example, a <i>driver</i> who fails to <i>monitor</i> the roadway during engagement of a level 1 adaptive cruise control (ACC) system still has the role of <i>driver</i>, even while s/he is neglecting it.</div> <div class="htmlview paragraph"><i>Active safety systems</i>, such as electronic stability control and automated emergency braking, and certain types of driver assistance systems, such as lane keeping assistance, are excluded from the scope of this <i>driving automation</i> taxonomy because they do not perform part or all of the <i>DDT</i> on a <i>sustained</i> basis and, rather, merely provide momentary intervention during potentially hazardous situations. Due to the momentary nature of the actions of <i>active safety systems</i>, their intervention does not change or eliminate the role of the <i>driver</i> in performing part or all of the <i>DDT,</i> and thus are not considered to be <i>driving automation</i>.</div> <div class="htmlview paragraph">It should, however, be noted that crash avoidance features, including intervention-type <i>active safety systems</i>, may be included in <i>vehicles</i> equipped with <i>driving automation systems</i> at any level. For <i>ADS</i>-equipped <i>vehicles</i> (i.e., levels 3-5) that perform the complete <i>DDT</i>, crash avoidance capability is part of <i>ADS</i> functionality.</div></div>

<div class="section abstract"> <div class="htmlview paragraph">This SAE Recommended Practice provides common data output formats and definitions for a variety of data elements that may be useful for analyzing the performance of automated driving system (ADS) during an event that meets the trigger threshold criteria specified in this document. The document is intended to govern data element definitions, to provide a minimum data element set, and to specify a common ADS data logger record format as applicable for motor vehicle applications.</div> <div class="htmlview paragraph">Automated driving systems (ADSs) perform the complete dynamic driving task (DDT) while engaged. In the absence of a human “driver,” the ADS itself could be the only witness of a collision event. As such, a definition of the ADS data recording is necessary in order to standardize information available to the accident reconstructionist. For this purpose, the data elements defined herein supplement the SAE J1698-1 defined EDR in order to facilitate the determination of the background and events leading up to a collision in an ADS-operated vehicle.</div> <div class="htmlview paragraph">The data elements defined in this document are unique to Level 3, 4, or 5 ADS features, as defined by SAE J3016, and provide additional background of the events leading up to a crash or crash-like event. The data from sensors such as camera(s), LiDAR(s) etc. will provide information in the absence of a human driver. The data included in the ADS data logger is expected to be used in conjunction with the SAE J1698 event data recorder (EDR) record and traditional accident reconstruction analysis. The EDR and ADS data logger will capture information leading up to the triggered event, at a minimum. There are no facts to support that recording data for greater than 5 seconds pre-event would change the outcome of any crash analysis. Thus, the recommended recording duration for a data logger is 5 seconds pre-event, same as an EDR. Due to the potential for sensor and/or communication failure during a crash event, the recommendation is that data should be collected post-crash for impact and rollover sensors for up to 250 ms.</div> <div class="htmlview paragraph">ADS technology is still being developed and is not yet commercially deployed. Therefore, this SAE Recommended Practice is intended as a guide toward standard practice and is subject to change to keep pace with experience and technical advances.</div> </div>

<div class="section abstract"> <div class="htmlview paragraph">This document provides preliminary<sup><span class="xref">1</span></sup> safety-relevant guidance for in-vehicle fallback test driver training and for on-road testing of vehicles being operated by prototype conditional, high, and full (Levels 3 to 5) ADS, as defined by SAE J3016. It does not include guidance for evaluating the performance of post-production ADS-equipped vehicles. Moreover, this guidance only addresses testing of ADS-operated vehicles as overseen by in-vehicle fallback test drivers (IFTD).</div> <div class="htmlview paragraph">These guidelines do not address:</div> <ul class="list disc"> <li class="list-item"> <div class="htmlview paragraph">Remote driving, including remote fallback test driving of prototype ADS-operated test vehicles in driverless operation. (Note: The term “remote fallback test driver” is included as a defined term herein and is intended to be addressed in a future iteration of this document. However, at this time, too little is published or known about this type of testing to provide even preliminary guidance.)</div></li> <li class="list-item"> <div class="htmlview paragraph">Testing of driver support features (i.e., Levels 1 and 2), which rely on a human driver to perform part of the dynamic driving task (DDT) and to supervise the driving automation feature’s performance in real time. (Refer to SAE J3016.)</div></li> <li class="list-item"> <div class="htmlview paragraph">Closed-course testing.</div></li> <li class="list-item"> <div class="htmlview paragraph">Simulation testing (except for training purposes).</div></li> <li class="list-item"> <div class="htmlview paragraph">Component-level testing.</div></li></ul> <div class="htmlview paragraph">These guidelines also do not address prototype vehicle and IFTD performance data collection and retention. The collection of data invokes various legal and risk management considerations that users of this document should nevertheless bear in mind, such as:</div> <ul class="list disc"> <li class="list-item"> <div class="htmlview paragraph">Maintaining auditable procedures and documentation.</div></li> <li class="list-item"> <div class="htmlview paragraph">Adhering to applicable privacy laws and principles.</div></li> <li class="list-item"> <div class="htmlview paragraph">Ensuring adequate data collection and recording integrity to support post-crash forensic analysis.</div></li></ul></div>

<div class="section abstract"><div class="htmlview paragraph">Advanced driver assistance systems (ADAS) and automated driving systems (ADS) continue to expand into the market at a rapid pace. As improved (i.e., next generation) versions of these systems become available, they will continue to face many challenges in their implementation and benefits for safety and driving operations. The solution will involve many parties, including road safety professionals and researchers who see the potential in these systems but may have difficulties keeping up with them, and safety advocates who are calling for these systems to achieve higher levels of safety now.</div><div class="htmlview paragraph"><b>The Challenges of Next-gen ADAS and ADS and Related Vehicle Safety Topics</b> explores these challenges that will fall on the National Highway Traffic Safety Administration (NHTSA) and automakers as they balance costs and benefits; establish reasonable regulations and standards; and determine how to improve, test, deliver, and use these systems successfully. Perhaps the most formidable challenge will be overcoming the expectation that ADAS—and especially ADS—will always work perfectly in every scenario.</div><div class="htmlview paragraph"><a href="https://www.sae.org/publications/edge-research-reports" target="_blank">Click here to access the full SAE EDGE</a><sup>TM</sup><a href="https://www.sae.org/publications/edge-research-reports" target="_blank"> Research Report portfolio.</a></div></div>

<div class="section abstract"><div class="htmlview paragraph">One major challenge in designing SAE level 3-5 Automated Driving Systems (ADS) is to define requirements for the perception system that would enable argumentation for safe operation. The safety requirements on the perception system can only be fulfilled through redundancy in the sensor hardware. It is, however, a challenge to specify the redundancy that is required in the sensor system. Safe operation for an ADS is significantly more difficult compared to advanced driver assistance systems (ADAS). The safety argumentation for ADAS typically argues that in case of a failure in the sensor array a fail-silent behavior is acceptable because the human driver can take control of the vehicle back. This argumentation however is not possible when developing level 4 or higher automation. This paper investigates prerequisites for applying a systematic methodology for analyzing redundancy in a multi-sensor system and the relation to a conceptual ADS functional architecture. This analysis must address the complexity that comes with partially overlapping sensor data from different sensors and considers variations in performance and characteristics due to changes in the environmental conditions. The paper introduces the term incomplete redundancy and presents a systematic methodology for analyzing redundancy. The aim is to provide arguments for how several sensors in a system, when appropriately combined, meet an assigned safety requirement on a higher level. Each sensor will then be assigned a certain responsibility and contributes with a sub-set of information. A set of questions of importance to address as a foundation for such a methodology are defined and discussed. The definitions of redundancy and independence between sensors are discussed as well as contract-based functional safety to adapt to different environmental and operating conditions.</div></div>

<div class="section abstract"><div class="htmlview paragraph">Curbs are as key to automated driving system (ADS) navigation, operation, and safety as they are for human driven vehicles. The design, maintenance, and management of curbs and adjacent infrastructure can make the difference in whether ADS vehicles can pick up and deliver passengers and goods safely, efficiently, and effectively. Curbs may also be key to integrating ADS services with other forms of active and human-driven transportation. Benefits from accessibility, reduced emissions, and strong supply chains require that ADS vehicles be able to dock curbside in a manner that does not disrupt traffic or impede safe movement of people walking, biking, or using a mobility device.</div><div class="htmlview paragraph"><b>Automated Vehicles and Infrastructure Enablers: Curbs and Curbside Management</b> addresses considerations regarding the curb with respect to pick up and drops for passengers and freight, as well as managing and designing both sides of the curb with respect to automated vehicles and other types of shared mobility. These issues are examined in the context of the US and emerging practices in the UK, Europe, and Asia.</div><div class="htmlview paragraph"><a href="https://www.sae.org/publications/edge-research-reports" target="_blank">Click here to access the full SAE EDGE</a><sup>TM</sup><a href="https://www.sae.org/publications/edge-research-reports" target="_blank"> Research Report portfolio.</a></div></div>

<div class="section abstract"><div class="htmlview paragraph">With the advancement of automated driving system levels, corner scenarios characterized by low probability and high risk have become critical for the safety validation of automated vehicles. However, due to the typical long-tail distribution of such scenarios, data-driven mining approaches face significant challenges in achieving efficient generation. To address this issue, this study proposes a feature-optimized combination-based method for generating corner scenarios in automated driving systems. Key scenario features related to functional failures are first identified using a combined approach of system theoretic process analysis (STPA) and hazard and operability analysis (HAZOP). Based on these features, an adaptive genetic algorithm is employed to optimize feature combinations and generate large numbers of corner scenario types that meet specified constraints. The proposed method is validated using cut-in and pedestrian-crossing scenarios as baseline cases. The results show that this method enables large-scale generation of corner scenario types grounded in regulatory scenarios and provides significant support for the development of comprehensive corner scenario libraries for automated vehicle testing.</div></div>

As Automated Driving Systems (ADS) advance toward higher levels of automation (SAE Levels 3-5), the role of human drivers is shifting from active control to supervision and intervention. However, traditional human-automation interaction frameworks do not fully account for the dynamic team-based coordination required for effective ADS integration. Poor adaptability and coordination between drivers and ADS can result in critical safety failures, as seen in real-world incidents. This research focuses on human-autonomy teaming (HATs) in ADS, emphasizing collaborative adaptation, shared decision-making, and real-time interaction metrics to enhance safety and user experience. By incorporating team cognition theory and layered dynamical models, this research aims to develop human-machine teaming metrics that facilitate adaptive and context-aware cooperation. Key research questions include: (1) How do human drivers and ADS dynamically interact? (2) What team cognition metrics effectively measure human-ADS adaptability? (3) How can real-time analytics improve driver-ADS collaboration and safety?

<div class="section abstract"><div class="htmlview paragraph">Intersections have a high risk of vehicle-to-vehicle conflicts because of the overlapping traffic flow from multiple roads. To understand the factors contributing to the crashes, this study examines the common characteristics in intersection-related crash and near- crash events, such as the existence of traffic control devices, the driver at fault, and occurrence of visual obstructions. The descriptive data of the crash and near-crash events recorded in the Second Strategic Highway Research Program Naturalistic Driving Study (SHRP 2 NDS) database is used in categorization and statistical analysis in this study. First, the events are divided into seven categories based on trajectories of the conflicting vehicles. The categorization provides the basis for in-depth analysis of crash-contributing factors in specific confliction patterns. Subsequently, descriptive statistics are used to portray each of the categories. The severity of the categories is determined based of the frequency of occurrences of the crashes and near-crashes. Factors contributing to each crash category are then examined based on their frequency of occurrence with the categories. This study reveals the severity of different intersection-related crash patterns, as well as the effects of some crash-contributing factors. The information presented in this paper has the potential to guide investigators interested in developing simulation and test track scenarios to evaluate vehicles equipped with automated driving systems (ADS).</div></div>

<div class="section abstract"><div class="htmlview paragraph">Effectively determining automated driving system (ADS)-equipped vehicle (AV) safety without relying on testing an infeasibly large number of driving scenarios is a challenge with wide recognition in industry and academia. The following paper builds on previous work by the Institute of Automated Mobility (IAM) and Science Foundation Arizona (SFAz), and proposes a test selection and scoring methodology (TSSM) as part of a safety case-based framework being developed by the SFAz to ensure the safety of AVs while addressing the scenario testing challenge. The TSSM is an AV verification and validation (V&V) process that relies, in part, on iterative, partially random generation of AV driving scenarios. These scenarios are generated using an operational design domain (ODD) and behavioral competency portfolio, which expresses the vehicle ODD and behavioral competencies in terms of quantifiable amounts or intensities of discrete components. Once generated, these scenarios are subjected to filters based on their relevance to the AV ODD and behavioral competency portfolio that preserves the robustness of the generated test set; after filtration, scenarios are assigned to a test method and executed. Further, these scenarios may be generated entirely by the TSSM or may be drawn from a preexisting scenario database and subjected to the same filtration process. After the scenarios assembled by the TSSM are executed, the methodology aggregates their driving assessment (DA) scores into a single numerical value. We outline the overall safety case-based framework, the TSSM, including its role in the framework as well as planned future work, and outline two proofs of concept: (1) a demonstration of the ability of the TSSM to pare down the space of scenarios in a scenario database; and (2) a specification form which may be used to solicit a description of the AV ODD and behavioral competency portfolio from the AV developer.</div></div>

The technological barriers to automated driving systems (ADS) are being quickly overcome to deploy on–road vehicles that do not require a human driver on–board. ADS have opened up possibilities to improve mobility, productivity, logistics planning, and energy consumption. However, further enhancements in productivity and energy consumption are required to reach CO2–reduction goals, owing to increased demands on transportation. In particular, in the freight sector, incorporation of automation with electrification can meet necessities of sustainable transport. However, the profitability of battery electric heavy vehicles (BEHVs) remains a concern. This study found that ADS led to profitability of BEHVs, which remained profitable for increased travel ranges by a factor of four compared to that of BEHVs driven by humans. Up to 20% reduction in the total cost of ownership of BEHVs equipped with ADS could be achieved by optimizing the electric propulsion system along with the infrastructure for a given transportation task. In that case, the optimized propulsion system might not be similar to that of a BEHV with a human driver. To obtain the results, the total cost of ownership was minimized numerically for 3072 different transportation scenarios that showed the effects of travel distance, road hilliness, average reference speed, and vehicle size on the incorporated electrification and automation, and compared to that of conventional combustion–powered heavy vehicles.

<div class="section abstract"><div class="htmlview paragraph">Currently, there are no safe and suitable fuel sources with comparable power density to traditional combustible fuels capable of replacing Internal Combustion Engines (ICEs). For the foreseeable future, civilian and military systems are likely to be reliant on traditional combustible fuels. Hybridization of the vehicle powertrains is the most likely avenue which can reduce emissions, minimize system inefficiencies, and build more sustainable vehicle systems that support the United States Army modernization priorities. Vehicle systems may further be improved by the creation and implementation of artificial intelligence and machine learning (AI/ML) in the form of advanced predictive capabilities and more robust control policies. AI/ML requires numerous characterized and complete datasets, given the sensitive nature of military systems, such data is unlikely to be known or accessible limiting the reach to develop and deploy AI/ML to military systems. With the absence of data, AI/ML may still be developed and deployed to military systems if supported by near-real-time or real-time computationally efficient and effective hardware and software or cloud-based computing. In this research, an OPAL real-time (OPAL-RT) simulator was used to emulate a compression ignition (CI) engine simulation architecture capable of developing and deploying advanced AI/ML predictive algorithms. The simulation architecture could be used for developing online predictive capabilities required to maximize the effectiveness or efficiency of a vehicle. The architecture includes a real-time simulator (RTS), a host PC, and a secondary PC. The RTS simulates a crank angle resolved engine model which utilized pseudo engine dynamometer data in the form of multi-dimensional matrices to emulate quasi-steady state conditions of the engine. The host PC was used to monitor and control the engine while the secondary PC was used to train the AI/ML to predict the per-cylinder generated torque from the crank shaft torque, which was then used to predict the in-cylinder temperature and pressure. The results indicate that using minimal sensor data and pretrained predictive algorithms, in-cylinder characterizations for unobserved engine variables may be achievable, providing an approximate characterization of quasi-steady state in-cylinder conditions.</div></div>

<div class="section abstract"><div class="htmlview paragraph">Accelerated adoption of electric propulsion system in mobility industry has stressed the time and iterations of product development cycle which was traditionally known to go over multiple iterations and phases. Current market demands a timely introduction of compelling products that brings high value to end user. Further, a growing emphasis over reducing mineral content using sustainable options and process, adds further complexity to multi-objective-optimization of electric drive systems. At BorgWarner our engineers use Digital-Twins, physics-based models which closely represent BorgWarner products in greater dept (physics) thus allowing an improved assessment of product design (components and systems) to target application at very early stage in product development. The spring success with Digital-Twin, BorgWarner furthered enhanced the model through introducing Artificial Intelligent (AI) and Machine Learning (ML) technologies in both modelling and virtual sensing.</div><div class="htmlview paragraph">This paper will provide reader an in-depth view of technology aspects with Digital-Twin and introduce AI and ML algorithms within Digital-Twin, beginning with a brief introduction on AI and ML technologies, the paper will also go in depth on in-use applications of these technologies at BorgWarner, such as, Deep learning virtual sensors usage. The paper will include a clear description of specialty tools and methods adopted by BorgWarner for data cleaning, model training and validation. The paper will provide reader an insight into how such trained AI-ML models were developed and trained using data provided by validated 1D and high-fidelity models from Amesim and COMSOL respectively, where each of those reference datasets were verified using real hardware tested in a vehicle environment. Although scope within BorgWarner product development was surrounding integrated drive module, for the purpose of showcasing an example the paper will provide technical insight to “Deep Learning Virtual Sensor for Power Module”. Towards conclusion, our goal is to showcase the method of integrating AI-ML models in Digital Twin and how those learnings can be translated into a product feature which could minimize controls complexity while enhancing accuracy and safety of the electrified propulsion system.</div></div>

<div class="section abstract"><div class="htmlview paragraph">Advanced technologies that assist the human driver or reduce (or even eliminate) the human driver’s role are becoming increasingly prevalent in new light-duty vehicles used by the general public. These technologies are divided between Active Safety features that monitor the human driver and vehicle motion and act intermittently to mitigate and avoid crashes, and Driving Automation features that assume some or all of the dynamic driving task from the human driver. Both types of technologies have the potential to reduce injuries and save lives by reducing the frequency and/or severity of crashes.</div><div class="htmlview paragraph"><b>Safety Impacts of Active Safety and Driving Automation Features</b> addresses the current capabilities and future potential for Active Safety and Driving Automation features to reduce crash frequency and severity and provides an overview of the state of the industry for both types of features, including current deployments, trends, and anticipated rollouts. Gaps in knowledge, unsettled issues, and suggestions for future research will also be discussed.</div><div class="htmlview paragraph"><a href="https://www.sae.org/publications/edge-research-reports" target="_blank">Click here to access the full SAE EDGE</a><sup>TM</sup><a href="https://www.sae.org/publications/edge-research-reports" target="_blank"> Research Report portfolio.</a></div></div>

<div class="section abstract"><div class="htmlview paragraph">Several challenges remain in deploying Machine Learning (ML) into safety critical applications. We introduce a safe machine learning approach tailored for safety-critical industries including automotive, autonomous vehicles, defense and security, healthcare, pharmaceuticals, manufacturing and industrial robotics, warehouse distribution, and aerospace. Aiming to fill a perceived gap within Artificial Intelligence and ML standards, the described approach integrates ML best practices with the proven Process Failure Mode & Effects Analysis (PFMEA) approach to create a robust ML pipeline. The solution views ML development holistically as a value-add, feedback process rather than the resulting model itself. By applying PFMEA, the approach systematically identifies, prioritizes, and mitigates risks throughout the ML development pipeline. The paper outlines each step of a typical pipeline, highlighting potential failure points and tailoring known best practices to minimize identified risks. As an additional contribution, a populated ML FMEA Template is provided. The ML FMEA captures the method into a modified PFMEA framework that connects each pipeline step with failure causes with known mitigations. The ML FMEA Template is designed as a handy tool for development teams to identify, manage, and communicate risk and to enable risk transparency with safety experts.</div></div>