On Perception Safety Requirements and Multi Sensor Systems for Automated Driving Systems

<div class="section abstract"><div class="htmlview paragraph">Advanced driver assistance systems (ADAS) and automated driving systems (ADS) continue to expand into the market at a rapid pace. As improved (i.e., next generation) versions of these systems become available, they will continue to face many challenges in their implementation and benefits for safety and driving operations. The solution will involve many parties, including road safety professionals and researchers who see the potential in these systems but may have difficulties keeping up with them, and safety advocates who are calling for these systems to achieve higher levels of safety now.</div><div class="htmlview paragraph"><b>The Challenges of Next-gen ADAS and ADS and Related Vehicle Safety Topics</b> explores these challenges that will fall on the National Highway Traffic Safety Administration (NHTSA) and automakers as they balance costs and benefits; establish reasonable regulations and standards; and determine how to improve, test, deliver, and use these systems successfully. Perhaps the most formidable challenge will be overcoming the expectation that ADAS—and especially ADS—will always work perfectly in every scenario.</div><div class="htmlview paragraph"><a href="https://www.sae.org/publications/edge-research-reports" target="_blank">Click here to access the full SAE EDGE</a><sup>TM</sup><a href="https://www.sae.org/publications/edge-research-reports" target="_blank"> Research Report portfolio.</a></div></div>

<div class="section abstract"> <div class="htmlview paragraph">This Recommended Practice provides a taxonomy for motor vehicle <i>driving automation systems</i> that perform part or all of the <i>dynamic driving task</i> (<i>DDT</i>) on a <i>sustained</i> basis and that range in level from <i>no driving automation</i> (level 0) to <i>full driving automation</i> (level 5). It provides detailed definitions for these six levels of <i>driving automation</i> in the context of <i>motor vehicles</i> (hereafter also referred to as “<i>vehicle</i>” or “<i>vehicles</i>”) and their <i>operation</i> on roadways. These level definitions, along with additional supporting terms and definitions provided herein, can be used to describe the full range of <i>driving automation features</i> equipped on <i>motor vehicles</i> in a functionally consistent and coherent manner. “On-road” refers to publicly accessible roadways (including parking areas and private campuses that permit public access) that collectively serve users of <i>vehicles</i> of all classes and <i>driving automation</i> levels (including no <i>driving automation</i>), as well as motorcyclists, pedal cyclists, and pedestrians.</div> <div class="htmlview paragraph">The levels apply to the <i>driving automation feature(s)</i> that are engaged in any given instance of on-road <i>operation</i> of an equipped <i>vehicle</i>. As such, although a given <i>vehicle</i> may be equipped with a <i>driving automation system</i> that is capable of delivering multiple <i>driving automation features</i> that perform at different levels, the level of <i>driving automation</i> exhibited in any given instance is determined by the <i>feature(s)</i> that are engaged.</div> <div class="htmlview paragraph">This document also refers to three primary actors in <i>driving</i>: the (human) <i>driver</i>, the <i>driving automation system</i>, and other <i>vehicle</i> systems and components. These other <i>vehicle</i> systems (or the <i>vehicle</i> in general terms) do not include the <i>driving automation system</i> in this model, even though as a practical matter a <i>driving automation system</i> may actually share hardware and software components with other <i>vehicle</i> systems, such as a processing module(s) or operating code.</div> <div class="htmlview paragraph">The levels of <i>driving automation</i> are defined by reference to the specific role played by each of the three primary actors in performance of the <i>DDT</i>. “Role” in this context refers to the expected role of a given primary actor, based on the design of the <i>driving automation system</i> in question and not necessarily to the actual performance of a given primary actor. For example, a <i>driver</i> who fails to <i>monitor</i> the roadway during engagement of a level 1 adaptive cruise control (ACC) system still has the role of <i>driver</i>, even while s/he is neglecting it.</div> <div class="htmlview paragraph"><i>Active safety systems</i>, such as electronic stability control and automated emergency braking, and certain types of driver assistance systems, such as lane keeping assistance, are excluded from the scope of this <i>driving automation</i> taxonomy because they do not perform part or all of the <i>DDT</i> on a <i>sustained</i> basis and, rather, merely provide momentary intervention during potentially hazardous situations. Due to the momentary nature of the actions of <i>active safety systems</i>, their intervention does not change or eliminate the role of the <i>driver</i> in performing part or all of the <i>DDT,</i> and thus are not considered to be <i>driving automation</i>.</div> <div class="htmlview paragraph">It should, however, be noted that crash avoidance features, including intervention-type <i>active safety systems</i>, may be included in <i>vehicles</i> equipped with <i>driving automation systems</i> at any level. For <i>ADS</i>-equipped <i>vehicles</i> (i.e., levels 3-5) that perform the complete <i>DDT</i>, crash avoidance capability is part of <i>ADS</i> functionality.</div></div>

<div class="section abstract"> <div class="htmlview paragraph">This SAE Recommended Practice provides common data output formats and definitions for a variety of data elements that may be useful for analyzing the performance of automated driving system (ADS) during an event that meets the trigger threshold criteria specified in this document. The document is intended to govern data element definitions, to provide a minimum data element set, and to specify a common ADS data logger record format as applicable for motor vehicle applications.</div> <div class="htmlview paragraph">Automated driving systems (ADSs) perform the complete dynamic driving task (DDT) while engaged. In the absence of a human “driver,” the ADS itself could be the only witness of a collision event. As such, a definition of the ADS data recording is necessary in order to standardize information available to the accident reconstructionist. For this purpose, the data elements defined herein supplement the SAE J1698-1 defined EDR in order to facilitate the determination of the background and events leading up to a collision in an ADS-operated vehicle.</div> <div class="htmlview paragraph">The data elements defined in this document are unique to Level 3, 4, or 5 ADS features, as defined by SAE J3016, and provide additional background of the events leading up to a crash or crash-like event. The data from sensors such as camera(s), LiDAR(s) etc. will provide information in the absence of a human driver. The data included in the ADS data logger is expected to be used in conjunction with the SAE J1698 event data recorder (EDR) record and traditional accident reconstruction analysis. The EDR and ADS data logger will capture information leading up to the triggered event, at a minimum. There are no facts to support that recording data for greater than 5 seconds pre-event would change the outcome of any crash analysis. Thus, the recommended recording duration for a data logger is 5 seconds pre-event, same as an EDR. Due to the potential for sensor and/or communication failure during a crash event, the recommendation is that data should be collected post-crash for impact and rollover sensors for up to 250 ms.</div> <div class="htmlview paragraph">ADS technology is still being developed and is not yet commercially deployed. Therefore, this SAE Recommended Practice is intended as a guide toward standard practice and is subject to change to keep pace with experience and technical advances.</div> </div>

<div class="section abstract"> <div class="htmlview paragraph">This document provides preliminary<sup><span class="xref">1</span></sup> safety-relevant guidance for in-vehicle fallback test driver training and for on-road testing of vehicles being operated by prototype conditional, high, and full (Levels 3 to 5) ADS, as defined by SAE J3016. It does not include guidance for evaluating the performance of post-production ADS-equipped vehicles. Moreover, this guidance only addresses testing of ADS-operated vehicles as overseen by in-vehicle fallback test drivers (IFTD).</div> <div class="htmlview paragraph">These guidelines do not address:</div> <ul class="list disc"> <li class="list-item"> <div class="htmlview paragraph">Remote driving, including remote fallback test driving of prototype ADS-operated test vehicles in driverless operation. (Note: The term “remote fallback test driver” is included as a defined term herein and is intended to be addressed in a future iteration of this document. However, at this time, too little is published or known about this type of testing to provide even preliminary guidance.)</div></li> <li class="list-item"> <div class="htmlview paragraph">Testing of driver support features (i.e., Levels 1 and 2), which rely on a human driver to perform part of the dynamic driving task (DDT) and to supervise the driving automation feature’s performance in real time. (Refer to SAE J3016.)</div></li> <li class="list-item"> <div class="htmlview paragraph">Closed-course testing.</div></li> <li class="list-item"> <div class="htmlview paragraph">Simulation testing (except for training purposes).</div></li> <li class="list-item"> <div class="htmlview paragraph">Component-level testing.</div></li></ul> <div class="htmlview paragraph">These guidelines also do not address prototype vehicle and IFTD performance data collection and retention. The collection of data invokes various legal and risk management considerations that users of this document should nevertheless bear in mind, such as:</div> <ul class="list disc"> <li class="list-item"> <div class="htmlview paragraph">Maintaining auditable procedures and documentation.</div></li> <li class="list-item"> <div class="htmlview paragraph">Adhering to applicable privacy laws and principles.</div></li> <li class="list-item"> <div class="htmlview paragraph">Ensuring adequate data collection and recording integrity to support post-crash forensic analysis.</div></li></ul></div>

<div class="section abstract"><div class="htmlview paragraph">Automated driving systems (ADSs) for road vehicles are being developed that can perform the entire dynamic driving task without a human driver in the loop. However, current regulatory frameworks for assuring vehicle safety may restrict the deployment of ADSs that can use machine learning to modify their functionality while in service. A review was undertaken to identify and assess key initiatives and research relevant to the safety assurance of adaptive safety-critical systems that use machine learning, and to highlight assurance concepts that could benefit from further research. The primary objective was to produce findings and recommendations that can inform policy and regulatory reform relating to ADS safety assurance. Due to the almost infinite number and combination of scenarios that an ADS could encounter, the review found much support for concepts that involve the use of simulation data as virtual evidence of safety compliance, with suggestions of a need to assure simulation tools and models. Real-world behavioural competency testing was also commonly proposed, although noting this concept has its limitations. The concept of whole-of-life assurance was identified, supported by various versions of dynamic runtime monitoring, verification and validation. Concerns regarding Artificial Intelligence (AI) robustness were raised, particularly regarding adversarial inputs and unmodelled scenarios that are essentially unknown unknowns. Further, the concept of explainable AI was highlighted as having potential to provide evidence from an ADS that could support safety assurance and regulatory compliance. While each of the identified assurance concepts should be considered when developing ADS safety assurance framework options, it is recommended that further research on each concept should be progressed.</div></div>

When arguing safety for an autonomous road vehicle it is considered very hard to show that the sensing capability is sufficient for all possible scenarios that might occur. Already for today’s manually driven road vehicles equipped with advanced driver assistance systems (ADAS), it is far from trivial how to argue that the sensor systems are sufficiently capable of enabling a safe behavior. In this paper, we argue that the transition from ADAS to automated driving systems (ADS) enables new solution patterns for the safety argumentation dependent on the sensor systems. A key factor is that the ADS itself can compensate for a lower sensor capability, by for example lowering the speed or increasing the distances. The proposed design strategy allocates safety requirements on the sensors to determine their own capability. This capability is then to be balanced by the tactical decisions of the ADS equipped road vehicle.

<div class="section abstract"><div class="htmlview paragraph">Automotive industry is inclined towards connected, comfortable, environment friendly, efficient and smarter systems. Advanced Driver Assist System (ADAS) technology assist drivers to achieve a safer as well as better ride by automation and improvisation of the vehicular systems. With the advent of ADAS system, there is a significant focus not only in the development of Electronic Control Units (ECUs) and its features to cater to the emerging market but also on the information that could be displayed to meet the functional as well as safety requirements. This ADAS information display ensures timely notification to the driver with unique alerts that can be acoustic or visual. These systems should be tested thoroughly to ensure reliability as failures may impose severe risk on the OEM. Hardware in the loop testing has been largely adopted by industry against manual testing in lieu of the testing constraints imposed by the latter. This paper discusses on the existing test methods for ADAS information display system testing using HIL. It also details on the automation trends adopted and highlights the key challenges associated with the development as well as reusability of the conventional test methods. It discusses an Artificial Intelligence based strategy for automation of cluster objects teaching, evaluation and illustrates the adoption of the same for ADAS HIL testing and brings out the efficiency and benefits achieved by the proposed strategy.</div></div>

<div class="section abstract"><div class="htmlview paragraph">Curbs are as key to automated driving system (ADS) navigation, operation, and safety as they are for human driven vehicles. The design, maintenance, and management of curbs and adjacent infrastructure can make the difference in whether ADS vehicles can pick up and deliver passengers and goods safely, efficiently, and effectively. Curbs may also be key to integrating ADS services with other forms of active and human-driven transportation. Benefits from accessibility, reduced emissions, and strong supply chains require that ADS vehicles be able to dock curbside in a manner that does not disrupt traffic or impede safe movement of people walking, biking, or using a mobility device.</div><div class="htmlview paragraph"><b>Automated Vehicles and Infrastructure Enablers: Curbs and Curbside Management</b> addresses considerations regarding the curb with respect to pick up and drops for passengers and freight, as well as managing and designing both sides of the curb with respect to automated vehicles and other types of shared mobility. These issues are examined in the context of the US and emerging practices in the UK, Europe, and Asia.</div><div class="htmlview paragraph"><a href="https://www.sae.org/publications/edge-research-reports" target="_blank">Click here to access the full SAE EDGE</a><sup>TM</sup><a href="https://www.sae.org/publications/edge-research-reports" target="_blank"> Research Report portfolio.</a></div></div>

<div class="section abstract"><div class="htmlview paragraph">New challenges and complexities are continuously increasing in advanced driver assistance systems (ADAS) development (e.g. active safety, driver assistant and autonomous vehicle systems). Therefore, the health management of ADAS’ components needs special improvements. Since software contribution in ADAS’ development is increasing significantly, remote diagnosis and maintenance for ADAS become more important. Furthermore, it is highly recommended to predict the remaining useful life (RUL) for the prognosis of ADAS’ safety critical components; e.g. (Ultrasonic, Cameras, Radar, LIDAR). This paper presents a remote diagnosis, maintenance and prognosis (RDMP) framework for ADAS, which can be used during development phase and mainly after production. An overview of RDMP framework’s elements is explained to demonstrate how/when this framework is connected to database servers and remote analysis servers. Moreover, Sensors fusion is used in RDMP to detect some sensor failures and even to predict their RUL. Additionally, some well-known machine learning algorithms (MLA) are used to predict RUL of ADAS’ components, and different types of input attributes to these MLA are proposed for some basic ADAS’ components. MLA use training data set, which shall be constructed ideally from actual records reported remotely by RDMP (Prognosis Analysis and Self-Learning System). However, initial dataset before production of the vehicle can be created from ADAS laboratory tests (e.g. Assessments on test tracks), ADAS simulation and theoretical analytical methods. Also, experiments of using the proposed RDMP in some ADAS’ components (Sensor fusion and Braking system as ADAS actuator) are presented. Summary, conclusion with proven results and future work are explained.</div></div>

<div class="section abstract"><div class="htmlview paragraph">With the advancement of automated driving system levels, corner scenarios characterized by low probability and high risk have become critical for the safety validation of automated vehicles. However, due to the typical long-tail distribution of such scenarios, data-driven mining approaches face significant challenges in achieving efficient generation. To address this issue, this study proposes a feature-optimized combination-based method for generating corner scenarios in automated driving systems. Key scenario features related to functional failures are first identified using a combined approach of system theoretic process analysis (STPA) and hazard and operability analysis (HAZOP). Based on these features, an adaptive genetic algorithm is employed to optimize feature combinations and generate large numbers of corner scenario types that meet specified constraints. The proposed method is validated using cut-in and pedestrian-crossing scenarios as baseline cases. The results show that this method enables large-scale generation of corner scenario types grounded in regulatory scenarios and provides significant support for the development of comprehensive corner scenario libraries for automated vehicle testing.</div></div>

<div class="section abstract"><div class="htmlview paragraph">Now-a-days, Advanced driver-assistance systems (ADAS) is equipping cars and drivers with advance information and technology to make them become aware of the environment and handle potential situations in better way semi-autonomously. High-quality training and test data is essential in the development and validation of ADAS systems which lay the foundation for autonomous driving technology.</div><div class="htmlview paragraph">ADAS uses the technology like radar, vision and combinations of various sensors including LIDAR to automatize dynamic driving tasks like steering, braking, and acceleration of vehicle for controlled and safe driving. And to integrate these advance technologies, the ADAS needs labeled data to train the algorithm to detect the various objects and moments of driver. Image annotation is one the well-known service to create such training data for computer vision.</div><div class="htmlview paragraph">There are number of open source annotated datasets available viz. COCO, KITTI etc. But these datasets are limited to either US or European road environment scenarios. There is hardly any dataset available for Indian Specific road conditions. In order to capture all the India Specific objects on road, we are capturing data through various environmental conditions, road conditions and annotating objects on the collected camera images. 2-D bounding boxes are being used to annotate objects in an image.</div><div class="htmlview paragraph">These annotated images will then be used to train a machine learning model. Finally this trained model will be then used to detect Indian specific objects with real time data acquired through camera sensors mounted on the vehicle. Object detection is one of the prime aspects for ADAS functionality. The complete process will demonstrate end to end solution for object detection techniques from data generation to real-time object identification and recognition.</div></div>

As Automated Driving Systems (ADS) advance toward higher levels of automation (SAE Levels 3-5), the role of human drivers is shifting from active control to supervision and intervention. However, traditional human-automation interaction frameworks do not fully account for the dynamic team-based coordination required for effective ADS integration. Poor adaptability and coordination between drivers and ADS can result in critical safety failures, as seen in real-world incidents. This research focuses on human-autonomy teaming (HATs) in ADS, emphasizing collaborative adaptation, shared decision-making, and real-time interaction metrics to enhance safety and user experience. By incorporating team cognition theory and layered dynamical models, this research aims to develop human-machine teaming metrics that facilitate adaptive and context-aware cooperation. Key research questions include: (1) How do human drivers and ADS dynamically interact? (2) What team cognition metrics effectively measure human-ADS adaptability? (3) How can real-time analytics improve driver-ADS collaboration and safety?

<div class="htmlview paragraph">Digital maps have large potential to enhance or enable Advanced Driver Assistance Systems (ADAS) by extending driver horizon and therefore to contribute to safer, smarter and cleaner road transport. As a predictive sensor called ADAS Horizon, in-vehicle digital maps are an important source of information providing look-ahead capability for ADAS applications and providing further information for on-board sensors to enhance environment perception.</div> <div class="htmlview paragraph">This paper reports on results from the European Commission project MAPS&ADAS, part of PReVENT Integrated Project, dedicated to the development, test and validation of a standardized interface (ADAS Interface) between ADAS applications and ADAS map data sources for accessing map data regarding vehicle position.</div>

<div class="htmlview paragraph">The aim of this paper is to present the Hermes Thermal Control Concept which constitutes the reference baseline for the phase 2 of the Hermes program.</div> <div class="htmlview paragraph">The technical solutions based on active and passive means, are dictated by a variety of requirements including temperature and heat flux limits, mass & power minimization, availability of space for accommodation of TCS devices.</div> <div class="htmlview paragraph">Furthermore thecomplex mission profile (including atmospheric flight and several modes of operation o002d;orbit) and the configuration of the space-vehicle, require the adoption of a particularly flexible thermal control.</div> <div class="htmlview paragraph">The <b>ATCS (Active Thermal Control Section)</b> is based on a dual loop concept, using both water and Freon R114 cooling loops for heat collection from the various sources, heat transportation and heat rejection through dedicated devices.</div> <div class="htmlview paragraph">The main heat sources to be considered are those extracted from the cabin air loop, the electronics, the power generation items (fuel cells and lithium batteries) and the environmental loads during ascent and re-entry.</div> <div class="htmlview paragraph">The major heat rejection devices are the Ground Support Equipment Heat Exchangers (on ground, before launch and after landing), the Ammonia Boilers during ascent or re-entry (low altitude), the Hermes Radiators on orbit, the Water Evaporators during ascent and re-entry (high altitudes) and to complement the radiator rejection capability.</div> <div class="htmlview paragraph">The <b>PTCS (Passive Thermal Control Section)</b> relies on different concepts of insulations together with a system of temperature sensors and heaters, to control the thermal excursions of the space-plane components and structural parts, within the specified limits.</div> <div class="htmlview paragraph">Thermal insulation blankets (foams, thermal control coatings, multi layer insulations) involve an extensive material research and testing activity, since they are subjected to quite different conditions: vacuum, atmospheric, internal, external and very different level of temperatures depending on the location.</div> <div class="htmlview paragraph">The present design of the Thermal Control Subsystem is oriented to the minimization of mass and power budgets and to respect the quantitative and qualitative safety and reliability requirements.</div>

<div class="section abstract"><div class="htmlview paragraph">As a result of the development of Event Data Recorders (EDR) and the recent FMVSS regulation 49 CFR 563, today’s automobiles provide a limited subset of electronic data measurements of a vehicle’s state before and during a crash. Prior to this data, the only information available about the vehicle movements before or during a collision had come from physical evidence (e.g. tire marks), witnesses, aftermarket camera systems on vehicles, and ground-based cameras that were monitoring vehicle traffic or used for security surveillance. Today’s vehicles equipped with Advanced Driver Assistance Systems (ADAS) have vehicle-based sensors that measure information about the environment around a vehicle including other vehicles, pedestrians, and fixed wayside objects. Vehicles equipped with these ADAS systems use primarily radar, lidar, ultrasonic, and/or image sensors either in standalone operation or in combination to establish the range and movement of potential hazardous objects (e.g. other vehicles, poles, and pedestrians) around a vehicle. The data from these sensors is unique because it measures objective environment information surrounding a vehicle, which can play a role in reconstructing and understanding the contributing factors of an accident. As more vehicles become equipped with advanced safety systems and the requisite sensors for automated vehicles, accident investigators will need to become familiar with system functionality, the data that the sensors measure and may record, as well as the limitations of that data in order to effectively and accurately analyze a crash.</div><div class="htmlview paragraph">This paper reviews the sensors in ADAS collision avoidance systems that may be present on current vehicles and how the data from a vehicle based radar may be used to reconstruct a collision. A radar sensor mounted on a host vehicle is experimentally evaluated using stationary objects, and a stationary and moving radar. The results demonstrate how an investigator would use the radar data to recognize a fixed object in the environment and analyze the relative position with respect to the host vehicle. The analysis of this radar data includes the limitations, accuracy, and validation of the particular radar sensor data. In addition, the authors propose in general how data from radars could be used to elucidate the location of fixed objects in the environment surrounding a vehicle for the purpose of understanding an accident.</div></div>