Remote Diagnosis, Maintenance and Prognosis for Advanced Driver Assistance Systems Using Machine Learning Algorithms

<div class="section abstract"><div class="htmlview paragraph">The testing of vision sensor-based Advanced Driver Assistance (ADA) systems and Electronic Control Units (ECUs) is quickly becoming a major requirement in the automotive industry. ADA ECU development teams require a verification and validation (V&V) infrastructure early in the development cycle, and traditional approaches are not suitable for this purpose. This paper proposes the design of an integrated test suite for the functional and real-time testing of ADA algorithms and ECUs. The proposed test suite consists of a hardware bridge built using a Field Programmable Gate Array (FPGA), a Universal Serial Bus (USB) 3.0 interface, and a software application running on the Personal Computer (PC). By using an FPGA to build the hardware bridge, we get a compact, real-time, and reconfigurable platform to support the desired test requirements. The USB3.0 interface is specifically proposed to support the high data rates evident in ADA systems based on multiple vision sensor and sensor fusion technologies. The PC-based software application has a modular, extensible, and scalable architecture to support current and future test requirements, as well as several features that improve the end user's productivity. The proposed test suite can be used for: (a) acquisition and storage of sensor data (recording mode), (b) visualization of stored sensor data (replay mode), (c) offline evaluation and improvement of ADA algorithms (functional test mode), (d) online testing of advanced driver assistance system (ADAS) ECUs (real-time test mode). The major contribution of this paper is the design proposal for supporting the real-time test mode for laboratory testing of ADA ECUs, and it concludes with a qualitative discussion of potential enhancements.</div></div>

<div class="htmlview paragraph">Digital maps have large potential to enhance or enable Advanced Driver Assistance Systems (ADAS) by extending driver horizon and therefore to contribute to safer, smarter and cleaner road transport. As a predictive sensor called ADAS Horizon, in-vehicle digital maps are an important source of information providing look-ahead capability for ADAS applications and providing further information for on-board sensors to enhance environment perception.</div> <div class="htmlview paragraph">This paper reports on results from the European Commission project MAPS&ADAS, part of PReVENT Integrated Project, dedicated to the development, test and validation of a standardized interface (ADAS Interface) between ADAS applications and ADAS map data sources for accessing map data regarding vehicle position.</div>

Modern automobiles are increasingly equipped with Advanced Driver Assistance Systems (ADAS), which improve passenger safety and comfort. Each of these ADAS can rely on a multitude of environmental sensors, which creates the need of cross-linking ADAS with respect to their access to environment information. This idea is reflected in the so-called environment model (EM) that centralizes the access of all ADAS functions to the perceived but not yet interpreted scene in the vehicle environment. In this paper, we propose an architectural framework that concretizes the EM in terms of separate modules for signal processing and sensor data fusion, as well as well-defined generic input and output interfaces. Moreover, we extend the EM by including additional components and output interfaces for centralized scene interpretation (CSI). The architectural framework allows streamlining the development process and increasing computational efficiency, as software modules for scene interpretation can be shared among multiple ADAS. In addition, the consistency of the overall ADAS behavior is improved, as all ADAS functions rely on the same scene information and scene interpretation. We illustrate our concept of centralized scene interpretation using the case study of a module for the detection of directional roadways, a generalization of motorways. The algorithm we present analyzes multiple perceptual indicators in favor or against a directional roadway and computes an overall decision, along with a foresight value for the expected duration of the directional roadway, to be used in various ADAS functions.

The use of intelligent technology has been widely applied to vehicles. ADAS (Advanced Driver Assistance System) is one of the smart features embedded in vehicles, generally directed to assist in driving a vehicle and for safety, such as braking. The development of ADAS is the first step toward an autonomous vehicle system. The braking system currently relies on several detectors, namely Radar, LiDAR, and cameras. The current use of ADAS usually uses a radar system or the integration of radar and cameras. This article aims to analyze the impact of using ADAS in braking assistance with the help of a stereo camera as the object detector to know how much influence intelligent systems for ADAS in driving. Several speeds are compared to see the vehicle’s response and the effect of the applied speed with the camera properly. The results show that the speed response can be increased by 0.3 seconds to reduce the delay in the driver’s awareness response to identify the object in front of him. Furthermore, the use of a camera-based system as a tool in ADAS in this study can be used for medium speeds of about 50 km/hour.

<div class="section abstract"><div class="htmlview paragraph">Now-a-days, Advanced driver-assistance systems (ADAS) is equipping cars and drivers with advance information and technology to make them become aware of the environment and handle potential situations in better way semi-autonomously. High-quality training and test data is essential in the development and validation of ADAS systems which lay the foundation for autonomous driving technology.</div><div class="htmlview paragraph">ADAS uses the technology like radar, vision and combinations of various sensors including LIDAR to automatize dynamic driving tasks like steering, braking, and acceleration of vehicle for controlled and safe driving. And to integrate these advance technologies, the ADAS needs labeled data to train the algorithm to detect the various objects and moments of driver. Image annotation is one the well-known service to create such training data for computer vision.</div><div class="htmlview paragraph">There are number of open source annotated datasets available viz. COCO, KITTI etc. But these datasets are limited to either US or European road environment scenarios. There is hardly any dataset available for Indian Specific road conditions. In order to capture all the India Specific objects on road, we are capturing data through various environmental conditions, road conditions and annotating objects on the collected camera images. 2-D bounding boxes are being used to annotate objects in an image.</div><div class="htmlview paragraph">These annotated images will then be used to train a machine learning model. Finally this trained model will be then used to detect Indian specific objects with real time data acquired through camera sensors mounted on the vehicle. Object detection is one of the prime aspects for ADAS functionality. The complete process will demonstrate end to end solution for object detection techniques from data generation to real-time object identification and recognition.</div></div>

<div class="section abstract"><div class="htmlview paragraph">Automotive industry is inclined towards connected, comfortable, environment friendly, efficient and smarter systems. Advanced Driver Assist System (ADAS) technology assist drivers to achieve a safer as well as better ride by automation and improvisation of the vehicular systems. With the advent of ADAS system, there is a significant focus not only in the development of Electronic Control Units (ECUs) and its features to cater to the emerging market but also on the information that could be displayed to meet the functional as well as safety requirements. This ADAS information display ensures timely notification to the driver with unique alerts that can be acoustic or visual. These systems should be tested thoroughly to ensure reliability as failures may impose severe risk on the OEM. Hardware in the loop testing has been largely adopted by industry against manual testing in lieu of the testing constraints imposed by the latter. This paper discusses on the existing test methods for ADAS information display system testing using HIL. It also details on the automation trends adopted and highlights the key challenges associated with the development as well as reusability of the conventional test methods. It discusses an Artificial Intelligence based strategy for automation of cluster objects teaching, evaluation and illustrates the adoption of the same for ADAS HIL testing and brings out the efficiency and benefits achieved by the proposed strategy.</div></div>

<div class="section abstract"><div class="htmlview paragraph">As a result of the development of Event Data Recorders (EDR) and the recent FMVSS regulation 49 CFR 563, today’s automobiles provide a limited subset of electronic data measurements of a vehicle’s state before and during a crash. Prior to this data, the only information available about the vehicle movements before or during a collision had come from physical evidence (e.g. tire marks), witnesses, aftermarket camera systems on vehicles, and ground-based cameras that were monitoring vehicle traffic or used for security surveillance. Today’s vehicles equipped with Advanced Driver Assistance Systems (ADAS) have vehicle-based sensors that measure information about the environment around a vehicle including other vehicles, pedestrians, and fixed wayside objects. Vehicles equipped with these ADAS systems use primarily radar, lidar, ultrasonic, and/or image sensors either in standalone operation or in combination to establish the range and movement of potential hazardous objects (e.g. other vehicles, poles, and pedestrians) around a vehicle. The data from these sensors is unique because it measures objective environment information surrounding a vehicle, which can play a role in reconstructing and understanding the contributing factors of an accident. As more vehicles become equipped with advanced safety systems and the requisite sensors for automated vehicles, accident investigators will need to become familiar with system functionality, the data that the sensors measure and may record, as well as the limitations of that data in order to effectively and accurately analyze a crash.</div><div class="htmlview paragraph">This paper reviews the sensors in ADAS collision avoidance systems that may be present on current vehicles and how the data from a vehicle based radar may be used to reconstruct a collision. A radar sensor mounted on a host vehicle is experimentally evaluated using stationary objects, and a stationary and moving radar. The results demonstrate how an investigator would use the radar data to recognize a fixed object in the environment and analyze the relative position with respect to the host vehicle. The analysis of this radar data includes the limitations, accuracy, and validation of the particular radar sensor data. In addition, the authors propose in general how data from radars could be used to elucidate the location of fixed objects in the environment surrounding a vehicle for the purpose of understanding an accident.</div></div>

<div class="section abstract"><div class="htmlview paragraph">Automotive industries focus on driver safety leading to raising improvements and advancements in Advanced Driver Assistance Systems (ADAS) to avoid collisions and provide safety and comfort to the drivers. This paper proposes a novel approach toward Driver health and fatigue monitoring systems that uses cabin cameras and biometric sensors communicating continuously with vehicle telematics systems to enhance real-time monitoring and predictive intervention. The data from the camera and biometric sensors is sent to the machine learning algorithm (LSBoost) which processes the data and if anything is wrong concerning the driver's behavior then immediately it communicates with vehicle telematics and sends information to the emergency services. This approach enhances driver safety and reduces accidents caused due to health-related driver impairment. This system comprises several sensors and fusion algorithms are applied between different sensors like cabin camera and biometric sensors, all these sensors are placed inside the driver compartment without disturbing the driver's comfort and functionality. The input from all these sensors is feed to a centralized processing unit, where advanced sensor fusion technology and machine learning algorithms are used for processing the raw data information. Especially selective machine learning models are used to detect patterns indicative of driver behavior such as drowsiness, fatigue, etc. The alerts will initially notify the driver. If the driver doesn’t respond, the alerts will then be forwarded to the vehicle’s telematics system, which in turn will notify emergency services like ambulance, or designated contacts.</div><div class="htmlview paragraph">This paper presents the architecture of the driver health monitoring system and the effectiveness of the proposed system is validated to the simulated model in the MATLAB environment showcasing its potential and ability to significantly enhance driver safety.</div></div>

<div class="section abstract"><div class="htmlview paragraph">Advanced driver assistance systems (ADAS) and automated driving systems (ADS) continue to expand into the market at a rapid pace. As improved (i.e., next generation) versions of these systems become available, they will continue to face many challenges in their implementation and benefits for safety and driving operations. The solution will involve many parties, including road safety professionals and researchers who see the potential in these systems but may have difficulties keeping up with them, and safety advocates who are calling for these systems to achieve higher levels of safety now.</div><div class="htmlview paragraph"><b>The Challenges of Next-gen ADAS and ADS and Related Vehicle Safety Topics</b> explores these challenges that will fall on the National Highway Traffic Safety Administration (NHTSA) and automakers as they balance costs and benefits; establish reasonable regulations and standards; and determine how to improve, test, deliver, and use these systems successfully. Perhaps the most formidable challenge will be overcoming the expectation that ADAS—and especially ADS—will always work perfectly in every scenario.</div><div class="htmlview paragraph"><a href="https://www.sae.org/publications/edge-research-reports" target="_blank">Click here to access the full SAE EDGE</a><sup>TM</sup><a href="https://www.sae.org/publications/edge-research-reports" target="_blank"> Research Report portfolio.</a></div></div>

Advanced Driver Assistance Systems (ADAS) have emerged as a critical technology in enhancing vehicle safety, providing convenience to drivers, and facilitating the transition toward fully autonomous vehicles. The core of these systems relies heavily on computer vision (CV) technologies, which enable vehicles to interpret and understand their surrounding environment in real time. The integration of computer vision in ADAS allows vehicles to recognize and process various objects such as pedestrians, vehicles, traffic signs, and lane markings, significantly improving both safety and the driving experience. In this paper, we explore the pivotal role of computer vision in the development and implementation of ADAS in autonomous vehicles. We begin by reviewing the evolution of ADAS, detailing the different sub-systems such as lane departure warning, automatic emergency braking, adaptive cruise control, and parking assistance, all of which depend on sophisticated image processing techniques. Furthermore, the paper examines the intersection of computer vision with other emerging technologies such as machine learning, deep learning, and sensor fusion. We discuss how these innovations enable ADAS to function more effectively in complex, dynamic environments. The methodology section focuses on cutting-edge algorithms used for object detection, semantic segmentation, and real-time image processing, which are integral to the functionality of modern ADAS. Additionally, we evaluate the challenges that computer vision systems face in real-world applications, including issues with sensor calibration, environmental conditions, computational limitations, and the integration of data from multiple sensors (e.g., cameras, LiDAR, radar). The paper also presents an analysis of the results obtained from various real-world implementations of ADAS technologies, highlighting improvements in driver safety and the reduction of traffic-related accidents. We explore the potential of computer vision in enabling fully autonomous driving by addressing the current gaps in technology and the advancements needed to close these gaps. Finally, the paper concludes with a discussion on the future of ADAS and the role of computer vision in realizing fully autonomous, reliable, and efficient transportation systems. We argue that continued research and development in computer vision, combined with innovations in deep learning and sensor technologies, are critical to advancing the capabilities and reliability of autonomous vehicles in the near future.

<div class="section abstract"><div class="htmlview paragraph">Given the current proliferation of active safety features on new vehicles, especially for Advanced Driver Assistance Systems (ADAS) and Highly Automated Driving (HAD) technologies, it is evident that there is a need for testing methods beyond a vehicle level physical test. This paper will discuss the current state of the art in the industry for simulation-based verification and validation (V&V) testing methods. These will include, but are not limited to, "Hardware-in-the-Loop (HIL)", “Software-in-the-Loop (SIL)”, “Model-in-the-Loop (MIL)”, “Driver-in-the-Loop (DIL)”, and any other suitable combinations of the aforementioned (XIL). Aspects of the test processes and needed components for simulation will be addressed, detailing the scope of work needed for various types of testing. The paper will provide an overview of standardized test aspects, active safety software validation methods, recommended practices and standards. The focus will be on trade-offs of cost and performance, as well as providing general guidance for the type of testing necessary for different phases of product development and implementation. Specific example applications, such as Forward Collision Warning (FCW) and Automatic Emergency Braking (AEB) functions, will be examined relative to the V&V processes.</div></div>

<div class="section abstract"><div class="htmlview paragraph">Acceptance criteria and validation target are the most important metrics used to measure/assure safety of the intended function (SOTIF) of an autonomous vehicle or advanced driver assistance system (ADAS). Often acceptance criteria are defined as acceptable number of fatalities, injuries or property damage events in certain hours of operation or for certain mileage driven. Validation target on the other hand is the amount of effort required in terms of hours of operation or mileage to be driven to show that the acceptance criteria is met. Although existing research details about potential values for acceptance criteria and validation target, they overlook various factors such as operational design domain, operational lifetime of a vehicle, average mileage of a vehicle, and length of roads. As a result, often acceptance criteria values are very small (e.g., 10<sup>-12</sup> incidents/h or mi) and validation targets are very large (e.g., 10<sup>13</sup> miles). However, these targets may not be practical to distribute within various testing avenues/platforms and provide as evidence for a given version. In this paper, we discuss the various factors and strategies we can consider for defining reasonable and practically achievable acceptance criteria and validation targets. For example, the number of miles driven by motor vehicles in United States in 2020 and the number of accidents occurred in those miles is given based on distance travelled by around 276 million vehicles. Will the distance and number of accidents remain the same even as the fleet size is 100 vehicles? In this paper, we aim to resolve such questions. Using an illustrative example, we demonstrate the usefulness of our proposed factors and strategies.</div></div>

<div class="section abstract"><div class="htmlview paragraph">The current approach for new Advanced Driver Assistance System (ADAS) and Connected and Automated Driving (CAD) function development involves a significant amount of public road testing which is inefficient due to the number miles that need to be driven for rare and extreme events to take place, thereby being very costly also, and unsafe as the rest of the road users become involuntary test subjects. A new development, evaluation and demonstration method for safe, efficient, and repeatable development, demonstration and evaluation of ADAS and CAD functions called Vehicle-in-Virtual –Environment (VVE) was recently introduced as a solution to this problem. The vehicle is operated in a large, empty, and flat area during VVE while its localization and perception sensor data is fed from the virtual environment with other traffic and rare and extreme events being generated as needed. The virtual environment can be easily configured and modified to construct different testing scenarios on demand. This paper focuses on the VVE approach and introduces the coordinate transformations needed to sync pose (location and orientation) in the virtual and physical worlds and handling of localization and perception sensor data using the highly realistic 3D simulation model of a recent autonomous shuttle deployment site in Columbus, Ohio as the virtual world. As a further example that uses multiple actors, the use of VVE for Vehicle-to-VRU communication based Vulnerable Road User (VRU) safety is presented in the paper using VVE experiments and real pedestrian(s) in a safe and repeatable manner. VVE experiments are used to demonstrate the efficacy of the method.</div></div>

<div class="section abstract"><div class="htmlview paragraph">This paper presents a cost effective development platform, named FEV-Driver, for Advanced Driver Assistance Systems (ADAS) and autonomous driving (AD). The FEV-Driver platform is an electric go-kart that was converted into an x-by-wire vehicle which represents the behavior of a full-scale electric vehicle. FEV-Driver has the advantage of being a small-scale vehicle that can be used with a significant lower safety risk compared to full-sized vehicles. The ADAS/AD algorithms for this platform were developed in both Simulink and C++ software and implemented within the Robot Operating System (ROS) middleware. Besides the description of the platform, Lane Keep Assist (LKA) and Automatic Emergency Braking (AEB) algorithms are discussed, followed by a path planning algorithm which enables the vehicle to drive autonomously after a manually controlled training lap. The modular system architecture allows for complete controller exchange or adaptation to different vehicles. The adaptation and implementation of the platform into a full-scale passenger vehicle is described in the last section of this paper. The presented platform has proven to be a low-cost scalable platform for development and verification of ADAS and AD functions.</div></div>

<div class="section abstract"><div class="htmlview paragraph">Safety is the cornerstone for Advanced Driver Assistance Systems (ADAS) and Autonomous Driving Systems (ADS). To assess the safety of a traffic situation, it is essential to predict motion states of traffic participants in the future with mathematic models. Accurate vehicle trajectory prediction is an important prerequisite for reasonable traffic situation risk assessment and appropriate decision making. Vehicle trajectory prediction methods can be generally divided into motion model based methods and maneuver model based methods. Vehicle trajectory prediction based on motion models can be accurate and reliable only in the short term. While vehicle trajectory prediction based on maneuver models present more satisfactory performance in the long term, these maneuver models rely on machine learning methods. Abundant data should be collected to train the maneuver recognition model, which increases complexity and lowers real-time performance. In this paper, a vehicle trajectory prediction method based on motion model and maneuver model fusion with Interactive Multiple Model (IMM) is proposed. Firstly, Constant Turn Rate and Acceleration (CTRA) motion model and Unscented Kalman Filter (UKF) are used to predict vehicle trajectory with uncertainty in the future. Then, vehicle trajectory prediction based on simplified maneuver recognition model is conducted, using temporal and spatial relationship between vehicle historical trajectory and lane lines. After that, vehicle trajectory prediction by integrating motion model and maneuver model with IMM is conducted. Finally, the proposed method is compared with CTRA motion model based vehicle trajectory prediction and lane keeping model (LKM) based vehicle trajectory prediction in two simulation test scenarios. The simulation results indicates that the IMM-based method achieves both excellent prediction accuracy and appropriate prediction uncertainty in the whole prediction horizon. This research can be used to support decision making for Advanced Driver Assistance Systems (ADAS) and Autonomous Driving Systems and leads to improvement of traffic safety.</div></div>