Detection & Tracking of Multi-Scenic Lane Based on Segnet-LSTM Semantic Split Network

<div class="section abstract"><div class="htmlview paragraph">New challenges and complexities are continuously increasing in advanced driver assistance systems (ADAS) development (e.g. active safety, driver assistant and autonomous vehicle systems). Therefore, the health management of ADAS’ components needs special improvements. Since software contribution in ADAS’ development is increasing significantly, remote diagnosis and maintenance for ADAS become more important. Furthermore, it is highly recommended to predict the remaining useful life (RUL) for the prognosis of ADAS’ safety critical components; e.g. (Ultrasonic, Cameras, Radar, LIDAR). This paper presents a remote diagnosis, maintenance and prognosis (RDMP) framework for ADAS, which can be used during development phase and mainly after production. An overview of RDMP framework’s elements is explained to demonstrate how/when this framework is connected to database servers and remote analysis servers. Moreover, Sensors fusion is used in RDMP to detect some sensor failures and even to predict their RUL. Additionally, some well-known machine learning algorithms (MLA) are used to predict RUL of ADAS’ components, and different types of input attributes to these MLA are proposed for some basic ADAS’ components. MLA use training data set, which shall be constructed ideally from actual records reported remotely by RDMP (Prognosis Analysis and Self-Learning System). However, initial dataset before production of the vehicle can be created from ADAS laboratory tests (e.g. Assessments on test tracks), ADAS simulation and theoretical analytical methods. Also, experiments of using the proposed RDMP in some ADAS’ components (Sensor fusion and Braking system as ADAS actuator) are presented. Summary, conclusion with proven results and future work are explained.</div></div>

<div class="section abstract"><div class="htmlview paragraph">The camera-based advanced driver assistance systems (ADAS) like lane departure warning system (LDWS) and lane keeping assist (LKA) can make vehicles safer and driving easier. Lane detection is indispensable for these lane-based systems for achieving vehicle local localization and behavior prediction. Since the vision is vulnerable to the variable environment conditions such as bad weather, occlusions and illumination, the robustness is important. In this paper, a robust algorithm for detecting and tracking multiple lanes with arbitrary shape is proposed. We extend the previously lane detection and tracking process from the space domain to the temporal-spatial domain by using a more robust and general multi-lane model. First, new slice images containing temporal information are generated from image sequences. Instead of binarization process, we use a more general detector for extracting the lane marker candidates with prior knowledge to generate the binary slice image. Then, all the lane marker candidates are clustered into many lanes and as the initialization of the following particle filtering tracking process. We calculate the distance map in the binary slice image by using a modified distance algorithm and sample particles in the distance map. Finally, we can track the lane markers with particle filtering successfully. A range of experiment results indicate that the proposed algorithm can detect and track multiple lanes correctly and robustly. Comparing with other method, it has enough tolerance to variant illumination and occlusion.</div></div>

<div class="section abstract"><div class="htmlview paragraph">Lane detection plays a critical role in autonomous vehicles for safe and reliable navigation. Lane detection is traditionally accomplished using a camera sensor and computer vision processing. The downside of this traditional technique is that it can be computationally intensive when high quality images at a fast frame rate are used and has reliability issues from occlusion such as, glare, shadows, active road construction, and more. This study addresses these issues by exploring alternative methods for lane detection in specific scenarios caused from road construction-induced lane shift and sun glare. Specifically, a U-Net, a convolutional network used for image segmentation, camera-based lane detection method is compared with a radar-based approach using a new type of sensor previously unused in the autonomous vehicle space: radar retro-reflectors. This evaluation is performed using ground truth data, obtained by measuring the lane positions and transforming them into pixel coordinates. The performance of each method is assessed using the statistical <i>R</i><sup>2</sup> score, indicating the correlation between the detected lane lines and the ground truth. The results show that the U-Net camera-based method exhibits limitations in accurately detecting and aligning the lane lines, particularly in challenging scenarios. However, the radar-based lane detection method demonstrates a strong correlation with the ground truth which implies that the use of this sensor may improve current reliability issues from conventional camera lane detection approach. Furthermore, the study highlights the limitations of the U-Net model for camera lane detection, especially in scenarios with sun glare. This study shows that infrastructure-based radar retro-reflectors can improve autonomous vehicle lane detection reliability. The integration of different sensor modalities and the development of advanced computer vision algorithms are crucial for improving the accuracy, reliability, and energy efficiency of lane detection systems. Addressing these challenges contributes to the advancement of autonomous vehicles and the realization of safer and more efficient transportation systems.</div></div>

<div class="section abstract"><div class="htmlview paragraph">Now-a-days, Advanced driver-assistance systems (ADAS) is equipping cars and drivers with advance information and technology to make them become aware of the environment and handle potential situations in better way semi-autonomously. High-quality training and test data is essential in the development and validation of ADAS systems which lay the foundation for autonomous driving technology.</div><div class="htmlview paragraph">ADAS uses the technology like radar, vision and combinations of various sensors including LIDAR to automatize dynamic driving tasks like steering, braking, and acceleration of vehicle for controlled and safe driving. And to integrate these advance technologies, the ADAS needs labeled data to train the algorithm to detect the various objects and moments of driver. Image annotation is one the well-known service to create such training data for computer vision.</div><div class="htmlview paragraph">There are number of open source annotated datasets available viz. COCO, KITTI etc. But these datasets are limited to either US or European road environment scenarios. There is hardly any dataset available for Indian Specific road conditions. In order to capture all the India Specific objects on road, we are capturing data through various environmental conditions, road conditions and annotating objects on the collected camera images. 2-D bounding boxes are being used to annotate objects in an image.</div><div class="htmlview paragraph">These annotated images will then be used to train a machine learning model. Finally this trained model will be then used to detect Indian specific objects with real time data acquired through camera sensors mounted on the vehicle. Object detection is one of the prime aspects for ADAS functionality. The complete process will demonstrate end to end solution for object detection techniques from data generation to real-time object identification and recognition.</div></div>

<div class="htmlview paragraph">Digital maps have large potential to enhance or enable Advanced Driver Assistance Systems (ADAS) by extending driver horizon and therefore to contribute to safer, smarter and cleaner road transport. As a predictive sensor called ADAS Horizon, in-vehicle digital maps are an important source of information providing look-ahead capability for ADAS applications and providing further information for on-board sensors to enhance environment perception.</div> <div class="htmlview paragraph">This paper reports on results from the European Commission project MAPS&ADAS, part of PReVENT Integrated Project, dedicated to the development, test and validation of a standardized interface (ADAS Interface) between ADAS applications and ADAS map data sources for accessing map data regarding vehicle position.</div>

A robust and real-time lane detection method in low-light scenarios to advanced driver assistance systems

<div class="section abstract"><div class="htmlview paragraph">As a result of the development of Event Data Recorders (EDR) and the recent FMVSS regulation 49 CFR 563, today’s automobiles provide a limited subset of electronic data measurements of a vehicle’s state before and during a crash. Prior to this data, the only information available about the vehicle movements before or during a collision had come from physical evidence (e.g. tire marks), witnesses, aftermarket camera systems on vehicles, and ground-based cameras that were monitoring vehicle traffic or used for security surveillance. Today’s vehicles equipped with Advanced Driver Assistance Systems (ADAS) have vehicle-based sensors that measure information about the environment around a vehicle including other vehicles, pedestrians, and fixed wayside objects. Vehicles equipped with these ADAS systems use primarily radar, lidar, ultrasonic, and/or image sensors either in standalone operation or in combination to establish the range and movement of potential hazardous objects (e.g. other vehicles, poles, and pedestrians) around a vehicle. The data from these sensors is unique because it measures objective environment information surrounding a vehicle, which can play a role in reconstructing and understanding the contributing factors of an accident. As more vehicles become equipped with advanced safety systems and the requisite sensors for automated vehicles, accident investigators will need to become familiar with system functionality, the data that the sensors measure and may record, as well as the limitations of that data in order to effectively and accurately analyze a crash.</div><div class="htmlview paragraph">This paper reviews the sensors in ADAS collision avoidance systems that may be present on current vehicles and how the data from a vehicle based radar may be used to reconstruct a collision. A radar sensor mounted on a host vehicle is experimentally evaluated using stationary objects, and a stationary and moving radar. The results demonstrate how an investigator would use the radar data to recognize a fixed object in the environment and analyze the relative position with respect to the host vehicle. The analysis of this radar data includes the limitations, accuracy, and validation of the particular radar sensor data. In addition, the authors propose in general how data from radars could be used to elucidate the location of fixed objects in the environment surrounding a vehicle for the purpose of understanding an accident.</div></div>

In recent years, many companies are working on the development of autonomous cars. Every citizen on the planet is concerned about their safety. In the framework of Advanced Driver Assistance Systems, one of the main goals is to improve safety and reduce road accidents, ultimately saving lives. One of the most difficult jobs in an autonomous driving system is detecting road lanes or road boundaries. Lane and object detection may be a crucial component of collision prevention in driving assistance systems. With the growth in traffic, there is a demand for more security and comfort when driving, it impose the development of new technology. Computer vision is one of the ways that may be utilized to assist the driver in difficult scenarios in order to improve his safety and comfort. The initial layer of autonomous vehicles' capabilities is lane tracking. Many sensors, including as lasers, radar, and vision sensors, are commonly employed for obstacle and lane detection. Computer vision is one of the primary method for detecting road limits and lanes using a vehicle's vision system. The technology uses a camera installed on the vehicle to capture the front view, then uses a few algorithms to detect lanes and objects. The lanes and objects are detected using a flexible algorithm. This paper focuses on a computer vision-based object detection technique by using CNN algorithm.

<div class="section abstract"><div class="htmlview paragraph">Advanced driver assistance systems (ADAS) and automated driving systems (ADS) continue to expand into the market at a rapid pace. As improved (i.e., next generation) versions of these systems become available, they will continue to face many challenges in their implementation and benefits for safety and driving operations. The solution will involve many parties, including road safety professionals and researchers who see the potential in these systems but may have difficulties keeping up with them, and safety advocates who are calling for these systems to achieve higher levels of safety now.</div><div class="htmlview paragraph"><b>The Challenges of Next-gen ADAS and ADS and Related Vehicle Safety Topics</b> explores these challenges that will fall on the National Highway Traffic Safety Administration (NHTSA) and automakers as they balance costs and benefits; establish reasonable regulations and standards; and determine how to improve, test, deliver, and use these systems successfully. Perhaps the most formidable challenge will be overcoming the expectation that ADAS—and especially ADS—will always work perfectly in every scenario.</div><div class="htmlview paragraph"><a href="https://www.sae.org/publications/edge-research-reports" target="_blank">Click here to access the full SAE EDGE</a><sup>TM</sup><a href="https://www.sae.org/publications/edge-research-reports" target="_blank"> Research Report portfolio.</a></div></div>

<div class="section abstract"><div class="htmlview paragraph">Automotive industry is inclined towards connected, comfortable, environment friendly, efficient and smarter systems. Advanced Driver Assist System (ADAS) technology assist drivers to achieve a safer as well as better ride by automation and improvisation of the vehicular systems. With the advent of ADAS system, there is a significant focus not only in the development of Electronic Control Units (ECUs) and its features to cater to the emerging market but also on the information that could be displayed to meet the functional as well as safety requirements. This ADAS information display ensures timely notification to the driver with unique alerts that can be acoustic or visual. These systems should be tested thoroughly to ensure reliability as failures may impose severe risk on the OEM. Hardware in the loop testing has been largely adopted by industry against manual testing in lieu of the testing constraints imposed by the latter. This paper discusses on the existing test methods for ADAS information display system testing using HIL. It also details on the automation trends adopted and highlights the key challenges associated with the development as well as reusability of the conventional test methods. It discusses an Artificial Intelligence based strategy for automation of cluster objects teaching, evaluation and illustrates the adoption of the same for ADAS HIL testing and brings out the efficiency and benefits achieved by the proposed strategy.</div></div>

<div class="section abstract"><div class="htmlview paragraph">The testing of vision sensor-based Advanced Driver Assistance (ADA) systems and Electronic Control Units (ECUs) is quickly becoming a major requirement in the automotive industry. ADA ECU development teams require a verification and validation (V&V) infrastructure early in the development cycle, and traditional approaches are not suitable for this purpose. This paper proposes the design of an integrated test suite for the functional and real-time testing of ADA algorithms and ECUs. The proposed test suite consists of a hardware bridge built using a Field Programmable Gate Array (FPGA), a Universal Serial Bus (USB) 3.0 interface, and a software application running on the Personal Computer (PC). By using an FPGA to build the hardware bridge, we get a compact, real-time, and reconfigurable platform to support the desired test requirements. The USB3.0 interface is specifically proposed to support the high data rates evident in ADA systems based on multiple vision sensor and sensor fusion technologies. The PC-based software application has a modular, extensible, and scalable architecture to support current and future test requirements, as well as several features that improve the end user's productivity. The proposed test suite can be used for: (a) acquisition and storage of sensor data (recording mode), (b) visualization of stored sensor data (replay mode), (c) offline evaluation and improvement of ADA algorithms (functional test mode), (d) online testing of advanced driver assistance system (ADAS) ECUs (real-time test mode). The major contribution of this paper is the design proposal for supporting the real-time test mode for laboratory testing of ADA ECUs, and it concludes with a qualitative discussion of potential enhancements.</div></div>

The advanced driver assistance system (ADAS) for adaptive cruise control and collision avoidance is strongly dependent upon the robust image recognition technology such as lane detection, vehicle/pedestrian detection, and traffic sign recognition. However, the conventional ADAS cannot realize more advanced collision evasion in real environments due to the absence of intelligent vehicle/pedestrian behavior analysis. Moreover, accurate distance estimation is essential in ADAS applications and semiglobal matching (SGM) is most widely adopted for high accuracy, but its system-on-chip (SoC) implementation is difficult due to the massive external memory bandwidth. In this paper, an ADAS SoC with behavior analysis with Artificial Intelligence functions and hardware implementation of SGM is proposed. The proposed SoC has dual-mode operations of highperformance operation for intelligent ADAS with real-time SGM in D-Mode (d-mode) and ultralow-power operation for black box system in parking-mode. It features: 1) task-level pipelined SGM processor to reduce external memory bandwidth by 85.8%; 2) region-of-interest generation processor to reduce 86.2% of computation; 3) mixed-mode intention prediction engine for dualmode intelligence; and 4) dynamic voltage and frequency scaling control to save 36.2% of power in d-mode. The proposed ADAS processor achieves 862 GOPS/W energy efficiency and 31.4GOPS/ mm(2) area efficiency, which are 1.53x and 1.75x improvements than the state of the art, with 30 frames/s throughput under 720p stereo inputs.

An advanced driver-assistance system (ADAS), based on lane detection technology, detects dangerous situations through various sensors and either warns the driver or takes over direct control of the vehicle. At present, cameras are commonly used for lane detection; however, their performance varies widely depending on the lighting conditions. Consequently, many studies have focused on using radar for lane detection. However, when using radar, it is difficult to distinguish between the plain road surface and painted lane markers, necessitating the use of radar reflectors for guidance. Previous studies have used long-range radars which may receive interference signals from various objects, including other vehicles, pedestrians, and buildings, thereby hampering lane detection. Therefore, we propose a lane detection method that uses an impulse radio ultra-wideband radar with high-range resolution and metal lane markers installed at regular intervals on the road. Lane detection and departure is realized upon using the periodically reflected signals as well as vehicle speed data as inputs. For verification, a field test was conducted by attaching radar to a vehicle and installing metal lane markers on the road. Experimental scenarios were established by varying the position and movement of the vehicle, and it was demonstrated that the proposed method enables lane detection based on the data measured.

There have been many studies that developed approaches for real-time lane and vehicle detection. However, the majority of implementations that have been made with these approaches are focused on cars. On this paper, the existing approach of object detection is implemented for motorcycle detection. Lane detection is also incorporated for the validation of object's position. We employ Haar cascade classifier approach for motorcycle detection and edge detection approach for lane detection. In order to refine the detection result, angle-based elimination is applied to lines in lane detection. For the refinement of motorcycle detection, object candidates are eliminated by object's position on lane, frequency of occurrence, and variation of horizontal edges. Experimental results show that these approaches followed by the refinement method results provide better precision of motorcycle detection. This implementation is also able to operate in real-time.

<div class="section abstract"><div class="htmlview paragraph">In order to improve drivers’ acceptance to advanced driver assistance systems (ADAS) with better adaptation, drivers’ driving behavior should play key role in the design of control strategy. Adaptive cruise control systems (ACC) have many factors that can be influenced by different driving behavior. It is important to recognize drivers’ driving behavior and take human-like parameters to the adaptive cruise control systems to assist different drivers effectively via their driving characteristics.</div><div class="htmlview paragraph">The paper proposed a method to recognize drivers’ behavior and intention based on Gaussian Mixture Model. By means of a fuzzy PID control method, a personalized ACC control strategy was designed for different kinds of drivers to improve the adaptabilities of the systems.</div><div class="htmlview paragraph">Several typical testing scenarios of longitudinal case were created with a host vehicle and a traffic vehicle. Some high precision sensors were mounted in both test vehicles, such as RT 3002<sup>®</sup>, RT range<sup>®</sup>, radar and pedal pressure sensor. The traffic vehicle was set in different driving modes, include cruise mode, decelerating slowly, accelerating slowly, hard braking, hard acceleration, cutting out the lane, cutting into the line and sinusoidal movement. The host vehicle was operated behind the traffic vehicle by 84 drivers selected randomly. The drivers’ driving styles were evaluated via both subjective and objective indicators. The subjective indicators were based on the questionnaires by interviewing the drivers and the passengers. The objective indicators were based on the parameters extracted from the specific test conditions. The vehicle-following data were collected and analyzed via MATLAB<sup>®</sup> which was further trained by Gaussian Mixture Model. With regarding to the multi-mode switching strategy, the parameters of different driver’s characteristics were considered in the fuzzy PID control method to improve the adaptivity and accuracy in the switching process. Meanwhile, different fuzzy rules and fuzzy membership functions were applied for every type of drivers to calculate the best parameters for the personalized ACC control strategy. The control modes include cruise, steady following, approaching and hard braking. The different driving characteristics were also added into every control strategy via road experimental data and analyzed results from Gaussian Mixture Model. Compared with the existing ACC, the system proposed in this paper has great advantages on not only the adaptivity and human acceptance, but also the performance of the system itself which was verified on the simulation platform PanoSim<sup>®</sup>.</div></div>