Switching Trackers for Effective Sensor Fusion in Advanced Driver Assistance Systems

Made available in DSpace on 2020-12-12T02:40:27Z (GMT). No. of bitstreams: 0\n Previous issue date: 2020-03-01

<div class="section abstract"><div class="htmlview paragraph">New challenges and complexities are continuously increasing in advanced driver assistance systems (ADAS) development (e.g. active safety, driver assistant and autonomous vehicle systems). Therefore, the health management of ADAS’ components needs special improvements. Since software contribution in ADAS’ development is increasing significantly, remote diagnosis and maintenance for ADAS become more important. Furthermore, it is highly recommended to predict the remaining useful life (RUL) for the prognosis of ADAS’ safety critical components; e.g. (Ultrasonic, Cameras, Radar, LIDAR). This paper presents a remote diagnosis, maintenance and prognosis (RDMP) framework for ADAS, which can be used during development phase and mainly after production. An overview of RDMP framework’s elements is explained to demonstrate how/when this framework is connected to database servers and remote analysis servers. Moreover, Sensors fusion is used in RDMP to detect some sensor failures and even to predict their RUL. Additionally, some well-known machine learning algorithms (MLA) are used to predict RUL of ADAS’ components, and different types of input attributes to these MLA are proposed for some basic ADAS’ components. MLA use training data set, which shall be constructed ideally from actual records reported remotely by RDMP (Prognosis Analysis and Self-Learning System). However, initial dataset before production of the vehicle can be created from ADAS laboratory tests (e.g. Assessments on test tracks), ADAS simulation and theoretical analytical methods. Also, experiments of using the proposed RDMP in some ADAS’ components (Sensor fusion and Braking system as ADAS actuator) are presented. Summary, conclusion with proven results and future work are explained.</div></div>

Worldwide, traffic safety is a strong concern as traffic accidents are one of the leading causes of death. In this context, advanced driver assistance systems (ADAS) and autonomous vehicles are traffic management measures aimed at improving road safety and flow. Automatic detection and recognition of traffic signs are important for intelligent vehicles and ADAS systems. This work proposes a pipeline of digital image processing (DIP) techniques, machine learning (ML), and temporal coherence to perform the detection and recognition of Brazilian traffic signs in videos aiming an application for real-time systems to help in traffic safety and to reduce the number of fatal accidents. We are mainly interested in recognizing signs of speed limit group, no overtaking and obligatory passage, thus our detection considers the traffic sign with a circular shape and red border. For detection, the red color segmentation and the Hough transform are used to find circular regions that will be classified through the SVM algorithm in sign and not sign. For recognition of these signs, the support vector machines (SVM) are used. For speed limit signs the thresholding and contours are used to segment the digits for later classification. Our proposed method achieved an accuracy of 0.82 in detection, an increase of 18% in the number of recognized frames and 0.96 in the recognition stage using temporal coherence.

Advanced driver assistance systems (ADASs) and autonomous vehicles are expected to increase safety, lower energy and fuel consumption, and lower pollutants from road traffic. The advanced driver assistance system’s major features include lane detection and tracking. Finding color line marks on the road is the technique of lane detection. The process of lane tracking aims to help the vehicle continue traveling along a predetermined course. Hence, automatic detection of lanes using convolutional neural networks (CNNs) models has gained popularity in the current economic development. This paper also aims at detecting various objects using convolutional neural networks. In this work, we offer object classification and detection, a demanding topic in computer vision and image processing. As a result, we deployed convolutional neural networks on the Keras platform with TensorFlow support. The experimental results illustrate the amount of time needed to train, test, and generate the model using a constrained computing environment. Here, we trained the system for about 80 images which have taken a couple of seconds to detect with better accuracy. Traffic sign recognition is carried out, which is a significant area of research in ADAS. It is crucial for driverless vehicles and is frequently used to read stationary or moving road signs along the side of the road. A comprehensive recognition system is made up of traffic sign detection (TSD) and categorization (TSC). The paper also aimed at traffic sign recognition, which is crucial to consider, because traffic sign recognition is typically applied to portable devices. The model’s detection accuracy is ensured as long as the speed is maintained. The model developed in this study is 99.89% accurate.

Modern automobiles are increasingly equipped with Advanced Driver Assistance Systems (ADAS), which improve passenger safety and comfort. Each of these ADAS can rely on a multitude of environmental sensors, which creates the need of cross-linking ADAS with respect to their access to environment information. This idea is reflected in the so-called environment model (EM) that centralizes the access of all ADAS functions to the perceived but not yet interpreted scene in the vehicle environment. In this paper, we propose an architectural framework that concretizes the EM in terms of separate modules for signal processing and sensor data fusion, as well as well-defined generic input and output interfaces. Moreover, we extend the EM by including additional components and output interfaces for centralized scene interpretation (CSI). The architectural framework allows streamlining the development process and increasing computational efficiency, as software modules for scene interpretation can be shared among multiple ADAS. In addition, the consistency of the overall ADAS behavior is improved, as all ADAS functions rely on the same scene information and scene interpretation. We illustrate our concept of centralized scene interpretation using the case study of a module for the detection of directional roadways, a generalization of motorways. The algorithm we present analyzes multiple perceptual indicators in favor or against a directional roadway and computes an overall decision, along with a foresight value for the expected duration of the directional roadway, to be used in various ADAS functions.

The advanced driver assistance system (ADAS) for adaptive cruise control and collision avoidance is strongly dependent upon the robust image recognition technology such as lane detection, vehicle/pedestrian detection, and traffic sign recognition. However, the conventional ADAS cannot realize more advanced collision evasion in real environments due to the absence of intelligent vehicle/pedestrian behavior analysis. Moreover, accurate distance estimation is essential in ADAS applications and semiglobal matching (SGM) is most widely adopted for high accuracy, but its system-on-chip (SoC) implementation is difficult due to the massive external memory bandwidth. In this paper, an ADAS SoC with behavior analysis with Artificial Intelligence functions and hardware implementation of SGM is proposed. The proposed SoC has dual-mode operations of highperformance operation for intelligent ADAS with real-time SGM in D-Mode (d-mode) and ultralow-power operation for black box system in parking-mode. It features: 1) task-level pipelined SGM processor to reduce external memory bandwidth by 85.8%; 2) region-of-interest generation processor to reduce 86.2% of computation; 3) mixed-mode intention prediction engine for dualmode intelligence; and 4) dynamic voltage and frequency scaling control to save 36.2% of power in d-mode. The proposed ADAS processor achieves 862 GOPS/W energy efficiency and 31.4GOPS/ mm(2) area efficiency, which are 1.53x and 1.75x improvements than the state of the art, with 30 frames/s throughput under 720p stereo inputs.

Cameras are an essential component of many Advanced Driver Assistance Systems (ADAS) such as Lane Departure Warning, Lane Keeping Assistance, Automatic Emergency Braking, Traffic Sign Recognition and Night Vision. With the ongoing evolution of ADAS towards Automated Driving, the amount of cameras installed in a vehicle is expected to increase. At the same time, the performance of such cameras needs to match more stringent requirements. Nowadays automotive cameras are typically sensitive beyond the visual range (i.e. into the near infrared range). They may also use up to 20 bits precision for their internal signal. In order to support the development process of ADAS and Automated Driving Systems by means of simulation, it is required to have detailed physically correct simulation models of these automotive cameras. This paper describes the need for physics-based simulation and introduces a new spectral HDR camera model in the ADAS simulation platform PreScan. The traditional RGB-based camera implementation created images limited to the visual spectrum. In contrast, the new PreScan spectral camera model allows for high color depth, broad spectrum and physically correct image generation. The model provides a real-time simulation of light in the 320-1500nm spectral range which traverses through the various media and lenses before the sensor produces the final image signal. Other important features of this new spectral HDR camera model are a depth resolution of 20 bit per waveband as well as an increased dynamic range.

There are many ideas to enhance the safety riding. Advanced Driver Assistance System (ADAS) is a system to help the driver more safety. ADAS has a purpose to assist and direct the driver in order to improve traffic safety. Traffic sign recognition is one important part of ADAS. Traffic sign is a warning sign which placed at the side or above the road to provide detail road information to the driver. In this study, we propose an unsupervised approach for traffic sign recognition based on bag-of-visual-word model. Unsupervised approach does not require label data and training process for traffic sign recognition. It helps when we have many data without label. Our experiment was conducted with dataset from German Traffic Sign Recognition Benchmark (GTSRB). It is a public dataset for traffic sign recognition. The results show that, large number of visual word enable the proposed method produce high accuracy and good quality cluster. Although the accuracy in this study is still very low. The reason for that is each image only produces 3–4 number of keypoints. Small size of the images affect the small number of keypoints.

The transportation system has become a fascinating and active research topic due to its multiple problems; most prior research has focused on traffic forecasting, the advanced driver assistant system (ADAS), and self-driving vehicles. Traffic sign recognition (TSR) is an essential sub-system in ADAS that helps a driver better understand the surrounding environment (obstacles, frost, pedestrians). Automatic recognition of traffic signs is a real-world computer vision challenge and pattern recognition problem. Recently, deep architecture neural networks have shown robust solutions in many areas (health care, agriculture, transportation) due to their ability to handle large amounts of data and excel in complex systems. Therefore, a convolutional neural network (CNN) has been adopted, one of the best deep learning approaches in pattern recognition and image classification for TSR. The proposed architecture has been trained and tested on the German traffic sign recognition benchmark dataset (GTSRB). The results reported accuracy of 99.43% that outperformed human accuracy.KeywordsTraffic sign recognition (TSR)Advanced driver assistant system (ADAS)Deep learningConvolutional neural network (CNN)

Locating roadworks sites via detecting change in lateral positions of traffic signs measured relative to the ego-car

Traffic sign recognition is a important component of environmental perception technology, with significant potential in the field of autonomous vehicles. Traditional recognition methods are susceptible to factors such as lighting conditions, extreme weather, and distance, leading to mismatches. To tackle this problem, this study proposed an improved YOLOv7-based algorithm for traffic sign recognition, aimed at enhancing the accuracy of small traffic sign recognition. Firstly, an ACmix attention mechanism module is inserted after the SPPCSPC layer in the backbone of the YOLOv7 network to enhance feature extraction capability and adaptability to complex scenarios. Secondly, the SIoU loss function is employed to replace the original YOLOv7 bounding box loss function, thus improving the precision of bounding box regression and localization. Extensive experiments are conducted to validate the effectiveness of the proposed algorithm. The improved algorithm is tested and evaluated using the TT100K dataset, achieving a recognition accuracy of 94.23%, a recall rate of 78.1%, [email protected] of 86.5%, and [email protected]:0.95 of 65.6%, surpassing the original YOLOv7 network. Moreover, the frame rate remains stable at 82 frames per second. The evaluations on traffic sign datasets demonstrate the accuracy and effectiveness of the proposed method.

Advanced Driver Assistance Systems (ADAS) have emerged as a critical technology in enhancing vehicle safety, providing convenience to drivers, and facilitating the transition toward fully autonomous vehicles. The core of these systems relies heavily on computer vision (CV) technologies, which enable vehicles to interpret and understand their surrounding environment in real time. The integration of computer vision in ADAS allows vehicles to recognize and process various objects such as pedestrians, vehicles, traffic signs, and lane markings, significantly improving both safety and the driving experience. In this paper, we explore the pivotal role of computer vision in the development and implementation of ADAS in autonomous vehicles. We begin by reviewing the evolution of ADAS, detailing the different sub-systems such as lane departure warning, automatic emergency braking, adaptive cruise control, and parking assistance, all of which depend on sophisticated image processing techniques. Furthermore, the paper examines the intersection of computer vision with other emerging technologies such as machine learning, deep learning, and sensor fusion. We discuss how these innovations enable ADAS to function more effectively in complex, dynamic environments. The methodology section focuses on cutting-edge algorithms used for object detection, semantic segmentation, and real-time image processing, which are integral to the functionality of modern ADAS. Additionally, we evaluate the challenges that computer vision systems face in real-world applications, including issues with sensor calibration, environmental conditions, computational limitations, and the integration of data from multiple sensors (e.g., cameras, LiDAR, radar). The paper also presents an analysis of the results obtained from various real-world implementations of ADAS technologies, highlighting improvements in driver safety and the reduction of traffic-related accidents. We explore the potential of computer vision in enabling fully autonomous driving by addressing the current gaps in technology and the advancements needed to close these gaps. Finally, the paper concludes with a discussion on the future of ADAS and the role of computer vision in realizing fully autonomous, reliable, and efficient transportation systems. We argue that continued research and development in computer vision, combined with innovations in deep learning and sensor technologies, are critical to advancing the capabilities and reliability of autonomous vehicles in the near future.

The three-dimensional stochastic vehicle–bridge coupled vibration model is developed in this study to systematically investigate the combined effects of wind, wave, and stochastic traffic. The stochastic traffic flow is simulated using a mass-spring-damper system, including the five degrees of freedom and the 12[Formula: see text]degrees of freedom. The wind speed data at key nodes of the bridge tower are obtained through on-site measurements to ensure realistic wind loading conditions. The wave action is applied to the bridge substructure in the form of pulsating forces. The vibration response of the bridge is solved by forming coupled vibration equations, where wind, wave and traffic actions are simultaneously introduced into the bridge vibration equations. The dynamic response characteristics of the bridge are analyzed under varying traffic density, wind speed, and wave height to assess their relative influences. The results demonstrate that the bridge’s dynamic response is positively correlated with traffic density, wind speed, and wave height, with traffic density having the most significant impact on bridge vibration. Specifically, under dense traffic conditions, the maximum vertical displacement in the third span of the bridge is found to be 2.47[Formula: see text]times that under sparse traffic conditions. Similarly, the maximum transverse displacement is 1.98[Formula: see text]times that under sparse traffic conditions.

Deep learning has developed rapidly and made unprecedented achievements especially in the field of image cognition, since Alex et al. proposed AlexNet in 2012. This paper focuses on the mainstream research methods of traffic sign detection and recognition, including the traditional feature-based image processing method and the target detection method based on deep learning. The traffic sign detection and recognition method based on deep learning is divided into two categories for discussion and analysis, namely, the Anchor based and Anchor-Free neural network architectures. Finally, the paper makes a brief summary and prospects the future development.

Traffic sign recognition plays an important role in autonomous vehicles as well as advanced driver assistance systems. Although various methods have been developed, it is still difficult for the state-of-the-art algorithms to obtain high recognition precision with low computational costs. In this paper, based on the investigation on the influence that color spaces have on the representation learning of convolutional neural network, a novel traffic sign recognition approach called DP-KELM is proposed by using a kernel-based extreme learning machine (KELM) classifier with deep perceptual features. Unlike the previous approaches, the representation learning process in DP-KELM is implemented in the perceptual Lab color space. Based on the learned deep perceptual feature, a kernel-based ELM classifier is trained with high computational efficiency and generalization performance. Through the experiments on the German traffic sign recognition benchmark, the proposed method is demonstrated to have higher precision than most of the state-of-the-art approaches. In particular, when compared with the hinge loss stochastic gradient descent method which has the highest precision, the proposed method can achieve a comparable recognition rate with significantly fewer computational costs.