Multi-Sensor Data Fusion for Robust Environment Reconstruction in Autonomous Vehicle Applications

For the past few years, the automation of transportation becomes a hot research topic for smart cities. Intelligent Transportation System (ITS) aims to manage and optimize the traffic congestion, road accidents, parking allocation using Autonomous Vehicles (AV) system, where the AVs are internally connected for message passing and critical decision making in time-sensitive applications. The data security in such applications can be offered using Blockchain (BC) technology. But, as per the existing literature, there exists no system which can call AVs automatically based on the situation, i.e., call an ambulance in case of an accident, call logistic service in case of home transfer, and call the traffic department in case of traffic jam. Motivated from the aforementioned reasons, in this article, we propose a BC-based secure and intelligent sensing and tracking architecture for AV system using beyond 5G communication network. Recently, AVs are facing issues with sensing and tracking technology as well as the data thefts. AV system contains sensitive information and transfers it through a communication channel to Connected AVs (CAVs), where the corrupted information or delay of a fraction of a second can lead to a critical situation. So, here we present possib the attacks and safety countermeasures using BC technology to protect the AV system. The proposed architecture ensures secure sensing and tracking of an object through BC by deploying AI algorithms at the edge servers. Also, the beyond 5G network enables communications with low latency and high reliability to meet the desires of the aforementioned time-sensitive applications. The proposed system is evaluated by considering the parameters as mobility and data transfer time against the traditional LTE-A and 5G communication networks. The proposed system outperforms traditional systems and can be suitable for diverse applications where latency, reliability, and security are the prime concerns.

Tracking of multiple extended three-dimensional targets in the presence of partial occlusions from point cloud measurement sets remains a challenge but has obvious applications in autonomous vehicles. We demonstrate that existing range-image based approaches to point cloud segmentation and motion detection can be fused with fast point cloud registration techniques to aid track association. Consequently, multiple generic extended targets in point clouds can be tracked. We also show that robust occlusion handling can be achieved using temporally accumulated geometric information from co-registered point clouds in a static local reference frame. The occlusion logic is based on a novel probabilistic occlusion reasoning approach combined with a simulated geometric handling of self-occlusions. The proposed tracking pipeline was tested on synthetic target geometries superimposed on real-world backgrounds, as well as against measured data from benchmark autonomous driving datasets. The targets tracked encountered both artificially introduced and naturally occurring foreground occlusions as well as self-occlusions. Results show notable robustness to foreground occlusions. Incorrect track associations are possible when multiple targets in proximity display similar dynamics and possess nearly identical geometries. Our sequence of tracking algorithms relies on multiple tunable hyper-parameters. These parameters require further automated runtime-optimization before robust, real-time field application can be achieved.

Transitioning to a driverless city: Evaluating a hybrid system for autonomous and non-autonomous vehicles

The rapid advancement of deep learning has significantly transformed the development of next-generation autonomous vehicles, enhancing both safety and efficiency. This paper explores the integration of deep learning techniques, including convolutional neural networks (CNNs), recurrent neural networks (RNNs), and reinforcement learning, in perception, decision-making, and control systems of autonomous vehicles. By leveraging vast datasets and real-time processing, deep learning enables precise object detection, path planning, and adaptive driving strategies. Furthermore, the implementation of sensor fusion techniques combining LiDAR, radar, and cameras enhances situational awareness, reducing the risk of accidents. Despite these advancements, challenges such as computational complexity, adversarial robustness, and ethical considerations remain key research areas. This study provides an overview of the current state-of-the-art deep learning applications in autonomous vehicles and discusses future directions toward fully autonomous, safer, and more efficient transportation systems.

The application of autonomous vehicles has grown dramatically in recent years. Not only has the rail-guided vehicle (RGV) been used widely in traditional production lines, the automatic guided vehicle (AGV) has also been increasingly used. Positioning and path planning are two major functions of autonomous vehicles; however, there are many ways to fulfill the above requirements. The infrared remote control has been heavily and successfully used in home appliances for decades, which has encouraged us to apply this mature and cost-effective technology to an autonomous vehicle. By decoding the coded signal from the infrared light-emitting diode (LED), which is equipped on the ceiling, the autonomous vehicle can be positioned with an accuracy of less than 50 mm. On the other hand, by changing the beam pattern of infrared light from the ceiling, an invisible route can be produced on the ground. That is to say, instead of the traditional rail-guided method, these invisible paths can guide the autonomous vehicle. We have implemented a prototype of an autonomous vehicle system based on the above concept, with the aim of creating a simple and reliable approach for the navigation of an indoor autonomous vehicle.

Underwater imaging is being increasingly helpful for the autonomous robots to reconstruct and map the marine environments which is fundamental for searching for pipelines or wreckages in depth waters. In this context, the accuracy of the information obtained from the environment is of extremely importance. This work presents a study about the accuracy of a reconfigurable stereo vision system while determining a dense disparity estimation for underwater imaging. The idea is to explore the advantage of this kind of system for underwater autonomous vehicles (AUV) since varying parameters like the baseline and the pose of the cameras make possible to extract accurate 3D information at different distances between the AUV and the scene. Therefore, the impact of these parameters is analyzed using a metric error of the point cloud acquired by a stereoscopic system. Furthermore, results obtained directly from an underwater environment proved that a reconfigurable stereo system can have some advantages for autonomous vehicles since, in some trials, the error was reduced by 0.05m for distances between 1.125 and 2.675 m.

A training dataset for semantic segmentation of urban point cloud map for intelligent vehicles

According to the world health organization, there were 1.25 million road accident deaths globally in 2013. To decrease this count, it has been said that autonomous vehicles can play a major role. The autonomous vehicle uses different sensors such as LIDAR, RADAR, Ultrasonic sensors, and cameras for detection of lane lines, objects, traffic sign, traffic lights, and driving path. There are several successful projects on autonomous vehicles, but the main limitations are high costs and regulations. The camera can be a cost-effective solution as compare to LIDAR in all the basic detection needed for autonomous driving. The proper placement of sensors on the vehicle plays an important role in efficient design, working and reducing the cost of the overall autonomous vehicle system. This paper discusses the parameters considered while deciding the positioning of the camera, and the geometric calculation for the best positioning of a monocular camera on an autonomous vehicle.

The performance of autonomous vehicle systems is currently limited by their inability to accurately analyze their surrounding environment. In order to function in a dynamic real world environment, an autonomous vehicle system must be capable of interpreting terrain based upon predetermined mission goals. This paper describes a concept of knowledge-based terrain analysis currently being developed to support the information needs of an autonomous helicopter system. The terrain analysis system consists of five integrated processing stages. Each process is discussed in detail and supported by a number of mission oriented examples. Int roduction One of the most unique system concepts being explored today is that of the development of totally autonomous land, sea, and air vehicles [1 - 3]. For military applications, autonomous vehicles provide a mechanism for removing humans from modern day battlefields while not impacting the tactical capabilities of the battle force. Civilian applications for such a device entail the investigation of hazardous areas (e.g. nuclear reactors and toxic waste dumps), search over large areas (e.g. downed aircraft or lost campers) and monitoring of large facilities (e.g. robot security police). As military funding for these systems is currently at an alltime high, a variety of autonomous system concepts are continually being presented in the technical literature. The primary limitation of an autonomous vehicle still lies in its inability to accurately analyze surrounding terrain information in order to generate tactical plans of action. This paper describes activities being performed in the Artificial Intelligence Branch of the Georgia Tech Research Institute on knowledge-based terrain analysis, particularly in support of the autonomous helicopter program.System OverviewThe knowledge-based tactical terrain analysis system consists of five separate processing phases: [1] image segmentation, [2] region classification, [3] visual model, [4] route planning, [5] threat location and coverage. Each processing phase is briefly discussed in the following subsections with actual image results given where appropriate.

Localization and mapping are key requirements for autonomous mobile systems to perform navigation and interaction tasks. Iterative Closest Point (ICP) is widely applied for LiDAR scan-matching in the robotic community. In addition, the standard ICP algorithm only considers geometric information when iteratively searching for the nearest point. However, ICP individually cannot achieve accurate point-cloud registration performance in challenging environments such as dynamic environments and highways. Moreover, the computation of searching for the closest points is an expensive step in the ICP algorithm, which is limited to meet real-time requirements, especially when dealing with large-scale point-cloud data. In this paper, we propose a segment-based scan-matching framework for six degree-of-freedom pose estimation and mapping. The LiDAR generates a large number of ground points when scanning, but many of these points are useless and increase the burden of subsequent processing. To address this problem, we first apply an image-based ground-point extraction method to filter out noise and ground points. The point cloud after removing the ground points is then segmented into disjoint sets. After this step, a standard point-to-point ICP is applied into to calculate the six degree-of-freedom transformation between consecutive scans. Furthermore, once closed loops are detected in the environment, a 6D graph-optimization algorithm for global relaxation (6D simultaneous localization and mapping (SLAM)) is employed. Experiments based on publicly available KITTI datasets show that our method requires less runtime while at the same time achieves higher pose estimation accuracy compared with the standard ICP method and its variants.

Improving performance of deep learning models for 3D point cloud semantic segmentation via attention mechanisms

Numerous developments in technology toward autonomous vehicle systems (AVSs) have been performed for so many years all over the world. As our day‐to‐day life is becoming progressively dependent on automation vehicle system and control devices, the craze on automation advancements is expected to move closer through scientific technologies like artificial intelligence and robotics. From another point of view, the cyber threat to the AVS causes drastic accidents and traffic congestion by varying the speed differences among the vehicles. To overcome such shortcomings, this paper presented a convolutional neural network‐oppositional‐based Henry gas solubility optimization (CNN‐OHGS) algorithm for an autonomous vehicle control system to enhance the robustness of the vehicle. At the same time, the attackers attempt to embed the faulty or defective data into the sensor readings of the autonomous vehicle to interrupt the optimal distances among the automated vehicles. Therefore to minimize such issues, our proposed framework employs the CNN‐OHGS algorithm to reduce the distance variations among the vehicles thus ensuring the safety and optimal distance variation. Finally, the experimental analysis is conducted and the performance evaluation for various attacks, FID evaluation, and remorse function, and distance deviation for all sensor signal attacks are evaluated. The comparative analysis is made and we can clearly state that the proposed work has outperformed other existing approaches.

In this paper, we propose an efficient way to produce an initial transposed matrix for two point clouds, which can effectively avoid the drawback of local optimism caused by using standard Iterative Closest Points (ICP)[ algorithm when matching two point clouds. In our approach, the correspondences used to calculate the transposed matrix are confirmed before the point cloud forms. We use the depth images which have been carefully target-segmented to find the boundaries of the shapes that reflect different views of the same target object. Then each contour is affected by curvature scale space (CSS)[ method to find a sequence of characteristic points. After that, our method is applied on these characteristic points to find the most matching pairs of points. Finally, we convert the matched characteristic points to 3D points, and the correspondences are there being confirmed. We can use them to compute an initial transposed matrix to tell the computer which part of the first point cloud should be matched to the second. In this way, we put the two point clouds in a correct initial location, so that the local optimism of ICP and its variations can be excluded.

With the rapid development of computer technology and Artificial Intelligence (AI), unmanned systems have been widely applied and studied in various fields. For unmanned systems based on Reinforcement Learning (RL) principles, user satisfaction evaluation of their performance can provide accurate reward functions, which deeply influence the optimization of system control strategies. This paper takes the towing and following system of autonomous vehicles as the research object, and proposes a user satisfaction evaluation method for autonomous vehicle behavior based on statistical characteristics and data-driven approach, realizing the mapping from raw sequence data of towing and following to user satisfaction. The core innovation of this paper lies in the first attempt to use the dynamic state of the tow rope (length, angular velocity, etc.) as a key feature source, which is fused with vehicle kinematic data to evaluate user subjective experience; and a self-supervised feedback mechanism is designed to optimize model parameters, enabling the model to have certain self-learning ability. Experimental results show that this method can effectively reflect user satisfaction with autonomous vehicle driving strategies, and achieve progressive optimization of satisfaction evaluation through self-supervised feedback mechanism. This study lays a theoretical foundation for building more intelligent and human-oriented human-machine collaborative following systems, and can be embedded as a reward function in reinforcement learning frameworks for training and optimizing towing and following strategies.

The registration of point clouds in a three-dimensional space is an important task in many areas of computer vision, including robotics and autonomous driving. The purpose of registration is to find a rigid geometric transformation to align two point clouds. The registration problem can be affected by noise and partiality (two point clouds only have a partial overlap). The Iterative Closed Point (ICP) algorithm is a common method for solving the registration problem. Recently, artificial neural networks have begun to be used in the registration of point clouds. The drawback of ICP and other registration algorithms is the possible convergence to a local minimum. Thus, an important characteristic of a registration algorithm is the ability to avoid local minima. In this paper, we propose an ICP-type registration algorithm (λ-ICP) that uses a multiparameter functional (λ-functional). The proposed λ-ICP algorithm generalizes the NICP algorithm (normal ICP). The application of the λ-functional requires a consistent choice of the eigenvectors of the covariance matrix of two point clouds. The paper also proposes an algorithm for choosing the directions of eigenvectors. The performance of the proposed λ-ICP algorithm is compared with that of a standard point-to-point ICP and neural network Deep Closest Points (DCP).