Probabilistic Semantic Mapping for Autonomous Driving in Urban Environments.

A high-definition (HD) map is becoming an integral component of future mobility systems such as autonomous and connected vehicles. Advances in computing systems, LiDAR technologies, and vehicle communication technologies have enabled the HD map to directly treat a point cloud map (PCM), modeling road environments as LiDAR signal-level data. However, if actual road environments are changed, the PCM, modeling the environments before changes, can not be used for vehicle applications. Accordingly, the PCM has to stay up-to-date states by reflecting the environment changes continuously. This paper presents a crowd-sourcing framework to update the PCM from environment changes continuously using LiDAR and vehicle communication. Multiple intelligent vehicles installed with the LiDAR sensors download the PCM from a map server via wireless vehicle communication. To minimize the effects of environment changes, a robust localization based on a hierarchical Simultaneous Localization and Mapping (SLAM) estimates the pose (position and direction). The estimated pose is used to detect the differences between the PCM and environments, which are defined as map changes. The map changes are detected by the probabilistic and evidential theory considering the LiDAR characteristics, such as beam divergence and multi-echo. The detected map changes are uploaded to the map cloud server and merged into the PCM. The proposed crowd-sourcing framework to keep the PCM up-to-date is verified and evaluated via simulations and experiments in sites with road environment changes.

CAOM: Change-aware online 3D mapping with heterogeneous multi-beam and push-broom LiDAR point clouds

A training dataset for semantic segmentation of urban point cloud map for intelligent vehicles

Review and challenge: High definition map technology for intelligent connected vehicle

Navigation system for orchard spraying robot based on 3D LiDAR SLAM with NDT_ICP point cloud registration

Autonomous vehicles are the future intelligent vehicles, which are expected to reduce the number of human drivers, improve efficiency, avoid collisions, and become the ideal city vehicles of the future. To achieve this goal, vehicle manufacturers have started to work in this field to harness the potential and solve current challenges to achieve the desired results. In this sense, the first challenge is transforming conventional vehicles into autonomous ones that meet users’ expectations. The evolution of conventional vehicles into autonomous vehicles includes the adoption and improvement of different technologies and computer algorithms. The essential task affecting the autonomous vehicle’s performance is its localization, apart from perception, path planning, and control, and the accuracy and efficiency of localization play a crucial role in autonomous driving. In this paper, we describe the implementation of map-based localization using point cloud matching for autonomous vehicles. The Robot Operating System (ROS) along with Autoware, which is an open-source software platform for autonomous vehicles, are utilized for the implementation of the vehicle localization system presented in this paper. Point cloud maps are generated based on 3D lidar points, and a normal distributions transform (NDT) matching algorithm is used for localizing the test vehicle through matching real-time lidar measurements with the pre-built point cloud maps. The experiment results show that the map-based localization system using 3D lidar scans enables real-time localization performance that is sufficiently accurate and efficient for autonomous driving in a campus environment. The paper comprises the methods used for point cloud map generation and vehicle localization as well as the step-by-step procedure for the implementation with a ROS-based system for the purpose of autonomous driving.

Smart cities are emerging and under development worldwide. As one of the core components for achieving the goal of smart transportation, autonomous vehicles are undoubtedly on the blueprint for smart cities. The development of autonomous driving cars leads to the demand for highly accurate and detailed machine-readable maps. To meet this demand, the high-definition (HD) maps are being developed. The HD maps are extended three-dimensional maps that tend to contain enough detailed information in the driving environment to be consumed by machines. Both commercial companies and research communities got a recent interest in building and are exploring research problems on this kind of mapping data. Several companies have already defined their commercial HD map formats, but there are arguments that the existing HD map formats are not research-friendly enough. In this paper, we propose to define a research-friendly HD map data model by extending the popular node-edge model that is widely used by researchers. Our model considers both on-road and off-road data to be extensible for future use cases and information that will be available on various three-dimensional objects in the outdoor space. This is partially motivated by several use cases that will benefit from HD mapping data other than the self-driving technology. We give a brief discussion for such potential use cases and how our model will fit their requirements. We also discuss that our model can be still used for human-machine interactions for compatibility with existing techniques and applications.

High-Definition (HD) maps are needed for robust navigation of autonomous vehicles, limited by the on-board storage capacity. To solve this, we propose a novel framework, Environment-Aware Normal Distributions Transform (EA-NDT), that significantly improves compression of standard NDT map representation. The compressed representation of EA-NDT is based on semantic-aided clustering of point clouds resulting in more optimal cells compared to grid cells of standard NDT. To evaluate EA-NDT, we present an open-source implementation that extracts planar and cylindrical primitive features from a point cloud and further divides them into smaller cells to represent the data as an EA-NDT HD map. We collected an open suburban environment dataset and evaluated EA-NDT HD map representation against the standard NDT representation. Compared to the standard NDT, EA-NDT achieved consistently at least 1.5x higher map compression while maintaining the same descriptive capability. Moreover, we showed that EA-NDT is capable of producing maps with significantly higher descriptivity score when using the same number of cells than the standard NDT.

High Definition (HD) maps are necessary for many applications of automated driving (AD), but their manual creation and maintenance is very costly. Vehicle fleet data from series production vehicles can be used to automatically generate HD maps, but the data is often incomplete and noisy. We propose a system for the generation of HD maps from vehicle fleet data, which is tolerant to missing or misclassified detections and can handle drives with multiple routes, generating a single complete map, model-free and without prior reference lines. Using randomly selected drives as pivot drives, a step-wise lateral sampling of detections is performed. These sampled points are then clustered and aligned using Expectation Maximization (EM), estimating a lateral offset for each drive to compensate localization errors. The clustered points are replaced with the maxima of their probability density function (PDF) and connected to form polylines using a modified rectangular linear assignment algorithm. The data from vehicles on varying routes is then fused into a hierarchical singular map graph. The proposed approach achieves an average accuracy below 0.5 meters compared to a hand annotated ground truth map, as well as correctly resolving lane splits and merges, proving the feasibility of the use of vehicle fleet data for the generation of highway HD maps.

Autonomous vehicle technology has been advancing over the past decade with many challenges and obstacles. One obstacle to achieving fully autonomous driving is the feasibility of efficiently producing robust high definition (HD) maps of existing road infrastructure in a sustainable manner. HD maps are digital twins of road infrastructure that include 3-D representation of road features critical to a vehicle’s ability to navigate a road and maintain a consistent trajectory. This includes lane markings, barriers, and road edges. HD maps are often produced by first surveying roads using remote sensing technology. The collected data are then segmented using computer vision and machine learning algorithms to produce an HD map of the features of concern. One common challenge that is specific to extracting lane marking information is that the markings are occasionally occluded or degraded by high traffic volumes. This results in failure to detect lane markings and the existence of significant gaps in the extracted information. These gaps are manually filled in by quality control staff in a tedious and time-consuming process. To help overcome such challenges, this paper proposes a novel algorithm through which Kalman filtering is combined with Bézier curve fitting to automatically refine lane marking information. The proposed algorithm is tested on multiple roads where mobile lidar data were collected and segmentation was first applied using deep learning technology. The proposed algorithm was then used to refine the lane marking information which helped close all the gaps and recreate missing portions of the extracted lane markings.

Perception of the indoor environment is the basis of mobile robot localization, navigation, and path planning, and it is particularly important to construct semantic maps in real time using minimal resources. The existing methods are too dependent on the graphics processing unit (GPU) for acquiring semantic information about the indoor environment, and cannot build the semantic map in real time on the central processing unit (CPU). To address the above problems, this paper proposes a non-use GPU for lightweight indoor semantic map construction algorithm, named NGLSFusion. In the VO method, ORB features are used for the initialization of the first frame, new keyframes are created by optical flow method, and feature points are extracted by direct method, which speeds up the tracking speed. In the semantic map construction method, a pretrained model of the lightweight network LinkNet is optimized to provide semantic information in real time on devices with limited computing power, and a semantic point cloud is fused using OctoMap and Voxblox. Experimental results show that the algorithm in this paper ensures the accuracy of camera pose while speeding up the tracking speed, and obtains a reconstructed semantic map with complete structure without using GPU.

Significant research issues and experimental possibilities on autonomous vehicles are in vogue around the world. Among these, collision-free navigation has constituted one of the significant fields to research. Along with the vigorous emergence of definite sensors, challenges are coming in new avenues. This paper proposes a keen way to detect on-route obstacles using training of model through adversarial neural network along with 3D reconstruction of a surrounding under GPS-denied Indoor Environment (IE) using point cloud map. The depth sensor used here has been systematically analysed to tackle the challenge. This paper also studies the possible challenges and hurdles faced by the customized mobile robot to create point cloud map using depth sensor integrated with Robot Operating System (ROS) platform to reconstruct the surrounding. The reconstructed visuals been used as memory trail of environment along with visual reference. This array of references would prove to be credential for path planning algorithm and promises finding near optimal collision-free Indoor Mobile Robot Navigation (IMRN).KeywordsAdversarial neural network3D reconstructionPoint cloud mapIndoor Mobile Robot Navigation

Point cloud semantic segmentation in urban scenes plays a vital role in intelligent city modeling, autonomous driving, and urban planning. Point cloud semantic segmentation based on deep learning methods has achieved significant improvement. However, it is also challenging for accurate semantic segmentation in large scenes due to complex elements, variety of scene classes, occlusions, and noise. Besides, most methods need to split the original point cloud into multiple blocks before processing and cannot directly deal with the point clouds on a large scale. We propose a novel context-aware network (CAN) that can directly deal with large-scale point clouds. In the proposed network, a Local Feature Aggregation Module (LFAM) is designed to preserve rich geometric details in the raw point cloud and reduce the information loss during feature extraction. Then, in combination with a Global Context Aggregation Module (GCAM), capture long-range dependencies to enhance the network feature representation and suppress the noise. Finally, a Context-Aware Upsampling Module (CAUM) is embedded into the proposed network to capture the global perception from a broad perspective. The ensemble of low-level and high-level features facilitates the effectiveness and efficiency of 3D point cloud feature refinement. Comprehensive experiments were carried out on three large-scale point cloud datasets in both outdoor and indoor environments to evaluate the performance of the proposed network. The results show that the proposed method outperformed the state-of-the-art representative semantic segmentation networks, and the overall accuracy (OA) of Tongji-3D, Semantic3D, and S3DIS is 96.01%, 95.0%, and 88.55%, respectively.

In recent years, high-definition (HD) maps are considered to be significant complements to the on-board sensors of autonomous vehicles. Due to the highly dynamic characteristic of HD map, how to update HD maps fast and efficiently has become the pivotal issue which attracts great attentions. Currently, most of the research on this issue focuses on the crowdsourcing protocol design or specific updating method meanwhile disregarding the need of communication and computing resource in HD map update. Therefore, in this paper, we propose an integrated update framework including both an adaptive data transmission scheme and an adaptive task scheduling scheme to realize fast data collection and fast data processing for HD map update. To the best of our knowledge, this is the first work that addresses the problem of HD map update which takes the need of communication and computing resource into consideration. In this paper, we firstly illustrate the system architecture of our HD map update framework. Based on that, to realize fast data collection, we propose an adaptive data transmission scheme, which optimizes the transmission rate on different links of each vehicle considering the wireless channel capacity and fronthaul link capacity of the access points. Besides, to realize fast data processing, we propose an adaptive task scheduling scheme, which enables cooperative computing between the edge nodes and cloud computing server, considering the constraints of computing capacity and backhaul link capacity. To the end, we carry out simulations to verify the effectiveness of our proposed policies.

One of the challenges of outdoor robots is developing effective portable Human-Robot-Interaction (HRI) frameworks. Hand-held devices offer a practical solution. By equipping these devices with robot software, they can be made to interact with the outdoor robots. Android devices are ideal as they are open source and can be integrated with robots powered by the Robot Operating System (ROS), also open source. However, due to the limits of rosjava, the mechanism that links ROS with android, and the conflicting modes of operation between ROS and android, current implementations of ROS-android offer limited robot applications that do not support advanced operations such as autonomous navigation and others. This paper implements selective compartmentalization to overcome these limitations, by combining ROS with android through a number of ROS and android bridges that would facilitate the development of advanced robot applications. Through the proposed method, authors were able to develop a portable HRI framework that allowed human operators to supervise an outdoor mobile robot while it performed an autonomous task. From their mobile devices, users were able to initialize the robot, configure its motion, and monitor its progress. Also, users were able to reprogram the robot to perform new tasks (not previously planned) through a creative use of features offered in the developed HRI framework. Also, user cognitive effort was reported to be low as evident by the positive score on the NASA-TLX scale test which was corroborated with robot performance data. This paper presents the detailed development and implementation steps.