Automatic Building and Labeling of HD Maps with Deep Learning

Review and challenge: High definition map technology for intelligent connected vehicle

Smart cities are emerging and under development worldwide. As one of the core components for achieving the goal of smart transportation, autonomous vehicles are undoubtedly on the blueprint for smart cities. The development of autonomous driving cars leads to the demand for highly accurate and detailed machine-readable maps. To meet this demand, the high-definition (HD) maps are being developed. The HD maps are extended three-dimensional maps that tend to contain enough detailed information in the driving environment to be consumed by machines. Both commercial companies and research communities got a recent interest in building and are exploring research problems on this kind of mapping data. Several companies have already defined their commercial HD map formats, but there are arguments that the existing HD map formats are not research-friendly enough. In this paper, we propose to define a research-friendly HD map data model by extending the popular node-edge model that is widely used by researchers. Our model considers both on-road and off-road data to be extensible for future use cases and information that will be available on various three-dimensional objects in the outdoor space. This is partially motivated by several use cases that will benefit from HD mapping data other than the self-driving technology. We give a brief discussion for such potential use cases and how our model will fit their requirements. We also discuss that our model can be still used for human-machine interactions for compatibility with existing techniques and applications.

High Definition (HD) maps are necessary for many applications of automated driving (AD), but their manual creation and maintenance is very costly. Vehicle fleet data from series production vehicles can be used to automatically generate HD maps, but the data is often incomplete and noisy. We propose a system for the generation of HD maps from vehicle fleet data, which is tolerant to missing or misclassified detections and can handle drives with multiple routes, generating a single complete map, model-free and without prior reference lines. Using randomly selected drives as pivot drives, a step-wise lateral sampling of detections is performed. These sampled points are then clustered and aligned using Expectation Maximization (EM), estimating a lateral offset for each drive to compensate localization errors. The clustered points are replaced with the maxima of their probability density function (PDF) and connected to form polylines using a modified rectangular linear assignment algorithm. The data from vehicles on varying routes is then fused into a hierarchical singular map graph. The proposed approach achieves an average accuracy below 0.5 meters compared to a hand annotated ground truth map, as well as correctly resolving lane splits and merges, proving the feasibility of the use of vehicle fleet data for the generation of highway HD maps.

Autonomous vehicle technology has been advancing over the past decade with many challenges and obstacles. One obstacle to achieving fully autonomous driving is the feasibility of efficiently producing robust high definition (HD) maps of existing road infrastructure in a sustainable manner. HD maps are digital twins of road infrastructure that include 3-D representation of road features critical to a vehicle’s ability to navigate a road and maintain a consistent trajectory. This includes lane markings, barriers, and road edges. HD maps are often produced by first surveying roads using remote sensing technology. The collected data are then segmented using computer vision and machine learning algorithms to produce an HD map of the features of concern. One common challenge that is specific to extracting lane marking information is that the markings are occasionally occluded or degraded by high traffic volumes. This results in failure to detect lane markings and the existence of significant gaps in the extracted information. These gaps are manually filled in by quality control staff in a tedious and time-consuming process. To help overcome such challenges, this paper proposes a novel algorithm through which Kalman filtering is combined with Bézier curve fitting to automatically refine lane marking information. The proposed algorithm is tested on multiple roads where mobile lidar data were collected and segmentation was first applied using deep learning technology. The proposed algorithm was then used to refine the lane marking information which helped close all the gaps and recreate missing portions of the extracted lane markings.

Lane-level localization is critical for autonomous vehicles (AVs). However, complex urban scenarios, particularly tunnels, pose significant challenges to AVs’ localization systems. In this paper, we propose a fusion localization method that integrates multiple mass-production sensors, including Global Navigation Satellite Systems (GNSSs), Inertial Measurement Units (IMUs), cameras, and high-definition (HD) maps. Firstly, we use a novel electronic horizon module to assess GNSS integrity and concurrently load the HD map data surrounding the AVs. This map data are then transformed into a visual space to match the corresponding lane lines captured by the on-board camera using an improved BiSeNet. Consequently, the matched HD map data are used to correct our localization algorithm, which is driven by an extended Kalman filter that integrates multiple sources of information, encompassing a GNSS, IMU, speedometer, camera, and HD maps. Our system is designed with redundancy to handle challenging city tunnel scenarios. To evaluate the proposed system, real-world experiments were conducted on a 36-kilometer city route that includes nine consecutive tunnels, totaling near 13 km and accounting for 35% of the entire route. The experimental results reveal that 99% of lateral localization errors are less than 0.29 m, and 90% of longitudinal localization errors are less than 3.25 m, ensuring reliable lane-level localization for AVs in challenging urban tunnel scenarios.

In recent years, high-definition (HD) maps are considered to be significant complements to the on-board sensors of autonomous vehicles. Due to the highly dynamic characteristic of HD map, how to update HD maps fast and efficiently has become the pivotal issue which attracts great attentions. Currently, most of the research on this issue focuses on the crowdsourcing protocol design or specific updating method meanwhile disregarding the need of communication and computing resource in HD map update. Therefore, in this paper, we propose an integrated update framework including both an adaptive data transmission scheme and an adaptive task scheduling scheme to realize fast data collection and fast data processing for HD map update. To the best of our knowledge, this is the first work that addresses the problem of HD map update which takes the need of communication and computing resource into consideration. In this paper, we firstly illustrate the system architecture of our HD map update framework. Based on that, to realize fast data collection, we propose an adaptive data transmission scheme, which optimizes the transmission rate on different links of each vehicle considering the wireless channel capacity and fronthaul link capacity of the access points. Besides, to realize fast data processing, we propose an adaptive task scheduling scheme, which enables cooperative computing between the edge nodes and cloud computing server, considering the constraints of computing capacity and backhaul link capacity. To the end, we carry out simulations to verify the effectiveness of our proposed policies.

High-Definition (HD) Maps are indispensable components of an autonomous vehicle software stack, containing a precise representation of the static surroundings. Prediction, motion planning and vehicle behavior heavily rely on the accuracy of the HD Map. However, a key problem in the mapping process is that the environment itself changes over time, leading to inconsistencies between the real world and the outdated knowledge in the HD Map. Therefore, validating the HD Map features becomes a deciding factor in the accuracy and safety of a self-driving vehicle. Intuitively, the validation can be done by correlating the map information with the acquired sensor measurements. Although, individual sensors are subject to errors, integrating the measurements from various sensor creates a more accurate representation of the vehicle surroundings which subsequently can be used for a more reliable map validation mechanism. In this paper, we propose such a real-time method to validate the HD map by using as input more accurate estimations provided by sensor fusion. The proposed solution is decomposing the high dimensional map validation problem into multiple independent one-dimensional estimators that are able to provide validity probabilities for every independent map feature, i.e., lines, curbs and lane markings.

State estimation plays an important role in most navigation tasks for mobile robots. In the case of autonomous vehicles, localization is primarily performed using measurements from Global Navigation Satellite System (GNSS) devices, high-definition (HD) maps, or both. On the other hand, aerial and derived maps, or higher-level maps, such as OpenStreetMap or Google Maps, represent more financially viable alternatives and, depending on how they are used, with more stable characteristics. However, the task of associating low-level sensor measurements with high-level features of these maps tends to be more complex given that the data representation domains tend to be different. Therefore, in this thesis, we propose to combine techniques to associate low-level sensor data with a more abstract map for localization in the context of intelligent vehicles. Among the results obtained with the last of the proposed methods, we achieved a localization with an average error of 3:4m using the trajectories from the Kitti dataset, surpassing the popular method proposed in (MILLER et al., 2021) by more than 1:7m on the same dataset. Finally, the methods proposed in this thesis aimed at modularization and integration with more complex systems, so that - even without the intention of merging with HD or GNSS maps - these can be integrated into the same system.

Deploying fully autonomous vehicles has been a subject of intense research in both industry and academia. However, the majority of these efforts have relied heavily on High Definition (HD) prior maps. These are necessary to provide the planning and control modules a rich model of the operating environment. While this approach has shown success, it drastically limits both the scale and scope of these deployments as creating and maintaining HD maps for very large areas can be prohibitive. In this work, we present a new method for building the HD map online by starting with a Standard Definition (SD) prior map such as a navigational road map, and incorporating onboard sensors to infer the local HD map. We evaluate our method extensively on 100 sequences of real-world vehicle data and demonstrate that it can infer a highly structured HD map-like model of the world accurately using only SD prior maps and onboard sensors.

High-Definition (HD) maps are needed for robust navigation of autonomous vehicles, limited by the on-board storage capacity. To solve this, we propose a novel framework, Environment-Aware Normal Distributions Transform (EA-NDT), that significantly improves compression of standard NDT map representation. The compressed representation of EA-NDT is based on semantic-aided clustering of point clouds resulting in more optimal cells compared to grid cells of standard NDT. To evaluate EA-NDT, we present an open-source implementation that extracts planar and cylindrical primitive features from a point cloud and further divides them into smaller cells to represent the data as an EA-NDT HD map. We collected an open suburban environment dataset and evaluated EA-NDT HD map representation against the standard NDT representation. Compared to the standard NDT, EA-NDT achieved consistently at least 1.5x higher map compression while maintaining the same descriptive capability. Moreover, we showed that EA-NDT is capable of producing maps with significantly higher descriptivity score when using the same number of cells than the standard NDT.

CAOM: Change-aware online 3D mapping with heterogeneous multi-beam and push-broom LiDAR point clouds

A high-definition (HD) map is becoming an integral component of future mobility systems such as autonomous and connected vehicles. Advances in computing systems, LiDAR technologies, and vehicle communication technologies have enabled the HD map to directly treat a point cloud map (PCM), modeling road environments as LiDAR signal-level data. However, if actual road environments are changed, the PCM, modeling the environments before changes, can not be used for vehicle applications. Accordingly, the PCM has to stay up-to-date states by reflecting the environment changes continuously. This paper presents a crowd-sourcing framework to update the PCM from environment changes continuously using LiDAR and vehicle communication. Multiple intelligent vehicles installed with the LiDAR sensors download the PCM from a map server via wireless vehicle communication. To minimize the effects of environment changes, a robust localization based on a hierarchical Simultaneous Localization and Mapping (SLAM) estimates the pose (position and direction). The estimated pose is used to detect the differences between the PCM and environments, which are defined as map changes. The map changes are detected by the probabilistic and evidential theory considering the LiDAR characteristics, such as beam divergence and multi-echo. The detected map changes are uploaded to the map cloud server and merged into the PCM. The proposed crowd-sourcing framework to keep the PCM up-to-date is verified and evaluated via simulations and experiments in sites with road environment changes.

High‐definition (HD) maps are essential for autonomous driving and intelligent transportation systems. However, their high acquisition cost, significant economic value, and sensitive information content expose them to serious security threats during storage, transmission, and application. Existing encryption and watermarking methods for HD maps rarely achieve both commutativity and lossless, limiting their practical deployment for OpenDRIVE format maps. To address this issue, we propose a commutative encryption and watermarking (CEW) algorithm based on DNA dynamic coding. In the scheme, four parameters of parameterized cubic curves are first scrambled through DNA base complementarity. The parameters are then converted into binary, dynamically encoded using DNA, and diffused via arithmetic operations, ensuring effective encryption. Because encryption does not alter the number of curves, a zero‐watermarking strategy is introduced by analyzing the parity of curve counts at each node, thus achieving commutativity between encryption and watermarking. Experimental results demonstrate the robustness and high security of the proposed algorithm. The encrypted HD maps can be accurately decrypted even under various attacks while maintaining efficient encryption and decryption processes. Compared with existing methods, the watermarking scheme embedded in the algorithm exhibits superior robustness and can be reliably detected under common attack scenarios, highlighting its potential for secure management and transmission of HD maps.

Automated driving technology has attracted extensive interests in both academia and industry. High-definition (HD) map plays the critical basis in ensuring the accuracy perception and localization for automated driving. Considering the time-sensitivity, location and direction-dependency, the dissemination of HD map is a challenging task. In this paper, we address the efficient transmission of HD map data from road side units (RSUs) to vehicles passing through their communication coverage. By jointly considering the age of information (AoI) and the direction of information (DoI) for all vehicles, we propose a deep-reinforcement learning based method to achieve efficient dissemination of HD map data, where we use the deep reinforcement learning to optimize the selection of target vehicles, then optimize the power allocation among all selected vehicles. Simulation results show the performance improvement of our proposed method compared with several existing counterparts. In specific, our proposed method can improve the ratio of vehicles that obtains the map data of the next area by 229% compared with random scheme with the number of vehicles n=4.

Abstract. The concept of Autonomous vehicles or self-driving cars has recently been gaining a lot of popularity. Because of this, a lot of research is being done to develop the technology. One of which is High Definition (HD) Maps, which are centimeter-level precision 3D maps that contain a lot of geometric and semantic information about the road which can assist the AV when driving. An important component of HD maps is the road markings which indicates a set of rules on how a vehicle should navigate itself on the road. For example, lane lines indicate which part of the road a vehicle can drive on in a certain direction. This research proposes a methodology that uses deep learning techniques to detect road arrows, road markings that show possible driving directions, on LIDAR derived images, and extract them as polyline vector shapefiles. The general workflow consists of (1) converting the LIDAR point cloud to images, (2) training and applying U-Net – a fully convolutional neural network, (3) creating masks from image segmentation results that have been transformed to fit the local coordinates, (4) extracting the polygons and polylines, and finally (5) exporting the vectors in shapefile format. The proposed methodology has shown promising results with object segmentation accuracies comparable with previous related works.