Point Cloud Map Generation and Localization for Autonomous Vehicles Using 3D Lidar Scans

Navigation system for orchard spraying robot based on 3D LiDAR SLAM with NDT_ICP point cloud registration

Accurate vehicle ego-localization is key for autonomous vehicles to complete high-level navigation tasks. The state-of-the-art localization methods adopt visual and light detection and ranging (LiDAR) simultaneous localization and mapping (SLAM) to estimate the position of the vehicle. However, both of them may suffer from error accumulation due to long-term running without loop optimization or prior constraints. Actually, the vehicle cannot always return to the revisited location, which will cause errors to accumulate in Global Navigation Satellite System (GNSS)-challenged environments. To solve this problem, we proposed a novel localization method with prior dense visual point cloud map constraints generated by a stereo camera. Firstly, the semi-global-block-matching (SGBM) algorithm is adopted to estimate the visual point cloud of each frame and stereo visual odometry is used to provide the initial position for the current visual point cloud. Secondly, multiple filtering and adaptive prior map segmentation are performed on the prior dense visual point cloud map for fast matching and localization. Then, the current visual point cloud is matched with the candidate sub-map by normal distribution transformation (NDT). Finally, the matching result is used to update pose prediction based on the last frame for accurate localization. Comprehensive experiments were undertaken to validate the proposed method, showing that the root mean square errors (RMSEs) of translation and rotation are less than 5.59 m and 0.08°, respectively.

Significant research issues and experimental possibilities on autonomous vehicles are in vogue around the world. Among these, collision-free navigation has constituted one of the significant fields to research. Along with the vigorous emergence of definite sensors, challenges are coming in new avenues. This paper proposes a keen way to detect on-route obstacles using training of model through adversarial neural network along with 3D reconstruction of a surrounding under GPS-denied Indoor Environment (IE) using point cloud map. The depth sensor used here has been systematically analysed to tackle the challenge. This paper also studies the possible challenges and hurdles faced by the customized mobile robot to create point cloud map using depth sensor integrated with Robot Operating System (ROS) platform to reconstruct the surrounding. The reconstructed visuals been used as memory trail of environment along with visual reference. This array of references would prove to be credential for path planning algorithm and promises finding near optimal collision-free Indoor Mobile Robot Navigation (IMRN).KeywordsAdversarial neural network3D reconstructionPoint cloud mapIndoor Mobile Robot Navigation

In recent years, due to the significant advancements in hardware sensors and software technologies, 3D environmental point cloud modeling has gradually been applied in the automation industry, autonomous vehicles, and construction engineering. With the high-precision measurements of 3D LiDAR, its point clouds can clearly reflect the geometric structure and features of the environment, thus enabling the creation of high-density 3D environmental point cloud models. However, due to the enormous quantity of high-density 3D point clouds, storing and processing these 3D data requires a considerable amount of memory and computing time. In light of this, this paper proposes a real-time 3D point cloud environmental contour modeling technique. The study uses the point cloud distribution from the 3D LiDAR body frame point cloud to establish structured edge features, thereby creating a 3D environmental contour point cloud map. Additionally, unstable objects such as vehicles will appear during the mapping process; these specific objects will be regarded as not part of the stable environmental model in this study. To address this issue, the study will further remove these objects from the 3D point cloud through image recognition and LiDAR heterogeneous matching, resulting in a higher quality 3D environmental contour point cloud map. This 3D environmental contour point cloud not only retains the recognizability of the environmental structure but also solves the problems of massive data storage and processing. Moreover, the method proposed in this study can achieve real-time realization without requiring the 3D point cloud to be organized in a structured order, making it applicable to unorganized 3D point cloud LiDAR sensors. Finally, the feasibility of the proposed method in practical applications is also verified through actual experimental data.

A high-definition (HD) map is becoming an integral component of future mobility systems such as autonomous and connected vehicles. Advances in computing systems, LiDAR technologies, and vehicle communication technologies have enabled the HD map to directly treat a point cloud map (PCM), modeling road environments as LiDAR signal-level data. However, if actual road environments are changed, the PCM, modeling the environments before changes, can not be used for vehicle applications. Accordingly, the PCM has to stay up-to-date states by reflecting the environment changes continuously. This paper presents a crowd-sourcing framework to update the PCM from environment changes continuously using LiDAR and vehicle communication. Multiple intelligent vehicles installed with the LiDAR sensors download the PCM from a map server via wireless vehicle communication. To minimize the effects of environment changes, a robust localization based on a hierarchical Simultaneous Localization and Mapping (SLAM) estimates the pose (position and direction). The estimated pose is used to detect the differences between the PCM and environments, which are defined as map changes. The map changes are detected by the probabilistic and evidential theory considering the LiDAR characteristics, such as beam divergence and multi-echo. The detected map changes are uploaded to the map cloud server and merged into the PCM. The proposed crowd-sourcing framework to keep the PCM up-to-date is verified and evaluated via simulations and experiments in sites with road environment changes.

A training dataset for semantic segmentation of urban point cloud map for intelligent vehicles

The fundamental task in an autonomous vehicle navigation system is localization from the available sensor measurements. GPS in the vehicles locates it with error of 1 to 10 meters so localization process should be performed to avoid fatal accidents. The realization of algorithms to estimate our vehicle’s position precisely is Localization. Odometry, Kalman Filter, Particle Filter and SLAM(Simultaneous Localization And Mapping) are the techniques used in an autonomous vehicle to localize itself in the map. Among these the particle filter is widely employed in the localization of autonomous vehicles as it provides accurate position of the vehicle in the environment. This paper aims at a localization technique for autonomous vehicles or robots using Particle Filter algorithm. The position estimator is implemented using the GPS and IMU sensor measurements. The map contains specific landmarks identified such as buildings and poles which assist the vehicle to know its position accurately by matching the distance between them in the particle filtering process. The results show that this algorithm can deliver accurate vehicle positioning even in erroneous GPS data.

Statistical learning techniques and increased computational power have facilitated the development of self-driving car technology. However, a limiting factor has been the high expense of scaling and maintaining high-definition (HD) maps. These maps are a crucial backbone for many approaches to self-driving technology. In response to this challenge, we present an approach that fuses pre-built point cloud map data with images to automatically and accurately identify static landmarks such as roads, sidewalks, and crosswalks. Our pipeline utilizes semantic segmentation of 2D images, associates semantic labels with points in point cloud maps to pinpoint locations in the physical world, and employs a confusion matrix formulation to generate a probabilistic bird's-eye view semantic map from semantic point clouds. The approach has been tested in an urban area with different segmentation networks to generate a semantic map with road features. The resulting map provides a rich context of the environment that is valuable for downstream tasks such as trajectory generation and intent prediction. Moreover, it has the potential to be extended to the automatic generation of HD maps for semantic features. The entire software pipeline is implemented in the robot operating system (ROS), a widely used robotics framework, and made available.

In response to the navigation requirements of robots in orchard environments, this paper presents a navigation method for orchard robots based on point cloud maps. Firstly, pre-acquired point cloud maps are preprocessed with pass-through filtering and PCA algorithms to obtain maps suitable for path planning. Additionally, tree rows within the orchard are clustered and segmented based on map orientations. Subsequently, using a combination of the least squares method and the A* algorithm, global operation path planning is conducted within the local maps. Finally, the TEB local path planning algorithm is employed to ensure that the robot navigates along the operation path. Experimental results indicate that the robot can successfully navigate orchards at speeds ranging from 0.4 to 1.0 m/s. The average longitudinal deviation obtained under these conditions is 26.7 cm, with a maximum value not exceeding 46.2 cm. The average heading angle deviation is 4.09°, with a maximum value of 8.65°. In conclusion, this approach guarantees full-coverage navigation operations for robots in commercial orchard environments and provides a solid foundation for further research.

Previous urban structure classification methods are intractable for practical robots in two viewpoints: storing point clouds and complex computation for using conditional random fields. This paper presents a classification method based on normal distribution transform (NDT) grids for practical robots. NDT grids store point clouds in the form of the mean and covariance, instead of directly dealing with huge point clouds. By taking the advantage of NDT grids, we design geometric-featured voxel (GFV) based on NDT grids to represent urban structures as a voxel model. The proposed method consists of three steps: GFV generation, segmentation, and classification. In the segmentation, GFVs are clustered according to units of urban structures by spectral clustering. For the classification, the clustered GFVs are classified as one kind of urban structures by supervised learning. Geometric characteristics of urban structures are expressed by a histogram of geometric words. Experimental results prove that the proposed method based on NDT grids is suitable for practical robots in terms of memory requirement, computation time, and even classification accuracy.

Accurate odometry is crucial for mapping and localization of autonomous vehicles in unknown environment. Deep neural networks have shown significant promise in self-supervised odometry, enabling pose estimation from consecutive sensor inputs. However, existing self-supervised visual odometry methods face scale ambiguity issue due to the inaccuracy of relative depth estimation, and self-supervised LiDAR odometry methods suffer from sparse data and the difficulty of correspondence searching. To address these challenges, we propose Self-FO, a self-supervised multi-sensor fusion odometry framework that effectively integrates image and point cloud data and overcomes the limitations of single-sensor methods. First, we utilize the inferred depth map as the fusion medium for image and point cloud, and realize multi-layer visual-LiDAR feature extraction and fusion through a novel homogeneous and heterogeneous channel exchange strategies. Then, we introduce a feature alignment and pose refinement module to fine-tune the coarse pose. By leveraging 2D-3D correspondences and a cross-modal attention mechanism, this module guides the image and point cloud features to focus on consistent scene-level observations and aligns features adaptively, significantly improving the accuracy of correspondence searching and enhancing the robustness and consistency of multimodal feature representations. Extensive experiments on KITTI and KITTI-360 dataset demonstrate that Self-FO outperforms existing learning-based odometry methods, delivering superior performance and scalability.

Vehicle localization is one of the key technical factors for autonomous vehicles. It requires high accuracy, precision, and robustness towards various road conditions. Popular localization methods include global navigation satellite system (GNSS) and visual methods, but their accuracy can degrade in some conditions. This work proposes to use the environmental magnetic field (EMF) for localization to complement the shortcomings of existing methods. EMF is a combination of the Earth’s geomagnetic field and magnetic field induced by man-made objects. It has local fluctuations that can be paired with coordinate positions and is time-invariant within a practical timescale. Past works considering its use in road vehicles had problems when applying them to the localization of autonomous vehicles. This work overcomes the problems by creating a two-dimensional magnetic field map using Gaussian process regression, using magnetic markers to enhance EMF fluctuations, and utilizing the Monte Carlo localization (MCL) algorithm. In this work, a 2-D EMF map was generated using measurements from our experiment vehicles. The generated map retained the original magnetic features. Next, the proposed method was validated through simulation and actual vehicle tests using the generated EMF map as a reference. The results showed that the MCL can accurately localize the vehicle and demonstrated the feasibility of the proposed method.

Accurate locating is an important task for autonomous vehicles' safely driving. In order to realize the precise locating autonomous vehicles without Global Positioning System(GPS) signal, a Kernelized Renyi Distance(KRD) based simultaneous localization and mapping(SLAM) algorithm is proposed in this paper. In our Algorithm, pose estimation are computed by optimizing KRD between two groups of laser point(in odometry process) or between laser points and the local map(in mapping process). The experimental results indicate that the proposed algorithm can accurately locate autonomous vehicle and build the traveled environment map.

Normal Distribution Transform (NDT) algorithm plays the role of detecting the relative pose of the vehicle in high-precision map. In this paper, a method of setting downsampling factor and grid factor for NDT relocation algorithm in dynamic environment is proposed, which can solve the problems of excessive NDT relocation error and location loss caused by dynamic objects accounting for 1 % to 35 % of the volume of scanning point cloud in vehicle environment. To simulate a real dynamic point cloud environment, the single-frame LiDAR point cloud space is voxelized into a mesh. Each grid is assigned a random number evenly distributed between 0 and 1. The threshold value for whether to add a Gaussian noise point is also set. Seven representative dynamic objects on the highway are selected. The threshold value of probability distribution function of Gaussian noise object needs to be set. Then the volume content of the dynamic object in the single frame point cloud space is calculated according to the set threshold value by using the definite integral. By changing the content and volume of dynamic obstacles in a dynamic environment, the effects of the downsampling factor and the grid factor on the accuracy of the repositioning trajectory are obtained. The resampling and mesh coefficients are optimized based on the analysis of the repositioning trajectory accuracy. The results show that When the current sampling factor is fixed, the grid factor of the NDT algorithm is proportional to the RMSE factor. When the NDT grid factor is fixed and the down sampling factor is equal to the side length of the obstacle, the NDT relocation accuracy is the highest and reaches the local optimum.

This study presents an innovative, intelligent obstacle avoidance module intended to significantly enhance the collision prevention capabilities of the robotic arm mechanism onboard a high-speed rail tunnel lining inspection train. The proposed module employs a fusion of ORB-SLAM3 and Normal Distribution Transform (NDT) point cloud registration techniques to achieve real-time point cloud densification, ensuring reliable detection of small-volume targets. By leveraging spatial filtering, cluster computation, and feature extraction, precise obstacle localization information is further obtained. A fusion of multi-modal data is achieved by jointly calibrating 3D LiDAR and camera images. Upon validation through field testing, it is demonstrated that the module can effectively detect obstacles with a minimum diameter of 0.5 cm, with an average deviation controlled within a 1–2 cm range and a safety margin of 3 cm, effectively preventing collisions. Compared to traditional obstacle avoidance sensors, this module provides information across more dimensions, offering robust support for the construction of powerful automated tunnel inspection control systems and digital twin lifecycle analysis techniques for railway tunnels.