Deep Place Recognition for 3D LiDAR Point Cloud with Transformer

In recent years, the Internet of things (IOT), big data and artificial intelligence are evolving rapidly. The Self-driving vehicle is an important issue. One of the biggest advantages of self-driving vehicles is to reduce traffic accidents. The goal of our efforts is to provide more convenient and safe transportation. Therefore, we present a “high-accuracy mapping design based on multi-view images and three-dimensional (3D) LiDAR point clouds” in this paper. The proposed system for 3D LiDAR combines color camera and three dimensional space coordinate corresponding to the two-dimensional space coordinates technology.

In recent years, due to the significant advancements in hardware sensors and software technologies, 3D environmental point cloud modeling has gradually been applied in the automation industry, autonomous vehicles, and construction engineering. With the high-precision measurements of 3D LiDAR, its point clouds can clearly reflect the geometric structure and features of the environment, thus enabling the creation of high-density 3D environmental point cloud models. However, due to the enormous quantity of high-density 3D point clouds, storing and processing these 3D data requires a considerable amount of memory and computing time. In light of this, this paper proposes a real-time 3D point cloud environmental contour modeling technique. The study uses the point cloud distribution from the 3D LiDAR body frame point cloud to establish structured edge features, thereby creating a 3D environmental contour point cloud map. Additionally, unstable objects such as vehicles will appear during the mapping process; these specific objects will be regarded as not part of the stable environmental model in this study. To address this issue, the study will further remove these objects from the 3D point cloud through image recognition and LiDAR heterogeneous matching, resulting in a higher quality 3D environmental contour point cloud map. This 3D environmental contour point cloud not only retains the recognizability of the environmental structure but also solves the problems of massive data storage and processing. Moreover, the method proposed in this study can achieve real-time realization without requiring the 3D point cloud to be organized in a structured order, making it applicable to unorganized 3D point cloud LiDAR sensors. Finally, the feasibility of the proposed method in practical applications is also verified through actual experimental data.

Towards 3D lidar point cloud registration improvement using optimal neighborhood knowledge

This study explores the use of point clouds for both representation and genetic morphogenesis of complex geometry. The accurate representation of existing objects of complex curved geometry, which are subsequently geometrically modified by evolutionary morphogenetic processes, is analysed. To this end, as a method of representation and generation of complex geometries, a point-based genetic algorithm and the use of large unstructured point clouds are proposed. A study of convergence and diversity of the implemented algorithm is detailed, as well as a comparison with the Coyote optimisation algorithm in terms of representation error demonstrating its efficiency. Some commonly used three-dimensional formats in architecture, such as NURBS and polygon meshes, are analysed, and compared against point clouds. This study also includes an evaluation regarding whether the use of point clouds is a more suitable format for realistic representation, rationalisation and genetic morphogenesis.

Abstract. Automatic 3D point cloud registration is a main issue in computer vision and photogrammetry. The most commonly adopted solution is the well-known ICP (Iterative Closest Point) algorithm. This standard approach performs a fine registration of two overlapping point clouds by iteratively estimating the transformation parameters, and assuming that good a priori alignment is provided. A large body of literature has proposed many variations of this algorithm in order to improve each step of the process. The aim of this paper is to demonstrate how the knowledge of the optimal neighborhood of each 3D point can improve the speed and the accuracy of each of these steps. We will first present the geometrical features that are the basis of this work. These low-level attributes describe the shape of the neighborhood of each 3D point, computed by combining the eigenvalues of the local structure tensor. Furthermore, they allow to retrieve the optimal size for analyzing the neighborhood as well as the privileged local dimension (linear, planar, or volumetric). Besides, several variations of each step of the ICP process are proposed and analyzed by introducing these features. These variations are then compared on real datasets, as well with the original algorithm in order to retrieve the most efficient algorithm for the whole process. Finally, the method is successfully applied to various 3D lidar point clouds both from airborne, terrestrial and mobile mapping systems.

LiDAR point cloud-based place recognition (LPR) in unstructured natural environments remains an open challenge with limited existing research. To address the limitations of unstructured environments, such as sparse structural features, uneven point cloud density, and significant viewpoint variations, we present BSPR-Net, a dual-branch point cloud feature extraction approach for point cloud place recognition, which consists of a BEV - projection rotation - invariant convolution branch and a point cloud sparse convolution branch. This design enhances the representation capability of geometric structural features while aggregating rotation-invariant characteristics of point clouds, thereby better addressing the challenge of large viewpoint disparities in reverse-revisited unstructured environments. The proposed network was tested on multiple reverse-revisited sequences of the Wild-Places data set, a benchmark for unstructured natural environment place recognition. It achieved a maximum F1 score of 85.46 %, exceeding other classical methods by more than 4 %. The ablation experiments further confirmed the effectiveness of each module in improving place recognition performance.

This work presents a new method of automatic window detection in 3D LiDAR point clouds obtained from mobile terrestrial data acquisition systems in the urban environment. The proposed method first segments out 3D points belonging to the building facade from the 3D urban point cloud and then projects them onto a 2D plane parallel to the building facade. After point inversion within a watertight boundary, windows are segmented out based on geometrical information. The window features are then estimated exploiting both symmetrically corresponding windows in the facade as well as temporally corresponding windows in successive passages, based on ANOVA measurements. This unique fusion of information not only accommodates for lack of symmetry but also helps complete missing features due to occlusions. The estimated windows are then used to refine the 3D point cloud of the building facade. The results, evaluated on real data using different standard evaluation metrics, demonstrate the efficacy of the method.

Global Navigation Satellite System (GNSS) can provide accurate absolute position but requires fine observational conditions. In urban canyons and indoor scenes, GNSS suffers from severe performance degradation and is even unavailable. Thereby, to provide ubiquitous positioning solutions, it is very necessary to incorporate other complementary sensors. A popular option is the 3D LiDAR sensor, which emits its own light and thus is robust to illumination variation. 3D LiDAR sensors can efficiently capture plentiful point patterns of the ambient environment. Besides perception tasks, the obtained point clouds are also used for relative localization in a registration fashion, known as the LiDAR odometry (LO). Generally, LO is based on structural information, where edge and planar features are extensively exploited.In this work, we propose a novel method aiming to efficiently extract planes from the sparse and noisy 3D LiDAR point clouds. To fully exploit the scanning pattern of this sensor, our method follows a framework of point-to-line-to-plane. The point cloud is firstly projected onto a range image by investigating the azimuth and elevation of each point. In the point-to-line stage, consecutive flat points in a column are grouped into line segments, where a new flat point detector is introduced. In the line-to-plane stage, we extend the classical line extraction method, i.e., Douglas-Peucker algorithm, to find planes in line segments. Considering the over-segmentation caused by occlusion and deformation, we finally conduct region growing and merging to acquire more complete results. Most importantly, we bridge the measurement noise model and the parameter uncertainty via error propagation to determine reasonable thresholds throughout our method.We test the proposed method on datasets collected by various LiDAR sensors. The experiments are conducted in indoor scenes and urban scenes, which contains abundant planar objects such as walls and building facades. Three point-level metrics, namely positive predictive value (PPV), true positive rate (TPR) and F1 score, are employed for quantitative evaluation. The average PPV, TPR, F1 of the proposed method are 89.92%, 86.38% and 88.11%, respectively. The results show that the proposed method is able to recover the dominant planar structure, which is valuable to LO. Moreover, compared to the rotation rate of the LiDAR sensor, which is generally set to 10 Hz, the average runtime of the proposed method is 15.6 ms/frame, so it is qualified for online operation.

3D LiDAR point clouds, obtained through scanning by LiDAR devices, contain rich information such as 3D coordinates (X, Y, Z), color, classification values, intensity values, and time. However, the original collected 3D LiDAR point clouds often exhibit significant disparities in instance counts, which can hinder the effectiveness of point cloud segmentation. PolarMix, a data augmentation algorithm for 3D LiDAR point cloud datasets, addresses this issue by rotating and pasting selected class instances around the Z axis multiple times to enrich the distribution of the point cloud. However, PolarMix does not adequately consider the substantial variations in instance counts within the original point clouds, leading to an imbalance in the dataset. To address this limitation, we propose a modified version of PolarMix's instance-level rotation and pasting method that dynamically adjusts the number of rotations and pastes based on the proportion of each instance's point cloud count relative to the total. This adaptive class-balancing approach ensures a more balanced distribution of instances across the entire dataset. We term our new algorithm Dynamic Adaptive Class-Balanced PolarMix (DACB-PolarMix). Experimental results demonstrate the effectiveness of DACB-PolarMix in balancing class distribution and enhancing model performance. The results on the SemanticKitti dataset are particularly significant. Under the MinkNet model, our method improved the mIoU from 65% to 67.9%, and under the SPVCNN model, our method increased the mIoU from 66.2% to 67.5%.

Water segmentation is essential for the autonomous driving system of unmanned surface vehicles (USVs), which provides reliable navigation for making safety decisions. However, existing methods have only used monocular images as input, which often suffer from the changes in illumination and weather. Compared with monocular images, LiDAR point clouds can be collected independently of ambient light and provide sufficient 3D information but lack the color and texture that images own. Thus, in this paper, we propose a novel camera-LiDAR cross-modality fusion water segmentation method, which combines the data characteristics of the 2D image and 3D LiDAR point cloud in water segmentation for the first time. Specifically, the 3D point clouds are first supplemented with 2D color and texture information from the images and then distinguished into water surface points and non-water points by the early 3D cross-modality segmentation module. Subsequently, the 3D segmentation results and features are fed into the late 2D cross-modality segmentation module to perform 2D water segmentation. Finally, the 2D and 3D water segmentation results are fused for the refinement by an uncertainty-aware cross-modality fusion module. We further collect, annotate and present a novel Cross-modality Water Segmentation (CMWS) dataset to validate our proposed method. To the best of our knowledge, this is the first water segmentation dataset for USVs in inland waterways consisting of images and corresponding point clouds. Extensive experiments on the CMWS dataset demonstrate that our proposed method can significantly improve image-only-based methods, achieving improvements in accuracy and MaxF of approximately 2% for all the image-only-based methods.

Aiming at the problem of accurately segmenting the ground in real-time from 3D LiDAR point cloud, a ground segmentation algorithm based on the features of scanning line segments is proposed. The algorithm first performs de-noising and pose correction on the 3D point cloud, then divides the scanning line according to the Euclidean distance and absolute height difference between adjacent points. Next, the characteristics of the adjacent line segments such as spacing, slope, and absolute height difference are analyzed. The maximum likelihood estimation is used to solve the feature threshold function, which improves the adaptability of threshold. Finally, comprehensively considering the undulating and inclined complex terrain, the scanning line segments are marked as segments of flat ground, segments of slope and segments of obstacle by formulating the new horizontal and vertical classification strategies. This algorithm has been successfully applied to the unmanned ground platform. The usage and comparative test show that the algorithm can detect the ground stably and efficiently in both urban and field scenarios.

3D point clouds, a new primitive representation for objects, are spreading among thousands of people through internet software. Thus, the privacy preserving problem of the 3D point cloud should be widely concerned by more and more people. To ensure the safe transmission and use of point cloud, two schemes of encryption have been proposed by using chaotic cat mapping in this paper. The two encryption schemes are tested by using various types of 3D point clouds. In addition, these proposed encryption algorithms are analyzed through key space, sensibility, statistical and encryption time analysis. These analysis results show that the two proposed schemes can resist the common existing cipher attacks and are effective encryption methods for 3D point cloud encryption. At the same time, the two promising encryption algorithms can guarantee the security of the 3D point cloud model transmitted on the Internet.

With benefits from wide-range detection and accurate measurements, LiDAR has been widely studied in the field of autonomous driving. Yet many challenges remain in modern LiDAR 3D point clouds processing algorithms. One is ground segmentation due to the real-time requirement dealing with huge input data from LiDAR. In this study, we propose a method to separate ground points from a point cloud in a hybrid way by adopting dynamic section division, height-based conditional filter, and multi-lines linear regression. Where, physical characteristics of LiDAR mounted on a vehicle have been introduced in the dynamic section division. Thereafter, we raise a conditional filter algorithm for filtering outliers of point clouds, and use multi-lines linear regression to generate the ground skeleton. In the end, the qualitative and quantitative experiments validate the performance by using two datasets and indicate that our proposed method outperforms state-of-the-art methods on KITTI in terms of accuracy 94.1% and runtime 432ms. The source code is publicly available under https://github.com/0-0cj/sgs

Converting three-dimensional (3D) point cloud data to two-dimensional (2D) image based on virtual structured-light system is a popular method in 3D data compression because the image format is easier to process by the exiting storing and transmitting methods. When this method is used in the 3D point cloud compression, the quantization error is introduced during the coordinate mapping. To solve this problem, a virtual structured-light 3D point cloud compression algorithm based on geometric reshaping is presented. And a least-square system parameter optimization method is proposed to further improve the data decoding accuracy. In the proposed method, the 3D spatial coordinates are reshaped to a 2D matrix first, and then the 2D matrix is stored as a “Holoimage” by using the parameter optimized virtual structured-light system. The quantization error introduced by coordinate mapping between X and Y coordinates of the 3D point cloud and the 2D image pixel coordinates is suppressed, so the decoding accuracy is improved. In addition, the geometric information of the 3D point cloud is hidden synchronously when the 3D point cloud is compressed, which is of great significance in the copyright protection of the point cloud data. Experiments verify the effectiveness of the proposed algorithm. The decoding root mean square error (RMSE) of the proposed method is decreased by 79.86% on average compared with the traditional one under the same compression ratio, and the compression ratio of the proposed method is 1.96 times bigger than the traditional one under the similar decoding accuracy.

Traditionally, road detection approaches mostly capitalize on RGB images, 3D LiDAR point cloud or their fusion. However, RGB camera is sensitive to light conditions, while LiDAR point cloud is sparse compared with dense image pixels. In this work, a new hybrid image dataset is provided for the task of road detection based on cameras. In this dataset, the hybrid images are acquired by an optically aligned hybrid imaging device, consisting of a far-infrared (FIR) imager and an RGB camera to output pixel-wise registration of thermal and RGB frames. Then we investigate on three methods based on fully convolutional neural network (F-CNN) to demonstrate the advantages by fusing RGB-FIR images in road detection. First, a middle-fusion based model is built, where the output feature maps of encoder branches from RGB and FIR images are directly concatenated into a single-fusion branch as the decoder. Next, the originally discarded layers after fusion operation for both RGB and FIR branches are recovered as the mimic branches to imitate the distributions of the fusion outputs, which constitutes an extended cross model (ECM). Moreover, the outputs of mimic branches at different scales are also used to imitate the corresponding outputs in the fusion branch, called a hierarchical cross model (HCM). The experimental results demonstrate the effectiveness and efficiency of our fusion strategies.