Intensity Triangle Descriptor Constructed From High-Resolution Spinning LiDAR Intensity Image for Loop Closure Detection

In this work, fast and efficient appearance-based methods for visual loop-closure detection are proposed. The widely used technique based on the Bag-of-Words image representation has shown some limitations especially with aliasing problem. In this work, however, an appearance-based approach for loop closure detection using local invariant and colour features is proposed. The first technique uses Bayes Decision Theory for loop closure detection based on Gaussian Mixture Model (GMM). A new technique based on the combination of GMM with the KD-Tree data structure is presented as well. The techniques have been validated using monocular image sequences from several environments.

This paper presents a novel statistical shape model that can be used to detect and localise feature points of a class of objects in images. The shape model is inspired from the 3D morphable model (3DMM) and has the property to be viewpoint invariant. This shape model is used to estimate the probability of the position of a feature point given the position of reference feature points, accounting for the uncertainty of the position of the reference points and of the intrinsic variability of the class of objects. The viewpoint invariant detection algorithm maximises a foreground/background likelihood ratio of the relative position of the feature points, their appearance, scale, orientation and occlusion state. Computational efficiency is obtained by using the Bellman principle and an early rejection rule based on 3D to 2D projection constraints. Evaluations of the detection algorithm on the CMU-P1E face images and on a large set of non-face images show high levels of accuracy (zero false alarms for more than 90% detection rate). As well as locating feature points, the detection algorithm also estimates the pose of the object and a few shape parameters. It is shown that it can be used to initialise a 3DMM fitting algorithm and thus enables a fully automatic viewpoint and lighting invariant image analysis solution.

The loop-closure detection module in visual SLAM eliminates cumulative error from the front-end by detecting the fact that the camera passes through the same place as it passed before, providing the back-end optimization with constraints among longer intervals apart from neighboring frames, resulting in globally consistent trajectories and maps. In order to solve the problem of time-consuming and poor robustness existed in feature point matching based loop-closure detection algorithm, the article uses the Bag-of-words model based on ORB features to find loop-closure and performs spatial consistency verification on results. Firstly, the ORB feature points of image data sets are clustered to construct a dictionary. In order to complete query operations within logarithmic time, the data structure of k-ary tree is adopted. K-means++ clustering is implemented layer by layer to ensure uniform clustering. Afterwards with words bag, an image is expressed by a vector. The article uses Term Frequency-Inverse Document Frequency to assess the importance of each word so as to obtain an more effective description. Finally, the article select key frames to implement the detection. In this way repetitive detection of similar loops can be avoided and coverage of the entire environment can be guaranteed .The L1 norm is used to calculate the similarity scores between key frames. The scores are normalized using a priori similarity thus avoiding the introduction of absolute similarity thresholds, which makes the algorithm more adaptable. To remove wrong loop-closure, the spatial consistency test based on Pose Graph optimization is performed on detection results. Experiments show that the loop-closure detection algorithm used in the article has a good recall rate when the accuracy rate is high.

Point cloud registration is important for the processing of 3D model reconstruction, but with the challenge of low registration accuracy. In order to overcome this obstacle, this paper proposed a dual attention mechanism registration (DAMR) method for the point cloud registration. Firstly, the dual attention mechanism is utilized to extract key point features with different dimensions via assigning weights to the input point clouds. Secondly, the matching parameters are obtained by estimating the position correspondence between feature points and gaussian mixture model. Finally, the parameter unit blocks of two models are applied to recover the optimal transformation by matching parameters. In order to test the performance of our method, four groups of datasets include public data models and cultural relic models are adopted, respectively. Compared with traditional methods, our method has shown better performance to effectively ensured the registration accuracy of 3D point cloud models.

Terrestrial laser scanning (TLS) is an important part of urban reconstruction and terrain surveying. In TLS applications, 4-point congruent set (4PCS) technology is widely used for the global registration of point clouds. However, TLS point clouds usually enjoy enormous data and uneven density. Obtaining the congruent set of tuples in a large point cloud scene can be challenging. To address this concern, we propose a registration method based on the voxel grid of the point cloud in this paper. First, we establish a voxel grid structure and index structure for the point cloud and eliminate uneven point cloud density. Then, based on the point cloud distribution in the voxel grid, keypoints are calculated to represent the entire point cloud. Fast query of voxel grids is used to restrict the selection of calculation points and filter out 4-point tuples on the same surface to reduce ambiguity in building registration. Finally, the voxel grid is used in our proposed approach to perform random queries of the array. Using different indoor and outdoor data to compare our proposed approach with other 4-point congruent set methods, according to the experimental results, in terms of registration efficiency, the proposed method is more than 50% higher than K4PCS and 78% higher than Super4PCS.

Under low-light nighttime conditions, building images often suffer from poor quality, making high-precision integration with point cloud data challenging. This paper presents a comprehensive solution for enhancing nighttime building images and fusing them with point clouds for modeling. We developed an unsupervised learning framework that in-corporates prior knowledge of building structures. Based on RetinexDIP theory, the framework includes a structural edge enhancement module. It improves the structure of initial lighting and refines it with edge-aware adjustments using reflectance information, significantly enhancing the clarity and detail of building facades and outlines. Experi-mental results show that this method significantly improves image quality on both the MIT-Adobe FiveK dataset and a self-collected dataset. The average PSNR reached 12.02 dB, which is 0.35 dB higher than baseline methods, and the information entropy increased by 0.08. Moreover, it effectively preserves the integrity of building edge structures. We also proposed a Line-Feature-Constrained Registration (LFR) method for fusing image-based point clouds and LiDAR point clouds. This approach identifies building outline feature point clouds, extracts corresponding lines, and decomposes 3D registration into planar and elevation registration. In three test areas, the LFR method achieved a registration ac-curacy (RMSE) of 0.04-0.06 meters and an overlap rate of approximately 95%, showing significant improvements in accuracy and robustness over traditional ICP and NDT methods. Finally, we performed fusion modeling using the enhanced images and regis-tered point clouds. The reconstructed building models outperform comparative methods in both geometric completeness and texture authenticity. The number of tie points in-creased by up to 27.6%, while the median errors in plane and elevation were reduced to 4.15 cm and 5.53 cm, respectively.

Target recognizing of unmanned aerial vehicle (UAV) based on image processing take the advantage of 2D information containing in the image for identifying the target. Compare to single UAV with electrical optical tracking system (EOTS), multi-UAV with EOTS is able to take a group of image focused on the suspected target from multiple view point. Benefit from matching each couple of image in this group, points set constituted by matched feature points implicates the depth of each point. Coordinate of target feature points could be computing from depth of feature points. This depth information makes up a cloud of points and reconstructed an exclusive 3D model to recognizing system. Considering the target recognizing do not require precise target model, the cloud of feature points was regrouped into n subsets and reconstructed to a semi-3D model. Casting these subsets in a Cartesian coordinate and applying these projections in convolutional neural networks (CNN) respectively, the integrated output of networks is the improved result of recognizing.

Most simultaneous localization and mapping (SLAM) systems assume that SLAM is conducted in a static environment. When SLAM is used in dynamic environments, the accuracy of each part of the SLAM system is adversely affected. We term this problem as dynamic SLAM. In this study, we propose solutions for three main problems in dynamic SLAM: camera tracking, three-dimensional map reconstruction, and loop closure detection. We propose to employ geometry-based method, deep learning-based method, and the combination of them for object segmentation. Using the information from segmentation to generate the mask, we filter the keypoints that lead to errors in visual odometry and features extracted by the CNN from dynamic areas to improve the performance of loop closure detection. Then, we validate our proposed loop closure detection method using the precision-recall curve and also confirm the framework’s performance using multiple datasets. The absolute trajectory error and relative pose error are used as metrics to evaluate the accuracy of the proposed SLAM framework in comparison with state-of-the-art methods. The findings of this study can potentially improve the robustness of SLAM technology in situations where mobile robots work together with humans, while the object-based point cloud byproduct has potential for other robotics tasks.

Segmenting or detecting objects in sparse Lidar point clouds are two important tasks in autonomous driving to allow a vehicle to act safely in its 3D environment. The best performing methods in 3D semantic segmentation or object detection rely on a large amount of annotated data. Yet annotating 3D Lidar data for these tasks is tedious and costly. In this context, we propose a self-supervised pretraining method for 3D perception models that is tailored to autonomous driving data. Specifically, we leverage the availability of synchronized and calibrated image and Lidar sensors in autonomous driving setups for distilling self-supervised pre-trained image representations into 3D models. Hence, our method does not require any point cloud nor image annotations. The keyingredient of our method is the use of superpixels which are used to pool 3D point features and 2D pixel features in visually similar regions. We then train a 3D network on the self-supervised task of matching these pooled point features with the corresponding pooled image pixel features. The advantages of contrasting regions obtained by superpixels are that: (1) grouping together pixels and points of visually coherent regions leads to a more meaningful contrastive task that produces features well adapted to 3D semantic segmentation and 3D object detection; (2) all the different regions have the same weight in the contrastive loss regardless of the number of 3D points sampled in these regions; (3) it mitigates the noise produced by incorrect matching of points and pixels due to occlusions between the different sensors. Extensive experiments on autonomous driving datasets demonstrate the ability of our image-to-Lidar distillation strategy to produce 3D representations that transfer well on semantic segmentation and object detection tasks.

A method for automatic extraction of the feature points of the human vertebrae three-dimensional model is presented. Firstly, the statistical model of vertebrae feature points is established based on the results of manual vertebrae feature points extraction. Then anatomical axial analysis of the vertebrae model is performed according to the physiological and morphological characteristics of the vertebrae. Using the axial information obtained from the analysis, a projection relationship between the statistical model and the vertebrae model to be extracted is established. According to the projection relationship, the statistical model is matched with the vertebrae model to get the estimated position of the feature point. Finally, by analyzing the curvature in the spherical neighborhood with the estimated position of feature points, the final position of the feature points is obtained. According to the benchmark result on multiple test models, the mean relative errors of feature point positions are less than 5.98%. At more than half of the positions, the error rate is less than 3% and the minimum mean relative error is 0.19%, which verifies the effectiveness of the method.

LiDAR sensors play a crucial role in autonomous driving and advanced driver assistance systems. By firing high-rate laser beams, a LiDAR device is able to project its surroundings as 2D or 3D point cloud, which can be used for different purposes such as object detection, map generation, localization, and navigation. Autonomous vehicles are often equipped with at least one multi-layer LiDAR sensor with 360-degree coverage to include as much information as possible in the point cloud. Such a device generates enormous amount of data which poses a challenge for data storage, real-time computation, and data transmission, as autonomous vehicles are typically resource-constrained systems. This paper proposes a lightweight and adaptive point cloud data structure to reduce the size of a 3D point cloud. The suggested data structure can be flexibly configured with different parameters to adapt for precision, distance coverage, and reflectivity resolution. The precision of the data structure is evaluated using a 16-layer Velodyne LiDAR sensor (VLP-16) to collect data in the city area of AstaZero proving ground and Gothenburg downtown. Our results show that the adaptive data structure can consume only 1/8th of the original point cloud size and hence, it is particularly suitable for applications with limited hardware resources or certain tolerance to precision of the point cloud. The suggested concept is also generalizable to other types of point cloud providing sensors.

This paper focuses on the train under-vehicle inspection robot equipped with 3D solid-state LiDAR and laser distance sensor. Dedicated 3D Simultaneous Localization and Mapping (SLAM) framework and motion control algorithm are proposed for the robot operating in the hazardous environment of the inspection pit. The constrained working environment includes repetitive and monotonous spatial features makes autonomous localization and navigation task a challenging issue. The proposed SLAM localization framework has a complete architecture that includes frontend laser odometry and loop closure detection. The method optimized for 3D solid-state LiDAR is utilized for feature extraction. The pose optimization problem is initialized with the motion estimation provided by wheel odometry. The wheel odometry interpolation is utilized to increase the pose update frequency. Moreover, the features of the vehicle wheels in the measurements of laser distance sensor are utilized for loop closure detection. The proposed motion control algorithm extracts the side features of the inspection pit for feedback of angular velocity closed-loop control. The experimental results from simulation and real-world show that the robot can be driven accurately along a straight trajectory in the inspection pit with a maximum integrated localization and navigation error of less than 0.015m in movement over 200m, which demonstrates the effectiveness of the proposed algorithms.

LiDAR sensors are used to percept a scene by its 3D point cloud data. But LiDAR sensor measurement data are not homogeneous, they have uneven density and subject to occlusion problems. And the technical challenge lies along object segmentation because well-known clustering methods can not perform well with stand-alone 3D point cloud data. In this study, an approach is proposed by defining two classes of objects that are ground and nonground objects. The points are labeled ground and nonground points for DBSCAN based method. Then, sensitivity is adjusted for each class to improve the segmentation performance of ground and non-ground objects. For the CCL based method, three bird's eye view matrices are derived from point cloud data. The first matrix is used to separate ground and nonground areas and to merge other two matrix results. The ER-CCL algorithm applied with a smaller sided cell for segmentation of ground objects and a bigger sided cell for segmentation of nonground objects. The presented approach is benchmarked with a DBSCAN based method and a CCL based method. The presented method outperforms the DBSCAN method by 4% and 13% and outperforms the CCL based method by 13% and %20 in v-measure and completeness clustering metrics, respectively. Overall, an approach is proposed to segment 3D terrains in an urban area with different segmentation methods.

In reverse engineering, the point cloud data processing is the key, it will directly affect the quality and efficiency of the subsequent reconstruction of the surface. However, the amount of data obtained by three-dimensional laser scanning is often larger and redundant, which will increase the difficulty of the surface reconstruction, therefore, the point cloud data is needed to streamline. Depth analysis of the efficiency and accuracy of curvature grid method, a secondary streamlined algorithm is proposed, which take use of non-uniform grid streamline law based on octree to streamline to the point cloud, and re-use curvature streamline method based on B-spline surface fitting to streamline it. This method, whose streamline ratio is relatively high, can effectively retain the feature point cloud data, providing a great convenience for the follow-up of reverse engineering.

Image registration is an important basis of image processing, which is of great significance in image mosaicking, target recognition, and change detection. Aiming at the automatic registration problem of multi-angle optical images for ground scenes, a registration method combining point features and line features to register images is proposed. Firstly, the LSD (Line Segment Detector) algorithm is used to extract line features of images. The obtained line segments whose length are less than a given threshold are eliminated by a visual significant algorithm. Then, an affine transform model obtained by estimating a Gaussian mixture model (GMM) is applied to the image to be matched. Lastly, Harris point features are utilized in fine matching to overcome shortages of methods based on line features. In experiments, the proposed algorithm is compared with popular feature-based registration algorithms. The results indicate that the proposed algorithm in this work has obvious advantages in terms of registration accuracy and reliability for optical images acquired at different angles.