Aggregate Point Cloud Geometric Features for Processing

LiDAR-based Global Localization (LGL) is an essential ingredient for autonomous robots. However, existing LGL methods typically consider only partial information (e.g., geometric features) from LiDAR observations or are designed for homogeneous LiDAR sensors, overlooking the uniformity in LGL. In this work, a uniform LGL method is proposed, termed UniLGL, which simultaneously achieves spatial and material uniformity, as well as sensor-type uniformity. The key idea of the proposed method is to encode the complete point cloud, which contains both geometric and material information, into a pair of Bird's Eye View (BEV) images (i.e., a spatial BEV image and an intensity BEV image), thereby transforming the LGL problem into a cascaded LiDAR Place Recognition (LPR) and pose estimation problem from the perspective of image fusion. An end-to-end multi-BEV fusion network is designed to extract uniform features, equipping UniLGL with spatial and material uniformity. To ensure robust LGL across heterogeneous LiDAR sensors, a viewpoint invariance hypothesis is introduced, which replaces the conventional translation equivariance assumption commonly used in existing LPR networks and supervises UniLGL to achieve sensor type uniformity in both global descriptors and local feature representations. Moreover, UniLGL introduces a pipeline that leverages a pre-trained single-image Vision Foundation Model (VFM) for feature extraction to enhance the multi-BEV fusion LPR network, enabling strong generalization with only a few LiDAR data for fine-tuning. Finally, based on the mapping between local features on the 2D BEV image and the point cloud, a robust global pose estimator is derived that determines the global minimum of the global pose on <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\text{SE}(3)$</tex-math></inline-formula> without requiring additional registration. To validate the effectiveness of the proposed uniform LGL, extensive benchmarks are conducted in real-world environments, and the results show that the proposed UniLGL is demonstratively competitive compared to other State-of-the-Art (SOTA) LGL methods. Furthermore, UniLGL has been deployed on diverse platforms, including full-size trucks and agile Micro Aerial Vehicles (MAVs), to enable high-precision localization and mapping as well as multi-MAV collaborative exploration in port and forest environments, demonstrating the applicability of UniLGL in industrial and field scenarios. The code will be released at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/shenhm516/UniLGL</uri>.

In recent years, machine vision has played an important role in product surface quality detection. The promotion and use of this technology have largely avoided the subjectivity caused by human detection and improved detection efficiency and accuracy. Different from the image data commonly used in machine vision, point cloud can describe the spatial structure of an object, provide more information than image data, and have the ability to expand the data to build multi-dimensional data models. Due to the strong anti-interference ability of point cloud sensors and the high accuracy of three-dimensional positioning information point cloud, nondestructive testing technology based on point cloud has received more and more attention. This paper summarizes the research progress of product surface quality detection methods based on 3D point cloud in recent years. According to different data processing methods, the detection research is divided into five categories: based on point cloud contour, based on local geometric feature, based on template matching, based on multimodal point cloud, and based on deep learning. The five methods are reviewed and summarized respectively. Finally, the key problems of each detection method and the future trend of product surface quality detection technology based on 3D point cloud are discussed.

BackgroundLaserscanning recently has become a powerful and common method for plant parameterization and plant growth observation on nearly every scale range. However, 3D measurements with high accuracy, spatial resolution and speed result in a multitude of points that require processing and analysis. The primary objective of this research has been to establish a reliable and fast technique for high throughput phenotyping using differentiation, segmentation and classification of single plants by a fully automated system. In this report, we introduce a technique for automated classification of point clouds of plants and present the applicability for plant parameterization.ResultsA surface feature histogram based approach from the field of robotics was adapted to close-up laserscans of plants. Local geometric point features describe class characteristics, which were used to distinguish among different plant organs. This approach has been proven and tested on several plant species. Grapevine stems and leaves were classified with an accuracy of up to 98%. The proposed method was successfully transferred to 3D-laserscans of wheat plants for yield estimation. Wheat ears were separated with an accuracy of 96% from other plant organs. Subsequently, the ear volume was calculated and correlated to the ear weight, the kernel weights and the number of kernels. Furthermore the impact of the data resolution was evaluated considering point to point distances between 0.3 and 4.0 mm with respect to the classification accuracy.ConclusionWe introduced an approach using surface feature histograms for automated plant organ parameterization. Highly reliable classification results of about 96% for the separation of grapevine and wheat organs have been obtained. This approach was found to be independent of the point to point distance and applicable to multiple plant species. Its reliability, flexibility and its high order of automation make this method well suited for the demands of high throughput phenotyping.Highlights• Automatic classification of plant organs using geometrical surface information• Transfer of analysis methods for low resolution point clouds to close-up laser measurements of plants• Analysis of 3D-data requirements for automated plant organ classification

In the agricultural field, concrete headworks is the most important structure for the irrigation system. In recent years, several agricultural concrete infrastructures aging for a long-term period has been increasing. For maintenance and management, conventional inspection methods are time-consuming and costly, such as the electromagnetic wave method and elastic wave method. The detection of surface damage is more effective, safe and reliable than before since the laser scanning method provides detailed geometric information about the structure. The fundamental studies on point cloud data have been conducted in the civil engineering fields, nevertheless, the characteristics of point cloud in agricultural infrastructures, such as dam, headworks and canal, have not been discussed. In this study, 3D point clouds are generated for a concrete irrigation structure using the laser scanning method. We focused on the characteristics of surface damage, which are quantitatively evaluated using point cloud information, geometric information and intensity parameter. The types of detected damage are efflorescence and cracks. Which aligned point clouds or point clouds from a single scan is effective for high-precision detection is confirmed. The characteristics of surface damage are evaluated by geometric features. The distance between the fitted plane and points is calculated by RANSAC algorithm and roughness parameter. The amount of efflorescence is detected by the distance between the fitted plane from RANSAC algorithm and points. The crack is detected by the local plane fitting method. The types of damage are characterized by the intensity parameter which is related to the color, roughness and moisture of the object. The surface damage and condition are evaluated by both geometric information and intensity parameter. These results show the unique parameters of point clouds from laser scanning methods, such as geometric features and intensity parameter, are useful to evaluate the characteristics of surface damage.

Aiming at the problem that the relationship between geometric features and semantic features is ignored in the point cloud data downsampling process, which leads to inaccurate segmentation of object boundaries and structural details, this paper proposes a 3D point cloud semantic segmentation network based on multi-scale dense nested type. Firstly, a dense nested network architecture is constructed by nesting multiple multi-scale feature fusion modules to fuse multi-scale features of different directions between encoder-decoder paths, so as to effectively propagate local ge-ometric context information and enhance the ability of cross-scale information interaction. Secondly, a local feature aggregation unit is constructed in the multi-scale feature fusion module, which strengthens the structural awareness within the local point set based on graph convolution and attention mechanism, and promotes the complementarity of local geometric features and abstract semantic information. Then, the cross-layer multi-loss supervision module is combined to further optimize the multi-scale feature propagation, which makes the network training more stable and improves the point cloud segmentation accuracy. Finally, this paper verified the proposed network on the S3DIS dataset. The experimental results show that the proposed network has the mean intersection over Union of 71.2% and the overall accuracy of 88.7%, which is 1.2 and 0.7 percentage points higher than RandLA-Net, respectively, which proves that the proposed network can effectively improve the accuracy of 3D point cloud semantic segmentation.

This paper reports on a novel nonparametric rigid point cloud registration framework, Semantic Continuous Visual Odometry (CVO), that jointly integrates geometric and semantic measurements such as color or semantic labels into the alignment process and does not require explicit data association. The point clouds are represented as nonparametric functions in a reproducible kernel Hilbert space. The alignment problem is formulated as maximizing the inner product between two functions, essentially a sum of weighted kernels, each of which exploits the local geometric and semantic features. As a result of the continuous models, analytical gradients can be computed, and a local solution can be obtained by optimization over the rigid body transformation group. Besides, we present a new point cloud alignment metric that is intrinsic to the proposed framework and takes into account geometric and semantic information. The evaluations using publicly available stereo and RGB-D datasets show that the proposed method outperforms state-of-the-art outdoor and indoor frame-to-frame registration methods. An open-source GPU implementation is also provided.

LESA-Net: Semantic segmentation of multi-type road point clouds in complex agroforestry environment

Although many mesh generation methods have been developed, difficulties still exist in generating high-quality structured hex-meshes for specific local geometric features such as holes and columns in three-dimensional models. In this paper, a mesh generation approach is proposed to overcome the problem of meshing local features having regular structure with high quality. Firstly, analyze the feature's geometry information and customize its mesh process. Secondly, introduce parametric design into the optimization of mesh customization. Scaled Jacobian and aspect ratio are taken as indexes of the objective functions. Thirdly, an optimized module of the feature is set up by integrating the meshing scheme into program. By establishing a knowledge base of optimized modules corresponding to geometric features, a local feature can be automatically meshed according to its geometry information. Experiments in this paper reveal that the method is applicable and stable for generating high-quality meshes for local geometric features with regular structure.

Coarse pose registration is a crucial step in 6DOF pose estimation. Assuming the objects to be estimated are regular and their 3D models are known, this paper presents local geometric features (LGFs) to estimate the poses of objects in point cloud. Coordinate frames describing the object's pose are defined according to geometric information. Pose estimation are performed by calculating the transformation between the frames in the scene point cloud and the reference model. Two kinds of LGFs are proposed: one is the corner edge feature (CEF) which exploits corner and radiating edges, and the other is the circle normal feature (CNF) which exploits circle center and normal vector of the plane where the circle is located. Included angle sequence of adjacent edges and circle radius, which are invariant to SE (3) transformation, are selected to identify CEF and CNF, respectively. Simulation is performed to verify the feasibility of the proposed features.

Learning discriminative shape representation directly on point clouds is still challenging in 3D shape analysis and understanding. Recent studies usually involve three steps: first splitting a point cloud into some local regions, then extracting the corresponding feature of each local region, and finally aggregating all individual local region features into a global feature as shape representation using simple max-pooling. However, such pooling-based feature aggregation methods do not adequately take the spatial relationships (e.g. the relative locations to other regions) between local regions into account, which greatly limits the ability to learn discriminative shape representation. To address this issue, we propose a novel deep learning network, named Point2SpatialCapsule, for aggregating features and spatial relationships of local regions on point clouds, which aims to learn more discriminative shape representation. Compared with the traditional max-pooling based feature aggregation networks, Point2SpatialCapsule can explicitly learn not only geometric features of local regions but also the spatial relationships among them. Point2SpatialCapsule consists of two main modules. To resolve the disorder problem of local regions, the first module, named geometric feature aggregation, is designed to aggregate the local region features into the learnable cluster centers, which explicitly encodes the spatial locations from the original 3D space. The second module, named spatial relationship aggregation, is proposed for further aggregating the clustered features and the spatial relationships among them in the feature space using the spatial-aware capsules developed in this paper. Compared to the previous capsule network based methods, the feature routing on the spatial-aware capsules can learn more discriminative spatial relationships among local regions for point clouds, which establishes a direct mapping between log priors and the spatial locations through feature clusters. Experimental results demonstrate that Point2SpatialCapsule outperforms the state-of-the-art methods in the 3D shape classification, retrieval and segmentation tasks under the well-known ModelNet and ShapeNet datasets.

Faced with the deficiency of storing the massive unstructured LiDAR point cloud data with local file system, a cloudizing storage method is proposed for unstructured LiDAR point cloud data based on MongoDB database system. Firstly, this paper designs the architecture of cloudizing storage server cluster for LiDAR point cloud data in the light of distributed storage framework of MongoDB. Secondly, this paper puts forward an organization model of point cloud data file chunks, in accordance with MongoDB distributed file system GridFS, based on which the point cloud file chunks' parallel sharding is achieved. Finally, the prototype system is designed and accomplished for point cloud data cloudizing storage, which is used to carry out experiments on accessing the point cloud data in GridFS, and the results approve that the cloudizing storage proposed in this paper performs better than local file system in the accessing point cloud data.

The challenges of point cloud registration in intelligent vehicle driving lie in the large scale, complex distribution, high noise, and strong sparsity of lidar point cloud data. This paper proposes an efficient registration algorithm for large-scale outdoor road scenes by selecting the continuous distribution of key area laser point clouds as the registration point cloud. The algorithm extracts feature descriptions of the key point cloud and introduces local geometric features of the point cloud to complete rough and fine registration under constraints of key point clouds and point cloud features. The algorithm is verified through extensive experiments under multiple scenarios, with an average registration time of 0.5831 s and an average accuracy of 0.06996 m, showing significant improvement compared to other algorithms. The algorithm is also validated through real-vehicle experiments, demonstrating strong versatility, reliability, and efficiency. This research has the potential to improve environment perception capabilities of autonomous vehicles by solving the point cloud registration problem in large outdoor scenes.

This paper addresses the challenge of insufficient multi-scale feature representation and limited local geometric modeling in sparse point cloud 3D object detection. We propose an efficient sparse voxel detection network, SAVF-Net (Sparse Attention and Voxel Fusion Network). Building on a fully sparse detection framework, SAVF-Net introduces a Neighborhood Sparse Attention (NSA) module and a parallel dual-tower multi-scale voxel fusion module (MSVF-SSM). The NSA module employs a local self-attention mechanism on sparse neighborhood indices, effectively capturing geometric correlations between sparse voxels and enhancing representations of small or boundary objects. The MSVF-SSM module uses parallel sparse convolutions with different dilation rates and a channel-spatial dualattention mechanism to adaptively fuse global semantic and local geometric features, thereby improving the network's perception of objects across different scales. Experimental results on the KITTI dataset demonstrate that SAVF-Net significantly outperforms mainstream methods. Specifically, SAVF-Net improves the mean average precision of 3D detection and BEV detection by approximately 2.20% and 2.14%, respectively. Notably, it achieves a 2%-4% accuracy gain on small-object categories such as pedestrians and cyclists. Furthermore, field tests on real-world roads validate the model's generalization and stability in complex dynamic scenes. Overall, SAVF-Net achieves improved accuracy and robustness for sparse point cloud 3D detection while preserving its sparse and efficient computation, providing strong support for efficient sensor perception in autonomous driving systems.

Point clouds have been gaining importance as a solution to the problem of efficient representation of 3D geometric and visual information. They are commonly represented by large amounts of data, and compression schemes are important for their manipulation transmission and storing. However, the selection of appropriate compression schemes requires effective quality evaluation. In this work a subjective quality evaluation of point clouds using a surface representation is analyzed. Using a set of point cloud data objects encoded with the popular octree pruning method with different qualities, a subjective evaluation was designed. The point cloud geometry was presented to observers in the form of a movie showing the 3D Poisson reconstructed surface without textural information with the point of view changing in time. Subjective evaluations were performed in three different laboratories. Scores obtained from each test were correlated and no statistical differences were observed. Scores were also correlated with previous subjective tests and a good correlation was obtained when compared with mesh rendering in 2D monitors. Moreover, the results were correlated with state of the art point cloud objective metrics revealing poor correlation. Likewise, the correlation with a subjective test using a different representation of the point cloud data also showed poor correlation. These results suggest the need for more reliable objective quality metrics and further studies on adequate point cloud data representations.

3D point cloud semantic segmentation is a fundamental task for 3D scene understanding. However, most existing pipelines usually use k-NN or ball query operation to form hard neighborhoods, which may cross different semantic objects, resulting low-quality local features. To address this issue, we propose a multi-scale superpoint network that gradually generates multi-scale soft neighborhoods to extract geometric local features, thereby boosting the 3D semantic segmentation performance. Specifically, we present a simple yet efficient superpoint merging module that merge small-scale superpoints to obtain large-scale superpoint by considering the feature similarity of superpoints, so that we can obtain multi-scale geometric features of point clouds. We also develop a superpoint upsampling module that adopt inverse mapping function to propagate multi-scale features from low-resolution point cloud to high-resolution point cloud. By integrating our multi-scale superpoint network into a simple point based semantic segmentation network, our method can obtain SOTA results on S3DIS Area 5 and 6-fold, and competitive results on ScanNet v2.