A Fuzzy Spatial Relationship Graph for Point Clouds Using Bounding Boxes

This paper presents a novel approach to recognize and estimate pose of the 3D objects in cluttered range images. The key technical breakthrough of the developed approach can enable robust object recognition and localization under undesirable condition such as environmental illumination variation as well as optical occlusion to viewing the object partially. First, the acquired point clouds are segmented into individual object point clouds based on the developed 3D object segmentation for randomly stacked objects. Second, an efficient shape-matching algorithm called Sub-OBB based object recognition by using the proposed oriented bounding box (OBB) regional area-based descriptor is performed to reliably recognize the object. Then, the 3D position and orientation of the object can be roughly estimated by aligning the OBB of segmented object point cloud with OBB of matched point cloud in a database generated from CAD model and 3D virtual camera. To detect accurate pose of the object, the iterative closest point (ICP) algorithm is used to match the object model with the segmented point clouds. From the feasibility test of several scenarios, the developed approach is verified to be feasible for object pose recognition and localization.

With the advancement of industrial automation, the frequency of human–robot interaction (HRI) has significantly increased, necessitating a paramount focus on ensuring human safety throughout this process. This paper proposes a simulation‐assisted neural network for point cloud segmentation in HRI, specifically distinguishing humans from various surrounding objects. During HRI, readily accessible prior information, such as the positions of background objects and the robot's posture, can generate a simulated point cloud and assist in point cloud segmentation. The simulation‐assisted neural network utilizes simulated and actual point clouds as dual inputs. A simulation‐assisted edge convolution module in the network facilitates the combination of features from the actual and simulated point clouds, updating the features of the actual point cloud to incorporate simulation information. Experiments of point cloud segmentation in industrial environments verify the efficacy of the proposed method.

Tree species diversity mapping using UAS-based digital aerial photogrammetry point clouds and multispectral imageries in a subtropical forest invaded by moso bamboo (Phyllostachys edulis)

Point cloud segmentation is a key part of geometric processing. In this paper, a point cloud segmentation method based on the Fast Point Feature Histograms (FPFH) is proposed. Fast Point Feature Histograms (FPFH) is used to extract features of point clouds, and then Gaussian Mixture Model (GMM) is employed to cluster the point clouds. Experimental results on SHREC2014 dataset show the effectiveness of the proposed method. It can avoid both under- segmentation and over-segmentation.

Terrestrial laser scanning (TLS) is an important part of urban reconstruction and terrain surveying. In TLS applications, 4-point congruent set (4PCS) technology is widely used for the global registration of point clouds. However, TLS point clouds usually enjoy enormous data and uneven density. Obtaining the congruent set of tuples in a large point cloud scene can be challenging. To address this concern, we propose a registration method based on the voxel grid of the point cloud in this paper. First, we establish a voxel grid structure and index structure for the point cloud and eliminate uneven point cloud density. Then, based on the point cloud distribution in the voxel grid, keypoints are calculated to represent the entire point cloud. Fast query of voxel grids is used to restrict the selection of calculation points and filter out 4-point tuples on the same surface to reduce ambiguity in building registration. Finally, the voxel grid is used in our proposed approach to perform random queries of the array. Using different indoor and outdoor data to compare our proposed approach with other 4-point congruent set methods, according to the experimental results, in terms of registration efficiency, the proposed method is more than 50% higher than K4PCS and 78% higher than Super4PCS.

In recent years, point cloud segmentation technology has increasingly played a pivotal role in tunnel construction and maintenance. Currently, traditional methods for segmenting point clouds in tunnel scenes often rely on a multitude of attribute information, including spatial distribution, color, normal vectors, intensity, and density. However, the underground tunnel scenes show greater complexity than road tunnel scenes, such as dim light, indistinct boundaries of tunnel walls, and disordered pipelines. Furthermore, issues pertaining to data quality, such as the lack of color information and insufficient annotated data, contribute to the subpar performance of conventional point cloud segmentation algorithms. To address this issue, a 3D point cloud segmentation framework specifically for underground tunnels is proposed based on the Segment Anything Model (SAM). This framework effectively leverages the generalization capability of the visual foundation model to automatically adapt to various scenes and perform efficient segmentation of tunnel point clouds. Specifically, the tunnel is first sliced along its direction on the tunnel line. Then, each sliced point cloud is projected onto a two-dimensional plane. Various projection methods and point cloud coloring techniques are employed to enhance SAM’s segmentation performance in images. Finally, the semantic segmentation of the entire underground tunnel is achieved by a small set of manually annotated semantic labels used as prompts in a progressive and recursive manner. The key feature of this method lies in its independence from model training, as it directly and efficiently addresses tunnel point cloud segmentation challenges by capitalizing on the generalization capability of foundation model. Comparative experiments against classical region growing algorithms and PointNet++ deep learning algorithms demonstrate the superior performance of our proposed algorithm.

Abstract. Utilizing ground control points (GCPs) to georeference photogrammetry-based point cloud data is a common practice in unmanned aerial system (UAS) mapping. Direct georeferencing or integrated sensor orientation (ISO) can be used to obtain georeferenced point clouds from UAS without relying heavily on GCPs. However, the accuracy of the point cloud may be impacted by the accuracy of the trajectory solution obtained by GNSS. To improve point cloud accuracy, post-processing kinematic (PPK) solutions can be applied to the UAS trajectory, which may provide higher accuracy than low-accuracy trajectory solutions and minimize the reliance on GCPs. This study compares the accuracy and precision of two different point clouds generated using different methods. One point cloud was generated using traditional photogrammetric methods with low accuracy Global Navigation Satellite System (GNSS) observations from the UAS and GCPs that have an average accuracy of one to two centimeters, while the other was generated using PPK trajectory solution for the UAS’s trajectory with two software: open-source Emlid Studio and the widely used Inertial Explorer. The use of PPK techniques in UAS mapping may have several potential benefits over traditional methods. By correcting the errors in the UAS's trajectory, a user may only need to depend on fewer ground control points, which can reduce the time and cost associated with fieldwork. This is particularly useful in areas that are difficult to access or have limited ground control point options, such as in urban or forested areas. To evaluate performance, a GNSS receiver is used to obtain measurements on checkpoints, which are used to assess the accuracies of the point clouds. In our experiments, the accuracy of the point clouds generated using PPK trajectory solution with high accuracy GCPs was found to be higher than those generated with low accuracy GNSS observations while aided with high accuracy ground control points. While the use of PPK with GCPs is generally expected to provide more accurate and reliable data than low-accuracy GNSS observations even after adjusting with GCPs, the number and distribution of GCPs can still significantly impact overall accuracy. Therefore, careful consideration of the number of GCPs and their placement is essential to achieve the desired level of efficiency and effectiveness in UAS mapping.

The accurate segmentation of roof point cloud will directly affect the quality of building model reconstruction. Because of the complexity of building roof structure, the uneven distribution of point cloud density obtained by lidar, and the interference of external environment, it is very difficult to accurately segment the point cloud on the roof. In order to improve the quality of point cloud segmentation, this paper proposes a joint optimization method of region growth and RANSAC for Airborne LIDAR point cloud segmentation. Firstly, the point cloud is voxeled, and the normal vector and curvature of the point cloud are calculated based on principal component analysis (PCA). After setting the correlation threshold, the region growing algorithm is used to extract the initial reliable patch. Then, RANSAC algorithm is used to regularize the irregular or deviation areas in the extracted patch, optimize and set new roof surface parameters. Finally, for the unclassified points, the Euclidean distance to each plane is calculated. According to the minimum distance principle and the set distance threshold, they are classified to the corresponding roof surface, and the segmentation results are refined. Tests on several representative building point clouds show that this method significantly improves the accuracy, recall and overall accuracy of roof point cloud segmentation, and provides high-quality basic data for building 3D model reconstruction

Global localization in a prior map is an important field in virtual and augmented reality systems, but it is always a challenge to conduct point cloud based localization in the large-scale scene prior map. A large-scale prior map usually means huge amount of calculation for point cloud processing, which leads to the long time required for global localization. To deal with this problem, we propose a fast point cloud global localization method based on point clouds segmentation and learning-based descriptor. On the one hand, cylindrical filtering, ground-point removal and point cloud segmentation are adopted to eliminate a large number of useless points and retain points with rich structures, which improves the efficiency of point cloud registration. On the other hand, reliable 3D point cloud descriptor, two-phase search strategy for place recognition and geometric consistency verification are used to ensure the localization accuracy. Experiments prove that the proposed method achieves good localization effect on both KITTI and MVSEC datasets. Under the condition of ensuring the high localization accuracy, the time for point clouds to complete the global localization is greatly reduced.

For more than a decade, Terrestrial Laser Scanning (TLS) has been a primary remote sensing technique for disciplines related to archaeology, architecture, built heritage, earth science, metrology, and land survey. The increasing precision, range, and survey speed of TLS make this technology even more viable for large-scale data capturing in the Age of Sensing. This chapter reviews the state of the art of Terrestrial Laser Scanning in 2015 with the aim to assess its applications in a context of lower data capturing costs for alternative technologies, such as new commodity sensors, Image-based 3D Modeling, Unmanned Aerial Systems (UAS), optical 3D scanning, and Airborne Laser Scanning. More specifically, TLS still maintains a fundamental role in the documentation and interpretation of archaeological contexts at intrasite scale: (i) Terrestrial Laser Scanning delivers high-fidelity data of surfaces and structures of buildings as well as ultra-precise measurements of the morphology of stratigraphic layers; (ii) research in remote sensing proved that TLS point clouds can be successfully interpolated with data recorded with other instruments and techniques, such as magnetometry, Ground Penetrating Radar, Unmanned Aerial Vehicles, Image-Based Modeling, in order to generate hybrid documentation and new knowledge on natural and cultural heritage sites. Inevitably, the current advancements in TLS bring new questions. For example, how can micro-differences only visible in the point clouds change the analysis and interpretation of layers and buildings? How to improve the monitoring and conservation of a site via automated analysis of TLS data? How to enhance the mapping process of built-heritage using data segmentation or semi-automatic feature extraction of TLS point clouds? This chapter proposes a new approach to TLS based on multi-modal capture workflows, semi-automated post processing, online archiving, and online visualization and management of point clouds with the aim to open new horizons for digital archaeology, architectural survey, and heritage conservation .

The advancement of deep learning technology has been concentrating on deploying end-to-end solutions using high dimensional data, such as images. Recently, a number of methods have been proposed for reconstructing 3D objects using deep learning. One such technique is the method that involves recovering 3D objects as voxel grids using one or multiple images. However, there has been very little work directed towards the generation of 3D objects represented by a set of points, i.e. point cloud, from voxel grids which are ambiguous and coarse. The development of a deep learning model that generates point clouds with details from coarse voxel grids has numerous benefits to quality of life. For example, design professionals can use this model to generate detailed 3D point clouds using a sketched and coarse voxel to enable their creativities. This paper presents an algorithm to generate point clouds from voxel grids. The algorithm explicitly loads 3D objects into voxels without projection operations and associated information loss. To obtain a comprehensive understanding of the voxel grid, the grid is analyzed through various angles, as inspired by how humans observe 3D objects. The features from various angles are passed into a GRU layer to extract patterns across views, which will then be passed to a channel-wise convolutional layer and graph convolution to generated the predicted point cloud. The experimental result of the algorithm indicates that the algorithm is capable of generating high-quality point clouds by understanding the semantic features of the voxel grid.

Fitting and recognition of geometric primitives in segmented 3D point clouds using a localized voting procedure

A shape-based segmentation method for mobile laser scanning point clouds

Multi-sensor point cloud data fusion for precise 3D mapping

Point cloud has emerged as the most popular three-dimensional (3D) data format in recent years for several scientific and industrial applications. Point cloud semantic segmentation has piqued the researcher’s interest, which is a crucial stage in 3D analysis and scene comprehension. Deep learning-based processing is more feasible to increase the availability of point cloud acquisition tools that is LiDAR systems at the user end. The point cloud learning achieves tremendous success in object detection, object categorization, and semantic segmentation. To summarize the recent works with chronological development, comprehensive review of projection-, voxel-, and direct point-based point cloud semantic segmentation methods is performed from various perspectives. The commonly used point cloud benchmark datasets with their characteristics are discussed, and they are used for the performance analysis and comparison of several state-of-the-art segmentation methods. The quantitative performance analysis of these deep learning models summarizes the trend of semantic segmentation of point clouds. In the context of point cloud semantic segmentation, the various methods have specific roles. Based on the review of methods working and their performance analysis, it is concluded that the projection-based methods prioritize efficiency, which is ideal in unavailability of high-performance computing system. Voxel-based methods capture overall context, serving well in 3D object classification. Point-based approaches excel in fine details and efficiency, suited for tasks like 3D semantic segmentation. Choosing the suitable method depends on the task, data, and resources. KPConv and DGCNN are popular choices, especially for precision and adaptability to point density. However, method performance varies, underlining the need for tailored selection. Hybrid approaches, combining method strengths, promise superior results.