Exploring Constructive Solid Geometry for Building Reconstruction from Point Clouds: Preliminary Results

In recent years, point cloud segmentation technology has increasingly played a pivotal role in tunnel construction and maintenance. Currently, traditional methods for segmenting point clouds in tunnel scenes often rely on a multitude of attribute information, including spatial distribution, color, normal vectors, intensity, and density. However, the underground tunnel scenes show greater complexity than road tunnel scenes, such as dim light, indistinct boundaries of tunnel walls, and disordered pipelines. Furthermore, issues pertaining to data quality, such as the lack of color information and insufficient annotated data, contribute to the subpar performance of conventional point cloud segmentation algorithms. To address this issue, a 3D point cloud segmentation framework specifically for underground tunnels is proposed based on the Segment Anything Model (SAM). This framework effectively leverages the generalization capability of the visual foundation model to automatically adapt to various scenes and perform efficient segmentation of tunnel point clouds. Specifically, the tunnel is first sliced along its direction on the tunnel line. Then, each sliced point cloud is projected onto a two-dimensional plane. Various projection methods and point cloud coloring techniques are employed to enhance SAM’s segmentation performance in images. Finally, the semantic segmentation of the entire underground tunnel is achieved by a small set of manually annotated semantic labels used as prompts in a progressive and recursive manner. The key feature of this method lies in its independence from model training, as it directly and efficiently addresses tunnel point cloud segmentation challenges by capitalizing on the generalization capability of foundation model. Comparative experiments against classical region growing algorithms and PointNet++ deep learning algorithms demonstrate the superior performance of our proposed algorithm.

This paper presents a novel approach to recognize and estimate pose of the 3D objects in cluttered range images. The key technical breakthrough of the developed approach can enable robust object recognition and localization under undesirable condition such as environmental illumination variation as well as optical occlusion to viewing the object partially. First, the acquired point clouds are segmented into individual object point clouds based on the developed 3D object segmentation for randomly stacked objects. Second, an efficient shape-matching algorithm called Sub-OBB based object recognition by using the proposed oriented bounding box (OBB) regional area-based descriptor is performed to reliably recognize the object. Then, the 3D position and orientation of the object can be roughly estimated by aligning the OBB of segmented object point cloud with OBB of matched point cloud in a database generated from CAD model and 3D virtual camera. To detect accurate pose of the object, the iterative closest point (ICP) algorithm is used to match the object model with the segmented point clouds. From the feasibility test of several scenarios, the developed approach is verified to be feasible for object pose recognition and localization.

The accurate segmentation of roof point cloud will directly affect the quality of building model reconstruction. Because of the complexity of building roof structure, the uneven distribution of point cloud density obtained by lidar, and the interference of external environment, it is very difficult to accurately segment the point cloud on the roof. In order to improve the quality of point cloud segmentation, this paper proposes a joint optimization method of region growth and RANSAC for Airborne LIDAR point cloud segmentation. Firstly, the point cloud is voxeled, and the normal vector and curvature of the point cloud are calculated based on principal component analysis (PCA). After setting the correlation threshold, the region growing algorithm is used to extract the initial reliable patch. Then, RANSAC algorithm is used to regularize the irregular or deviation areas in the extracted patch, optimize and set new roof surface parameters. Finally, for the unclassified points, the Euclidean distance to each plane is calculated. According to the minimum distance principle and the set distance threshold, they are classified to the corresponding roof surface, and the segmentation results are refined. Tests on several representative building point clouds show that this method significantly improves the accuracy, recall and overall accuracy of roof point cloud segmentation, and provides high-quality basic data for building 3D model reconstruction

Global localization in a prior map is an important field in virtual and augmented reality systems, but it is always a challenge to conduct point cloud based localization in the large-scale scene prior map. A large-scale prior map usually means huge amount of calculation for point cloud processing, which leads to the long time required for global localization. To deal with this problem, we propose a fast point cloud global localization method based on point clouds segmentation and learning-based descriptor. On the one hand, cylindrical filtering, ground-point removal and point cloud segmentation are adopted to eliminate a large number of useless points and retain points with rich structures, which improves the efficiency of point cloud registration. On the other hand, reliable 3D point cloud descriptor, two-phase search strategy for place recognition and geometric consistency verification are used to ensure the localization accuracy. Experiments prove that the proposed method achieves good localization effect on both KITTI and MVSEC datasets. Under the condition of ensuring the high localization accuracy, the time for point clouds to complete the global localization is greatly reduced.

With the advancement of industrial automation, the frequency of human–robot interaction (HRI) has significantly increased, necessitating a paramount focus on ensuring human safety throughout this process. This paper proposes a simulation‐assisted neural network for point cloud segmentation in HRI, specifically distinguishing humans from various surrounding objects. During HRI, readily accessible prior information, such as the positions of background objects and the robot's posture, can generate a simulated point cloud and assist in point cloud segmentation. The simulation‐assisted neural network utilizes simulated and actual point clouds as dual inputs. A simulation‐assisted edge convolution module in the network facilitates the combination of features from the actual and simulated point clouds, updating the features of the actual point cloud to incorporate simulation information. Experiments of point cloud segmentation in industrial environments verify the efficacy of the proposed method.

Fitting and recognition of geometric primitives in segmented 3D point clouds using a localized voting procedure

A shape-based segmentation method for mobile laser scanning point clouds

Point cloud has emerged as the most popular three-dimensional (3D) data format in recent years for several scientific and industrial applications. Point cloud semantic segmentation has piqued the researcher’s interest, which is a crucial stage in 3D analysis and scene comprehension. Deep learning-based processing is more feasible to increase the availability of point cloud acquisition tools that is LiDAR systems at the user end. The point cloud learning achieves tremendous success in object detection, object categorization, and semantic segmentation. To summarize the recent works with chronological development, comprehensive review of projection-, voxel-, and direct point-based point cloud semantic segmentation methods is performed from various perspectives. The commonly used point cloud benchmark datasets with their characteristics are discussed, and they are used for the performance analysis and comparison of several state-of-the-art segmentation methods. The quantitative performance analysis of these deep learning models summarizes the trend of semantic segmentation of point clouds. In the context of point cloud semantic segmentation, the various methods have specific roles. Based on the review of methods working and their performance analysis, it is concluded that the projection-based methods prioritize efficiency, which is ideal in unavailability of high-performance computing system. Voxel-based methods capture overall context, serving well in 3D object classification. Point-based approaches excel in fine details and efficiency, suited for tasks like 3D semantic segmentation. Choosing the suitable method depends on the task, data, and resources. KPConv and DGCNN are popular choices, especially for precision and adaptability to point density. However, method performance varies, underlining the need for tailored selection. Hybrid approaches, combining method strengths, promise superior results.

Point cloud segmentation neural network with same-type point cloud assistance

Semantic segmentation of large-scale point clouds by integrating attention mechanisms and transformer models

Abstract. Segmentation of point clouds is recently used in many Geomatics Engineering applications such as the building extraction in urban areas, Digital Terrain Model (DTM) generation and the road or urban furniture extraction. Segmentation is a process of dividing point clouds according to their special characteristic layers. The present paper discusses K-means and self-organizing map (SOM) which is a type of ANN (Artificial Neural Network) segmentation algorithm which treats the segmentation of point cloud. The point clouds which generate with photogrammetric method and Terrestrial Lidar System (TLS) were segmented according to surface normal, intensity and curvature. Thus, the results were evaluated. LIDAR (Light Detection and Ranging) and Photogrammetry are commonly used to obtain point clouds in many remote sensing and geodesy applications. By photogrammetric method or LIDAR method, it is possible to obtain point cloud from terrestrial or airborne systems. In this study, the measurements were made with a Leica C10 laser scanner in LIDAR method. In photogrammetric method, the point cloud was obtained from photographs taken from the ground with a 13 MP non-metric camera.

Plant phenotypic analysis is of great importance to the development of agricultural engineering, and is one of the core issues in crop science and plant breeding. Since plant growth is spatio-temporal and synchronous, understanding the growth and development of individual plants can help to reveal the growth potential of the whole plot and thus improve planting methods. In recent years, the technical means to analyze the growth situation using 3D point cloud data has received extensive attention. The plant point cloud obtained by scanning plants with LiDAR has the characteristics of high resolution, high precision, etc. Periodic scanning of the same plant for spatio-temporal point cloud data sets allows monitoring of growth through subtle changes of plant organs. Organ tracking of growing plants remains challenging due to the lightward nature of growth, the potential for topological changes and the unpredictability of plant growth over time, with the possibility of new leaf growth and leaf death. This paper designs a plant organ growth tracking method based on point cloud. First of all, for growing plants, we have established a crop point cloud spatio-temporal dataset based on two publicly available point cloud datasets. The data set includes four species, tomato, tobacco, sorghum and maize, each species contains complete organ instance labels, and each organ of the same plant has a unique label, which means that the labels of the same organ of an individual at different scan dates correspond one-to-one. Second, this paper proposes a point cloud data-based plant organ growth tracking method, which uses a cost correlation matrix to automatically track growing plant organs. Finally, based on the set of quantitative evaluation metrics, our algorithm achieves a matching accuracy of 82.89% on the plant spatio-temporal dataset and good growth tracking results in the qualitative analysis.

The automating classification of point clouds capturing urban scenes is critical for supporting applications that demand three-dimensional (3D) models. Achieving this goal, however, is met with challenges because of the varying densities of the point clouds and the complexity of the 3D data. In order to increase the level of automation in the point cloud classification, this study proposes a segment-based parameter learning method that incorporates a two-dimensional (2D) land cover map, in which a strategy of fusing the 2D land cover map and the 3D points is first adopted to create labelled samples, and a formalized procedure is then implemented to automatically learn the following parameters of point cloud classification: the optimal scale of the neighborhood for segmentation, optimal feature set, and the training classifier. It comprises four main steps, namely: (1) point cloud segmentation; (2) sample selection; (3) optimal feature set selection; and (4) point cloud classification. Three datasets containing the point cloud data were used in this study to validate the efficiency of the proposed method. The first two datasets cover two areas of the National University of Singapore (NUS) campus while the third dataset is a widely used benchmark point cloud dataset of Oakland, Pennsylvania. The classification parameters were learned from the first dataset consisting of a terrestrial laser-scanning data and a 2D land cover map, and were subsequently used to classify both of the NUS datasets. The evaluation of the classification results showed overall accuracies of 94.07% and 91.13%, respectively, indicating that the transition of the knowledge learned from one dataset to another was satisfactory. The classification of the Oakland dataset achieved an overall accuracy of 97.08%, which further verified the transferability of the proposed approach. An experiment of the point-based classification was also conducted on the first dataset and the result was compared to that of the segment-based classification. The evaluation revealed that the overall accuracy of the segment-based classification is indeed higher than that of the point-based classification, demonstrating the advantage of the segment-based approaches.

To quantify the architecture and select the ideal ideotype, it is vital to accurately measure the dimension of each part of the mantis shrimp. Point clouds have become increasingly popular in recent years as an efficient solution. However, the current manual measurement is labor intensive and costly and has high uncertainty. Automatic organ point cloud segmentation is a prerequisite and core step for phenotypic measurements of mantis shrimps. Nevertheless, little work focuses on mantis shrimp point cloud segmentation. To fill this gap, this paper develops a framework for automated organ segmentation of mantis shrimps from multiview stereo (MVS) point clouds. First, a Transformer-based MVS architecture is applied to generate dense point clouds from a set of calibrated phone images and estimated camera parameters. Next, an improved point cloud segmentation (named ShrimpSeg) that exploits both local and global features based on contextual information is proposed for organ segmentation of mantis shrimps. According to the evaluation results, the per-class intersection over union of organ-level segmentation is 82.4%. Comprehensive experiments demonstrate the effectiveness of ShrimpSeg, outperforming other commonly used segmentation methods. This work may be helpful for improving shrimp phenotyping and intelligent aquaculture at the level of production-ready.

Point cloud segmentation is essential for scene understanding, which provides advanced information for many applications, such as autonomous driving, robots, and virtual reality. To improve the accuracy and robustness of point cloud segmentation, many researchers have attempted to fuze camera images to complement the color and texture information. The common fusion strategy is the combination of convolutional operations with concatenation, element-wise addition or element-wise multiplication. However, conventional convolutional operators tend to confine the fusion of modal features within their receptive fields, which can be incomplete and limited. In addition, the inability of encoder–decoder segmentation networks to explicitly perceive segmentation boundary information results in semantic ambiguity and classification errors at object edges. These errors are further amplified in point cloud segmentation tasks, significantly affecting the accuracy of point cloud segmentation. To address the above issues, we propose a novel self-attention multi-modal fusion semantic segmentation network for point cloud semantic segmentation. Firstly, to effectively fuze different modal features, we propose a self-cross fusion module (SCF), which models long-range modality dependencies and transfers complementary image information to the point cloud to fully leverage the modality-specific advantages. Secondly, we design the salience refinement module (SR), which calculates the importance of channels in the feature maps and global descriptors to enhance the representation capability of salient modal features. Finally, we propose the local-aware anisotropy loss measure the element-level importance in the data and explicitly provide boundary information for the model, which alleviates the inherent semantic ambiguity problem in segmentation networks. Extensive experiments on two benchmark datasets demonstrate that our proposed method surpasses current state-of-the-art methods.