A Comparison of Segmentation Methods for Semantic OctoMap Generation

DLA-Net: Learning dual local attention features for semantic segmentation of large-scale building facade point clouds

Autonomous vehicles need to have a semantic understanding of the three-dimensional world around them in order to reason about their environment. State of the art methods use deep neural networks to predict semantic classes for each point in a LiDAR scan. A powerful and efficient way to process LiDAR measurements is to use two-dimensional, image-like projections. In this work, we perform a comprehensive experimental study of image-based semantic segmentation architectures for LiDAR point clouds. We demonstrate various techniques to boost the performance and to improve runtime as well as memory constraints. First, we examine the effect of network size and suggest that much faster inference times can be achieved at a very low cost to accuracy. Next, we introduce an improved point cloud projection technique that does not suffer from systematic occlusions. We use a cyclic padding mechanism that provides context at the horizontal field-of-view boundaries. In a third part, we perform experiments with a soft Dice loss function that directly optimizes for the intersection-over-union metric. Finally, we propose a new kind of convolution layer with a reduced amount of weight-sharing along one of the two spatial dimensions, addressing the large difference in appearance along the vertical axis of a LiDAR scan. We propose a final set of the above methods with which the model achieves an increase of 3.2% in mIoU segmentation performance over the baseline while requiring only 42% of the original inference time.

Semantic segmentation of large-scale point clouds by integrating attention mechanisms and transformer models

Point cloud has emerged as the most popular three-dimensional (3D) data format in recent years for several scientific and industrial applications. Point cloud semantic segmentation has piqued the researcher’s interest, which is a crucial stage in 3D analysis and scene comprehension. Deep learning-based processing is more feasible to increase the availability of point cloud acquisition tools that is LiDAR systems at the user end. The point cloud learning achieves tremendous success in object detection, object categorization, and semantic segmentation. To summarize the recent works with chronological development, comprehensive review of projection-, voxel-, and direct point-based point cloud semantic segmentation methods is performed from various perspectives. The commonly used point cloud benchmark datasets with their characteristics are discussed, and they are used for the performance analysis and comparison of several state-of-the-art segmentation methods. The quantitative performance analysis of these deep learning models summarizes the trend of semantic segmentation of point clouds. In the context of point cloud semantic segmentation, the various methods have specific roles. Based on the review of methods working and their performance analysis, it is concluded that the projection-based methods prioritize efficiency, which is ideal in unavailability of high-performance computing system. Voxel-based methods capture overall context, serving well in 3D object classification. Point-based approaches excel in fine details and efficiency, suited for tasks like 3D semantic segmentation. Choosing the suitable method depends on the task, data, and resources. KPConv and DGCNN are popular choices, especially for precision and adaptability to point density. However, method performance varies, underlining the need for tailored selection. Hybrid approaches, combining method strengths, promise superior results.

Semantic segmentation of 3D point cloud is an essential task for autonomous driving environment perception. The pipeline of most pointwise point cloud semantic segmentation methods includes points sampling, neighbor searching, feature aggregation, and classification. Neighbor searching method like K-nearest neighbors algorithm, KNN, has been widely applied. However, the complexity of KNN is always a bottleneck of efficiency. In this paper, we propose an end-to-end neural architecture, Multiple View Pointwise Net, MVP-Net, to efficiently and directly infer large-scale outdoor point cloud without KNN or any complex pre/postprocessing. Instead, assumption-based space filling curves and multi-rotation of point cloud methods are introduced to point feature aggregation and receptive field expanding. Numerical experiments show that the proposed MVP-Net is 11 times faster than the most efficient pointwise semantic segmentation method RandLA-Net [Qin20a] and achieves the same accuracy on the large-scale benchmark SemanticKITTI dataset.

As the demand for multi-scenario semantic updates in intelligent open-pit mining grows, existing semantic segmentation techniques encounter substantial challenges, particularly in handling visually indistinguishable, continuous geometric engineering objects. These approaches frequently fall short of meeting the accuracy standards required for practical applications. To tackle this challenge, the present study introduces a novel semantic segmentation method, CAD-to-PC matching and label propagation (CPMLP). The method constructs unified graph representations from 2D computer-aided design (CAD) drawings and 3D point clouds (PCs), extracts subgraph features using a graph-based feature encoding module, performs cross-graph alignment through contrastive learning, and propagates semantic labels from CAD to PC nodes. Experimental results demonstrate that CPMLP achieves high segmentation accuracy, with an average 3D mean Intersection over Union of approximately 0.7 across different acquisition routes, and robust generalization within complex mining environments. The proposed method provides a feasible solution for high-precision semantic segmentation in intelligent mining scenarios.

Segmentation of vegetation LiDAR point clouds is an important method for obtaining individual tree structure parameters. The current individual tree segmentation methods are mainly for airborne LiDAR point clouds, which use elevation information to form a grid map for segmentation, or use canopy vertices as seed points for clustering. Side-view LiDAR (vehicle LiDAR and hand-held LiDAR) can acquire more information about the lower layer of trees, but it is a challenge to perform the individual tree segmentation because the structure of side-view LiDAR point clouds is more complex. This paper proposes an individual tree segmentation method called Shadow-cut to extract the contours of the street tree point cloud. Firstly, we separated the region of the trees using the binary classifier (e.g., support vector machine) based on point cloud geometric features. Then, the optimal projection of the 3D point clouds to the 2D image is calculated and the optimal projection is the case where the pixels of the individual tree image overlap the least. Finally, after using the image segmentation algorithm to extract the tree edges in the 2D image, the corresponding 3D individual tree point cloud contours are matched with the pixels of individual tree edges in the 2D image. We conducted experiments with the proposed method on LiDAR data of urban street trees, and the correctness, completeness, and quality of the proposed individual tree segmentation method reached 91.67%, 85.33%, and 79.19%, which were superior to the CHM-based method by 2.70%, 6.19%, and 7.12%, respectively. The results show that this method is a practical and effective solution for individual tree segmentation in the LiDAR point clouds of street trees.

Point cloud data provide three-dimensional (3D) information about objects in the real world, containing rich semantic features. Therefore, the task of semantic segmentation of point clouds has been widely applied in fields such as robotics and autonomous driving. Although existing research has made unprecedented progress, achieving real-time semantic segmentation of point clouds on airborne devices still faces challenges due to excessive computational and memory requirements. To address this issue, we propose a novel sequence convolution semantic segmentation architecture that integrates Convolutional Neural Networks (CNN) with a sequence-to-sequence (seq2seq) structure, termed SeqConv-Net. This architecture views point cloud semantic segmentation as a sequence generation task. Based on our unique perspective of spatially ordered sequences, we use Recurrent Neural Networks (RNN) to encode elevation information, then input the structured hidden states into a CNN for planar feature extraction. The results are combined with the RNN’s encoded outputs via residual connections and are fed into a decoder for sequence prediction in a seq2seq manner. Experiments show that the SeqConv-Net architecture achieves 75.5% mean Intersection Over Union (mIOU) accuracy on the DALES dataset, with the total processing speed from data preprocessing to prediction being several to tens of times faster than existing methods. Additionally, SeqConv-Net can balance accuracy and speed by adjusting the hyperparameters and using different RNNs and CNNs, providing a new solution for real-time point cloud semantic segmentation in airborne environments.

3D semantic segmentation is a fundamental task for many applications like Autonomous Driving. Recent work shows the capability of Deep Neural Networks in labelling 3D point clouds of major sensors like: LiDAR and Radar. The main challenge that faces this task is the nature of 3D point clouds being unordered and spatially-uncorrelated, making it different in terms of processing algorithms than the images. In addition to that, a point cloud usually needs higher processing power than the images if it's processed in its raw nature. In this paper, we will review different deep learning methods for 3D semantic segmentation, examples of the widely used datasets in addition to the evaluation metrics.

The use of 3D point cloud-based technology for quantifying standing wood and stand parameters can play a key role in forestry ecological benefit assessment and standing tree cultivation and utilization. With the advance of 3D information acquisition techniques, such as light detection and ranging (LiDAR) scanning, the stand information of trees in large areas and complex terrain can be obtained more efficiently. However, due to the diversity of the forest floor, the morphological diversity of the trees, and the fact that forestry is often planted as large-scale plantations, efficiently segmenting the point cloud of artificially planted forests and extracting standing wood feature parameters remains a considerable challenge. An effective method based on energy segmentation and PointCNN is proposed in this work to address this issue. The network is enhanced for learning point cloud features by geometric feature balance model (GFBM), enabling the efficient segmentation of tree point clouds from forestry point cloud data collected by terrestrial laser scanning (TLS) in outdoor environments. The 3D Forest software is then used to obtain single wood point cloud after semantic segmentation, and the extracted single wood point cloud is finally employed to extract standing wood feature parameters using TreeQSM. The point cloud semantic segmentation method is the most important part of our research. According to our findings, this method can segment datasets of two different artificially planted woodland point clouds with an overall accuracy of 0.95 and a tree segmentation accuracy of 0.93. When compared with the manual measurements, the root-mean-square error (RMSE) for tree height in the two datasets are 0.30272 and 0.21015 m, and the RMSEs for the diameter at breast height are 0.01436 and 0.01222 m, respectively. Our method is a robust framework based on deep learning that is applicable to forestry for extracting the feature parameters of artificially planted trees. It solves the problem of segmenting tree point clouds in artificially planted trees and provides a reliable data processing method for tree information extraction, trunk shape analysis, etc.

LiDAR point cloud is essential for autonomous vehicles, but motion distortions from dynamic objects degrade the data quality. While previous work has considered distortions caused by ego motion, distortions caused by other moving objects remain largely overlooked, leading to errors in object shape and position. This distortion is particularly pronounced in high-speed environments such as highways and in multi-LiDAR configurations, a common setup for heavy vehicles. To address this challenge, we introduce HiMo, a pipeline that repurposes scene flow estimation for non-ego motion compensation, correcting the representation of dynamic objects in point clouds. During the development of HiMo, we observed that existing self-supervised scene flow estimators often produce degenerate or inconsistent estimates under high-speed distortion. We further propose SeFlow++, a real-time scene flow estimator that achieves state-of-the-art performance on both scene flow and motion compensation. Since well-established motion distortion metrics are absent in the literature, we introduce two evaluation metrics: compensation accuracy at a point level and shape similarity of objects. We validate HiMo through extensive experiments on Argoverse 2, ZOD and a newly collected real-world dataset featuring highway driving and multi-LiDAR-equipped heavy vehicles. Our findings show that HiMo improves the geometric consistency and visual fidelity of dynamic objects in LiDAR point clouds, benefiting downstream tasks such as semantic segmentation and 3D detection. See https://kin-zhang.github.io/HiMo for more details.

LESA-Net: Semantic segmentation of multi-type road point clouds in complex agroforestry environment

Reconstructing as‐built building information modeling from point clouds in underground garages holds great potential for enhancing facility management. However, this task is particularly challenging due to the complex and diverse architectural and mechanical, electrical, and plumbing (MEP) components present in such environments. Moreover, there is no publicly available dataset and no dedicated semantic segmentation algorithm for underground garages environments. To address this gap, this paper first builds an expert‐labeled underground garage dataset, which includes 10 categories of components. Subsequently, this paper introduces GarageNet, a deep learning (DL) network designed for semantic segmentation of point clouds for scenes including both building and MEP components. GarageNet integrates a residual squeeze‐and‐excitation module to enhance layer and channel connection and a multi‐head attention feature aggregation module to capture local geometric structures. The results indicate that the GarageNet achieved a high overall accuracy of 95.09% and a mean intersection over union of 87.15%, surpassing other DL algorithms such as PointNext. Ablation studies and cross‐validation further demonstrate the model's reliability and generalization capability.

In this paper, we present the PS^2-Net - a locally and globally aware deep learning framework for semantic segmentation on 3D scene-level point clouds. In order to deeply incorporate local structures and global context to support 3D scene segmentation, our network is built on four repeatedly stacked encoders, where each encoder has two basic components: EdgeConv that captures local structures and NetVLAD that models global context. Different from existing state-of-the-art methods for point-based scene semantic segmentation that either violate or do not achieve permutation invariance, our PS^2-Net is designed to be permutation invariant which is an essential property of any deep network used to process unordered point clouds. We further provide theoretical proof to guarantee the permutation invariance property of our network. We perform extensive experiments on two large-scale 3D indoor scene datasets and demonstrate that our PS^2-Net is able to achieve state-of-the-art performances as compared to existing approaches.

In modern greenhouses, complicated tasks and unstructured environments generate the imperious demand for advanced semantic information about each object at work scenes. A significant problem that mainstream methods intend to resolve is that the refinement and understanding of environmental information cannot efficiently cover the entire task in real time. Therefore, this paper proposes a panoptic semantic mapping method to identify each object that is supposed to be concerned in greenhouses. This method builds grid maps with advanced semantic information based on RGB and depth images. For the agricultural task with tomato as the working object, the categories of various objects in the grid map are divided into four groups: fruits, pedicels, stems and obstacles. This method consists of three steps: semantic segmentation from RGB images with K-Net, reconstruction of point cloud data based on depth images and semantic masks and transformation of the point cloud data into OctoMap. Experimental results show the semantic segmentation algorithm reaches a mean precision of semantic segmentation of 93.83%, a mean IoU of 88.39% and an average accuracy of 98.28%. Meanwhile, the refresh frequency of publishing point cloud data with advanced semantic information holds steady at 2 Hz with the resolution of 8 mm.