DFT-Net: Deep Feature Transformation Based Network for Object Categorization and Part Segmentation in 3-Dimensional Point Clouds

Analyzing the geometric and semantic properties of 3D point clouds through the deep networks is still challenging due to the irregularity and sparsity of samplings of their geometric structures. This paper presents a new method to define and compute convolution directly on 3D point clouds by the proposed annular convolution. This new convolution operator can better capture the local neighborhood geometry of each point by specifying the (regular and dilated) ring-shaped structures and directions in the computation. It can adapt to the geometric variability and scalability at the signal processing level. We apply it to the developed hierarchical neural networks for object classification, part segmentation, and semantic segmentation in large-scale scenes. The extensive experiments and comparisons demonstrate that our approach outperforms the state-of-the-art methods on a variety of standard benchmark datasets (e.g., ModelNet10, ModelNet40, ShapeNet-part, S3DIS, and ScanNet).

Object part segmentation is an important problem for many applications, but generating the annotations to train a part segmentation model is typically quite labor-intensive. Recently, Fang et al. augmented object part segmentation datasets by using keypoint locations as weak supervision to transfer a source object instance's part annotations to an unlabeled target object. We show that while their approach works well when the source and target objects have clearly visible keypoints, it often fails for severely articulated poses. Also, their model does not generalize well across multiple object classes, even if they are very similar. In this paper, we propose and evaluate a new model for transferring part segmentations using keypoints, even for complex object poses and across different object classes.

DC-GNN: drop channel graph neural network for object classification and part segmentation in the point cloud

Siamese KPConv: 3D multiple change detection from raw point clouds using deep learning

With the rapid advancement of 3D acquisition technologies, 3D sensors such as LiDARs, 3D scanners, and RGB-D cameras have become increasingly accessible and cost-effective. These sensors generate 3D point cloud data that require efficient algorithms for tasks such as 3D model classification and segmentation. While deep learning techniques have proven effective in these areas, existing models often rely on complex architectures, leading to high computational costs that are impractical for real-time applications like augmented reality and robotics. In this work, we propose the Multi-level Graph Convolutional Neural Network (MLGCN), an ultra-efficient model for 3D point cloud analysis. The MLGCN model utilizes shallow Graph Neural Network (GNN) blocks to extract features at various spatial locality levels, leveraging precomputed KNN graphs shared across GCN blocks. This approach significantly reduces computational overhead and memory usage, making the model well-suited for deployment on low-memory and low-CPU devices. Despite its efficiency, MLGCN achieves competitive performance in object classification and part segmentation tasks, demonstrating results comparable to state-of-the-art models while requiring up to a thousand times fewer floating-point operations and significantly less storage. The contributions of this paper include the introduction of a lightweight, multi-branch graph-based network for 3D shape analysis, the demonstration of the model's efficiency in both computation and storage, and a thorough theoretical and experimental evaluation of the model's performance. We also conduct ablation studies to assess the impact of different branches within the model, providing valuable insights into the role of specific components.

Recently, point cloud learning has become widely utilized in a variety of domains, including autonomous driving, robotics, and computer vision. PointNet, a pioneer in point cloud processing, uses max pooling to address the disorder of point clouds. However, PointNet's method of mapping points to high-dimensional space, and then obtaining global features through maximum pooling still leads to a large loss of feature information. To this end, we suggest a new PointNet-based segmentation and classification network called PointNetX. PointNetX expands the network’s depth and the number of neurons. Simultaneously, we extracted the features of different layers to compensate for the loss caused by pooling and used a better rotation matrix to adjust the Angle. On the other hand, we add label smoothing to the loss function, and use Ranger optimizer in training to realize various tasks of point clouds. Experiments show that PointNetX has better performance in part segmentation as well as object classification than PointNet.

While object part segmentation is useful for many applications, typical approaches require a large amount of labeled data to train a model for good performance. To reduce the labeling effort, weak supervision cues such as object keypoints have been used to generate pseudo-part annotations which can subsequently be used to train larger models. However, previous weakly-supervised part segmentation methods require the same object classes during both training and testing. We propose a new model to use key-point guidance for segmenting parts of novel object classes given that they have similar structures as seen objects - different types of four-legged animals, for example. We show that a non-parametric template matching approach is more effective than pixel classification for part segmentation, especially for small or less frequent parts. To evaluate the generalizability of our approach, we introduce two new datasets that contain 200 quadrupeds in total with both key-point and part segmentation annotations. We show that our approach can outperform existing models by a large margin on the novel object part segmentation task using limited part segmentation labels during training.

Nowadays, point clouds are frequently gathered by 3D scanners such as Lidar and Kinect, which produces thousands of point cloud models. Point cloud processing is vital to 3D vision, especially 3D object recognition, positioning, and navigation technology. Addressing uneven data density caused by coordinate frame transformations and the inherent problem of insufficient context connection in point clouds, the DiM-PCNet (Multi-scale and Multi-level Point Clouds Classification Network, and the Di is a prefix to represent double M in Multi-scale and Multi-level) is proposed in this paper. DiM-PCNet is provided for object classification with multi-scale and multi-level features. We encode the point cloud in multi-scale and fully fuse the features with the raw point cloud for keeping the context relationship. In DiM-PCNet, we sample the point clouds from eight parts for multi-scales feature extraction. The multi-scales features are fully fused by multi-level pyramid models. The multi-scale and multi-level strategies are applied in DiM-PCNet, in which the abundant and important features of point clouds are extracted and utilized in the 3D object classification. It is worth noting that the DiM-PCNet feature block can be embedded into the segmentation net, where the accuracy achieved is 87.1%. We conducted experiments on ShapeNet and ModelNet40 and the experimental results show that DiM-PCNet achieves state-of-the-art performance in 3D object classification. The experiment shows competitive performance on robustness and segmentation tasks.

The classification and part segmentation of point clouds have gained significant attention in the field of artificial intelligence (AI), especially in the construction industry. However, addressing the dataset directly in AI models remains a challenge, as most existing methods are not well-suited for processing point cloud data. PointNet has emerged as an AI architecture algorithm. It transforms individual points independently to learn local and global features. This research aims to develop a comprehensive framework for classification and part segmentation for point clouds of steel structure elements. The framework enhances the accuracy of the PointNet algorithm, and it consists of three stages: (1) dataset creation; (2) model classification; and (3) part segmentation. First, the dataset creation procedure encompasses modeling steel columns, beams, and braces using Autodesk Revit software. For the classification dataset, a dataset comprising 580 columns and 920 beams is obtained. In the part segmentation dataset, five categories of steel braced frame elements are generated, yielding a total of 21,870 elements for braced frame structures. Several point cloud experiments have been applied, including adjusting the number of points in the point cloud, altering the batch size, and fine-tuning the number of epochs. These experimental settings were systematically investigated to identify the optimal combination that yields the highest (AI) model accuracy. PointNet model achieved 100% accuracy across all classification experiments, while part segmentation experiments reached up to 97.10% accuracy, with a mean intersection over union (MIOU) of 93.70%. The comprehensive analysis of the point cloud dataset is applied on an actual case study to demonstrate the practical features of the proposed research.

The thesis is going to focus on point clouds analysis, especially on 3D object classification and part segmentation of point clouds. Point clouds analysis of 3D models and 3D scenes attracts extensive research attention recently, not only due to the availability of several large repositories but also because of its wide applications. Point clouds are commonly used in industry, e.g. autonomous vehicles and robotics use LIDAR to collect point clouds of scenes, which works as an auxiliary recognition tool of 3D space. Therefore, proposing realiable and feasible algorithms for point clouds recognition and analysis is significant and benefitial.

In recent years, due to the significant advancements in hardware sensors and software technologies, 3D environmental point cloud modeling has gradually been applied in the automation industry, autonomous vehicles, and construction engineering. With the high-precision measurements of 3D LiDAR, its point clouds can clearly reflect the geometric structure and features of the environment, thus enabling the creation of high-density 3D environmental point cloud models. However, due to the enormous quantity of high-density 3D point clouds, storing and processing these 3D data requires a considerable amount of memory and computing time. In light of this, this paper proposes a real-time 3D point cloud environmental contour modeling technique. The study uses the point cloud distribution from the 3D LiDAR body frame point cloud to establish structured edge features, thereby creating a 3D environmental contour point cloud map. Additionally, unstable objects such as vehicles will appear during the mapping process; these specific objects will be regarded as not part of the stable environmental model in this study. To address this issue, the study will further remove these objects from the 3D point cloud through image recognition and LiDAR heterogeneous matching, resulting in a higher quality 3D environmental contour point cloud map. This 3D environmental contour point cloud not only retains the recognizability of the environmental structure but also solves the problems of massive data storage and processing. Moreover, the method proposed in this study can achieve real-time realization without requiring the 3D point cloud to be organized in a structured order, making it applicable to unorganized 3D point cloud LiDAR sensors. Finally, the feasibility of the proposed method in practical applications is also verified through actual experimental data.

Point cloud is a significant data which is collected from various sensors. but due to the unordered and irregular data form, an efficient and fully exploitation of raw point cloud is still challenging. PointNet provides us a promising network architecture to utilize the raw point cloud directly, which can considered to be a global descriptor of point cloud in an efficient way. Nevertheless, the information contained in local geometry is ignored by PointNet, if properly used, which can lead to a more accurate result. In this paper, LSTM network is employed to acquire the relationship contained in the point cloud, and we put forward an innovative deep-learning network, termed PLSTMNet. In experiments, we apply our PLSTMNet in the semantic segmentation task, and achieve much better performance than PointNet. The result shows the potential of LSTM in processing of point cloud.

In the past decade, deep neural networks have achieved significant progress in point cloud learning. However, collecting large-scale precisely-annotated point clouds is extremely laborious and expensive, which hinders the scalability of existing point cloud datasets and poses a bottleneck for efficient exploration of point cloud data in various tasks and applications. Label-efficient learning offers a promising solution by enabling effective deep network training with much-reduced annotation efforts. This paper presents the first comprehensive survey of label-efficient learning of point clouds. We address three critical questions in this emerging research field: i) the importance and urgency of label-efficient learning in point cloud processing, ii) the subfields it encompasses, and iii) the progress achieved in this area. To this end, we propose a taxonomy that organizes label-efficient learning methods based on the data prerequisites provided by different types of labels. We categorize four typical label-efficient learning approaches that significantly reduce point cloud annotation efforts: data augmentation, domain transfer learning, weakly-supervised learning, and pretrained foundation models. For each approach, we outline the problem setup and provide an extensive literature review that showcases relevant progress and challenges. Finally, we share our views on the current research challenges and potential future directions.

3D Point clouds are irregular and unordered. Recently, neural networks applying on point clouds have shown superior performance on 3D object classification and segmentation. PointNet has achieved competitive performance in classification tasks, however, it ignores the important local information of point clouds, which carries the important information in 3D geometric space. The informations lack will reduce the accuracy of object classification. In this paper, a new deep neural network model is proposed. Specifically, a multiscale network is used to extract local features of points to improve the accuracy of PointNet and a deep residual network is used to avoid the vanishing gradient problem in the process of deep training network. Experiments show that the proposed network architecture achieves better performance than state-of-the-art methods on multiple challenging benchmark datasets and tasks.

Abstract. Point-wise classification of 3D point clouds is a challenging task in point cloud processing, whereas, in particular, its application to high-density point clouds needs special attention because a large number of point clouds affect computational efficiency negatively. Although deep learning based models have been gaining popularity in recent years and have reached state-of-the-art results in accuracy for point-wise classification, their requirements of the high number of training samples and computational resources make those models inefficient for high-density 3D point clouds. However, traditional machine learning classifiers require less training samples, so they are capable of reducing computational requirements, even considering the latest machine learning classifiers, particularly in ensemble learning of gradient boosting machines, the results can compete with deep learning models. In this study, we are studying the point-wise classification of high-density UAV LiDAR data and focusing on efficient feature extraction and a recent state-of-the-art gradient boosting machine learning classifier, LightGBM. Our proposed framework includes the following steps: at first, we are using point cloud sampling for creating sub-sampled point clouds, then we are calculating the features based on those scales implemented on GPU. Finally, we are using the LightGBM classifier for training and testing. For the evaluation of our framework, we used a publicly available benchmark dataset, Hessigheim 3D. According to the results, we achieved an overall accuracy of 87.59% and an average F1 score of 75.92%. Our framework has promising results and scores closer to deep learning models. However, more distinctive features are required to obtain more accurate results.