Point cloud completion network based on multiple-decoder

SPC: Self-supervised point cloud completion.

In this paper we study the task of a single-view image-guided point cloud completion. Existing methods have got promising results by fusing the information of image into point cloud explicitly or implicitly. However, given that the image has global shape information and the partial point cloud has rich local details, We believe that both modalities need to be given equal attention when performing modality fusion. To this end, we propose a novel dual-channel modality fusion network for image-guided point cloud completion(named DMF-Net), in a coarse-to-fine manner. In the first stage, DMF-Net takes a partial point cloud and corresponding image as input to recover a coarse point cloud. In the second stage, the coarse point cloud will be upsampled twice with shape-aware upsampling transformer to get the dense and complete point cloud. Extensive quantitative and qualitative experimental results show that DMF-Net outperforms the state-of-the-art unimodal and multimodal point cloud completion works on ShapeNet-ViPC dataset.

Point cloud data in the real world is often affected by occlusion and light reflection, leading to incompleteness of the data. Large‐region missing point clouds will cause great deviations in downstream tasks. A dual feature fusion network (DFF‐Net) is proposed to improve the accuracy of the completion of a large missing region of the point cloud. First, a dual feature encoder is designed to extract and fuse the global and local features of the input point cloud. Subsequently, a decoder is used to directly generate a point cloud of missing region that retains local details. In order to make the generated point cloud more detailed, a loss function with multiple terms is employed to emphasise the distribution density and visual quality of the generated point cloud. A large number of experiments show that the authors’ DFF‐Net is better than the previous state‐of‐the‐art methods in the aspect of point cloud completion.

Unsupervised point cloud completion aims at estimating the corresponding complete point cloud of a partial point cloud in an unpaired manner. It is a crucial but challenging problem since there is no paired partial-complete supervision that can be exploited directly. In this work, we pro-pose a novel framework, which learns a unified and structured latent space that encoding both partial and complete point clouds. Specifically, we map a series of related par-tial point clouds into multiple complete shape and occlusion code pairs and fuse the codes to obtain their repre-sentations in the unified latent space. To enforce the learning of such a structured latent space, the proposed method adopts a series of constraints including structured ranking regularization, latent code swapping constraint, and distribution supervision on the related partial point clouds. By establishing such a unified and structured latent space, better partial-complete geometry consistency and shape completion accuracy can be achieved. Extensive experi-ments show that our proposed method consistently outper-forms state-of-the-art unsupervised methods on both syn-thetic ShapeNet and real-world KITTI, ScanNet, and Mat- terport3D datasets.

Point cloud completion aims at completing geometric and topological shapes from a partial observation. However, some topology of the original shape is missing, existing methods directly predict the location of complete points, without predicting structured and topological information of the complete shape, which leads to inferior performance. To better tackle the missing topology part, we propose LAKe-Net, a novel topology-aware point cloud completion model by localizing aligned keypoints, with a novel Keypoints-Skeleton-Shape prediction manner. Specifically, our method completes missing topology using three steps: 1) Aligned Keypoint Localization. An asymmetric keypoint locator, including an unsupervised multi-scale keypoint detector and a complete keypoint generator, is proposed for localizing aligned keypoints from complete and partial point clouds. We theoretically prove that the detector can capture aligned keypoints for objects within a sub-category. 2) Surface-skeleton Generation. A new type of skeleton, named Surface-skeleton, is generated from keypoints based on geometric priors to fully represent the topological information captured from keypoints and better recover the local details. 3) Shape Refinement. We design a refinement subnet where multi-scale surface-skeletons are fed into each recursive skeleton-assisted refinement module to assist the completion process. Experimental results show that our method achieves the state-of-the-art performance on point cloud completion.

We consider the problem of online and real-time registration of partial point clouds obtained from an unseen real-world rigid object without knowing its 3D model. The point cloud is partial as it is obtained by a depth sensor capturing only the visible part of the object from a certain viewpoint. It introduces two main challenges: 1) two partial point clouds do not fully overlap and 2) keypoints tend to be less reliable when the visible part of the object does not have salient local structures. To address these issues, we propose DeepPRO, a keypoint-free and an end-to-end trainable deep neural network. Its core idea is inspired by how humans align two point clouds: we can imagine how two point clouds will look like after the registration based on their shape. To realize the idea, DeepPRO has inputs of two partial point clouds and directly predicts the point-wise location of the aligned point cloud. By preserving the ordering of points during the prediction, we enjoy dense correspondences between input and predicted point clouds when inferring rigid transform parameters. We conduct extensive experiments on the real-world Linemod and synthetic ModelNet40 datasets. In addition, we collect and evaluate on the PRO1k dataset, a large-scale version of Linemod meant to test generalization to real-world scans. Results show that DeepPRO achieves the best accuracy against thirteen strong baseline methods, e.g., 2.2mm ADD on the Linemod dataset, while running 50 fps on mobile devices.

Self-supervised learning on point clouds has gained a lot of attention recently, since it addresses the label-efficiency and domain-gap problems on point cloud tasks. In this paper, we propose a novel self-supervised framework to learn informative features from partial point clouds. We leverage partial point clouds scanned by LiDAR that contain both content and pose attributes, and we show that disentangling such two factors from partial point clouds enhances feature learning. To this end, our framework consists of three main parts: 1) a completion network to capture holistic semantics of point clouds; 2) a pose regression network to understand the viewing angle where partial data is scanned from; 3) a partial reconstruction network to encourage the model to learn content and pose features. To demonstrate the robustness of the learnt feature representations, we conduct several downstream tasks including classification, part segmentation, and registration, with comparisons against state-of-the-art methods. Our method not only outperforms existing self-supervised methods, but also shows a better generalizability across synthetic and real-world datasets.

DcTr: Noise-robust point cloud completion by dual-channel transformer with cross-attention

Point cloud completion recovers the complete point clouds from partial ones, providing numerous point cloud information for downstream tasks such as 3-D reconstruction and target detection. However, previous methods usually suffer from unstructured prediction of points in local regions and the discrete nature of the point cloud. To resolve these problems, we propose a point cloud completion network called TPDC. Representing the point cloud as a set of unordered features of points with local geometric information, we devise a Triangular Pyramid Extractor (TPE), using the simplest 3-D structure-a triangular pyramid-to convert the point cloud to a sequence of local geometric information. Our insight of revealing local geometric information in a complex environment is to design a Divide-and-Conquer Splitting Module in a Divide-and-Conquer Splitting Decoder (DCSD) to learn point-splitting patterns that can fit local regions the best. This module employs the Divide-and-Conquer approach to parallelly handle tasks related to fitting ground-truth values to base points and predicting the displacement of split points. This approach aims to make the base points align more closely with the ground-truth values while also forecasting the displacement of split points relative to the base points. Furthermore, we propose a more realistic and challenging benchmark, ShapeNetMask, with more random point cloud input, more complex random item occlusion, and more realistic random environmental perturbations. The results show that our method outperforms both widely used benchmarks as well as the new benchmark.

Point cloud classification is critical for three-dimensional scene understanding. However, in real-world scenarios, depth cameras often capture partial, single-view point clouds of objects with different poses, making their accurate classification a challenge. In this paper, we propose a novel point cloud classification network that captures the detailed spatial structure of objects by constructing tetrahedra, which is different from point-wise operations. Specifically, we propose a RISpaNet block to extract rotation-invariant features. A rotation-invariant property generation module is designed in RISpaNet for constructing rotation-invariant tetrahedron properties (RITPs). Meanwhile, a multi-scale pooling module and a hybrid encoder are used to process RITPs to generate integrated rotation-invariant features. Further, for single-view point clouds, a complete point cloud auxiliary branch and a part-whole correlation module are jointly employed to obtain complete point cloud features from partial point clouds. Experimental results show that this network performs better than other state-of-the-art methods, evaluated on four public datasets. We achieved an overall accuracy of 94.7% (+2.0%) on ModelNet40, 93.4% (+5.9%) on MVP, 94.7% (+6.3%) on PCN and 94.8% (+1.7%) on ScanObjectNN. Our project website is https://luxurylf.github.io/RISpaNet_project/.

Evaluation of static three-dimensional (3-D) laser scanning point cloud accuracy has become a topical research issue. Point cloud accuracy is typically estimated by comparing terrestrial laser scanning data related to a finite number of check point coordinates against those obtained by an independent source of higher accuracy. These methods can only estimate the point accuracy but not the point cloud accuracy, which is influenced by the positional error and sampling interval. It is proposed that the point cloud error ellipsoid is favorable for inspecting the point cloud accuracy, which is determined by the individual point error ellipsoid volume. The kernel of this method is the computation of the point cloud error ellipsoid volume and the determination of the functional relationship between the error ellipsoid and accuracy. The proposed point cloud accuracy evaluation method is particularly suited for small sampling intervals when there exists an intersection of two error ellipsoids, and is suited not only for planar but also for nonplanar target surfaces. The performance of the proposed method (PM) is verified using both planar and nonplanar board point clouds. The results demonstrate that the proposed evaluation method significantly outperforms the existing methods when the target surface is nonplanar or there exists an intersection of two error ellipsoids. The PM therefore has the potential for improving the reliability of point cloud digital elevation models and static 3-D laser scanning-based deformation monitoring.

In this paper, we tackle the challenging problem of point cloud completion from the perspective of feature learning. Our key observation is that to recover the underlying structures as well as surface details, given partial input, a fundamental component is a good feature representation that can capture both global structure and local geometric details. We accordingly first propose FSNet, a feature structuring module that can adaptively aggregate point-wise features into a 2D structured feature map by learning multiple latent patterns from local regions. We then integrate FSNet into a coarse-to-fine pipeline for point cloud completion. Specifically, a 2D convolutional neural network is adopted to decode feature maps from FSNet into a coarse and complete point cloud. Next, a point cloud upsampling network is used to generate a dense point cloud from the partial input and the coarse intermediate output. To efficiently exploit local structures and enhance point distribution uniformity, we propose IFNet, a point upsampling module with a self-correction mechanism that can progressively refine details of the generated dense point cloud. We have conducted qualitative and quantitative experiments on ShapeNet, MVP, and KITTI datasets, which demonstrate that our method outperforms state-of-the-art point cloud completion approaches.

According to the requirements of point cloud simplification for T-profile steel plate welding in shipbuilding, the disadvantages of the existing simplification algorithms are analyzed. In this paper, a point cloud simplification method is proposed based on octree coding and the threshold of the surface curvature feature. In this method, the original point cloud data are divided into multiple sub-cubes with specified side lengths by octree coding, and the points that are closest to the gravity center of the sub-cube are kept. The k-neighborhood method and the curvature calculation are performed in order to obtain the curvature features of the point cloud. Additionally, the point cloud data are divided into several regions based on the given adjustable curvature threshold. Finally, combining the random sampling method with the simplification method based on the regional gravity center, the T-profile point cloud data can be simplified. In this study, after obtaining the point cloud data of a T-profile plate, the proposed simplification method is compared with some other simplification methods. It is found that the proposed simplification method for the point cloud of the T-profile steel plate for shipbuilding is faster than the three existing simplification methods, while retaining more feature points and having approximately the same reduction rates.

Point cloud completion using multiscale feature fusion and cross-regional attention

The challenges of point cloud registration in intelligent vehicle driving lie in the large scale, complex distribution, high noise, and strong sparsity of lidar point cloud data. This paper proposes an efficient registration algorithm for large-scale outdoor road scenes by selecting the continuous distribution of key area laser point clouds as the registration point cloud. The algorithm extracts feature descriptions of the key point cloud and introduces local geometric features of the point cloud to complete rough and fine registration under constraints of key point clouds and point cloud features. The algorithm is verified through extensive experiments under multiple scenarios, with an average registration time of 0.5831 s and an average accuracy of 0.06996 m, showing significant improvement compared to other algorithms. The algorithm is also validated through real-vehicle experiments, demonstrating strong versatility, reliability, and efficiency. This research has the potential to improve environment perception capabilities of autonomous vehicles by solving the point cloud registration problem in large outdoor scenes.