Mutual Information Maximization Based Similarity Operation for 3D Point Cloud Completion Network

Point cloud completion aims to reconstruct the complete 3D shape from incomplete point clouds, and it is crucial for tasks such as 3D object detection and segmentation. Despite the continuous advances in point cloud analysis techniques, feature extraction methods are still confronted with apparent limitations. The sparse sampling of point clouds, used as inputs in most methods, often results in a certain loss of global structure information. Meanwhile, traditional local feature extraction methods usually struggle to capture the intricate geometric details. To overcome these drawbacks, we introduce PointCFormer, a transformer framework optimized for robust global retention and precise local detail capture in point cloud completion. This framework embraces several key advantages. First, we propose a relation-based local feature extraction method to perceive local delicate geometry characteristics. This approach establishes a fine-grained relationship metric between the target point and its k-nearest neighbors, quantifying each neighboring point's contribution to the target point's local features. Secondly, we introduce a progressive feature extractor that integrates our local feature perception method with self-attention. Starting with a denser sampling of points as input, it iteratively queries long-distance global dependencies and local neighborhood relationships. This extractor maintains enhanced global structure and refined local details, without generating substantial computational overhead. Additionally, we develop a correction module after generating point proxies in the latent space to reintroduce denser information from the input points, enhancing the representation capability of the point proxies. PointCFormer demonstrates state-of-the-art performance on several widely used benchmarks.

Point cloud completion is an interesting and challenging task in 3D vision, aiming to recover complete shapes from sparse and incomplete point clouds. Existing learning-based methods often require vast computation cost to achieve excellent performance, which limits their practical applications. In this paper, we propose a novel Recurrent Forward Network (RFNet), which is composed of three modules: Recurrent Feature Extraction (RFE), Forward Dense Completion (FDC) and Raw Shape Protection (RSP). The RFE extracts multiple global features from the incomplete point clouds for different recurrent levels, and the FDC generates point clouds in a coarse-to-fine pipeline. The RSP introduces details from the original incomplete models to refine the completion results. Besides, we propose a Sampling Chamfer Distance to better capture the shapes of models and a new Balanced Expansion Constraint to restrict the expansion distances from coarse to fine. According to the experiments on ShapeNet and KITTI, our network can achieve the state-of-the-art with lower memory cost and faster convergence.

SPC: Self-supervised point cloud completion.

In recent years, due to the significant advancements in hardware sensors and software technologies, 3D environmental point cloud modeling has gradually been applied in the automation industry, autonomous vehicles, and construction engineering. With the high-precision measurements of 3D LiDAR, its point clouds can clearly reflect the geometric structure and features of the environment, thus enabling the creation of high-density 3D environmental point cloud models. However, due to the enormous quantity of high-density 3D point clouds, storing and processing these 3D data requires a considerable amount of memory and computing time. In light of this, this paper proposes a real-time 3D point cloud environmental contour modeling technique. The study uses the point cloud distribution from the 3D LiDAR body frame point cloud to establish structured edge features, thereby creating a 3D environmental contour point cloud map. Additionally, unstable objects such as vehicles will appear during the mapping process; these specific objects will be regarded as not part of the stable environmental model in this study. To address this issue, the study will further remove these objects from the 3D point cloud through image recognition and LiDAR heterogeneous matching, resulting in a higher quality 3D environmental contour point cloud map. This 3D environmental contour point cloud not only retains the recognizability of the environmental structure but also solves the problems of massive data storage and processing. Moreover, the method proposed in this study can achieve real-time realization without requiring the 3D point cloud to be organized in a structured order, making it applicable to unorganized 3D point cloud LiDAR sensors. Finally, the feasibility of the proposed method in practical applications is also verified through actual experimental data.

Point cloud completion is essential for restoring 3D geometric data lost due to occlusions or sensor limitations. Existing methods often rely on k-nearest neighbor(KNN)-based local feature extraction, which focuses on neighborhoods around central points while neglecting critical global structural information. Additionally, Transformer-based approaches, while effective at modeling global context, typically use central point feature sequences to reduce computational complexity. This windowed attention strategy compromises the preservation of global context, leading to incomplete modeling of the point cloud's overall structure. To address these challenges, we propose SDA-Net, a dual-attention point cloud completion network utilizing multiple serialization strategies. These strategies transform unordered point clouds into structured sequences, enabling comprehensive modeling of inter-point relationships. Additionally, the dual-attention mechanism enhances global feature extraction through complementary spatial and channel-wise self-attention, effectively compensating for the loss of global context. Extensive experiments demonstrate that SDA-Net achieves state-of-the-art performance, including an average Chamfer Distance (CD) of 6.48 on the PCN dataset. Furthermore, it excels in real-world applications, accurately reconstructing fine-grained details in LiDAR-scanned point clouds.

3D point clouds are gradually becoming more widely used in the medical field, however, they are rarely used for 3D representation of intracranial vessels and aneurysms due to the time-consuming data reconstruction. In this paper, we simulate the incomplete intracranial vessels (including aneurysms) in the actual collection from different angles, then propose Multi-Scope Feature Extraction Network (MSENet) for Intracranial Aneurysm 3D Point Cloud Completion. MSENet adopts a multi-scope feature extraction encoder to extract the global features from the incomplete point cloud. This encoder utilizes different scopes to fuse the neighborhood information for each point fully. Then a folding-based decoder is applied to obtain the complete 3D shape. To enable the decoder to intuitively match the original geometric structure, we engage the original points coordinates input to perform residual linking. Finally, we merge and sample the complete but coarse point cloud from the decoder to obtain the final refined complete 3D point cloud shape. We conduct extensive experiments on both 3D intracranial aneurysm datasets and general 3D vision PCN datasets. The results demonstrate the effectiveness of the proposed method on three evaluation metrics compared to baseline: our model increases the F-score to ()/ (), reduces Chamfer Distance score to ()/ (), and reduces the Earth Mover’s Distance to ()/ ().

Point cloud data in the real world is often affected by occlusion and light reflection, leading to incompleteness of the data. Large‐region missing point clouds will cause great deviations in downstream tasks. A dual feature fusion network (DFF‐Net) is proposed to improve the accuracy of the completion of a large missing region of the point cloud. First, a dual feature encoder is designed to extract and fuse the global and local features of the input point cloud. Subsequently, a decoder is used to directly generate a point cloud of missing region that retains local details. In order to make the generated point cloud more detailed, a loss function with multiple terms is employed to emphasise the distribution density and visual quality of the generated point cloud. A large number of experiments show that the authors’ DFF‐Net is better than the previous state‐of‐the‐art methods in the aspect of point cloud completion.

DcTr: Noise-robust point cloud completion by dual-channel transformer with cross-attention

3D vehicle shape completion is a key task in urban environment reconstruction, since the available sensors and 3D scanning procedures (such as Mobile Laser Scanning) in an outdoor city scene cannot usually extract the entire car shapes, which can be necessary for specifying their geometric properties for further analysis or realistic visualization. In this paper, we propose a novel multi-view based 3D object point cloud completion technique. In contrast to existing approaches, our method operates on 2D images formed by projecting the point cloud from several virtual camera positions around the object of interest. Both color and geometrical information is considered during the process, generating dense textured point clouds, displaying realistic patterns in the missing regions from the partial inputs. We present both quantitative and qualitative tests on various synthetic and real laser scanned vehicle point clouds, which demonstrate that our method surpasses existing state-of-the-art approaches. By applying it to vehicles from the Shapenet dataset, our approach outperforms recent techniques in terms of Earth Mover’s Distance (EMD) and Chamfer Distance (CD) by 43.8% and 12.17%, respectively.

In this paper, we tackle the challenging problem of point cloud completion from the perspective of feature learning. Our key observation is that to recover the underlying structures as well as surface details, given partial input, a fundamental component is a good feature representation that can capture both global structure and local geometric details. We accordingly first propose FSNet, a feature structuring module that can adaptively aggregate point-wise features into a 2D structured feature map by learning multiple latent patterns from local regions. We then integrate FSNet into a coarse-to-fine pipeline for point cloud completion. Specifically, a 2D convolutional neural network is adopted to decode feature maps from FSNet into a coarse and complete point cloud. Next, a point cloud upsampling network is used to generate a dense point cloud from the partial input and the coarse intermediate output. To efficiently exploit local structures and enhance point distribution uniformity, we propose IFNet, a point upsampling module with a self-correction mechanism that can progressively refine details of the generated dense point cloud. We have conducted qualitative and quantitative experiments on ShapeNet, MVP, and KITTI datasets, which demonstrate that our method outperforms state-of-the-art point cloud completion approaches.

By formulating the data generation as a sequence procedure of denoising autoencoding, diffusion models have achieved superior in-painting performance on image data and beyond. Nevertheless, it is not trivial when capitalizing on diffusion models to generate missing 3D points. The difficulty originates from the intrinsic structure where 3D point cloud is a set of unordered and irregular coordinates. That motivates us to delve into the 3D structural information for designing point cloud encoder-decoder and shape latent generator, to precisely formulate the latent distribution of the complete point cloud and partial observation. In this paper, we propose Point cloud completion with Latent Diffusion Models (PointLDM), a new approach that leverages the conditional denoising diffusion probabilistic modeling (DDPM) in the 3D latent space for shape reconstruction. The architecture of PointLDM consists of a transformer-based variational auto-encoder (VAE) to model the complete shape latent, and a diffusion network for shape latent prediction. The encoder of VAE exploits both of global shape latent and local point features in shape distribution learning. With the learnt shape latent, the decoder first decodes the shape latent into coarse points, and then recovers the fine-grained details around each coarse point by deforming a 2D grid. To reconstruct the shape latent from partial observation, the diffusion network treats the partial observation as the conditional input and generates the shape latent via DDPM. Extensive experiments conducted on MVP, Completion3D, and KITTI quantitatively and qualitatively demonstrate the efficacy of PointLDM over the state-of-the-art shape completion approaches.

Three-dimensional (3D) point clouds have a wide range of applications in the field of 3D vision. The quality of the acquired point cloud data considerably impacts the subsequent work of point cloud processing. Due to the sparsity and irregularity of point cloud data, processing point cloud data has always been challenging. However, existing deep learning-based point cloud dense reconstruction methods suffer from excessive smoothing of reconstruction results and too many outliers. The reason for this is that it is not possible to extract features for local and global features at different scales and provide different levels of attention to different regions in order to obtain long-distance dependence for dense reconstruction. In this paper, we use a parallel multi-scale feature extraction module based on graph convolution and an upsampling method with an added multi-head attention mechanism to process sparse and irregular point cloud data to obtain extended point clouds. Specifically, a point cloud training patch with 256 points is inputted. The PMS module uses three residual connections in the multi-scale feature extraction stage. Each PMS module consists of three parallel DenseGCN modules with different size convolution kernels and different averaging pooling sizes. The local and global feature information of the augmented receptive field is extracted efficiently. The scale information is obtained by averaging the different pooled augmented receptive fields. The scale information was obtained using the different average pooled augmented receptive fields. The upsampling stage uses an upsampling rate of r=4, The self-attentive features with a different focus on different point cloud data regions obtained by fusing different weights make the feature representation more diverse. This operation avoids the bias of one attention, and each focuses on extracting valuable fine-grained feature information. Finally, the coordinate reconstruction module obtains 1024 dense point cloud data. Experiments show that the proposed method demonstrates good evaluation metrics and performance and is able to obtain better visual quality. The problems of over-smoothing and excessive outliers are effectively mitigated, and the obtained sparse point cloud is more dense.

Point cloud completion recovers the complete point clouds from partial ones, providing numerous point cloud information for downstream tasks such as 3-D reconstruction and target detection. However, previous methods usually suffer from unstructured prediction of points in local regions and the discrete nature of the point cloud. To resolve these problems, we propose a point cloud completion network called TPDC. Representing the point cloud as a set of unordered features of points with local geometric information, we devise a Triangular Pyramid Extractor (TPE), using the simplest 3-D structure-a triangular pyramid-to convert the point cloud to a sequence of local geometric information. Our insight of revealing local geometric information in a complex environment is to design a Divide-and-Conquer Splitting Module in a Divide-and-Conquer Splitting Decoder (DCSD) to learn point-splitting patterns that can fit local regions the best. This module employs the Divide-and-Conquer approach to parallelly handle tasks related to fitting ground-truth values to base points and predicting the displacement of split points. This approach aims to make the base points align more closely with the ground-truth values while also forecasting the displacement of split points relative to the base points. Furthermore, we propose a more realistic and challenging benchmark, ShapeNetMask, with more random point cloud input, more complex random item occlusion, and more realistic random environmental perturbations. The results show that our method outperforms both widely used benchmarks as well as the new benchmark.

3D human mesh recovery from point clouds is essential for various tasks, including AR/VR and human behavior understanding. Previous works in this field either require high-quality 3D human scans or sequential point clouds, which cannot be easily applied to low-quality 3D scans captured by consumer-level depth sensors. In this paper, we make the first attempt to reconstruct reliable 3D human shapes from single-frame partial point clouds. To achieve this, we propose an end-to-end learnable method, named VoteHMR. The core of VoteHMR is a novel occlusion-aware voting network that can first reliably produce visible joint-level features from the input partial point clouds, and then complete the joint-level features through the kinematic tree of the human skeleton. Compared with holistic features used by previous works, the joint-level features can not only effectively encode the human geometry information but also be robust to noisy inputs with self-occlusions and missing areas. By exploiting the rich complementary clues from the joint-level features and global features from the input point clouds, the proposed method encourages reliable and disentangled parameter predictions for statistical 3D human models, such as SMPL. The proposed method achieves state-of-the-art performances on two large-scale datasets, namely SURREAL and DFAUST. Furthermore, VoteHMR also demonstrates superior generalization ability on real-world datasets, such as Berkeley MHAD.

Point cloud is the popular researches in the industry, and the research on partial point cloud is favored by many researchers. In the acquisition process of original 3D point cloud data, there are partial or sparse problems due to occlusion, lighting and other reasons, which will lead to deviation in downstream tasks. We propose a network MDPCN using multiple decoders, which can make the partial point cloud complete. The network uses deep learning to get multiple features of the partial point cloud, and uses multiple identical decoders to decode. Each decoder uses the self-attention module and the Folding-Net to generate smooth point clouds from the features, and finally integrates the point clouds generated by each decoder to obtain dense point cloud. Compared with other mainstream point cloud completion network methods and ablation experiments, it can be proved that this network model can generate more accurate point cloud and effectively complete the 3D reconstruction task.