Surface reconstruction by layer peeling

Surface reconstruction by parallel and unified particle-based resampling from point clouds

Surface reconstruction from point clouds has been a well-studied research topic with applications in computer vision and computer graphics. Recently, several learningbased methods were proposed for 3D shape representation through implicit functions which among others can be used for point cloud-based reconstruction. Although delivering compelling results for synthetic object datasets of overseeable size, they fail to represent larger scenes accurately, presumably due to the use of only one global latent code for encoding an entire scene or object. We propose to encode only parts of objects with features attached to unstructured point clouds. To this end we use a hierarchical feature map in 3D space, extracted from the input point clouds, with which local latent shape encodings can be queried at arbitrary positions. We use a permutohedral lattice to process the hierarchical feature maps sparsely and efficiently. This enables accurate and detailed point cloud-based reconstructions for large amounts of points in a time-efficient manner, showing good generalization capabilities across different datasets. Experiments on synthetic and real world datasets demonstrate the reconstruction capability of our method and compare favorably to state-of-the-art methods.

We propose a level set-based semi-Lagrangian method on graded adaptive Cartesian grids to address the problem of surface reconstruction from point clouds. The goal is to obtain an implicit, high-quality representation of real shapes that can subsequently serve as computational domain for partial differential equation models. The mathematical formulation is variational, incorporating a curvature constraint that minimizes the surface area while being weighted by the distance of the reconstructed surface from the input point cloud. Within the level set framework, this problem is reformulated as an advection–diffusion equation, which we solve using a semi-Lagrangian scheme coupled with a local high-order interpolator. Building on the features of the level set and semi-Lagrangian method, we use quadtree and octree data structures to represent the grid and generate a mesh with the finest resolution near the zero level set and the point cloud data. The complete surface reconstruction workflow is described, including localization and reinitialization techniques, as well as strategies to handle complex and evolving topologies. A broad set of numerical tests in two and three dimensions is presented to assess the effectiveness of the method.

In reverse engineering, the point cloud data processing is the key, it will directly affect the quality and efficiency of the subsequent reconstruction of the surface. However, the amount of data obtained by three-dimensional laser scanning is often larger and redundant, which will increase the difficulty of the surface reconstruction, therefore, the point cloud data is needed to streamline. Depth analysis of the efficiency and accuracy of curvature grid method, a secondary streamlined algorithm is proposed, which take use of non-uniform grid streamline law based on octree to streamline to the point cloud, and re-use curvature streamline method based on B-spline surface fitting to streamline it. This method, whose streamline ratio is relatively high, can effectively retain the feature point cloud data, providing a great convenience for the follow-up of reverse engineering.

Surface reconstruction is a challenging task when input point clouds, especially real scans, are noisy and lack normals. Observing that the Multilayer Perceptron (MLP) and the implicit moving least-square function (IMLS) provide a dual representation of the underlying surface, we introduce Neural-IMLS, a novel approach that directly learns a noise-resistant signed distance function (SDF) from unoriented raw point clouds in a self-supervised manner. In particular, IMLS regularizes MLP by providing estimated SDFs near the surface and helps enhance its ability to represent geometric details and sharp features, while MLP regularizes IMLS by providing estimated normals. We prove that at convergence, our neural network produces a faithful SDF whose zero-level set approximates the underlying surface due to the mutual learning mechanism between the MLP and the IMLS. Extensive experiments on various benchmarks, including synthetic and real scans, show that Neural-IMLS can reconstruct faithful shapes even with noise and missing parts. The source code can be found at https://github.com/bearprin/Neural-IMLS.

Abstract. Laser scanning samples the surface geometry of objects efficiently and records versatile information as point clouds. However, often more scans are required to fully cover a scene. Therefore, a registration step is required that transforms the different scans into a common coordinate system. The registration of point clouds is usually conducted in two steps, i.e. coarse registration followed by fine registration. In this study an automatic marker-free coarse registration method for pair-wise scans is presented. First the two input point clouds are re-sampled as voxels and dimensionality features of the voxels are determined by principal component analysis (PCA). Then voxel cells with the same dimensionality are clustered. Next, the Extended Gaussian Image (EGI) descriptor of those voxel clusters are constructed using significant eigenvectors of each voxel in the cluster. Correspondences between clusters in source and target data are obtained according to the similarity between their EGI descriptors. The random sampling consensus (RANSAC) algorithm is employed to remove outlying correspondences until a coarse alignment is obtained. If necessary, a fine registration is performed in a final step. This new method is illustrated on scan data sampling two indoor scenarios. The results of the tests are evaluated by computing the point to point distance between the two input point clouds. The presented two tests resulted in mean distances of 7.6 mm and 9.5 mm respectively, which are adequate for fine registration.

Skeleton-based intrinsic symmetry detection on point clouds

The contours, one of the most significant human perceptual features, have a significant impact on point cloud processing. In urban scenes, contour extraction is quite challenging due to the enormous number of unstructured and irregular points (typically greater than 10 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">7</sup> points). In this paper, we propose a Large-scale 3D point cloud Contour Extraction Network (LCE-NET) to generate contours consistent with human perception of outdoor scenes. To our knowledge, it is the first time that an end-to-end learning-based framework has been proposed for contour extraction on point cloud at this scale. The proposed LCE-Net is essentially a two-phase system. In the first phase, potential vertexes are detected from the input point cloud by the vertex detection module, then in the second phase, a designed overcomplete line proposal set is generated, and invalid line segments are further suppressed by the line proposal discrimination module. The two phases are jointly trained by a uniform loss function to promote the information interchange, thus leading to extraction results with satisfied accurate and false alarm ratings. Since there is hardly any available dataset with labeled contours for the large-scale outdoor scene, we open sourced SemanticLine, the first dataset for large-scale point clouds with labeled contour information, based on re-annotation of previous mapping level point cloud dataset semantic3D. Experimental results demonstrate that LCE-NET can effectively extract parametric contour lines from large-scale point clouds of urban scenes. Additionally, it outperforms the state-of-the-art approaches. The code will be open source on GitHub soon.

In this paper, we address the problem of curve and surface reconstruction from sets of points. We introduce regular interpolants which are polygonal approximations of planar curves and surfaces verifying a local sampling criterion. Properties of regular interpolants lead to new polygonal reconstruction methods from sets of organized and unorganized points. These methods do not need any parameter of additional information apart from the original points and allow unorganized sets of points to be easily handled.

Self-supervised rigid transformation equivariance for accurate 3D point cloud registration

Integrated gradients is an explainable AI technique that aims to explain the relationship between a model’s predictions in terms of its features. Adapting this technique to point clouds and semantic segmentation models allows a class-wise attribution of the predictions with respect to the input features. This allows better insight into how a model reached a prediction. Furthermore, it allows a quantitative analysis of how much each feature contributes to a prediction. To obtain these attributions, a baseline with high entropy is generated and interpolated with the point cloud to be visualized. These interpolated point clouds are then run through the network and their gradients are collected. By observing the change in gradients during each iteration an attribution can be found for each input feature. These can then be projected back onto the original point cloud and compared to the predictions and input point cloud. These attributions are generated using RandLA-Net due to it being an efficient semantic segmentation model that uses comparatively few parameters, therefore keeping the number of gradients that must be stored at a reasonable level. The attribution was run on the public Semantic3D dataset and the SVGEO large-scale urban dataset.

Chimneys are architectural elements of difficult access and analysis in heritage buildings. This paper presents a method for automatic modelling of round brick chimneys and analysing their inclination, straightness and surface. The method uses as input data point clouds acquired with a Terrestrial Laser Scanner. Four chimneys have been surveyed on the north face of Bragança Dukes’ Palace in Guimarães, Portugal. The method consists of two main processes. First, the point cloud of each chimney is segmented into circumferential sections and adjusted to cylinders by applying Maximum Likelihood Sample Consensus. From the set of centroids, the inclination and the straightness of the chimney is estimated. Second, the point cloud chimney is segmented longitudinally and adjusted to finite planes. The distance estimation between the input point cloud and the modelled planes allows detecting the lack of bricks or mortar erosions. The proposed method allows estimating characteristics in chimneys even if the chimneys have not been completely acquired.

Coarse-to-fine pipeline for 3D wireframe reconstruction from point cloud

Deep learning on point clouds has attracted increasing attention in the fields of 3D computer vision and robotics. In particular, graph-based point-cloud deep neural networks (DNNs) have demonstrated promising performance in 3D object classification and scene segmentation tasks. However, the scattered and irregular graph-structured data in a graph-based point-cloud DNN cannot be computed efficiently by existing SIMD architectures and accelerators. We present Point-X, an energy-efficient accelerator architecture that extracts and exploits the spatial locality in point cloud data for efficient processing. Point-X uses a clustering method to extract fine-grained and coarse-grained spatial locality from the input point cloud. The clustering maps the point cloud into distributed compute tiles to maximize intra-tile computational parallelism and minimize inter-tile data movement. Point-X employs a chain network-on-chip (NoC) to further reduce the NoC traffic and achieve up to 3.2 × speedup over a traditional mesh NoC. Point-X’s multi-mode dataflow can support all common operations in a graph-based point-cloud DNN, i.e., edge convolution, shared multi-layer perceptron, and fully-connected layers. Point-X is synthesized in a 28nm technology and it demonstrates a throughput of 1307.1 inference/s and an energy efficiency of 604.5 inference/J on the DGCNN workload. Compared to the Nvidia GTX-1080Ti GPU, Point-X shows 4.5 × and 342.9 × improvement in throughput and efficiency, respectively.

We address an actively discussed problem in signal processing, recognizing patterns from spatial data in motion. In particular, we suggest a neural network architecture to recognize motion patterns from 4D point clouds. We demonstrate the feasibility of our approach with point cloud datasets of hand gestures. The architecture, PointGest, directly feeds on unprocessed timelines of point cloud data without any need for voxelization or projection. The model is resilient to noise in the input point cloud through abstraction to lower-density representations, especially for regions of high density. We evaluate the architecture on a benchmark dataset with ten gestures. PointGest achieves an accuracy of 98.8%, outperforming five state-of-the-art point cloud classification models.