Robust sharp features infer from point clouds

Sharp features of 3D point clouds play an important role in many geometric computations and modeling application. In this paper, a novel modified Partition of Unity (PoU) Based Sharp feature extraction algorithm is proposed, which is directly operated on discrete point clouds. For every point in target point cloud, spherical neighborhood with radius δ is acquired with the help of KD-Tree and weighted average position of points within the δ -neighborhood is computed using modified PoU method. Distance which is the projection of the displace between original point and its Weighted average position along normal direction is defined as the criteria for a point belong to sharp feature or crease line. Experiments on both synthetic data and practical scanner point clouds indicate that our algorithm are both efficient and effective to the task of sharp feature extraction from point clouds. Our method is easy to be implemented and more sensitive to sharp features as well as its low computational complexity.

Sharpness fields in point clouds using deep learning

3D point cloud data captured from Oil and Gas (O&G) facilities are becoming increasingly abundant. This data supports the interactive visualization of complex plants, which is central to numerous industrial advancements. As industries seek to innovate and optimize their processes, the versatile use of 3D point cloud data is proving invaluable beyond traditional O&G applications. The precision and comprehensiveness of 3D point cloud data facilitate efficient design, construction, maintenance and operational monitoring across various industrial sectors. Digital Twin (DT) solutions enable this by maintaining 3D representations, such as 3D CAD models, that accurately reflect the current state of facilities. As installations change, the process of 3D scanning is often repeated to capture the latest updates. Engineers and designers then use modelling software to visualize the resulting point cloud and update the facility's 3D representation, incorporating any new modifications. However, this involves interpreting the visual point cloud data and structuring it into meaningful objects (such as valves, tanks, etc.), a process that remains manual, requiring labor-intensive tasks that are prone to errors and incur significant costs due to the need for specialized labor. We propose an Artificial Intelligence approach, that uses a deep learning-based algorithm to automatically segment 3D objects within a point cloud. Our workflow recognizes instances of usual components of production systems, such as control valves, pipes, and transmitters. A state-of-the-art 3D instance segmentation approach is adapted to the O&G domain. Given the lack of publicly available instance segmentation datasets in this domain, we manually annotate 54 scenes from an O&G offshore plant, each of them around 20 m2 in footprint. To further enhance model training, we also incorporated synthetic point clouds. Practical results on real point clouds, acquired from operational facilities and pre-processed as required by commercial DT solutions, indicated a performance of approximately 70% success (f1-score) in segmenting components of O&G production systems, reaching up to 85% for components such as control valves. Accordingly, a 52.4% reduction in costs and efforts associated with manually interpreting 3D point clouds is estimated. Point cloud instance segmentation has attracted much attention in the computer vision research community, and many approaches have been proposed. However, results in the literature are heavily biased toward a few benchmarking datasets, which were acquired in indoor environments like offices and rooms. Few studies have addressed the O&G domain, creating a gap in the literature for achieving more competitive results. This lack of comprehensive research limits the potential for innovation and optimization in the industry. This paper contributes by (i) assessing the performance of state-of-the-art algorithms in the domain of O&G facilities, (ii) proposing modifications to better generalize to components of O&G production systems, (iii) objectively measuring the impact of this algorithm in reducing manual effort for engineers, and (iv) advancing towards a new DT functionality that derives an asset 3D representation, that is faithful to physical reality, through the automatic interpretation of point cloud scans.

Light Detection And Ranging (LiDAR) sensors can generate a number of sequential 3D point clouds, which are widely deployed in many real-world systems. 3D object detection in point clouds, is one of the most fundamental tasks. Unfortunately, the existing 3D object detection methods degrade in snowy weather, because in that situation the annotated samples are difficult to collect. To solve this issue, we propose a novel GAN-based Snowfall Point-cloud AugmentOR (GAN spaor ) to generate high-quality synthetic snowfall point clouds as augmentations. The basic idea of GAN spaor is to transfer annotated point clouds to snowfall versions by simultaneously learning the global style of real snowfall point clouds and the local details of physics-induced ones. Our framework fuses data-driven and physical modeling methods for rapidly generating data in snowy weather. To evaluate the effectiveness of GAN spaor , we employ a number of recent 3D object detection methods and train them by using the synthetic samples of GAN spaor as auxiliary augmentations. Moreover, we conduct a comparative analysis of the characteristics of the data distributions of the snowy point clouds synthesized by GAN spaor . Experimental results demonstrate that GAN spaor can improve the performance of 3D object detection methods compared with other existing snowfall point cloud simulators.

An adaptive normal estimation method for scanned point clouds with sharp features

Point cloud data, a flexible 3D object representation, is critical for various applications such as autonomous driving, robotics and remote sensing. Despite the recent success of deep neural networks (DNNs) on supervised point cloud analysis tasks, they still rely on tedious manual annotation of point clouds and cannot make predictions for new classes. Unlike few-shot learning for 2D images with the advantages of large-scale datasets and high-quality deep pre-trained models like ResNet, for 3D few-shot learning, obtaining discriminative representations of unseen classes with high intra-class similarity and inter-class difference is very challenging. To address this issue, this work proposes a novel cascade graph neural network for few-shot learning on point clouds, termed as CGNN, in which two cascade GNNs are adopted to extract the intra-object topological information and learn the inter-object relations respectively. To further increase the discriminability of point cloud features, we first design a novel discriminative edge label to model the intra-class similarity and inter-class dissimilarity based on channel-wise feature variance and class consistency. Second, we propose a novel few-shot circle loss which classifies the nodes into two subsets, i.e., support to support pairs and support to query pairs, and optimizes the pair-wise similarity on two subsets independently. Extensive experiments on benchmark CAD and real LiDAR point cloud datasets have demonstrated that CGNN improves accuracy by 5.98% over the state-of-the-art GNN-based few-shot classification methods.

Nowadays, mobile robot exploration needs a rangefinder to obtain a large number of measurement points to achieve a detailed and precise description of a surrounding area and objects, which is called the point cloud. However, a single point cloud scan does not cover the whole area, so multiple point cloud scans must be acquired and compared together to find the right matching between them in a process called registration method. This method requires further processing and places high demands on memory consumption, especially for small embedded devices in mobile robots. This paper describes a novel method to reduce the burden of processing for multiple point cloud scans. We introduce our approach to preprocess an input point cloud in order to detect planar surfaces, simplify space description, fill gaps in point clouds, and get important space features. All of these processes are achieved by applying advanced image processing methods in combination with the quantization of physical space points. The results show the reliability of our approach to detect close parallel walls with suitable parameter settings. More importantly, planar surface detection shows a 99% decrease in necessary descriptive points almost in all cases. This proposed approach is verified on the real indoor point clouds.

Abstract. This study introduces a novel image to a 3D point-cloud translation method with a conditional generative adversarial network that creates a large-scale 3D point cloud. This can generate supervised point clouds observed via airborne LiDAR from aerial images. The network is composed of an encoder to produce latent features of input images, generator to translate latent features to fake point clouds, and discriminator to classify false or real point clouds. The encoder is a pre-trained ResNet; to overcome the difficulty of generating 3D point clouds in an outdoor scene, we use a FoldingNet with features from ResNet. After a fixed number of iterations, our generator can produce fake point clouds that correspond to the input image. Experimental results show that our network can learn and generate certain point clouds using the data from the 2018 IEEE GRSS Data Fusion Contest.

Robust normal estimation for point clouds with sharp features

Three-dimensional information perception from point clouds is of vital importance for improving the ability of machines to understand the world, especially for autonomous driving and unmanned aerial vehicles. Data annotation for point clouds is one of the most challenging and costly tasks. In this paper, we propose a closed-loop and virtual–real interactive point cloud generation and model-upgrading framework called Parallel Point Clouds (PPCs). To our best knowledge, this is the first time that the training model has been changed from an open-loop to a closed-loop mechanism. The feedback from the evaluation results is used to update the training dataset, benefiting from the flexibility of artificial scenes. Under the framework, a point-based LiDAR simulation model is proposed, which greatly simplifies the scanning operation. Besides, a group-based placing method is put forward to integrate hybrid point clouds, via locating candidate positions for virtual objects in real scenes. Taking advantage of the CAD models and mobile LiDAR devices, two hybrid point cloud datasets, i.e., ShapeKITTI and MobilePointClouds, are built for 3D detection tasks. With almost zero labor cost on data annotation for newly added objects, the models (PointPillars) trained with ShapeKITTI and MobilePointClouds achieved 78.6% and 60.0% of the average precision of the model trained with real data on 3D detection, respectively.

Light detection and ranging (LiDAR) sensors provide accurate 3-D point clouds for noncooperative spacecraft pose estimation. Several robust methods, such as iterative closest point, exist to perform a local refinement of the pose starting from an initial estimate. However, finding the initial pose of the spacecraft is a global optimization problem, which is challenging to solve in real time. This is especially true on space hardware with limited computing power. In addition, many spacecrafts have a shape with multiple symmetries, making an unambiguous initial pose estimation impossible. This work introduces a convolutional-neural-network-based pose estimation method, accounting for potential symmetries of the target satellite. The point clouds are projected to a 2-D depth image before being processed by the network. To generate a sufficient amount of training data, a LiDAR simulator integrating multiple effects such as reflections or laser beam divergence is developed. While being trained solely on synthetic point clouds, the pose estimation method shows to be precise, efficient, and reliable when evaluated on real point clouds taken at a hardware-in-the-loop rendezvous test facility. A runtime evaluation on potential space computing hardware is also performed to demonstrate the applicability of the method to real-time onboard pose estimation.

ABSTRACTWe develop and evaluate a new individual tree detection (ITD) algorithm to automatically locate and estimate the number of individual trees within a Pinus radiata plantation from relatively sparse airborne LiDAR point cloud data. The area of interest comprised stands covering a range of age classes and stocking levels. Our approach is based on local maxima (LM) filtering that tackles the issue of selecting the optimal search radius from the LiDAR point cloud for every potential LM using metrics derived from local neighbourhood data points; thus, it adapts to the local conditions, irrespective of canopy variability. This was achieved through two steps: (i) logistic regression model development using simulated stands composed of individual trees derived from real LiDAR point cloud data and (ii) application testing of the model using real plantation LiDAR point cloud data and geolocated, tree-level reference crowns that were manually identified in the LiDAR imagery. Our ITD algorithm performed well compared with previous studies, producing RMSE of 5.7% and a bias of only −2.4%. Finally, we suggest that the ITD algorithm can be used for accurately estimating stocking and tree mapping, which in turn could be used to derive the plot-level metrics for an area-based approach for enhancing estimates of stand-level inventory attributes based on plot imputation.

Large-scale 3D-scanned point clouds enable the accurate and easy recording of complex 3D objects in the real world. The acquired point clouds often describe both the surficial and internal 3D structure of the scanned objects. The recently proposed edge-highlighted transparent visualization method is effective for recognizing the whole 3D structure of such point clouds. This visualization utilizes the degree of opacity for highlighting edges of the 3D-scanned objects, and it realizes clear transparent viewing of the entire 3D structures. However, for 3D-scanned point clouds, the quality of any edge-highlighting visualization depends on the distribution of the extracted edge points. Insufficient density, sparseness, or partial defects in the edge points can lead to unclear edge visualization. Therefore, in this paper, we propose a deep learning-based upsampling method focusing on the edge regions of 3D-scanned point clouds to generate more edge points during the 3D-edge upsampling task. The proposed upsampling network dramatically improves the point-distributional density, uniformity, and connectivity in the edge regions. The results on synthetic and scanned edge data show that our method can improve the percentage of edge points more than 15% compared to the existing point cloud upsampling network. Our upsampling network works well for both sharp and soft edges. A combined use with a noise-eliminating filter also works well. We demonstrate the effectiveness of our upsampling network by applying it to various real 3D-scanned point clouds. We also prove that the improved edge point distribution can improve the visibility of the edge-highlighted transparent visualization of complex 3D-scanned objects.

The identification of peeling off blocks is an important part of tunnel disease detection. In order to effectively solve the problem of accuracy of the identification of peeling off blocks in tunnels, this paper focuses on the use of the three-dimensional coordinates of the point cloud data obtained by the laser scanning system to realize the detection of peeling off blocks. A study of automatic detection of blocks. In the proposed automatic detection method for peeling off blocks, the original tunnel point cloud data is first filtered to remove outliers; then, the point cloud data is coordinately transformed to convert the cylindrical point cloud into a rectangular point cloud; the cloud constructs a triangular mesh, and by calculating the area of the triangular mesh and the projected area of the triangular mesh, the specific position of the peeling block can be obtained; finally, the point cloud is converted into an image, and the Hough transform is used to mark the pipeline and tunnel joints and get more intuitive detection results. This paper uses the real tunnel point cloud data to carry out the peeling off block detection experiment, and the results show the effectiveness and feasibility of the method.

Segmentation of large-scale masonry arch bridge point clouds with a synthetic simulator and the BridgeNet neural network