Advanced Methods for Point Cloud Processing and Simplification

3D point cloud data captured from Oil and Gas (O&G) facilities are becoming increasingly abundant. This data supports the interactive visualization of complex plants, which is central to numerous industrial advancements. As industries seek to innovate and optimize their processes, the versatile use of 3D point cloud data is proving invaluable beyond traditional O&G applications. The precision and comprehensiveness of 3D point cloud data facilitate efficient design, construction, maintenance and operational monitoring across various industrial sectors. Digital Twin (DT) solutions enable this by maintaining 3D representations, such as 3D CAD models, that accurately reflect the current state of facilities. As installations change, the process of 3D scanning is often repeated to capture the latest updates. Engineers and designers then use modelling software to visualize the resulting point cloud and update the facility's 3D representation, incorporating any new modifications. However, this involves interpreting the visual point cloud data and structuring it into meaningful objects (such as valves, tanks, etc.), a process that remains manual, requiring labor-intensive tasks that are prone to errors and incur significant costs due to the need for specialized labor. We propose an Artificial Intelligence approach, that uses a deep learning-based algorithm to automatically segment 3D objects within a point cloud. Our workflow recognizes instances of usual components of production systems, such as control valves, pipes, and transmitters. A state-of-the-art 3D instance segmentation approach is adapted to the O&G domain. Given the lack of publicly available instance segmentation datasets in this domain, we manually annotate 54 scenes from an O&G offshore plant, each of them around 20 m2 in footprint. To further enhance model training, we also incorporated synthetic point clouds. Practical results on real point clouds, acquired from operational facilities and pre-processed as required by commercial DT solutions, indicated a performance of approximately 70% success (f1-score) in segmenting components of O&G production systems, reaching up to 85% for components such as control valves. Accordingly, a 52.4% reduction in costs and efforts associated with manually interpreting 3D point clouds is estimated. Point cloud instance segmentation has attracted much attention in the computer vision research community, and many approaches have been proposed. However, results in the literature are heavily biased toward a few benchmarking datasets, which were acquired in indoor environments like offices and rooms. Few studies have addressed the O&G domain, creating a gap in the literature for achieving more competitive results. This lack of comprehensive research limits the potential for innovation and optimization in the industry. This paper contributes by (i) assessing the performance of state-of-the-art algorithms in the domain of O&G facilities, (ii) proposing modifications to better generalize to components of O&G production systems, (iii) objectively measuring the impact of this algorithm in reducing manual effort for engineers, and (iv) advancing towards a new DT functionality that derives an asset 3D representation, that is faithful to physical reality, through the automatic interpretation of point cloud scans.

Abstract. The automatic and accurate alignment of multiple point clouds is a basic requirement for an adequate digitization, reconstruction and interpretation of large 3D environments. Due to the recent technological advancements, modern devices are available which allow for simultaneously capturing intensity and range images with high update rates. Hence, such devices can even be used for dynamic scene analysis and for rapid mapping which is particularly required for environmental applications and disaster management, but unfortunately, they also reveal severe restrictions. Facing challenges with respect to noisy range measurements, a limited non-ambiguous range, a limited field of view and the occurrence of scene dynamics, the adequate alignment of captured point clouds has to satisfy additional constraints compared to the classical registration of terrestrial laser scanning (TLS) point clouds for describing static scenes. In this paper, we propose a new methodology for point cloud registration which considers such constraints while maintaining the fundamental properties of high accuracy and low computational effort without relying on a good initial alignment or human interaction. Exploiting 2D image features and 2D/2D correspondences, sparse point clouds of physically almost identical 3D points are derived. Subsequently, these point clouds are aligned with a fast procedure directly taking into account the reliability of the detected correspondences with respect to geometric and radiometric information. The proposed methodology is evaluated and its performance is demonstrated for data captured with a moving sensor platform which has been designed for monitoring from low altitudes. Due to the provided reliability and a fast processing scheme, the proposed methodology offers a high potential for dynamic scene capture and analysis.

The 3D laser scanning technology is a hot spot in developed measuring in recent years. In the surface reconstruction of reverse engineering, the 3D laser scanning point cloud data is too large, and is not conducive to the computation, storage and surface reconstruction. After understanding the research status of the point cloud data processing streamline method at home and abroad, and through the analysis of minimum distance algorithm and the angle-chord height combined code method applicable to engineering characteristics, at the same time, the combination algorithm, which is based on the minimum distance algorithm and the angle-chord height combined code method, is proposed to simplify the point cloud data. The scanning point cloud is simplified by using matlab line by line.

The density of acquired data by 3D laser scanner is huge and is not convenient for curved face reconstruction, model reconstruction, data storage, data display and transmission.Firstly, the conventional point cloud data classification and simplification method is introduced.Secondly, the angle deviation error and chord deviation error simplification method is researched based on the analysis of characteristic of 3D laser scan point cloud data.Finally, the angle--chord comprehensive simplification method is prompted.The example shows that this method could simplify the scan line point cloud data got by 3D laser scanner directly and effectively.This method could retain the point cloud data which reflects the surface character of object with higher simplification efficiency.

According to the requirements of point cloud simplification for T-profile steel plate welding in shipbuilding, the disadvantages of the existing simplification algorithms are analyzed. In this paper, a point cloud simplification method is proposed based on octree coding and the threshold of the surface curvature feature. In this method, the original point cloud data are divided into multiple sub-cubes with specified side lengths by octree coding, and the points that are closest to the gravity center of the sub-cube are kept. The k-neighborhood method and the curvature calculation are performed in order to obtain the curvature features of the point cloud. Additionally, the point cloud data are divided into several regions based on the given adjustable curvature threshold. Finally, combining the random sampling method with the simplification method based on the regional gravity center, the T-profile point cloud data can be simplified. In this study, after obtaining the point cloud data of a T-profile plate, the proposed simplification method is compared with some other simplification methods. It is found that the proposed simplification method for the point cloud of the T-profile steel plate for shipbuilding is faster than the three existing simplification methods, while retaining more feature points and having approximately the same reduction rates.

Three-dimensional (3D) mapping deals with the problem of building a map of the unknown environments explored by a mobile robot. In contrast to 2D maps, 3D maps contain richer information of the visited places. Besides enabling robot navigation in 3D, a 3D map of the robot surroundings could be of great importance for higher-level robotic tasks, like scene interpretation and object interaction or manipulation, as well as for visualization purposes in general, which are required in surveillance, urban search and rescue, surveying, and others. Hence, the goal of this thesis is to develop a system which is capable of reconstructing the surrounding environment of a mobile robot as a three-dimensional map. Microsoft Kinect camera is a novel sensing sensor that captures dense depth images along with RGB images at high frame rate. Recently, it has dominated the stage of 3D robotic sensing, as it is low-cost, low-power. For this work, it is used as the exteroceptive sensor and obtains 3D point clouds of the surrounding environment. Meanwhile, the wheel odometry of the robot is used to initialize the search for correspondences between different observations. As a single 3D point cloud generated by the Microsoft Kinect sensor is composed of many tens of thousands of data points, it is necessary to compress the raw data to process them efficiently. The method chosen in this work is to use a feature-based representation which simplifies the 3D mapping procedure. The chosen features are planar surfaces and orthogonal corners, which is based on the fact that indoor environments are designed such that walls, ground floors, pillars, and other major parts of the building structures can be modeled as planar surface patches, which are parallel or perpendicular to each other. While orthogonal corners are presented as higher features which are more distinguishable in indoor environment. In this thesis, the main idea is to obtain spatial constraints between pairwise frames by building correspondences between the extracted vertical plane features and corner features. A plane matching algorithm is presented that maximizes the similarity metric between a pair of planes within a search space to determine correspondences between planes. The corner matching result is based on the plane matching results. The estimated spatial constraints form the edges of a pose graph, referred to as graph-based SLAM front-end. In order to build a map, however, a robot must be able to recognize places that it has previously visited. Limitations in sensor processing problem, coupled with environmental ambiguity, make this difficult. In this thesis, we describe a loop closure detection algorithm by compressing point clouds into viewpoint feature histograms, inspired by their strong recognition ability. The estimated roto-translation between detected loop frames is added to the graph representing this newly discovered constraint. Due to the estimation errors, the estimated edges form a non-globally consistent trajectory. With the aid of a linear pose graph optimizing algorithm, the most likely configuration of the robot poses can be estimated given the edges of the graph, referred to as SLAM back-end. Finally, the 3D map is retrieved by attaching each acquired point cloud to the corresponding pose estimate. The approach is validated through different experiments with a mobile robot in an indoor environment

There is no single sensor can acquire the complete information for disaster monitoring. This study investigates the applicability of registering multiple point clouds obtained from unmanned aerial vehicle (UAV) images and terrestrial laser scanning (TLS). Low attitude images with high overlaps were collected by an eight-rotor UAV platform and image-based 3D modeling techniques are used to generate 3D point cloud, covering most of roof information of the damaged buildings. TLS was used to collect the side information of the damaged buildings with multiple scans. Point clouds from the two platforms are iteratively registered using a method, from coarse to fine, to get complete geometry of the study area. Geometric features are subsequently extracted to help for the identification of damage degree of buildings. Experimental result shows that by analyzing the intersection lines of plane features, we can further detect the building's inclination.

The effective extraction of shape features is an important requirement for the accurate and efficient classification of terrestrial laser scanning (TLS) point clouds. However, the challenge of how to obtain robust and discriminative features from noisy and varying density TLS point clouds remains. This paper introduces a novel multiscale and hierarchical framework, which describes the classification of TLS point clouds of cluttered urban scenes. In this framework, we propose multiscale and hierarchical point clusters (MHPCs). In MHPCs, point clouds are first resampled into different scales. Then, the resampled data set of each scale is aggregated into several hierarchical point clusters, where the point cloud of all scales in each level is termed a point-cluster set. This representation not only accounts for the multiscale properties of point clouds but also well captures their hierarchical structures. Based on the MHPCs, novel features of point clusters are constructed by employing the latent Dirichlet allocation (LDA). An LDA model is trained according to a training set. The LDA model then extracts a set of latent topics, i.e., a feature of topics, for a point cluster. Finally, to apply the introduced features for point-cluster classification, we train an AdaBoost classifier in each point-cluster set and obtain the corresponding classifiers to separate the TLS point clouds with varying point density and data missing into semantic regions. Compared with other methods, our features achieve the best classification results for buildings, trees, people, and cars from TLS point clouds, particularly for small and moving objects, such as people and cars.

In the process of using structured light technology to obtain indoor point clouds, due to the limited field of view of the device, it is necessary to obtain multiple point clouds of different wall surfaces. Therefore, merging the point cloud is necessary to acquire a complete point cloud. However, due to issues such as the sparse geometric features of the wall point clouds and the high similarity of multiple point clouds, the merging effect of point clouds is poor. In this paper, we leverage artificially enhanced features to improve the accuracy of registration in this scenario. Firstly, we design feature markers and present their layout criteria. Then, the feature information of the marker is extracted based on the Color Signature of Histograms of OrienTations (Color-SHOT) descriptor, and coarse registration is realized through the second-order similarity measure matrix. After that, precise registration is achieved using the Iterative Closest Point (ICP) method based on markers and overlapping areas. Finally, the global error of the point cloud registration is optimized by loop error averaging. Our method enables the high-precision reconstruction of integrated home design scenes lacking significant features at a low cost. The accuracy and validity of the method were verified through comparative experiments.

This paper presents a new and fully automatic method to extract cross-sectional contour profiles of a physical object from the point cloud data scanned from its surface. Correctly extracting the sectional contours is of particular importance in the quality inspection of airfoil blades as the tolerances specified on a manufactured aero-engine blade are generally imposed at specific blade sections. The collected point cloud via 3D laser scanning is, however, distributed all over the blade surface rather than at the desired specific sections. In fact, no point in the point cloud is located exactly on the sectional planes. The desired sectional data have to be extracted from the nearby data points. If the underlying smooth surface geometry of the point cloud in the vicinity of a nearby data point can be approximated by a mathematical function, the approximated local surface formulation can be used to project the nearby point onto the desired sectional plane along a curvilinear trajectory. This is achieved in this work by fitting a local quadric surface to the neighbouring points of the point of interest. A systematic approach to establish a balanced set of neighbouring points is employed to avoid bias in fitting the local quadric surface as well as to guide the selection of points to be projected onto the sectional plane. The projected points are then used to construct the desired sectional contour profile. Implementation results have demonstrated the superior performance of the proposed fully automatic method in comparison with the existing methods.

Abstract. Forest inventories collect feature data manually on terrestrial field plots. Measuring large amounts of breast height diameters and tree positions is time consuming. Terrestrial laser scanning could be an additional instrument to collect precise and full inventory data in the 3D space. As a preliminary assumption single scan data is used to evaluate a minimal data acquisition scheme. To extract features like trees and diameter from the scanned point cloud, a simple geometric model world is defined in 3D. Trees are cylinder shapes vertically located on a plane. Using a RANSAC-based segmentation approach, cylinders are fitted iteratively in the point cloud. Several threshold parameters increase the robustness of the segmentation model and extract point clouds of single trees, which still contain branches and the tree crown. Fitting circles along the stem using point cloud slices allows to refine the effective diameter for customized heights. The cross section of a single tree point cloud covers only the semi circle towards the scan location, but is still contiguous enough to estimate diameters by using a robust circle fitting method.

The objective of this study was to identify and quantify surface depressions on grass-covered land surfaces using a high-resolution terrestrial laser scanning (TLS) point cloud, and a triangulated irregular network (TIN). The entire grassy land surface in the study area was divided into five subwatersheds of different topographic attributes (i.e., depression depth and surface slope). Surface depressions were identified and quantified using a TIN generated from a high-resolution TLS point cloud. The results indicated that microtopography of the grassy land surface was well-characterized within each subwatershed in comparison with field observations. With the terrestrial light detection and ranging (LIDAR) point cloud of 15-mm point spacing and the TIN method, surface depression storage depths of the five subwatersheds ranged from 1.73 to 14.28 mm in the study area. The surface depression storage depth, as expected, increased with the maximum depth of surface depression. It was also found to increase when the land surface slope became milder. A sensitivity analysis indicated that a point cloud with a point spacing of 30 mm was sufficient to obtain an accurate representation of the terrain surface in the study area. This study also indicated the TIN method can represent the ground surface and the surface depression more realistically than the commonly used digital elevation model (DEM) method due to the TIN method’s higher capability of identifying and filtering out surface obstructions such as blades of grass. Moreover, by using the high-resolution TLS technology and the TIN method, our study provides an important and broad range of reference data on the surface depression storage depth commonly needed in application of the Storm Water Management Model (SWMM) and other watershed models.

Abstract. This study introduces a novel image to a 3D point-cloud translation method with a conditional generative adversarial network that creates a large-scale 3D point cloud. This can generate supervised point clouds observed via airborne LiDAR from aerial images. The network is composed of an encoder to produce latent features of input images, generator to translate latent features to fake point clouds, and discriminator to classify false or real point clouds. The encoder is a pre-trained ResNet; to overcome the difficulty of generating 3D point clouds in an outdoor scene, we use a FoldingNet with features from ResNet. After a fixed number of iterations, our generator can produce fake point clouds that correspond to the input image. Experimental results show that our network can learn and generate certain point clouds using the data from the 2018 IEEE GRSS Data Fusion Contest.

Abstract. This paper investigates the problem of local surface reconstruction and best fitting for planar surfaces from unorganized 3D point cloud data. Least Squares (LS), its equivalent Principal Component Analysis (PCA) and RANSAC are the three most popular techniques for fitting planar surfaces to 3D data. LS and PCA are sensitive to outliers and do not give reliable and robust parameter estimation. The RANSAC algorithm is robust but it is not completely free from the effect of outliers and is slow for large datasets. In this paper, we propose a diagnostic-robust statistical algorithm that uses both diagnostics and robust approaches in combination for fitting planar surfaces in the presence of outliers. Recently introduced high breakdown and fast Minimum Covariance Determinant (MCD) based location and scatter estimates are used for robust distance to identify outliers and a MCD based robust PCA approach is used as an outlier resistant technique for plane fitting. The benefits of the new diagnostic-robust algorithm are demonstrated with artificial and real laser scanning point cloud datasets. Results show that the proposed method is significantly better and more efficient than the other three methods for planar surface fitting. This method also has great potential for robust local normal estimation and for other surface shape fitting applications.

An adaptive normal estimation method for scanned point clouds with sharp features