Automatic Registration Method of Multi-Source Point Clouds Based on Building Facades Matching in Urban Scenes

The development and management of green open spaces are essential in overcoming environmental problems such as air pollution and urban warming. 3D modeling and biomass calculation are the example efforts in managing green open spaces. In this study, 3D modeling was carried out on point clouds data acquired by the UAV photogrammetry and UAV LiDAR methods. 3D modeling is done explicitly using the point clouds fitting method. This study uses three fitting methods: the spherical fitting method, the ellipsoid fitting method, and the spherical harmonics fitting method. The spherical harmonics fitting method provides the best results and produces an R2 value between 0.324 to 0.945. In this study, Above-Ground Biomass (AGB) calculations were also carried out from the modeling results using three methods with UAV LiDAR and Photogrammetry data. AGB calculation using UAV LiDAR data gives better results than using photogrammetric data. AGB calculation using UAV LiDAR data gives an accuracy of 78% of the field validation results. However, for visualization purposes with a not-too-wide area, a 3D model of photogrammetric data using the spherical harmonics method can be used.

Three-dimensional point clouds captured by sensors can quantify plant phenotype, which plays an important role in agricultural intelligence. Many scanned objects in agriculture and forestry are tall and obscured by leaves, so point clouds captured by either terrestrial or airborne methods may be incomplete. In order to obtain a more complete point cloud, this paper proposes a point cloud registration method based on the fast point feature histogram (FPFH), which aligns point clouds collected from different viewpoints. This method calculates the FPFH feature of each point and the Bhattacharyya distance between point pairs. Effective strategies pick out reliable sets of point pairs. Singular value decomposition is used to obtain the transformation relationship between point clouds. Experimental results show that the proposed method has high accuracy for plant point cloud registration in real scenes, and the root-mean-square error is smaller than that of common registration methods of SAC-IA and NDT.KeywordsFast point feature histogramBhattacharyya distanceRandom sample consensusPoint cloud registration

The dense high-rise buildings and multipath effects in urban areas significantly reduce the positioning signal accuracy of laser scanning systems, leading to layering and offset issues in the collected point cloud data on the same road. In order to acquire comprehensive and consistent three-dimensional information on the objects, thereby providing field inspection data for large-scale road traffic network scenarios, in this paper, an improved point cloud registration method is proposed to divide the registration process into two stages: elevation registration and plane registration. Elevation registration takes the ground point cloud as the registration primitive, reduces the number of point clouds through curvature down-sampling, and constrains the feature point sequence with a fixed range to provide a good initial pose for fine registration. The plane registration first inherits the elevation registration parameters, combining the dynamic distance parameters of spherical region step based on the median, using robust multi-scale loss functions to address residual points, effective adjacent point pairs are selected to obtain the spatial transformation matrix, and realizes accurate registration. Experimental results with multiple sets of urban point cloud data show that the root mean square error of point cloud registration can be controlled within 0.06 m, achieving a relatively superior registration accuracy, it can provide detailed prior data for measurement information analysis.

Terrestrial point cloud registration plays an important role in 3D reconstruction, heritage restoration and topographic mapping, etc. Unfortunately, current research studies heavily rely on matching the 3D features of overlapped areas between point clouds, which is error-prone and time-consuming. To this end, we propose an automatic point cloud registration method based on Gaussian-weighting projected image matching, which can quickly and robustly register multi-station terrestrial point clouds. Firstly, the point cloud is regularized into a 2D grid, and the point density of each cell in the grid is normalized by our Gaussian-weighting function. A grayscale image is subsequently generated by shifting and scaling the x-y coordinates of the grid to the image coordinates. Secondly, the scale-invariant features (SIFT) algorithm is used to perform image matching, and a line segment endpoint verification method is proposed to filter out negative matches. Thirdly, the transformation matrix between point clouds from two adjacent stations is calculated based on reliable image matching. Finally, a global least-square optimization is conducted to align multi-station point clouds and then obtain a complete model. To test the performance of our framework, we carry out the experiment on six datasets. Compared to previous work, our method achieves the state-of-the-art performance on both efficiency and accuracy. In terms of efficiency, our method is comparable to an existing projection-based methods and 4 times faster on the indoor datasets and 10 times faster on the outdoor datasets than 4PCS-based methods. In terms of accuracy, our framework is ~2 times better than the existing projection-based method and 6 times better than 4PCS-based methods.

With the rapid development of binocular reconstruction, fringe projection profilometry, and time of flight, 3D imaging technology has been widely applied in the field of 3D measurement. However, due to limited measurement range and self-occlusion, point cloud registration methods are often used to obtain larger or more complete 3D contours. Although many scholars have proposed various point cloud registration methods, the accuracy and efficiency of point cloud registration still need to be further improved, especially for point clouds with different density or non-rigid transformation. Image registration technology based on image correlation has been developed for many years and has achieved great success in fields such as computer vision, photomechanics, and photogrammetry. Therefore, a simple and direct idea in this paper is to transform the point cloud registration problem into volume image correlation problem. By this, an efficient image registration method based on fast Fourier transform and an inverse compositional Gaussian Newton optimization method that only needs to calculate the Hessian matrix once can be introduced into the point cloud registration field, which can greatly improve the speed and accuracy of point cloud registration. Comparative experiments have shown that our method has doubled the accuracy and efficiency compared to the iterative closest point (ICP) method, and its practicality has also been verified in impeller reconstruction experiments.

Abstract. The aim of the research is to evaluate the performance of the point cloud registration methods using mobile laser scanning data. The point cloud registration methods involved in this research are match bounding-box centres and iterative closest point (ICP). The research began with the two epoch’s mobile laser scanning survey using a Phoenix AL-3-32 system. At the same time, the stereo images of the study area were acquired using UAV Photogrammetric method. Both two epoch point cloud datasets were gone through the pre and post-processing stages to produce the cleaned and geo-referenced point clouds data. The data were then gone through the two registration methods and four Cloud-to-Cloud (C2C) distance methods. The 3D surface deviation results obtained from mobile laser scanning data was compared with the 3D surface deviation results from UAV data that undergoes the same registration and C2C distance computation methods. The study area involved in the research is an active landslide area that was located at Kulim Hi-Tech residential area in Kedah state, Malaysia. The study area exposed to the movement of the land which caused cracked to the buildings and drainages. The findings show that the ICP registration becomes the most suitable method to register point clouds dataset that was acquired using mobile laser scanning system. Among the four C2C distance computation methods that was involved in the testing, the least square plane method was the best method to calculate the distance between two sets of point clouds datasets which in turn gave the best results in the process of detecting the movement of the land in the study area.

An automatic 3D point cloud registration method based on regional curvature maps

Under low-light nighttime conditions, building images often suffer from poor quality, making high-precision integration with point cloud data challenging. This paper presents a comprehensive solution for enhancing nighttime building images and fusing them with point clouds for modeling. We developed an unsupervised learning framework that in-corporates prior knowledge of building structures. Based on RetinexDIP theory, the framework includes a structural edge enhancement module. It improves the structure of initial lighting and refines it with edge-aware adjustments using reflectance information, significantly enhancing the clarity and detail of building facades and outlines. Experi-mental results show that this method significantly improves image quality on both the MIT-Adobe FiveK dataset and a self-collected dataset. The average PSNR reached 12.02 dB, which is 0.35 dB higher than baseline methods, and the information entropy increased by 0.08. Moreover, it effectively preserves the integrity of building edge structures. We also proposed a Line-Feature-Constrained Registration (LFR) method for fusing image-based point clouds and LiDAR point clouds. This approach identifies building outline feature point clouds, extracts corresponding lines, and decomposes 3D registration into planar and elevation registration. In three test areas, the LFR method achieved a registration ac-curacy (RMSE) of 0.04-0.06 meters and an overlap rate of approximately 95%, showing significant improvements in accuracy and robustness over traditional ICP and NDT methods. Finally, we performed fusion modeling using the enhanced images and regis-tered point clouds. The reconstructed building models outperform comparative methods in both geometric completeness and texture authenticity. The number of tie points in-creased by up to 27.6%, while the median errors in plane and elevation were reduced to 4.15 cm and 5.53 cm, respectively.

A shape-based segmentation method for mobile laser scanning point clouds

DLA-Net: Learning dual local attention features for semantic segmentation of large-scale building facade point clouds

Collecting 3D point cloud data of buildings is important for many applications such as urban mapping, renovation, preservation, and energy simulation. However, laser-scanned point clouds are often difficult to analyze, visualize, and interpret due to incompletely scanned building facades caused by numerous sources of defects such as noise, occlusions, and moving objects. Several point cloud scene completion algorithms have been proposed in the literature, but they have been mostly applied to individual objects or small-scale indoor environments and not on large-scale scans of building facades. This paper introduces a method of performing point cloud scene completion of building facades using orthographic projection and generative adversarial inpainting methods. The point cloud is first converted into the 2D structured representation of depth and color images using an orthographic projection approach. Then, a data-driven 2D inpainting approach is used to predict the complete version of the scene, given the incomplete scene in the image domain. The 2D inpainting process is fully automated and uses a customized generative-adversarial network based on Pix2Pix that is trainable end-to-end. The inpainted 2D image is finally converted back into a 3D point cloud using depth remapping. The proposed method is compared against several baseline methods, including geometric methods such as Poisson reconstruction and hole-filling, as well as learning-based methods such as the point completion network (PCN) and TopNet. Performance evaluation is carried out based on the task of reconstructing real-world building facades from partial laser-scanned point clouds. Experimental results using the performance metrics of voxel precision, voxel recall, position error, and color error showed that the proposed method has the best performance overall.

Abstract. For obtaining a full coverage of 3D scans in a large-scale urban area, the registration between point clouds acquired via terrestrial laser scanning (TLS) is normally mandatory. However, due to the complex urban environment, the automatic registration of different scans is still a challenging problem. In this work, we propose an automatic marker free method for fast and coarse registration between point clouds using the geometric constrains of planar patches under a voxel structure. Our proposed method consists of four major steps: the voxelization of the point cloud, the approximation of planar patches, the matching of corresponding patches, and the estimation of transformation parameters. In the voxelization step, the point cloud of each scan is organized with a 3D voxel structure, by which the entire point cloud is partitioned into small individual patches. In the following step, we represent points of each voxel with the approximated plane function, and select those patches resembling planar surfaces. Afterwards, for matching the corresponding patches, a RANSAC-based strategy is applied. Among all the planar patches of a scan, we randomly select a planar patches set of three planar surfaces, in order to build a coordinate frame via their normal vectors and their intersection points. The transformation parameters between scans are calculated from these two coordinate frames. The planar patches set with its transformation parameters owning the largest number of coplanar patches are identified as the optimal candidate set for estimating the correct transformation parameters. The experimental results using TLS datasets of different scenes reveal that our proposed method can be both effective and efficient for the coarse registration task. Especially, for the fast orientation between scans, our proposed method can achieve a registration error of less than around 2 degrees using the testing datasets, and much more efficient than the classical baseline methods.

Multisensor systems with optical 3D sensors are frequently employed to capture complete surface information by measuring workpieces from different views. During coarse and fine registration the resulting datasets are afterward transformed into one common coordinate system. Automatic fine registration methods are well established in dimensional metrology, whereas there is a deficit in automatic coarse registration methods. The advantage of a fully automatic registration procedure is twofold: it enables a fast and contact-free alignment and further a flexible application to datasets of any kind of optical 3D sensor. In this paper, an algorithm adapted for a robust automatic coarse registration is presented. The method was originally developed for the field of object reconstruction or localization. It is based on a segmentation of planes in the datasets to calculate the transformation parameters. The rotation is defined by the normals of three corresponding segmented planes of two overlapping datasets, while the translation is calculated via the intersection point of the segmented planes. First results have shown that the translation is strongly shape dependent: 3D data of objects with non-orthogonal planar flanks cannot be registered with the current method. In the novel supplement for the algorithm, the translation is additionally calculated via the distance between centroids of corresponding segmented planes, which results in more than one option for the transformation. A newly introduced measure considering the distance between the datasets after coarse registration evaluates the best possible transformation. Results of the robust automatic registration method are presented on the example of datasets taken from a cutting tool with a fringe-projection system and a focus-variation system. The successful application in dimensional metrology is proven with evaluations of shape parameters based on the registered datasets of a calibrated workpiece.

Segmentation of vegetation LiDAR point clouds is an important method for obtaining individual tree structure parameters. The current individual tree segmentation methods are mainly for airborne LiDAR point clouds, which use elevation information to form a grid map for segmentation, or use canopy vertices as seed points for clustering. Side-view LiDAR (vehicle LiDAR and hand-held LiDAR) can acquire more information about the lower layer of trees, but it is a challenge to perform the individual tree segmentation because the structure of side-view LiDAR point clouds is more complex. This paper proposes an individual tree segmentation method called Shadow-cut to extract the contours of the street tree point cloud. Firstly, we separated the region of the trees using the binary classifier (e.g., support vector machine) based on point cloud geometric features. Then, the optimal projection of the 3D point clouds to the 2D image is calculated and the optimal projection is the case where the pixels of the individual tree image overlap the least. Finally, after using the image segmentation algorithm to extract the tree edges in the 2D image, the corresponding 3D individual tree point cloud contours are matched with the pixels of individual tree edges in the 2D image. We conducted experiments with the proposed method on LiDAR data of urban street trees, and the correctness, completeness, and quality of the proposed individual tree segmentation method reached 91.67%, 85.33%, and 79.19%, which were superior to the CHM-based method by 2.70%, 6.19%, and 7.12%, respectively. The results show that this method is a practical and effective solution for individual tree segmentation in the LiDAR point clouds of street trees.

Individual tree detection from airborne laser scanning data based on supervoxels and local convexity