Robust 3D Line Extraction from Stereo Point Clouds

In recent years, due to the significant advancements in hardware sensors and software technologies, 3D environmental point cloud modeling has gradually been applied in the automation industry, autonomous vehicles, and construction engineering. With the high-precision measurements of 3D LiDAR, its point clouds can clearly reflect the geometric structure and features of the environment, thus enabling the creation of high-density 3D environmental point cloud models. However, due to the enormous quantity of high-density 3D point clouds, storing and processing these 3D data requires a considerable amount of memory and computing time. In light of this, this paper proposes a real-time 3D point cloud environmental contour modeling technique. The study uses the point cloud distribution from the 3D LiDAR body frame point cloud to establish structured edge features, thereby creating a 3D environmental contour point cloud map. Additionally, unstable objects such as vehicles will appear during the mapping process; these specific objects will be regarded as not part of the stable environmental model in this study. To address this issue, the study will further remove these objects from the 3D point cloud through image recognition and LiDAR heterogeneous matching, resulting in a higher quality 3D environmental contour point cloud map. This 3D environmental contour point cloud not only retains the recognizability of the environmental structure but also solves the problems of massive data storage and processing. Moreover, the method proposed in this study can achieve real-time realization without requiring the 3D point cloud to be organized in a structured order, making it applicable to unorganized 3D point cloud LiDAR sensors. Finally, the feasibility of the proposed method in practical applications is also verified through actual experimental data.

Structured scenes are an important element of 3D reconstruction. Characteristics of structured scenes are analyzed, based on which a modified PEaRL algorithm is proposed to perform plane fitting on points provided by multiple view stereo algorithms. In this method, candidate planes are generated from both 3D lines and 3D points. The number of candidate planes is reduced using information provided by the 3D lines; furthermore, the reliability of the planes is improved. Subsequently, 3D lines are employed to obtain a more reliable neighborhood system, which leads to more accurate plane fitting results at plane intersections. For energy optimization, the smoothness term and label cost term are improved to better manage the irregularity of multi-view stereo(MVS)point clouds. Experiments on synthesized and actual data show that the proposed method provides satisfactory time efficiency, and obtains more reliable plane fitting results than other multi-model fitting methods, including PEaRL.

The automatic reconstruction of photogrammetric mesh models often suffers from issues such as erroneous adhesion of different planes, local distortions, and the loss of edge details. This study aims to improve mesh quality by incorporating three-dimensional (3D) line segments and plane regions as geometric constraints during mesh refinement. Using previously obtained 3D line segments, depth, and normal information, accurate 3D plane regions are extracted to complete missing sections of the dense point cloud, with depth-based outlier removal applied. Correspondences between mesh model vertices and the 3D line segments and plane regions are established, followed by mesh refinement through an energy function that integrates photo-consistency, line and plane constraints, and regularization. Experiments conducted on four open-source datasets demonstrate that the proposed method substantially improves 3D plane region extraction and point cloud completion. The refined mesh model achieves a 24.30% reduction in the average distance related to 3D lines and a 15.75% reduction for 3D planes compared with the original model, confirming the effectiveness of incorporating 3D line and plane constraints in enhancing reconstruction accuracy.

Stereo cameras, laser rangers and other time-of-flight ranging devices are utilized with increasing frequency as they can provide information in the 3D plane. The ability to perform real-time registration of the 3D point clouds obtained from these sensors is important in many applications. However, the tasks of locating accurate and dependable correspondences between point clouds and registration can be quite slow. Furthermore, any algorithm must be robust against artifacts in 3D range data as sensor motion, reflection and refraction are commonplace. The SIFT feature detector is a robust algorithm used to locate features, but cannot be extended directly to the 3D range point clouds since it requires dense pixel information, whereas the range voxels are sparsely distributed. This paper proposes an approach which enables SIFT application to locate scale and rotation invariant features in 3D point clouds. The algorithm then utilizes the known point correspondence registration algorithm in order to achieve real-time registration of 3D point clouds.KeywordsPoint CloudScale Invariant Feature TransformIterative Close Point AlgorithmScale Invariant Feature Transform FeatureRANSAC AlgorithmThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Our goal is the registration of multiple 3D point clouds obtained from LIDAR scans of underground mines. Such a capability is crucial to the surveying and planning operations in mining. Often, the point clouds only partially overlap and initial alignment is unavailable. Here, we propose an interactive user-assisted point cloud registration system. Guided by the system, the user's role is simply to identify and search for overlapping regions across the point clouds. Specifically, given two point sets, the user clicks on a point in one set, then simply hovers the mouse on the other set to find a matching point. Each mouse position gives rise to a translation, and our system instantly optimises the rotation that aligns the point clouds. Assuming that each individual point set is horizontally levelled with the ground by the level compensator on the LIDAR device, given a candidate 3D translation only one angular parameter needs to be estimated to rotationally align two 3D point sets. We propose a fast rotation search algorithm that delivers globally optimal results in real time. Our method conducts branch-and-bound optimisation with a novel bounding function whose evaluation amounts to simple sorted array operations. This provides smooth and accurate feedback to the user's search for overlapping regions. Our system is intuitive and helps to accelerate the registration of multiple scans.

Converting three-dimensional (3D) point cloud data to two-dimensional (2D) image based on virtual structured-light system is a popular method in 3D data compression because the image format is easier to process by the exiting storing and transmitting methods. When this method is used in the 3D point cloud compression, the quantization error is introduced during the coordinate mapping. To solve this problem, a virtual structured-light 3D point cloud compression algorithm based on geometric reshaping is presented. And a least-square system parameter optimization method is proposed to further improve the data decoding accuracy. In the proposed method, the 3D spatial coordinates are reshaped to a 2D matrix first, and then the 2D matrix is stored as a “Holoimage” by using the parameter optimized virtual structured-light system. The quantization error introduced by coordinate mapping between X and Y coordinates of the 3D point cloud and the 2D image pixel coordinates is suppressed, so the decoding accuracy is improved. In addition, the geometric information of the 3D point cloud is hidden synchronously when the 3D point cloud is compressed, which is of great significance in the copyright protection of the point cloud data. Experiments verify the effectiveness of the proposed algorithm. The decoding root mean square error (RMSE) of the proposed method is decreased by 79.86% on average compared with the traditional one under the same compression ratio, and the compression ratio of the proposed method is 1.96 times bigger than the traditional one under the similar decoding accuracy.

Along with increasingly popular virtual reality applications, the three-dimensional (3D) point cloud has become a fundamental data structure to characterize 3D objects and surroundings. To process 3D point clouds efficiently, a suitable model for the underlying structure and outlier noises is always critical. In this work, we propose a hypergraph-based new point cloud model that is amenable to efficient analysis and processing. We introduce tensor-based methods to estimate hypergraph spectrum components and frequency coefficients of point clouds in both ideal and noisy settings. We establish an analytical connection between hypergraph frequencies and structural features. We further evaluate the efficacy of hypergraph spectrum estimation in two common applications of sampling and denoising of point clouds for which we provide specific hypergraph filter design and spectral properties. Experimental results demonstrate the strength of hypergraph signal processing as a tool in characterizing the underlying properties of 3D point clouds.

The inclusion of thermal infrared (TIR) data in point clouds derived from unmanned aircraft system (UAS) imagery can benefit a variety of applications in which surface temperature and 3D geometry are both important discriminants of feature type and condition. Low resolution and narrow fields of view (FOV) of current consumer-grade TIR cameras on UAS, combined with the lack of sharpness and texture in many image regions, may cause failure or poor results from structure from motion (SfM) photogrammetric software, which has gained widespread use for generating point clouds from UAS imagery. This paper proposes a photogrammetric approach for generating 3D multispectral point clouds utilizing coacquired TIR-RGB images. A 3D point cloud is first generated from the RGB imagery using standard SfM procedures. Then the TIR attributes are assigned to points, where the image coordinates of the points in TIR images are estimated using transformation parameters obtained from co-registration procedures. To obtain RGB-to-TIR transformation parameters, this study tests 3D and 2D co-registration approaches. The latter produces better results due to the challenge of calibrating the TIR camera as required for the 3D approach. This proposed approach is advantageous for generating TIR point clouds without loss of photogrammetric precision compared with solely TIR-based SfM, as the 3D accuracy, point density, and reliability are greatly enhanced.

Automated recognition and rebar dimensional assessment of prefabricated bridge components from low-cost 3D laser scanner

The traditional point-based scene reconstruction scheme always leads to inaccurate 3D reconstruction results due to its lack of scene texture and object occlusions. With the help of geometric characteristics and line features of 3D point cloud scenes, a novel extraction, matching, and registration method of 3D line features is presented by fusing RGB gradients and the spatial distribution of neighboring point clouds. Firstly, the supporting region of 3D linear segments with similar gradient directions can be determined by the gradients of an RGB scene image. Secondly, owing to the neighboring geometric features contained in the depth map, the point set of boundary lines and folded edge lines can also be extracted according to the spatial distribution of the current point and its neighboring points. The 3D line features can thus be fitted by these extracted point sets respectively. Thirdly, taking each extracted 3D line feature as a central axis, the supporting region can be constructed as the co-axial cylinder with the same thickness of each cylindrical layer. According to the 3D gradient statistics of point cloud within each cylindrical layer, we can calculate an intrinsic descriptor for each 3D line, which can be adopted to match 3D line features between scene frames. Finally, a novel line-to-line iterative closest line registration algorithm is introduced to compute the rotation and translation transformations, and the inter-frame point cloud scene data can be registered efficiently. The experimental results illustrate that the proposed line feature-based scheme has high registration efficiency, less computational cost, accurate registration, and strong robustness when compared with the existing line feature extraction and point-based registration methods.

In recent years, there has been significant growth in the ubiquity and popularity of three-dimensional (3D) point clouds, with an increasing focus on the classification of 3D point clouds. To extract richer features from point clouds, many researchers have turned their attention to various point set regions and channels within irregular point clouds. However, this approach has limited capability in attending to crucial regions of interest in 3D point clouds and may overlook valuable information from neighboring features during feature aggregation. Therefore, this paper proposes a novel 3D point cloud classification method based on global attention and adaptive graph convolution (Att-AdaptNet). The method consists of two main branches: the first branch computes attention masks for each point, while the second branch employs adaptive graph convolution to extract global features from the point set. It dynamically learns features based on point interactions, generating adaptive kernels to effectively and precisely capture diverse relationships among points from different semantic parts. Experimental results demonstrate that the proposed model achieves 93.8% in overall accuracy and 90.8% in average accuracy on the ModeNet40 dataset.

ABSTRACTThis paper proposes a novel method for register partially overlapped three-dimensional (3D) point sets when a complete 3D scanned model is constructed from partially scanned 3D point sets. 3D point sets are the most primitive and fundamental surface representations. 3D acquisition devices such as laser scanners and range image finders are popular sources for acquiring point clouds. With the availability of low cost and highly efficient 3D laser scanners, the popularity of such devices is increasing rapidly in various fields. Generally, a standard scanner can obtain only a partial data to build a complete 3D model of the object. These partially scanned 3D patches need to be aligned together in the same coordinate system to construct a complete 3D model, and this process is widely known as registration. A coarse-to-fine alignment technique is proposed to register two partially overlapped point sets. First, the overlapping regions using a novel sampling strategy are identified and the overlapping regions of two point sets are aligned together. Second, fine registration is carried out, in which an auxiliary pair and surface normal constraints are used to find reliable corresponding points in order to achieve higher accuracy.

The present work proposes a new method Reg3DFacePtCd for registration of point clouds. The key contribution of the present method is that an unknown face in 3D point cloud form is given to the system and is registered to the already existing known 3D face point clouds using a fast 3D face registration method. The novelty of the present technique is that at first the alignment and registration parameters are found out by initially registering eight key points of the unknown source model to that of the known model. Next, the rest of the point clouds of the unknown model are registered to that of the known model using the same parameters found as above. The main method used for alignment is iterative closest point (ICP) using point-based technique followed by registration in the least squares sense. Mainly there are two significant contributions. Firstly, we have developed a new mathematical model facial landmark point based model across poses to obtain the target or the known model to which all the unknown models will be registered. Secondly, a novel way to accelerate point cloud matching by reducing the number of points has been proposed. Using a small number of points necessarily would speed up the registration process but may inculcate errors. So, to determine the registration quality of the fundamental eight key points on which the entire registration process is based, a new robust metric namely ICV (ICP certainty vector) consisting of several key components have been used. Finally, we have addressed several important face registration issues like pre-processing, convergence and quality of registration of the entire facial point cloud based on the eight key points. Extensive experimentation on Frav3D, GavabDB, and the Bosphorus databases on a high-performance computing environment show the novelty and robustness of the method.

Surface defect identification is a crucial task in many manufacturing systems, including automotive, aircraft, steel rolling, and precast concrete. Although image-based surface defect identification methods have been proposed, these methods usually have two limitations: images may lose partial information, such as depths of surface defects, and their precision is vulnerable to many factors, such as the inspection angle, light, color, noise, etc. Given that a three-dimensional (3D) point cloud can precisely represent the multidimensional structure of surface defects, we aim to detect and classify surface defects using a 3D point cloud. This has two major challenges: (i) the defects are often sparsely distributed over the surface, which makes their features prone to be hidden by the normal surface and (ii) different permutations and transformations of 3D point cloud may represent the same surface, so the proposed model needs to be permutation and transformation invariant. In this paper, a two-step surface defect identification approach is developed to investigate the defects’ patterns in 3D point cloud data. The proposed approach consists of an unsupervised method for defect detection and a multi-view deep learning model for defect classification, which can keep track of the features from both defective and non-defective regions. We prove that the proposed approach is invariant to different permutations and transformations. Two case studies are conducted for defect identification on the surfaces of synthetic aircraft fuselage and the real precast concrete specimen, respectively. The results show that our approach receives the best defect detection and classification accuracy compared with other benchmark methods.

Towards 3D lidar point cloud registration improvement using optimal neighborhood knowledge