Hypergraph Spectral Analysis and Processing in 3D Point Cloud.

In recent years, due to the significant advancements in hardware sensors and software technologies, 3D environmental point cloud modeling has gradually been applied in the automation industry, autonomous vehicles, and construction engineering. With the high-precision measurements of 3D LiDAR, its point clouds can clearly reflect the geometric structure and features of the environment, thus enabling the creation of high-density 3D environmental point cloud models. However, due to the enormous quantity of high-density 3D point clouds, storing and processing these 3D data requires a considerable amount of memory and computing time. In light of this, this paper proposes a real-time 3D point cloud environmental contour modeling technique. The study uses the point cloud distribution from the 3D LiDAR body frame point cloud to establish structured edge features, thereby creating a 3D environmental contour point cloud map. Additionally, unstable objects such as vehicles will appear during the mapping process; these specific objects will be regarded as not part of the stable environmental model in this study. To address this issue, the study will further remove these objects from the 3D point cloud through image recognition and LiDAR heterogeneous matching, resulting in a higher quality 3D environmental contour point cloud map. This 3D environmental contour point cloud not only retains the recognizability of the environmental structure but also solves the problems of massive data storage and processing. Moreover, the method proposed in this study can achieve real-time realization without requiring the 3D point cloud to be organized in a structured order, making it applicable to unorganized 3D point cloud LiDAR sensors. Finally, the feasibility of the proposed method in practical applications is also verified through actual experimental data.

Surface defect identification is a crucial task in many manufacturing systems, including automotive, aircraft, steel rolling, and precast concrete. Although image-based surface defect identification methods have been proposed, these methods usually have two limitations: images may lose partial information, such as depths of surface defects, and their precision is vulnerable to many factors, such as the inspection angle, light, color, noise, etc. Given that a three-dimensional (3D) point cloud can precisely represent the multidimensional structure of surface defects, we aim to detect and classify surface defects using a 3D point cloud. This has two major challenges: (i) the defects are often sparsely distributed over the surface, which makes their features prone to be hidden by the normal surface and (ii) different permutations and transformations of 3D point cloud may represent the same surface, so the proposed model needs to be permutation and transformation invariant. In this paper, a two-step surface defect identification approach is developed to investigate the defects’ patterns in 3D point cloud data. The proposed approach consists of an unsupervised method for defect detection and a multi-view deep learning model for defect classification, which can keep track of the features from both defective and non-defective regions. We prove that the proposed approach is invariant to different permutations and transformations. Two case studies are conducted for defect identification on the surfaces of synthetic aircraft fuselage and the real precast concrete specimen, respectively. The results show that our approach receives the best defect detection and classification accuracy compared with other benchmark methods.

Converting three-dimensional (3D) point cloud data to two-dimensional (2D) image based on virtual structured-light system is a popular method in 3D data compression because the image format is easier to process by the exiting storing and transmitting methods. When this method is used in the 3D point cloud compression, the quantization error is introduced during the coordinate mapping. To solve this problem, a virtual structured-light 3D point cloud compression algorithm based on geometric reshaping is presented. And a least-square system parameter optimization method is proposed to further improve the data decoding accuracy. In the proposed method, the 3D spatial coordinates are reshaped to a 2D matrix first, and then the 2D matrix is stored as a “Holoimage” by using the parameter optimized virtual structured-light system. The quantization error introduced by coordinate mapping between X and Y coordinates of the 3D point cloud and the 2D image pixel coordinates is suppressed, so the decoding accuracy is improved. In addition, the geometric information of the 3D point cloud is hidden synchronously when the 3D point cloud is compressed, which is of great significance in the copyright protection of the point cloud data. Experiments verify the effectiveness of the proposed algorithm. The decoding root mean square error (RMSE) of the proposed method is decreased by 79.86% on average compared with the traditional one under the same compression ratio, and the compression ratio of the proposed method is 1.96 times bigger than the traditional one under the similar decoding accuracy.

In recent years, there has been significant growth in the ubiquity and popularity of three-dimensional (3D) point clouds, with an increasing focus on the classification of 3D point clouds. To extract richer features from point clouds, many researchers have turned their attention to various point set regions and channels within irregular point clouds. However, this approach has limited capability in attending to crucial regions of interest in 3D point clouds and may overlook valuable information from neighboring features during feature aggregation. Therefore, this paper proposes a novel 3D point cloud classification method based on global attention and adaptive graph convolution (Att-AdaptNet). The method consists of two main branches: the first branch computes attention masks for each point, while the second branch employs adaptive graph convolution to extract global features from the point set. It dynamically learns features based on point interactions, generating adaptive kernels to effectively and precisely capture diverse relationships among points from different semantic parts. Experimental results demonstrate that the proposed model achieves 93.8% in overall accuracy and 90.8% in average accuracy on the ModeNet40 dataset.

Abstract. A series of panoramic images are usually used to generate a 720° panorama image. Although panoramic images are typically used for establishing tour guiding systems, in this research, we demonstrate the potential of using panoramic images acquired from multiple sites to create not only 720° panorama, but also three-dimensional (3D) point clouds and 3D indoor models. Since 3D modeling is one of the goals of this research, the location of the panoramic sites needed to be carefully planned in order to maintain a robust result for close-range photogrammetry. After the images are acquired, panoramic images are processed into 720° panoramas, and these panoramas which can be used directly as panorama guiding systems or other applications. In addition to these straightforward applications, interior orientation parameters can also be estimated while generating 720° panorama. These parameters are focal length, principle point, and lens radial distortion. The panoramic images can then be processed with closerange photogrammetry procedures to extract the exterior orientation parameters and generate 3D point clouds. In this research, VisaulSFM, a structure from motion software is used to estimate the exterior orientation, and CMVS toolkit is used to generate 3D point clouds. Next, the 3D point clouds are used as references to create building interior models. In this research, Trimble Sketchup was used to build the model, and the 3D point cloud was added to the determining of locations of building objects using plane finding procedure. In the texturing process, the panorama images are used as the data source for creating model textures. This 3D indoor model was used as an Augmented Reality model replacing a guide map or a floor plan commonly used in an on-line touring guide system. The 3D indoor model generating procedure has been utilized in two research projects: a cultural heritage site at Kinmen, and Taipei Main Station pedestrian zone guidance and navigation system. The results presented in this paper demonstrate the potential of using panoramic images to generate 3D point clouds and 3D models. However, it is currently a manual and labor-intensive process. A research is being carried out to Increase the degree of automation of these procedures.

Recent advances and commercialisation of unmanned aerial vehicle/red blue green (RGB) camera systems and digital photogrammetric techniques now provide a cheap and flexible alternative to higher-cost airborne platforms for routine monitoring of canopy health in timber plantations. Structure-from-Motion photogrammetry produces very dense three-dimensional (3D) point clouds which can be used to derive metrics for inventory estimation. Unmanned aerial vehicle RGB photography also captures data that can relate to tree health. In contrast to the more common use of orthorectified RGB photography to extract this spectral information, we used the software package Agisoft Photoscan to assign a simple Vegetation Index value directly to each point in the 3D point cloud. Using data acquired by a DJI Phantom 4 Pro, we present a simple processing and photogrammetric workflow solution for detecting dead and dying trees in a young Eucalyptus pellita plantation located in the provenance of Riau, Sumatra. Trees affected by the bacterial wilt Ralstonia sp. present symptoms of necrotic foliage on individual branches or the whole crown. Assigning the Visible Atmospheric Resistant Index Vegetation Index colour-coded values to individual points in the 3D point cloud significantly enhanced visualisation of necrotic foliage on individual trees in both the point cloud and the associated orthophoto compared to the RGB equivalent images. This approach could easily be operationally deployed for the rapid detection and mapping of unhealthy trees with symptoms of necrotic foliage.

Machine learning methods, in particular those based on deep learning, have gained in importance through the latest development of artificial intelligence and computer hardware. However, the direct application of deep learning methods to improve the results of 3D building reconstruction is often not possible due, for example, to the lack of suitable training data. To address this issue, we present RoofN3D which provides a three-dimensional (3D) point cloud training dataset that can be used to train machine learning models for different tasks in the context of 3D building reconstruction. The details about RoofN3D and the developed framework to automatically derive such training data are described in this paper. Furthermore, we provide an overview of other available 3D point cloud training data and approaches from current literature in which solutions for the application of deep learning to 3D point cloud data are presented. Finally, we exemplarily demonstrate how the provided data can be used to classify building roofs with the PointNet framework.

The inclusion of thermal infrared (TIR) data in point clouds derived from unmanned aircraft system (UAS) imagery can benefit a variety of applications in which surface temperature and 3D geometry are both important discriminants of feature type and condition. Low resolution and narrow fields of view (FOV) of current consumer-grade TIR cameras on UAS, combined with the lack of sharpness and texture in many image regions, may cause failure or poor results from structure from motion (SfM) photogrammetric software, which has gained widespread use for generating point clouds from UAS imagery. This paper proposes a photogrammetric approach for generating 3D multispectral point clouds utilizing coacquired TIR-RGB images. A 3D point cloud is first generated from the RGB imagery using standard SfM procedures. Then the TIR attributes are assigned to points, where the image coordinates of the points in TIR images are estimated using transformation parameters obtained from co-registration procedures. To obtain RGB-to-TIR transformation parameters, this study tests 3D and 2D co-registration approaches. The latter produces better results due to the challenge of calibrating the TIR camera as required for the 3D approach. This proposed approach is advantageous for generating TIR point clouds without loss of photogrammetric precision compared with solely TIR-based SfM, as the 3D accuracy, point density, and reliability are greatly enhanced.

Stereo cameras, laser rangers and other time-of-flight ranging devices are utilized with increasing frequency as they can provide information in the 3D plane. The ability to perform real-time registration of the 3D point clouds obtained from these sensors is important in many applications. However, the tasks of locating accurate and dependable correspondences between point clouds and registration can be quite slow. Furthermore, any algorithm must be robust against artifacts in 3D range data as sensor motion, reflection and refraction are commonplace. The SIFT feature detector is a robust algorithm used to locate features, but cannot be extended directly to the 3D range point clouds since it requires dense pixel information, whereas the range voxels are sparsely distributed. This paper proposes an approach which enables SIFT application to locate scale and rotation invariant features in 3D point clouds. The algorithm then utilizes the known point correspondence registration algorithm in order to achieve real-time registration of 3D point clouds.KeywordsPoint CloudScale Invariant Feature TransformIterative Close Point AlgorithmScale Invariant Feature Transform FeatureRANSAC AlgorithmThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Three-dimensional (3D) point clouds have a wide range of applications in the field of 3D vision. The quality of the acquired point cloud data considerably impacts the subsequent work of point cloud processing. Due to the sparsity and irregularity of point cloud data, processing point cloud data has always been challenging. However, existing deep learning-based point cloud dense reconstruction methods suffer from excessive smoothing of reconstruction results and too many outliers. The reason for this is that it is not possible to extract features for local and global features at different scales and provide different levels of attention to different regions in order to obtain long-distance dependence for dense reconstruction. In this paper, we use a parallel multi-scale feature extraction module based on graph convolution and an upsampling method with an added multi-head attention mechanism to process sparse and irregular point cloud data to obtain extended point clouds. Specifically, a point cloud training patch with 256 points is inputted. The PMS module uses three residual connections in the multi-scale feature extraction stage. Each PMS module consists of three parallel DenseGCN modules with different size convolution kernels and different averaging pooling sizes. The local and global feature information of the augmented receptive field is extracted efficiently. The scale information is obtained by averaging the different pooled augmented receptive fields. The scale information was obtained using the different average pooled augmented receptive fields. The upsampling stage uses an upsampling rate of r=4, The self-attentive features with a different focus on different point cloud data regions obtained by fusing different weights make the feature representation more diverse. This operation avoids the bias of one attention, and each focuses on extracting valuable fine-grained feature information. Finally, the coordinate reconstruction module obtains 1024 dense point cloud data. Experiments show that the proposed method demonstrates good evaluation metrics and performance and is able to obtain better visual quality. The problems of over-smoothing and excessive outliers are effectively mitigated, and the obtained sparse point cloud is more dense.

The present work proposes a new method Reg3DFacePtCd for registration of point clouds. The key contribution of the present method is that an unknown face in 3D point cloud form is given to the system and is registered to the already existing known 3D face point clouds using a fast 3D face registration method. The novelty of the present technique is that at first the alignment and registration parameters are found out by initially registering eight key points of the unknown source model to that of the known model. Next, the rest of the point clouds of the unknown model are registered to that of the known model using the same parameters found as above. The main method used for alignment is iterative closest point (ICP) using point-based technique followed by registration in the least squares sense. Mainly there are two significant contributions. Firstly, we have developed a new mathematical model facial landmark point based model across poses to obtain the target or the known model to which all the unknown models will be registered. Secondly, a novel way to accelerate point cloud matching by reducing the number of points has been proposed. Using a small number of points necessarily would speed up the registration process but may inculcate errors. So, to determine the registration quality of the fundamental eight key points on which the entire registration process is based, a new robust metric namely ICV (ICP certainty vector) consisting of several key components have been used. Finally, we have addressed several important face registration issues like pre-processing, convergence and quality of registration of the entire facial point cloud based on the eight key points. Extensive experimentation on Frav3D, GavabDB, and the Bosphorus databases on a high-performance computing environment show the novelty and robustness of the method.

The hyperloop transportation system has emerged as an innovative next-generation transportation system. In this system, a capsule-type vehicle inside a sealed near-vacuum tube moves at 1000 km/h or more. Not only must this transport tube span over long distances, but it must be clear of potential hazards to vehicles traveling at high speeds inside the tube. Therefore, an automated infrastructure anomaly detection system is essential. This study sought to confirm the applicability of advanced sensing technology such as Light Detection and Ranging (LiDAR) in the automatic anomaly detection of next-generation transportation infrastructure such as hyperloops. To this end, a prototype two-dimensional LiDAR sensor was constructed and used to generate three-dimensional (3D) point cloud models of a tube facility. A technique for detecting abnormal conditions or obstacles in the facility was used, which involved comparing the models and determining the changes. The design and development process of the 3D safety monitoring system using 3D point cloud models and the analytical results of experimental data using this system are presented. The tests on the developed system demonstrated that anomalies such as a 25 mm change in position were accurately detected. Thus, we confirm the applicability of the developed system in next-generation transportation infrastructure.

Estimation of plant parameters is prerequisite for fast and precise management of tomato crops on commercial farm level. Destructive estimation methods may be accurate but not suitable when large farm size, spatio-temporal variability or short period of time to obtain information are considered. Therefore, this study was aimed to estimate leaf area (LA) in mature tomato plants from reconstructed geometric plant model based on 3-dimensional (3D) point cloud. A 2D laser scanner was placed on a programable linear conveyor system to move at controlled forward speed. To achieve a high density 3D point cloud of whole plant, close range (< 2 m) scanning was performed at 10 mm/s forward speed. Plant parameters capturing points per plant (PPP), plant height, plant volume, and LA were extracted from the 3D point clouds. Destructive method was utilized in laboratory for reference LA data collection, whereas other required reference data (plant height and canopy area) were measured in-situ. Linear regression models were developed and cross validated to estimate LA using variables from the 3D point clouds and reference LA. Among the models, plant height showed strong correlation with reference LA (R <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> = 0.90, RMSE = 40 cm <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> ) and followed by PPP (R <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> = 0.89, RMSE = 45 cm <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> ), voxel grid approach (R <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> = 0.56, RMSE = 58 cm <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> ) and slicing approach (R <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> = 0.44, RMSE = 70 cm <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> ). Results indicate that mobile laser scanning could be a reliable non-destructive LA estimation tool for tomato and may be adapted by the commercial tomato growers for profitable management decisions.

Rockfall event is common on rocky terrain, and it threatens people. Recent advances in remote sensing technologies allow us to model the terrain with high accuracy and resolution. In this study, we show the application of modeling a rockfall site in three-dimensional (3D) using an unmanned aerial vehicle. The acquisition of information about discontinuities in the rock masses using 3D point clouds is vital to characterize rockfall dynamics. We obtained 3D spatial information and point clouds of a rocky slope from acquired images. We extracted 10 discontinuity sets from the 3D model as they have significant effects on the failure mechanism. Details in mapping discontinuities have improved significantly with the help of point clouds. The results are beneficial for modeling the terrain of inaccessible areas with high resolution and obtaining information from those areas. This paper shows how can point cloud analysis be applied to characterize discontinuity sets.

Introduction and aimsIntraoral scanning (IOS) captures real-time surface morphology of teeth and is widely used in clinical dentistry. However, due to the complex intraoral environment, data loss during scanning is common, leading to incomplete three-dimensional (3D) point clouds. This study aimed to develop and evaluate a deep learning–based method to automatically restore missing regions in intraoral 3D point clouds, thereby improving the accuracy and efficiency of digital orthodontic workflows.MethodsA Point Fractal Network architecture was adopted to reconstruct incomplete IOS data. A dataset comprising 314 IOS scans and 4162 individual teeth was used for training and validation. Missing data were simulated by removing random portions of point clouds (5%, 10%, 15%, and 20%). Model performance was assessed using Chamfer distance (CD) to measure the accuracy of point cloud completion across different levels of data loss.ResultsThe proposed method achieves robust performance, maintaining average CD values below 0.01 across most levels of simulated data loss. Visual comparisons confirmed high geometric fidelity between the completed and original point clouds. Furthermore, the model demonstrated efficient processing, completing each point cloud in approximately 0.5 seconds, enabling near real-time restoration during clinical scanning.ConclusionThe deep learning–based model accurately restores missing IOS data, improving the precision and efficiency of digital dental workflows. Its speed and accuracy support real-time clinical applications and reduce reliance on manual corrections.Clinical RelevanceThis method improves clinical efficiency, reduces chairside time, and enhances both patient comfort and treatment acceptance. In addition, it minimises human error and increases the precision of dental restorations. As digital dentistry continues to evolve, this approach holds great potential for improving the accuracy and efficiency of dental treatments, paving the way for broader artificial intelligence integration in clinical practice.