Extraction of damaged building's geometric features from multi-source point clouds

In recent years, low-cost unmanned aerial vehicles (UAVs) photogrammetry and terrestrial laser scanner (TLS) techniques have become very important non-contact measurement methods for obtaining topographic data about landslides. However, owing to the differences in the types of UAVs and whether the ground control points (GCPs) are set in the measurement, the obtained topographic data for landslides often have large precision differences. In this study, two types of UAVs (DJI Mavic Pro and DJI Phantom 4 RTK) with and without GCPs were used to survey a loess landslide. UAVs point clouds and digital surface model (DSM) data for the landslide were obtained. Based on this, we used the Geomorphic Change Detection software (GCD 7.0) and the Multiscale Model-To-Model Cloud Comparison (M3C2) algorithm in the Cloud Compare software for comparative analysis and accuracy evaluation of the different point clouds and DSM data obtained using the same and different UAVs. The experimental results show that the DJI Phantom 4 RTK obtained the highest accuracy landslide terrain data when the GCPs were set. In addition, we also used the Maptek I-Site 8,820 terrestrial laser scanner to obtain higher precision topographic point cloud data for the Beiguo landslide. However, owing to the terrain limitations, some of the point cloud data were missing in the blind area of the TLS measurement. To make up for the scanning defect of the TLS, we used the iterative closest point (ICP) algorithm in the Cloud Compare software to conduct data fusion between the point clouds obtained using the DJI Phantom 4 RTK with GCPs and the point clouds obtained using TLS. The results demonstrate that after the data fusion, the point clouds not only retained the high-precision characteristics of the original point clouds of the TLS, but also filled in the blind area of the TLS data. This study introduces a novel perspective and technical scheme for the precision evaluation of UAVs surveys and the fusion of point clouds data based on different sensors in geological hazard surveys.

Abstract. Soil erosion is a decisive earth surface process strongly influencing the fertility of arable land. Several options exist to detect soil erosion at the scale of large field plots (here 600 m²), which comprise different advantages and disadvantages depending on the applied method. In this study, the benefits of unmanned aerial vehicle (UAV) photogrammetry and terrestrial laser scanning (TLS) are exploited to quantify soil surface changes. Beforehand data combination, TLS data is co-registered to the DEMs generated with UAV photogrammetry. TLS data is used to detect global as well as local errors in the DEMs calculated from UAV images. Additionally, TLS data is considered for vegetation filtering. Complimentary, DEMs from UAV photogrammetry are utilised to detect systematic TLS errors and to further filter TLS point clouds in regard to unfavourable scan geometry (i.e. incidence angle and footprint) on gentle hillslopes. In addition, surface roughness is integrated as an important parameter to evaluate TLS point reliability because of the increasing footprints and thus area of signal reflection with increasing distance to the scanning device. The developed fusion tool allows for the estimation of reliable data points from each data source, considering the data acquisition geometry and surface properties, to finally merge both data sets into a single soil surface model. Data fusion is performed for three different field campaigns at a Mediterranean field plot. Successive DEM evaluation reveals continuous decrease of soil surface roughness, reappearance of former wheel tracks and local soil particle relocation patterns.

For more than a decade, Terrestrial Laser Scanning (TLS) has been a primary remote sensing technique for disciplines related to archaeology, architecture, built heritage, earth science, metrology, and land survey. The increasing precision, range, and survey speed of TLS make this technology even more viable for large-scale data capturing in the Age of Sensing. This chapter reviews the state of the art of Terrestrial Laser Scanning in 2015 with the aim to assess its applications in a context of lower data capturing costs for alternative technologies, such as new commodity sensors, Image-based 3D Modeling, Unmanned Aerial Systems (UAS), optical 3D scanning, and Airborne Laser Scanning. More specifically, TLS still maintains a fundamental role in the documentation and interpretation of archaeological contexts at intrasite scale: (i) Terrestrial Laser Scanning delivers high-fidelity data of surfaces and structures of buildings as well as ultra-precise measurements of the morphology of stratigraphic layers; (ii) research in remote sensing proved that TLS point clouds can be successfully interpolated with data recorded with other instruments and techniques, such as magnetometry, Ground Penetrating Radar, Unmanned Aerial Vehicles, Image-Based Modeling, in order to generate hybrid documentation and new knowledge on natural and cultural heritage sites. Inevitably, the current advancements in TLS bring new questions. For example, how can micro-differences only visible in the point clouds change the analysis and interpretation of layers and buildings? How to improve the monitoring and conservation of a site via automated analysis of TLS data? How to enhance the mapping process of built-heritage using data segmentation or semi-automatic feature extraction of TLS point clouds? This chapter proposes a new approach to TLS based on multi-modal capture workflows, semi-automated post processing, online archiving, and online visualization and management of point clouds with the aim to open new horizons for digital archaeology, architectural survey, and heritage conservation .

In the current modern era of information and technology, emerging remote advancements have been widely established for detailed virtual inspections and assessments of infrastructure assets, especially bridges. These technologies are capable of creating an accurate digital representation of the existing assets, commonly known as the digital twins. Digital twins are suitable alternatives to in-person and on-site based assessments that can provide safer, cheaper, more reliable, and less distributive bridge inspections. In the case of bridge monitoring, Unmanned Aerial Vehicle (UAV) photogrammetry and Terrestrial Laser Scanning (TLS) are among the most common advanced technologies that hold the potential to provide qualitative digital models; however, the research is still lacking a reliable methodology to evaluate the generated point clouds in terms of quality and geometric accuracy for a bridge size case study. Therefore, this paper aims to provide a comprehensive methodology along with a thorough bridge case study to evaluate two digital point clouds developed from an existing Australian heritage bridge via both UAV-based photogrammetry and TLS. In this regard, a range of proposed approaches were employed to compare point clouds in terms of points’ distribution, level of outlier noise, data completeness, surface deviation, and geometric accuracy. The comparative results of this case study not only proved the capability and applicability of the proposed methodology and approaches in evaluating these two voluminous point clouds, but they also exhibited a higher level of point density and more acceptable agreements with as-is measurements in TLS-based point clouds subjected to the implementation of a precise data capture and a 3D reconstruction model.

The accurate estimation of tree attributes is essential for sustainable forest management. Terrestrial Laser Scanning (TLS) is a viable remote sensing technology suitable for estimating under canopy structure. However, TLS measurements generally underestimate tree height in taller trees, which leads to the underestimation of other tree attributes (e.g., stem volume). The integration of information derived from TLS and Unmanned Aerial Vehicle (UAV) photogrammetry could potentially improve tree height estimation. This study investigated the applicability of integrating TLS and UAV photogrammetry to estimate individual tree attributes in managed coniferous forests of Japan. Diameter at breast height (DBH), tree height, and stem volume were estimated by (1) TLS data only, (2) integrating TLS and UAV data with TLS tree locations, and (3) integrating TLS and UAV data with treetop detections of the tree canopy. The TLS data only approach achieved high accuracy for DBH estimations with a root mean squared error (RMSE) of 2.36 cm (RMSE% 5.6%); however, tree height was greatly underestimated, with an RMSE of 8.87 m (28.9%) and a bias of −8.39 m. Integrating TLS and UAV photogrammetric data improved tree height estimation accuracy for both the TLS tree location (RMSE of 1.89 m and a bias of −0.46 m) and the treetop detection (RMSE of 1.77 m and a bias of 0.36 m) approaches. Integrating TLS and UAV photogrammetric data also improved the accuracy of the stem volume estimations with RMSEs of 0.21 m3 (10.8%) and 0.21 m3 (10.5%) for the TLS tree location and treetop detection approaches, respectively. Although the tree height of suppressed trees tended to be overestimated by TLS and UAV photogrammetric data integration, a good performance was obtained for dominant trees. The results of this study indicate that the integration of TLS and UAV photogrammetry is beneficial for the accurate estimation of tree attributes in coniferous forests.

Robust and rapid image dense matching is the key to large-scale three-dimensional (3D) reconstruction for multiple Unmanned Aerial Vehicle (UAV) images. However, the following problems must be addressed: (1) the amount of UAV image data is very large, but ordinary computer memory is limited; (2) the patch-based multi-view stereo-matching algorithm (PMVS) does not work well for narrow-baseline cases, and its computing efficiency is relatively low, and thus, it is difficult to meet the UAV photogrammetry’s requirements of convenience and speed. This paper proposes an Image-grouping and Self-Adaptive Patch-based Multi-View Stereo-matching algorithm (IG-SAPMVS) for multiple UAV imagery. First, multiple UAV images were grouped reasonably by a certain grouping strategy. Second, image dense matching was performed in each group and included three processes. (1) Initial feature-matching consists of two steps: The first was feature point detection and matching, which made some improvements to PMVS, according to the characteristics of UAV imagery. The second was edge point detection and matching, which aimed to control matching propagation during the expansion process; (2) The second process was matching propagation based on the self-adaptive patch. Initial patches were built that were centered by the obtained 3D seed points, and these were repeatedly expanded. The patches were prevented from crossing the discontinuous terrain by using the edge constraint, and the extent size and shape of the patches could automatically adapt to the terrain relief; (3) The third process was filtering the erroneous matching points. Taken the overlap problem between each group of 3D dense point clouds into account, the matching results were merged into a whole. Experiments conducted on three sets of typical UAV images with different texture features demonstrate that the proposed algorithm can address a large amount of UAV image data almost without computer memory restrictions, and the processing efficiency is significantly better than that of the PMVS algorithm and the matching accuracy is equal to that of the state-of-the-art PMVS algorithm.

Analysing the potential of UAV point cloud as input in quantitative structure modelling for assessment of woody biomass of single trees

To improve the efficiency and effectiveness of mosaicking unmanned aerial vehicle (UAV) images, we propose in this paper a rapid mosaicking method based on scale-invariant feature transform (SIFT) for mosaicking UAV images used for crop growth monitoring. The proposed method dynamically sets the appropriate contrast threshold in the difference of Gaussian (DOG) scale-space according to the contrast characteristics of UAV images used for crop growth monitoring. Therefore, this method adjusts and optimizes the number of matched feature point pairs in UAV images and increases the mosaicking efficiency. Meanwhile, based on the relative location relationship of UAV images used for crop growth monitoring, the random sample consensus (RANSAC) algorithm is integrated to eliminate the influence of mismatched point pairs in UAV images on mosaicking and to keep the accuracy and quality of mosaicking. Mosaicking experiments were conducted by setting three types of UAV images in crop growth monitoring: visible, near-infrared, and thermal infrared. The results indicate that compared to the standard SIFT algorithm and frequently used commercial mosaicking software, the method proposed here significantly improves the applicability, efficiency, and accuracy of mosaicking UAV images in crop growth monitoring. In comparison with image mosaicking based on the standard SIFT algorithm, the time efficiency of the proposed method is higher by 30%, and its structural similarity index of mosaicking accuracy is about 0.9. Meanwhile, the approach successfully mosaics low-resolution UAV images used for crop growth monitoring and improves the applicability of the SIFT algorithm, providing a technical reference for UAV application used for crop growth and phenotypic monitoring.

The paper presents an efficient photogrammetric workflow to improve the 3D reconstruction of scenes surveyed by integrating terrestrial and Unmanned Aerial Vehicle (UAV) images. In the last years, the integration of this kind of images has shown clear advantages for the complete and detailed 3D representation of large and complex scenarios. Nevertheless, their photogrammetric integration often raises several issues in the image orientation and dense 3D reconstruction processes. Noisy and erroneous 3D reconstructions are the typical result of inaccurate orientation results. In this work, we propose an automatic filtering procedure which works at the sparse point cloud level and takes advantage of photogrammetric quality features. The filtering step removes low-quality 3D tie points before refining the image orientation in a new adjustment and generating the final dense point cloud. Our method generalizes to many datasets, as it employs statistical analyses of quality feature distributions to identify suitable filtering thresholds. Reported results show the effectiveness and reliability of the method verified using both internal and external quality checks, as well as visual qualitative comparisons. We made the filtering tool publicly available on GitHub.

Recent advances in structure from motion (SfM) and dense matching algorithms enable surface reconstruction from unmanned aerial vehicle (UAV) images with high spatial resolution, allowing for new insights into earth surface processes. However, accuracy issues are inherent in parallel‐axes UAV image configurations. In this study, the quality of digital elevation models (DEMs) is assessed using images from a simulated UAV flight. Five different SfM tools and three different cameras are compared. If ground control points (GCPs) are not integrated into the adjustment process with parallel‐axes image configurations, significant dome‐effect systematic errors are observed, which can be reduced based on calibration parameters retrieved from a testfield captured with convergent images immediately before or after the UAV flight. A comparison between DEMs of a soil surface generated from UAV images and terrestrial laser‐scanning data show that natural surfaces can be very accurately reconstructed from UAV images, even when GCPs are missing and simple geometric camera models are considered.

For monitoring purposes and in the context of geomorphological research, Unmanned Aerial Vehicles (UAV) appear to be a promising solution to provide multi-temporal Digital Surface Models (DSMs) and orthophotographs. There are a variety of photogrammetric software tools available for UAV-based data. The objective of this study is to investigate the level of accuracy that can be achieved using two of these software tools: Agisoft PhotoScan® Pro and an open-source alternative, IGN© MicMac®, in sub-optimal survey conditions (rugged terrain, with a large variety of morphological features covering a range of roughness sizes, poor GPS reception). A set of UAV images has been taken by a hexacopter drone above the Rivière des Remparts, a river on Reunion Island. This site was chosen for its challenging survey conditions: the topography of the study area (i) involved constraints on the flight plan; (ii) implied errors on some GPS measurements; (iii) prevented an optimal distribution of the Ground Control Points (GCPs) and; (iv) was very complex to reconstruct. Several image processing tests are performed with different scenarios in order to analyze the sensitivity of each software package to different parameters (image quality, numbers of GCPs, etc.). When computing the horizontal and vertical errors within a control region on a set of ground reference targets, both methods provide rather similar results. A precision up to 3–4 cm is achievable with these software packages. The DSM quality is also assessed over the entire study area comparing PhotoScan DSM and MicMac DSM with a Terrestrial Laser Scanner (TLS) point cloud. PhotoScan and MicMac DSM are also compared at the scale of particular features. Both software packages provide satisfying results: PhotoScan is more straightforward to use but its source code is not open; MicMac is recommended for experimented users as it is more flexible.

To develop an intelligent recognition model based on deep learning algorithms of unmanned aerial vehicle (UAV) images, and to preliminarily explore the value of this model for remote identification, monitoring and management of cattle, a source of Schistosoma japonicum infection. Oncomelania hupensis snail-infested marshlands around the Poyang Lake area were selected as the study area. Image datasets of the study area were captured by aerial photography with UAV and subjected to augmentation. Cattle in the sample database were annotated with the annotation software VGG Image Annotator to create the morphological recognition labels for cattle. A model was created for intelligent recognition of livestock based on deep learning-based Mask R-convolutional neural network (CNN) algorithms. The performance of the model for cattle recognition was evaluated with accuracy, precision, recall, F1 score and mean precision. A total of 200 original UAV images were obtained, and 410 images were yielded following data augmentation. A total of 2 860 training samples of cattle recognition were labeled. The created deep learning-based Mask R-CNN model converged following 200 iterations, with an accuracy of 88.01%, precision of 92.33%, recall of 94.06%, F1 score of 93.19%, and mean precision of 92.27%, and the model was effective to detect and segment the morphological features of cattle. The deep learning-based Mask R-CNN model is highly accurate for recognition of cattle based on UAV images, which is feasible for remote intelligent recognition, monitoring, and management of the source of S. japonicum infection.

In forestry research, for forest inventories or other applications which require accurate 3D information on the forest structure, a Terrestrial Laser Scanner (TLS) is an efficient tool for vegetation structure estimation. Light Detection and Ranging (LiDAR) can even provide high-resolution information in tree canopies due to its high penetration capability. Depending on the forest plot size, tree density, and structure, multiple TLS scans are acquired to cover the forest plot in all directions to avoid any voids in the dataset that are generated. However, while increasing the number of scans, we often tend to increase the data redundancy as we keep acquiring data for the same region from multiple scan positions. In this research, an extensive qualitative analysis was carried out to examine the capability and efficiency of TLS to generate canopy top points in six different scanning combinations. A total of nine scans were acquired for each forest plot, and from these nine scans, we made six different combinations to evaluate the 3D vegetation structure derived from each scan combination, such as Center Scans (CS), Four Corners Scans (FCS), Four Corners with Center Scans (FCwCS), Four Sides Center Scans (FSCS), Four Sides Center with Center Scans (FSCwCS), and All Nine Scans (ANS). We considered eight forest plots with dimensions of 25 m × 25 m, of which four plots were of medium tree density, and the other four had a high tree density. The forest plots are located in central Slovakia; European beech was the dominant tree species with a mixture of European oak, Silver fir, Norway spruce, and European hornbeam. Altogether, 487 trees were considered for this research. The quantification of tree canopy top points obtained from a TLS point cloud is very crucial as the point cloud is used to derive the Digital Surface Model (DSM) and Canopy Height Model (CHM). We also performed a statistical evaluation by calculating the differences in the canopy top points between ANS and the five other combinations and found that the most significantly different combination was FSCwCS respective to ANS. The Root Mean Squared Error (RMSE) of the deviations in tree canopy top points obtained for plots TLS_Plot1 and TLS_Plot2 ranged from 0.89 m to 14.98 m and 0.61 m to 7.78 m, respectively. The relative Root Mean Squared Error (rRMSE) obtained for plots TLS_Plot1 and TLS_Plot2 ranged from 0.15% to 2.48% and 0.096% to 1.22%, respectively.

The quantitative, accurate and efficient acquisition of tree phenotypes is the basis for forest “gene-phenotype-environment” studies. It also offers significant support for clarifying the genetic control mechanisms of tree traits. The application of unmanned aerial vehicle (UAV) remote sensing technology to the collection of phenotypic traits at an individual tree level quantitatively analyses tree phenology and directionally evaluates tree growth, as well as accelerating the process of forest genetics and breeding. In this study, with the help of high-resolution, high-overlap, multispectral images obtained by an UAV, combined with digital elevation models (DEMs) extracted from point clouds acquired by a backpack LiDAR, a high-throughput tree structure and spectral phenotypic traits extraction and a genetic selection were conducted in a trial of Eucalyptus clones in the State-owned Dongmen Forest Farm in the Guangxi Zhuang Autonomous Region. Firstly, we validated the accuracy of extracting the phenotypic parameters of individual tree growth based on aerial stereo photogrammetry point clouds. Secondly, on this basis, the repeatability of the tree growth traits and vegetation indices (VIs), the genetic correlation coefficients between the traits were calculated. Finally, the eucalypt clones were ranked by integrating a selection index of traits, and the superior genotypes were selected and their genetic gain predicted. The results showed a high accuracy of the tree height (H) extracted from the digital aerial photogrammetry (DAP) point cloud based on UAV images (R2 = 0.91, and RMSE = 0.56 m), and the accuracy of estimating the diameter at breast height (DBH) was R2 = 0.71, and RMSE = 0.75 cm. All the extracted traits were significantly different within the tree species and among the clones. Except for the crown width (CW), the clonal repeatability (Rc) of the traits were all above 0.9, and the individual repeatability values (Ri) were all above 0.5. The genetic correlation coefficient between the tree growth traits and VIs fluctuated from 0.3 to 0.5, while the best clones were EA14-15, EA14-09, EC184, and EC183 when the selection proportion was 10%. The purpose of this study was to construct a technical framework for phenotypic traits extraction and genetic analysis of trees based on unmanned aerial stereo photography point clouds and high-resolution multispectral images, while also exploring the application potential of this approach in the selective breeding of eucalypt clones.

Stereo images have long been the main practical data source for the high-accuracy retrieval of 3-D information over large areas. However, stereoscopy has been surpassed by laser scanning (LS) techniques in recent years, particularly in forested areas, because the reflection of laser points from object surfaces directly provides 3-D geometric features and because the laser beam has good penetration capacity through forest canopies. In the last few years, image-based point clouds have become a more widely available data source because of advances in matching algorithms and computer hardware. This paper explores the possibility of using consumer cameras for forest field data collection and presents an application of terrestrial image-based point clouds derived from a handheld camera to forest plot inventories. In the experiment, the sample forest plot was photographed in a stop-and-go mode using different routes and camera settings. Five data sets were generated from photographs taken in the field, representing different photographic conditions. The stem detection accuracy ranged between 60% and 84%, and the root-mean-square errors of the estimated diameters at breast height were between 2.98 and 6.79 cm. The performance of image-based point clouds in forest data collection was compared with that of point clouds derived from two LS techniques, i.e., terrestrial LS (the professional level) and personal LS (an emerging technology). The study indicates that the construction of image-based point clouds of forest field data requires only low-cost, low-weight, and easy-to-use equipment and automated data processing. Photographic measurement is easy and relatively fast. The accuracy of tree attribute estimates is close to an acceptable level for forest field inventory but is lower than that achieved with the tested LS techniques.