Digital surface model generation from high-resolution satellite stereo imagery based on structural similarity

Geometric integration of high-resolution satellite imagery and airborne LiDAR data for improved geopositioning accuracy in metropolitan areas

In many cases, the Digital Surface Models (DSMs) and Digital Elevation Models (DEMs) are obtained with Light Detection and Ranging (LiDAR) or stereo matching. As an active method, LiDAR is very accurate but expensive, thus often limiting its use in small-scale acquisition. Stereo matching is suitable for large-scale acquisition of terrain information as the increase of satellite stereo sensors. However, underperformance of stereo matching easily occurs in textureless areas. Accordingly, this study proposed a Shading Aware DSM GEneration Method (SADGE) with high resolution multi-view satellite images. Considering the complementarity of stereo matching and Shape from Shading (SfS), SADGE combines the advantage of stereo matching and SfS technique. First, an improved Semi-Global Matching (SGM) technique is used to generate an initial surface expressed by a DSM; then, it is refined by optimizing the objective function which modeled the imaging process with the illumination, surface albedo, and normal object surface. Different from the existing shading-based DEM refinement or generation method, no information about the illumination or the viewing angle is needed while concave/convex ambiguity can be avoided as multi-view images are utilized. Experiments with ZiYuan-3 and GaoFen-7 images show that the proposed method can generate higher accuracy DSM (12.5–56.3% improvement) with sound overall shape and temporarily detailed surface compared with a software solution (SURE) for multi-view stereo.

The world is witnessing massive demographic residents transitions from rural to urban, resulting in alarming consequences. These shifts highlight the necessity of comprehensive urban land-use mapping for current assessment and future city planning and management. The integration of Light Detection and Ranging (LiDAR) and imagery data allows using their complementary geometric and radiometric characteristics, respectively, with a potential increase in urban mapping accuracies. However, airborne LiDAR data may not always come with onboard optical images collected during the same flight mission. Indirect geo-referencing can be adopted if ancillary imagery data are available. Nevertheless, automatic recognition of control primitives in LiDAR and imagery datasets becomes challenging when collected on different dates. This dissertation proposes a generic geo-registration mechanism using the Phase Congruency (PC) model and scene abstraction to overcome the stated challenges. The Root Mean Square Error (RMSE) is between one to two pixels. The proposed workflow is computationally efficient, especially with small datasets, and generic enough to apply to various urban morphologies. A point-based classification was then performed on a point cloud covering a residential area using machine learning. The data was geo-registered to an aerial image. The classification tried multiple combinations of the inherited image spectra and the height as the only geometric feature in the classifying spaces. The acquired radiometric features added to the height gave an overall accuracy above 97% using Multi-Layer Perceptron (MLP) neural networks, surpassing those achieved with height-based geometric-oriented feature spaces. However, misclassifications occurred among different classes due to the independent acquisition of the aerial and LiDAR data and shadow and orthorectification problems from the aerial images. The dissertation then moves on to modify the color-based segmentation algorithm for the object-based classification of the multispectral LiDAR version of the same point cloud. The dissertation proposes a multilevel classification through vectors of inherited and calculated height-based geometric and spectral features structured in different classification scenarios (combinations). An overall mapping accuracy higher than 98% was achieved using MLP neural networks, combining the entire feature vectors and their optimized output space projected by the Principal Component Analysis (PCA). This combination also eliminates some of the abovementioned misclassifications.

Differential LiDAR (Light Detection and Ranging) from repeated surveys has recently emerged as an effective tool to measure three-dimensional (3D) change for applications, such as quantifying slip and spatially distributed warping associated with earthquake ruptures, and examining the spatial distribution of beach erosion after hurricane impact. Currently, the primary method for determining 3D change from LiDAR is through the use of the iterative closest point (ICP) algorithm and its variants. However, all current studies using ICP have assumed that all LiDAR points in the compared point clouds have uniform accuracy. This assumption is simplistic given that the error for each LiDAR point is variable, and dependent upon time varying factors such as target range, angle of incidence, and aircraft trajectory accuracy. Therefore, to rigorously determine spatial change, it would be ideal to model the random error for every LiDAR observation in the differential point cloud, and use these error estimates as apriori weights in the ICP algorithm. To test this approach, we implemented a rigorous LiDAR observation error propagation method to generate estimated random error for each point in a LiDAR point cloud, and then determine 3D displacements between two point clouds using an anisotropic weighted ICP (A-ICP) algorithm. The algorithm was evaluated by qualitatively and quantitatively comparing point clouds with synthetic fault ruptures between a uniform weight and anistropically weighted ICP algorithm. Then post-earthquake slip is estimated for the 2010 El Mayor-Cucapah Earthquake (EMC), using pre- and post-event LiDAR. Based on the analysis, Moving Window A-ICP is able to better estimate the synthetic surface ruptures, and provides a smoother estimate of actual displacement for the EMC earthquake.

Routine bridge inspections require labor intensive and highly subjective visual interpretation to determine bridge deck surface condition. Light Detection and Ranging (LiDAR) a relatively new class of survey instrument has become a popular and increasingly used technology for providing as-built and inventory data in civil applications. While an increasing number of private and governmental agencies possess terrestrial and mobile LiDAR systems, an understanding of the technology’s capabilities and potential applications continues to evolve. LiDAR is a line-of-sight instrument and as such, care must be taken when establishing scan locations and resolution to allow the capture of data at an adequate resolution for defining features that contribute to the analysis of bridge deck surface condition. Information such as the location, area, and volume of spalling on deck surfaces, undersides, and support columns can be derived from properly collected LiDAR point clouds. The LiDAR point clouds contain information that can provide quantitative surface condition information, resulting in more accurate structural health monitoring. LiDAR scans were collected at three study bridges, each of which displayed a varying degree of degradation. A variety of commercially available analysis tools and an independently developed algorithm written in ArcGIS Python (ArcPy) were used to locate and quantify surface defects such as location, volume, and area of spalls. The results were visual and numerically displayed in a user-friendly web-based decision support tool integrating prior bridge condition metrics for comparison. LiDAR data processing procedures along with strengths and limitations of point clouds for defining features useful for assessing bridge deck condition are discussed. Point cloud density and incidence angle are two attributes that must be managed carefully to ensure data collected are of high quality and useful for bridge condition evaluation. When collected properly to ensure effective evaluation of bridge surface condition, LiDAR data can be analyzed to provide a useful data set from which to derive bridge deck condition information.

3D Modeling Approach of Building Construction based on Point Cloud Data Using LiDAR Byeongjun Oh, Minju Kim, Chanwoo Lee, Hunhee Cho and Kyung-In Kang Pages 906-912 (2019 Proceedings of the 36th ISARC, Banff, Canada, ISBN 978-952-69524-0-6, ISSN 2413-5844) Abstract: LiDAR (Light Detection and Ranging) emerges as a mapping technology that provides fast, accurate, and reliable data with geometric representation of construction facilities. The technology has received high recognition especially since the importance of digital transformation in construction projects has been emphasized. However, the current state of 3D technologies including LiDAR encounters difficulty in recognizing and extracting accurate information on the as-built status of the buildings and construction sites. As a preliminary study for the development of a 3D architectural geometric representation of building and environment, this paper proposes a 3D modeling approach based on point cloud data obtained by LiDAR technology. This paper suggests a novel method for data acquisition on as-built status of construction sites and buildings during construction-operation phases. Furthermore, a field test has been performed for visualizing and modeling of the indoors of a residential house. The results of this paper are expected to inaugurate adaptation of LiDAR technology during the as-built process, and further implementation of digital technology after construction phase in construction projects. Keywords: LiDAR; 3D modeling; building construction; digital transformation; point cloud data DOI: https://doi.org/10.22260/ISARC2019/0121 Download fulltext Download BibTex Download Endnote (RIS) TeX Import to Mendeley

The 3D object detection is becoming indispensable for environmental perception in autonomous driving. Light detection and ranging (LiDAR) point clouds often fail to distinguish objects with similar structures and are quite sparse for distant or small objects, thereby introducing false and missed detections. To address these issues, LiDAR is often fused with cameras due to the rich textural information provided by images. However, current fusion methods suffer the inefficient data representation and inaccurate alignment of heterogeneous features, leading to poor precision and low efficiency. To this end, we propose a plug-and-play module termed range-image fusion (RI-Fusion) to achieve an effective fusion of LiDAR and camera data, designed to be easily accessible by existing mainstream LiDAR-based algorithms. In this process, we design an image and point cloud alignment method by converting a point cloud into a compact range-view representation through a spherical coordinate transformation. The range image is then integrated with a corresponding camera image utilizing an attention mechanism. The original range image is then concatenated with fusion features to retain point cloud information, and the results are projected onto a spatial point cloud. Finally, the feature-enhanced point cloud can be input into a LiDAR-based 3D object detector. The results of validation experiments involving the KITTI 3D object detection benchmark showed that our proposed fusion method significantly enhanced multiple mainstream LiDAR-based 3D object detectors, PointPillars, SECOND, and Part <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\text{A}{^{2}}$ </tex-math></inline-formula> , improving the 3D mAP (mean Average Precision) by 3.61%, 2.98%, and 1.27%, respectively, particularly for small objects such as pedestrians and cyclists.

With an increase in the frequency and severity of wildfires across the globe and resultant changes to long-established fire regimes, the mapping of fire severity is a vital part of monitoring ecosystem resilience and recovery. The emergence of unoccupied aircraft systems (UAS) and compact sensors (RGB and LiDAR) provide new opportunities to map fire severity. This paper conducts a comparison of metrics derived from UAS Light Detecting and Ranging (LiDAR) point clouds and UAS image based products to classify fire severity. A workflow which derives novel metrics describing vegetation structure and fire severity from UAS remote sensing data is developed that fully utilises the vegetation information available in both data sources. UAS imagery and LiDAR data were captured pre- and post-fire over a 300 m by 300 m study area in Tasmania, Australia. The study area featured a vegetation gradient from sedgeland vegetation (e.g., button grass 0.2m) to forest (e.g., Eucalyptus obliqua and Eucalyptus globulus 50m). To classify the vegetation and fire severity, a comprehensive set of variables describing structural, textural and spectral characteristics were gathered using UAS images and UAS LiDAR datasets. A recursive feature elimination process was used to highlight the subsets of variables to be included in random forest classifiers. The classifier was then used to map vegetation and severity across the study area. The results indicate that UAS LiDAR provided similar overall accuracy to UAS image and combined (UAS LiDAR and UAS image predictor values) data streams to classify vegetation (UAS image: 80.6%; UAS LiDAR: 78.9%; and Combined: 83.1%) and severity in areas of forest (UAS image: 76.6%, UAS LiDAR: 74.5%; and Combined: 78.5%) and areas of sedgeland (UAS image: 72.4%; UAS LiDAR: 75.2%; and Combined: 76.6%). These results indicate that UAS SfM and LiDAR point clouds can be used to assess fire severity at very high spatial resolution.

Detection and delineation of individual trees is important for the estimation of forest information using Light Detection and Ranging (LiDAR) point clouds. However, over-segmentation of canopy height models (CHMs) is frequently caused by subcrowns and branches, whereas under-segmentation is caused by adjacent crowns, especially in deciduous forests. To eliminate such problems, an approach for delineating individual trees based on curvature attributes derived from LiDAR data is proposed. To detect accurate treetops for reducing over-segmentation, the dominant sizes of tree crowns are first determined and used to fit the dome-shaped tree crowns in CHMs, and then the minimum curvature values of fitted CHMs at different scales are calculated for treetop detection. For delineation of individual trees, tree features such as height variation and curvature are chosen to form a feature vector as a pixel-level logistic regression classifier. Then an ensemble classifier combining multiple logistic regression classifiers is trained to analyze the similarity between neighboring points to reduce under-segmentation. Finally, segment maps at different scales are integrated to generate the final segment map. Our experimental results on 2 test areas demonstrate that the proposed method has promising applications for eliminating over-segmentation and under-segmentation in deciduous forests.

The intelligent road is an important component of the intelligent vehicle infrastructure cooperative system, the latest development of intelligent transportation systems. As an advanced sensor, Light Detection and Ranging (LiDAR) has gradually been used to collect high-resolution micro-traffic data on the roadside of intelligent roads. Furthermore, a fusion of multiple LiDARs has become a current hot spot to extend the data collection range and improve detection accuracy. This paper focuses on point cloud registration in a complex traffic environment and proposes a three-dimensional (3D) registration method based on traffic signs and prior knowledge of traffic scenes. Traffic signs with their reflective films are used as reference targets to register 3D point cloud data from roadside LiDARs. The proposed method consists of a vertical registration and a horizontal registration. For the vertical registration, we propose a panel rotation algorithm to rotate the initial point cloud to register it vertically, converting the 3D point cloud registration into a two-dimensional (2D) rigid body transformation. For the vertical registration, our system registers traffic signs from different LiDARs. Our method has been verified in some actual scenarios. Compared with previous methods, the proposed method is automatic and does not need to search reference targets manually. Furthermore, it is suitable for actual engineering use and can be applied to sparse point cloud data from LiDAR with few beams, realizing point cloud registration of large disparity.

Robust multi-task learning network for complex LiDAR point cloud data preprocessing

Chapter 5 - Data Preprocessing and Feature Extraction

A Light Detection and Ranging (LiDAR) sensor mounted on an Unmanned Aerial Vehicle (UAV) can map the overflown environment in point clouds. Mapped canopy heights allow for the estimation of crop biomass in agriculture. The work presented in this paper contributes to sensory UAV setup design for mapping and textual analysis of agricultural fields. LiDAR data are combined with data from Global Navigation Satellite System (GNSS) and Inertial Measurement Unit (IMU) sensors to conduct environment mapping for point clouds. The proposed method facilitates LiDAR recordings in an experimental winter wheat field. Crop height estimates ranging from 0.35–0.58 m are correlated to the applied nitrogen treatments of 0–300 . The LiDAR point clouds are recorded, mapped, and analysed using the functionalities of the Robot Operating System (ROS) and the Point Cloud Library (PCL). Crop volume estimation is based on a voxel grid with a spatial resolution of 0.04 × 0.04 × 0.001 m. Two different flight patterns are evaluated at an altitude of 6 m to determine the impacts of the mapped LiDAR measurements on crop volume estimations.

This paper presents a new algorithm for building an indoor map by integrating point clouds of 2D light detection and ranging (LIDAR) scanners in indoor environments. Iterative closest point (ICP) algorithm is one of the well-known methods for such purpose and often used for mobile robot SLAM. However, the algorithm is designed based on dense (or continuous) measurement of the same space with known relative positions of measurement points and angles, and it does not often work efficiently if the measurement is sparse and/or LIDAR locations are unknown. Such situations are seen when some installed LIDARs in a room are used to build a background indoor map for object tracking, or mobile LIDARs are used by technicians to build a digital indoor map, where each space is captured only at a few locations with different angles. To tackle this issue, our method extracts line segments and edge points as features from given LIDAR point clouds and finds shape coincidences commonly contained in a pair of given point clouds to identify positional relationships between LIDARs. By this information, these point clouds can be integrated into common 2D coordinates. Indoor map generation is realized by sequentially applying this integration procedure to every pair of point clouds. Also, the experiments on real data show that our method can identify the relative positions of LIDARs with 10cm-order errors in average, and by sequentially applying the point-cloud integration, the generated maps have only 3% errors.

Single tree detection is important for monitoring tree plantations in order to estimate crop yield, which can be used for assessing food and biofuels. In the field of remote sensing, there have been studies regarding single tree extraction using Light Detection and Ranging (LiDAR) point cloud data. Canopy Height Model (CHM) can be simply derived from these LiDAR data where trees can be analysed using different physical characteristics such as tree crown width, and height. However, using these physical characteristics can’t be very useful for identifying which trees are within a plantation, a forest, or a single tree. This is the motivation for developing algorithms of automated differentiation of random trees from plantation trees. Object Based Image Analysis (OBIA) was used to acquire the locations of individual trees. Trees were separated from other land cover in the image using a CHM derived from LiDAR point cloud data. Using watershed algorithm in eCognition, trees with good canopy spacing was easily distinguished. A spatial point pattern analysis was done from the extracted locations of trees to differentiate regularly patterned points from randomly distributed points. Nearest neighbour statistics was done to provide measurement of the distribution of the extracted points. The Z-Score parameter from nearest neighbor was then added to the layers in eCognition as supplement to the available height derivatives such as mean, standard deviation, and different textural parameters. Supervised type of classification was used in order to classify the trees on the image. Training samples of each tree from the image were entered in Support Vector Machine (SVM) to classify the different trees. Two sets of validation were done to compare the results of using the object parameters only, and the object parameters and Z-Score. Addition of the Z-Score from the object parameters increased the obtained accuracy on both validation sets. Classification accuracy based on validation set 1 and set 2 increased by about 20% and 6%, respectively. Additional spatial pattern analysis methods and validation can further prove the use of such layers in improving classification of trees.