LiDAR-guided stereo matching with a spatial consistency constraint

In many cases, the Digital Surface Models (DSMs) and Digital Elevation Models (DEMs) are obtained with Light Detection and Ranging (LiDAR) or stereo matching. As an active method, LiDAR is very accurate but expensive, thus often limiting its use in small-scale acquisition. Stereo matching is suitable for large-scale acquisition of terrain information as the increase of satellite stereo sensors. However, underperformance of stereo matching easily occurs in textureless areas. Accordingly, this study proposed a Shading Aware DSM GEneration Method (SADGE) with high resolution multi-view satellite images. Considering the complementarity of stereo matching and Shape from Shading (SfS), SADGE combines the advantage of stereo matching and SfS technique. First, an improved Semi-Global Matching (SGM) technique is used to generate an initial surface expressed by a DSM; then, it is refined by optimizing the objective function which modeled the imaging process with the illumination, surface albedo, and normal object surface. Different from the existing shading-based DEM refinement or generation method, no information about the illumination or the viewing angle is needed while concave/convex ambiguity can be avoided as multi-view images are utilized. Experiments with ZiYuan-3 and GaoFen-7 images show that the proposed method can generate higher accuracy DSM (12.5–56.3% improvement) with sound overall shape and temporarily detailed surface compared with a software solution (SURE) for multi-view stereo.

Improvement to LiDAR-camera extrinsic calibration by using 3D–3D correspondences

Light Detection and Ranging (LIDAR) is a fast method for sampling the earthpsilas surface with a high density and high accuracy point cloud that is used to generate high density and high accuracy Digital Elevation Models (DEMs) and DSMs. In this research we obtained airborne LIDAR elevation data with spatial resolution of 1 m to reveal new details of mountain block morphology and structure to investigate the origin of uphill facing scarps. Quaternary fault scarps occur in several mountain blocks in the western Saint Elias and Eastern Chugach Mountains of southern Alaska. Possible mechanisms for formation of these scarps include deformation caused by active folding, or deformation caused by gravitational loading and strong ground motion during earthquakes. The field observations and LIDAR visualization lead us to propose a three-stage model for flexural toppling. Failure is by flexural toppling, with rotation and shearing of bedding planes under the influence of gravity where bedding dips steeply into the mountain side. Down-slope bending of bedding surfaces may initiate formation of a basal sliding surface beneath the toppled beds, leading to landsliding down slope. However, horizontal acceleration caused by strong ground motion enhances the probability of failure by flexural toppling, especially in the upper parts of mountain slopes, where ground motion is amplified.

Registering light detection and ranging (LiDAR) data with optical camera images enhances spatial awareness in autonomous driving, robotics, and geographic information systems. The current challenges in this field involve aligning 2D-3D data acquired from sources with distinct coordinate systems, orientations, and resolutions. This paper introduces a new pipeline for camera–LiDAR post-registration to produce colorized point clouds. Utilizing deep learning-based matching between 2D spherical projection LiDAR feature layers and camera images, we can map 3D LiDAR coordinates to image grey values. Various LiDAR feature layers, including intensity, bearing angle, depth, and different weighted combinations, are used to find correspondence with camera images utilizing state-of-the-art deep learning matching algorithms, i.e., SuperGlue and LoFTR. Registration is achieved using collinearity equations and RANSAC to remove false matches. The pipeline’s accuracy is tested using survey-grade terrestrial datasets from the TX5 scanner, as well as datasets from a custom-made, low-cost mobile mapping system (MMS) named Simultaneous Localization And Mapping Multi-sensor roBOT (SLAMM-BOT) across diverse scenes, in which both outperformed their baseline solutions. SuperGlue performed best in high-feature scenes, whereas LoFTR performed best in low-feature or sparse data scenes. The LiDAR intensity layer had the strongest matches, but combining feature layers improved matching and reduced errors.

Automatic building extraction is an important topic for many applications such as urban planning, disaster management, 3D building modeling and updating GIS databases. Its approaches mainly depend on two data sources: light detection and ranging (LiDAR) point cloud and aerial imagery both of which have advantages and disadvantages of their own. In this study, in order to benefit from the advantages of each data sources, LiDAR and image data combined together. And then, the building boundaries were extracted with the automated active contour algorithm implemented in MATLAB. Active contour algorithm uses initial contour positions to segment an object in the image. Initial contour positions were detected without user interaction by a series of image enhancements, band ratio and morphological operations. Four test areas with varying building and background levels of detail were selected from ISPRS’s benchmark Vaihingen and Istanbul datasets. Vegetation and shadows were removed from all the datasets by band ratio to improve segmentation quality. Subsequently, LiDAR point cloud data was converted to raster format and added to the aerial imagery as an extra band. Resulting merged image and initial contour positions were given to the active contour algorithm to extract building boundaries. In order to compare the contribution of LiDAR to the proposed method, the boundaries of the buildings were extracted from the input image before and after adding LiDAR data to the image as a layer. Finally extracted building boundaries were smoothed by the Awrangjeb (Int J Remote Sen 37(3): 551–579. https://doi.org/10.1080/01431161.2015.1131868 , 2016) boundary regularization algorithm. Correctness (Corr), completeness (Comp) and accuracy (Q) metrics were used to assess accuracy of segmented building boundaries by comparing extracted building boundaries with manually digitized building boundaries. Proposed approach shows the promising results with over 93% correctness, 92% completeness and 89% quality.

This paper presents a wavelet-based (WB) echo detector that can recover the echoes missed by a light detection and ranging (LiDAR) system via on-the-fly detection. An on-the-fly detection method normally utilizes a simple threshold (TH) to register a target point. Points that belong to weak and/or overlapping echoes are much complicated and are easily missed by TH approaches. The proposed detector based on wavelet transformation is robust to noise and is capable of resolving overlapping echoes. It is thus expected to be good at handling missing echoes. A simulated waveform data set and a real waveform data set of a forest area were both used in this paper. The simulated waveform data were utilized to compare the proposed detector with zero crossing (ZC) and Gaussian decomposition (GD) detectors in terms of their ability to deal with weak or overlapping echoes. The real waveform data set acquired from Leica ALS60 was used to demonstrate a WB algorithm for exploring the missing echoes. Experiments using the simulated data showed that the WB and GD detectors are superior to the ZC detector in finding overlapping echoes. The WB algorithm performs well when dealing with overlapping echoes with a low signal-to-noise ratio. Experiments using the real waveform data show that 31.5% additional weak or overlapping echoes can be detected by the WB detector compared with the point cloud provided by the system. With such additional points, the mean and root-mean-square errors of the digital elevation model differences can be improved from 0.72 and 0.79 m to 0.16 and 0.59 m, respectively.

The application of interferometric synthetic aperture radar (InSAR) has been challenged over hilly and vegetated regions due to significant phase decorrelation. The spatiotemporal filtering techniques can partially overcome this, whereas the so-called ‘homogeneous pixels’ are selected based on the synthetic aperture radar (SAR) datasets and the computation is not satisfactorily efficient. In this letter, we propose the idea of interferogram homogeneous filtering relying on Light Detection and Ranging (LiDAR) derived topography features, instead of using amplitude or phase from the SAR images. The method exploits terrain slope and aspect to cluster topographically homogeneous pixels (THPs) and uses a coherence weighted phase-link (WPL) algorithm to reconstruct pixel phase histories. We compare the proposed method with two representative strategies, i.e., the amplitude statistically homogeneous pixels (SHPs) and the similar time-series interferometric phase pixels (STIPs). Results are given using L-band ALOS-1 data covering the forested Island of Lantau, Hong Kong. It is demonstrated that the LiDAR-derived THP has substantially decreased the computation load whereas the coherence of phase and the InSAR parameter estimation have been improved.

Automatic 2-D LiDAR geometric calibration of installation bias

Among the methods proposed for identifying the number of latent traits in multidimensional IRT models, DETECT has attracted the attention of both methodologists and applied researchers as a nonparametric counterpart to other procedures. The current study investigated the overall performance of the DETECT procedure and its outcomes using a real-data sampling design recommended by MacCallum (2003) and compared the results from a purely simulated data set that was generated with a well-specified “perfect” model. The comparison revealed that the sampling behavior of the maximized DETECT value and R-ratio statistics was quite robust to minor factors and other model misspecifications that potentially exist in the real data set, as there were negligible differences between the results of the real and simulated data sets. Item classification accuracy was also nearly identical for the real and simulated data sets. The accuracy of the identified number of dimensions reported by DETECT was the only outcome with an obvious difference between the purely simulated data set and the real data set. While the difference was small for smaller sample sizes, the identified number of dimensions was more accurate for larger sample sizes when the population data set was purely simulated. In many instances, exploratory DETECT analysis outperformed the cross-validated DETECT analysis in terms of overall accuracy.

Normal estimation is a crucial first step for numerous light detection and ranging (LiDAR) data-processing algorithms, from building reconstruction, road extraction, and ground-cover classification to scene rendering. For LiDAR point clouds in urban environments, this paper presents a robust method to estimate normals by constructing an octree-based hierarchical representation for the data and detecting a group of large enough consistent neighborhoods at multiscales. Consistent neighborhoods are mainly determined based on the observation that an urban environment is typically comprised of regular objects, e.g., buildings, roads, and the ground surface, and irregular objects, e.g., trees and shrubs; the surfaces of most regular objects can be approximatively represented by a group of local planes. Even in the frequent presence of heavy noise and anisotropic point samplings in LiDAR data, our method is capable of estimating robust normals for kinds of objects in urban environments, and the estimated normals are beneficial to more accurately segment and identify the objects, as well as preserving their sharp features and complete outlines. The proposed method was experimentally validated both on synthetic and real urban LiDAR datasets, and was compared to state-of-the-art methods.

Light Detection and Ranging (LIDAR) has become a very important tool to many environmental applications. This work proposes to use LIDAR and image data fusion to develop high-resolution thematic maps. A novel methodology is presented which starts building a matrix of statistics from spectral and spatial information by feature extraction on the available bands (RGB from images, and intensity and height from LIDAR). Then, a contextual classification is applied to generate the final map using a support vector machine (SVM) to classify every cell and the nearest neighbor (NN) rule to sequentially reclassify each cell. The results obtained by this novel method, called SVMNNS (SVM and NN Stacking), are compared with non-contextual and contextual SVMs. It is shown that SVMNNS obtains the best results when applied to real data from the Iberian peninsula.

Hyperspectral imagery (HSI) and light detection and ranging (LiDAR) remote sensing technologies are important ways to obtain surface information. Combining the characteristics and advantages of hyperspectral images and LiDAR DSM data, we propose a classification method for co-classification of hyperspectral images and LiDAR data in this paper. The model uses a dual-branch HSI image and LiDAR data classification method. First, the features of shallow convolution and deep convolution are merged in the spatial feature extraction process of HSI, which combines to focus on more global information. Then the space and spectrum are combined, considering the characteristics of different layers, the network has the characteristics of multiscale. Finally, the LiDAR branch introduces dilated convolution to obtain a larger receptive field and reduce information loss. The classification experimental results on Houston and Trento dataset show the superiority of our proposed method in the collaborative classification of hyperspectral images and LiDAR data.

In this study, we proposed an automatic approach to extract DSMs from VHR satellite imagery. This study aims to apply the histogram of orientated phase congruency (HOPC) descriptor and SIFT-Flow algorithm to match multi-view VHR satellite images densely and extract detailed DSMs competitive with ones generated from Light Detection and Ranging (LiDAR) point clouds. The HOPC descriptor relies on detecting features by calculating the phase congruency model, which allows the image features to be highly robust against illumination changes. Moreover, we developed an automatic algorithm to extract precise and homogenously distributed tie-points and subsequently conduct bias correction of the Rational Polynomial Coefficients (RPC). We tested our proposed algorithm on four WorldViw-3 datasets captured from different viewpoints, and compared the results with the DSMs obtained by the Semi-Global Matching (SGM) algorithm. The experimental results indicated the proficiency of our proposed method in generating accurate and reliable DSMs using VHR satellite data.

Over-the-row harvester damage evaluation in super-high-density olive orchard by on-board sensing techniques

Recently, the Sentinel-2 (S2) satellite constellation was deployed for mapping and monitoring the Earth environment. Images acquired by the sensors mounted on the S2 platforms have three levels of spatial resolution: 10, 20, and 60 m. In many remote sensing applications, the availability of images at the highest spatial resolution (i.e., 10 m for S2) is often desirable. This can be achieved by generating a synthetic high-resolution image through data fusion. To this end, researchers have proposed techniques exploiting the spectral/spatial correlation inherent in multispectral data to sharpen the lower resolution S2 bands to 10 m. In this paper, we propose a novel method that formulates the sharpening process as a solution to an inverse problem. We develop a cyclic descent algorithm called S2Sharp and an associated tuning parameter selection algorithm based on generalized cross validation and Bayesian optimization. The tuning parameter selection method is evaluated on a simulated data set. The effectiveness of S2Sharp is assessed experimentally by comparisons to state-of-the-art methods using both simulated and real data sets.