Motion Estimation and Coding Structure for Inter-Prediction of LiDAR Point Cloud Geometry

With an increase in the frequency and severity of wildfires across the globe and resultant changes to long-established fire regimes, the mapping of fire severity is a vital part of monitoring ecosystem resilience and recovery. The emergence of unoccupied aircraft systems (UAS) and compact sensors (RGB and LiDAR) provide new opportunities to map fire severity. This paper conducts a comparison of metrics derived from UAS Light Detecting and Ranging (LiDAR) point clouds and UAS image based products to classify fire severity. A workflow which derives novel metrics describing vegetation structure and fire severity from UAS remote sensing data is developed that fully utilises the vegetation information available in both data sources. UAS imagery and LiDAR data were captured pre- and post-fire over a 300 m by 300 m study area in Tasmania, Australia. The study area featured a vegetation gradient from sedgeland vegetation (e.g., button grass 0.2m) to forest (e.g., Eucalyptus obliqua and Eucalyptus globulus 50m). To classify the vegetation and fire severity, a comprehensive set of variables describing structural, textural and spectral characteristics were gathered using UAS images and UAS LiDAR datasets. A recursive feature elimination process was used to highlight the subsets of variables to be included in random forest classifiers. The classifier was then used to map vegetation and severity across the study area. The results indicate that UAS LiDAR provided similar overall accuracy to UAS image and combined (UAS LiDAR and UAS image predictor values) data streams to classify vegetation (UAS image: 80.6%; UAS LiDAR: 78.9%; and Combined: 83.1%) and severity in areas of forest (UAS image: 76.6%, UAS LiDAR: 74.5%; and Combined: 78.5%) and areas of sedgeland (UAS image: 72.4%; UAS LiDAR: 75.2%; and Combined: 76.6%). These results indicate that UAS SfM and LiDAR point clouds can be used to assess fire severity at very high spatial resolution.

The geometry of a point cloud is commonly represented by an octree recursively decomposing a 3D volume into eight child sub-volumes. Said volumes and sub-volumes are associated with nodes and child-nodes of the octree. The geometry is defined by the occupancy information indicating the presence or not of a point in each of the sub-volumes. This naturally leads to an eight-bit occupancy information to be coded for each internal node of the tree. This paper introduces a new binarization scheme to efficiently compress the occupancy information using an optimal set of binary entropy coders. Then, it is shown how using the occupancy information of neighbouring nodes helps to compress the occupancy bits associated with the child nodes of the current node. This information is used to contextualise the binarization scheme by computing, firstly a neighbour configuration, secondly a number of neighbours with occupied child nodes adjacent to the current child node, and thirdly an intra predictor. Objective results show lossless geometry compression gains between 60% and 75% on virtual reality oriented dense point clouds used by MPEG, reaching sub-bit per point bit-rates for the lossless intra coding of such point clouds. Solid gains (between 5% and 25% depending upon the sampling) are also observed on sparse point clouds captured by a LiDAR (Light Detection and Ranging) device attached to a moving vehicle or representing 3D maps.

This paper presents a new algorithm for building an indoor map by integrating point clouds of 2D light detection and ranging (LIDAR) scanners in indoor environments. Iterative closest point (ICP) algorithm is one of the well-known methods for such purpose and often used for mobile robot SLAM. However, the algorithm is designed based on dense (or continuous) measurement of the same space with known relative positions of measurement points and angles, and it does not often work efficiently if the measurement is sparse and/or LIDAR locations are unknown. Such situations are seen when some installed LIDARs in a room are used to build a background indoor map for object tracking, or mobile LIDARs are used by technicians to build a digital indoor map, where each space is captured only at a few locations with different angles. To tackle this issue, our method extracts line segments and edge points as features from given LIDAR point clouds and finds shape coincidences commonly contained in a pair of given point clouds to identify positional relationships between LIDARs. By this information, these point clouds can be integrated into common 2D coordinates. Indoor map generation is realized by sequentially applying this integration procedure to every pair of point clouds. Also, the experiments on real data show that our method can identify the relative positions of LIDARs with 10cm-order errors in average, and by sequentially applying the point-cloud integration, the generated maps have only 3% errors.

Differential LiDAR (Light Detection and Ranging) from repeated surveys has recently emerged as an effective tool to measure three-dimensional (3D) change for applications, such as quantifying slip and spatially distributed warping associated with earthquake ruptures, and examining the spatial distribution of beach erosion after hurricane impact. Currently, the primary method for determining 3D change from LiDAR is through the use of the iterative closest point (ICP) algorithm and its variants. However, all current studies using ICP have assumed that all LiDAR points in the compared point clouds have uniform accuracy. This assumption is simplistic given that the error for each LiDAR point is variable, and dependent upon time varying factors such as target range, angle of incidence, and aircraft trajectory accuracy. Therefore, to rigorously determine spatial change, it would be ideal to model the random error for every LiDAR observation in the differential point cloud, and use these error estimates as apriori weights in the ICP algorithm. To test this approach, we implemented a rigorous LiDAR observation error propagation method to generate estimated random error for each point in a LiDAR point cloud, and then determine 3D displacements between two point clouds using an anisotropic weighted ICP (A-ICP) algorithm. The algorithm was evaluated by qualitatively and quantitatively comparing point clouds with synthetic fault ruptures between a uniform weight and anistropically weighted ICP algorithm. Then post-earthquake slip is estimated for the 2010 El Mayor-Cucapah Earthquake (EMC), using pre- and post-event LiDAR. Based on the analysis, Moving Window A-ICP is able to better estimate the synthetic surface ruptures, and provides a smoother estimate of actual displacement for the EMC earthquake.

Routine bridge inspections require labor intensive and highly subjective visual interpretation to determine bridge deck surface condition. Light Detection and Ranging (LiDAR) a relatively new class of survey instrument has become a popular and increasingly used technology for providing as-built and inventory data in civil applications. While an increasing number of private and governmental agencies possess terrestrial and mobile LiDAR systems, an understanding of the technology’s capabilities and potential applications continues to evolve. LiDAR is a line-of-sight instrument and as such, care must be taken when establishing scan locations and resolution to allow the capture of data at an adequate resolution for defining features that contribute to the analysis of bridge deck surface condition. Information such as the location, area, and volume of spalling on deck surfaces, undersides, and support columns can be derived from properly collected LiDAR point clouds. The LiDAR point clouds contain information that can provide quantitative surface condition information, resulting in more accurate structural health monitoring. LiDAR scans were collected at three study bridges, each of which displayed a varying degree of degradation. A variety of commercially available analysis tools and an independently developed algorithm written in ArcGIS Python (ArcPy) were used to locate and quantify surface defects such as location, volume, and area of spalls. The results were visual and numerically displayed in a user-friendly web-based decision support tool integrating prior bridge condition metrics for comparison. LiDAR data processing procedures along with strengths and limitations of point clouds for defining features useful for assessing bridge deck condition are discussed. Point cloud density and incidence angle are two attributes that must be managed carefully to ensure data collected are of high quality and useful for bridge condition evaluation. When collected properly to ensure effective evaluation of bridge surface condition, LiDAR data can be analyzed to provide a useful data set from which to derive bridge deck condition information.

3D Modeling Approach of Building Construction based on Point Cloud Data Using LiDAR Byeongjun Oh, Minju Kim, Chanwoo Lee, Hunhee Cho and Kyung-In Kang Pages 906-912 (2019 Proceedings of the 36th ISARC, Banff, Canada, ISBN 978-952-69524-0-6, ISSN 2413-5844) Abstract: LiDAR (Light Detection and Ranging) emerges as a mapping technology that provides fast, accurate, and reliable data with geometric representation of construction facilities. The technology has received high recognition especially since the importance of digital transformation in construction projects has been emphasized. However, the current state of 3D technologies including LiDAR encounters difficulty in recognizing and extracting accurate information on the as-built status of the buildings and construction sites. As a preliminary study for the development of a 3D architectural geometric representation of building and environment, this paper proposes a 3D modeling approach based on point cloud data obtained by LiDAR technology. This paper suggests a novel method for data acquisition on as-built status of construction sites and buildings during construction-operation phases. Furthermore, a field test has been performed for visualizing and modeling of the indoors of a residential house. The results of this paper are expected to inaugurate adaptation of LiDAR technology during the as-built process, and further implementation of digital technology after construction phase in construction projects. Keywords: LiDAR; 3D modeling; building construction; digital transformation; point cloud data DOI: https://doi.org/10.22260/ISARC2019/0121 Download fulltext Download BibTex Download Endnote (RIS) TeX Import to Mendeley

Detection and delineation of individual trees is important for the estimation of forest information using Light Detection and Ranging (LiDAR) point clouds. However, over-segmentation of canopy height models (CHMs) is frequently caused by subcrowns and branches, whereas under-segmentation is caused by adjacent crowns, especially in deciduous forests. To eliminate such problems, an approach for delineating individual trees based on curvature attributes derived from LiDAR data is proposed. To detect accurate treetops for reducing over-segmentation, the dominant sizes of tree crowns are first determined and used to fit the dome-shaped tree crowns in CHMs, and then the minimum curvature values of fitted CHMs at different scales are calculated for treetop detection. For delineation of individual trees, tree features such as height variation and curvature are chosen to form a feature vector as a pixel-level logistic regression classifier. Then an ensemble classifier combining multiple logistic regression classifiers is trained to analyze the similarity between neighboring points to reduce under-segmentation. Finally, segment maps at different scales are integrated to generate the final segment map. Our experimental results on 2 test areas demonstrate that the proposed method has promising applications for eliminating over-segmentation and under-segmentation in deciduous forests.

The work conducted in this thesis is a part of an EU Horizon 2020 research initiative project known as DigiArt. This part of the DigiArt project presents and explores the design, formulation and implementation of probabilistically orientated state estimation algorithms with focus towards unmanned system positioning and three-dimensional (3D) mapping. State estimation algorithms are considered an influential aspect of any dynamic system with autonomous capabilities. Possessing the ability to predictively estimate future conditions enables effective decision making and anticipating any possible changes in the environment. Initial experimental procedures utilised a wireless ultra-wide band (UWB) based communication network. This system functioned through statically situated beacon nodes used to localise a dynamically operating node. The simultaneous deployment of this UWB network, an unmanned system and a Robotic Total Station (RTS) with active and remote tracking features enabled the characterisation of the range measurement errors associated with the UWB network. These range error metrics were then integrated into an Range based Extended Kalman Filter (R-EKF) state estimation algorithm with active outlier identification to outperform the native approach used by the UWB system for two-dimensional (2D) pose estimation.The study was then expanded to focus on state estimation in 3D, where a Six Degreeof-Freedom EKF (6DOF-EKF) was designed using Light Detection and Ranging (LiDAR) as its primary observation source. A two step method was proposed which extracted information between consecutive LiDAR scans. Firstly, motion estimation concerning Cartesian states x, y and the unmanned system’s heading (ψ) was achieved through a 2D feature matching process. Secondly, the extraction and alignment of ground planes from the LiDAR scan enabled motion extraction for Cartesian position z and attitude angles roll (θ) and pitch (φ). Results showed that the ground plane alignment failed when two scans were at 10.5◦ offset. Therefore, to overcome this limitation an Error State Kalman Filter (ES-KF) was formulated and deployed as a sub-system within the 6DOF-EKF. This enabled the successful tracking of roll, pitch and the calculation of z. The 6DOF-EKF was seen to outperform the R-EKF and the native UWB approach, as it was much more stable, produced less noise in its position estimations and provided 3D pose estimation.

The intelligent road is an important component of the intelligent vehicle infrastructure cooperative system, the latest development of intelligent transportation systems. As an advanced sensor, Light Detection and Ranging (LiDAR) has gradually been used to collect high-resolution micro-traffic data on the roadside of intelligent roads. Furthermore, a fusion of multiple LiDARs has become a current hot spot to extend the data collection range and improve detection accuracy. This paper focuses on point cloud registration in a complex traffic environment and proposes a three-dimensional (3D) registration method based on traffic signs and prior knowledge of traffic scenes. Traffic signs with their reflective films are used as reference targets to register 3D point cloud data from roadside LiDARs. The proposed method consists of a vertical registration and a horizontal registration. For the vertical registration, we propose a panel rotation algorithm to rotate the initial point cloud to register it vertically, converting the 3D point cloud registration into a two-dimensional (2D) rigid body transformation. For the vertical registration, our system registers traffic signs from different LiDARs. Our method has been verified in some actual scenarios. Compared with previous methods, the proposed method is automatic and does not need to search reference targets manually. Furthermore, it is suitable for actual engineering use and can be applied to sparse point cloud data from LiDAR with few beams, realizing point cloud registration of large disparity.

Autonomous vehicles (AVs) and robots increasingly exploit light detection and ranging (LiDAR)-based 3-D object detection systems to detect obstacles in the environment. Correct detection and classification are important to ensure safe driving. Although previous work has demonstrated the feasibility of manipulating point clouds to spoof 3-D object detectors, most of these attempts are performed digitally. In this article, we investigate the possibility of physically fooling LiDAR-based 3-D object detection by injecting adversarial point clouds using lasers. First, we develop a laser transceiver that can inject up to 4200 points, and can measure the scanning cycle of victim LiDARs to schedule the spoofing laser signals. By designing a control signal method that converts the coordinates of point clouds to control signals and an adversarial point cloud optimization method with physical constraints of LiDARs and attack capabilities, we manage to inject spoofing point cloud with desired point cloud shapes into the victim LiDAR physically. We can launch four types of attacks, i.e., naive hiding, record-based creating, optimization-based hiding, and optimization-based creating. Extensive experiments demonstrate the effectiveness of our attacks against two commercial LiDAR and three detectors. We further analyze the impact of our attacks on four fusion-based detectors. This article concludes with experiments on defense methods and discussion on potential defense strategies at both the sensor and AV system levels.

With the development of fast motion estimation (ME) algorithms towards video compression standard H.264, burden on integer-pel ME is slightly reduced. However the integer-pel ME is becoming computationally complex due to demand of higher and higher resolution videos. On the other hand, there is always trade-off between the speed and performance of these search algorithms. This work improvises the performance of content awareness enabled integer-pel ME algorithm for encoding the fast and slow motion video sequences of high definition (HD) resolution category viz. 1280×720 and 1920×1080. The algorithm proposes a novel notion of 'integrity index' with focus on increasing the PSNR and reducing the bit rate. It is implemented in the frame work of x265 version 1.7 video encoder. Motion independent ME algorithm is analysed quantitatively in terms of parameters based on rate distortion and ME time. The proposed algorithm is found to be performing better on BD-Rate and BD-PSNR parameters for the videos of both resolutions under consideration. Observed increase in motion estimation time is less than four seconds compared to hexagonal search algorithm of ME which is the benchmark in fast ME algorithm.

Recent growth in unmanned aerial vehicle (UAV) technology have promoted the detailed mapping of individual tree species. However, the in-depth mining and comprehending of the significance of features derived from high-resolution UAV data for tree species discrimination remains a difficult task. In this study, a state-of-the-art approach combining UAV-borne light detection and ranging (LiDAR) and hyperspectral was used to classify 11 common tree species in a typical natural secondary forest in Northeast China. First, comprehensive relevant structural and spectral features were extracted. Then, the most valuable feature sets were selected by using a hybrid approach combining correlation-based feature selection with the optimized recursive feature elimination algorithm. The random forest algorithm was used to assess feature importance and perform the classification. Finally, the robustness of features derived from point clouds with different structures and hyperspectral images with different spatial resolutions was tested. Our results showed that the best classification accuracy was obtained by combining LiDAR and hyperspectral data (75.7%) compared to that based on LiDAR (60.0%) and hyperspectral (64.8%) data alone. The mean intensity of single returns and the visible atmospherically resistant index for red-edge band were the most influential LiDAR and hyperspectral derived features, respectively. The selected features were robust in point clouds with a density not lower than 5% (~5 pts/m2) and a resolution not lower than 0.3 m in hyperspectral data. Although canopy surface features were slightly different from original LiDAR features, canopy surface information was also important for tree species classification. This study proved the capabilities of UAV-borne LiDAR and hyperspectral data in natural secondary forest tree species discrimination and the potential for this approach to be transferable to other study areas.

Robust multi-task learning network for complex LiDAR point cloud data preprocessing

Chapter 5 - Data Preprocessing and Feature Extraction

A Light Detection and Ranging (LiDAR) sensor mounted on an Unmanned Aerial Vehicle (UAV) can map the overflown environment in point clouds. Mapped canopy heights allow for the estimation of crop biomass in agriculture. The work presented in this paper contributes to sensory UAV setup design for mapping and textual analysis of agricultural fields. LiDAR data are combined with data from Global Navigation Satellite System (GNSS) and Inertial Measurement Unit (IMU) sensors to conduct environment mapping for point clouds. The proposed method facilitates LiDAR recordings in an experimental winter wheat field. Crop height estimates ranging from 0.35–0.58 m are correlated to the applied nitrogen treatments of 0–300 . The LiDAR point clouds are recorded, mapped, and analysed using the functionalities of the Robot Operating System (ROS) and the Point Cloud Library (PCL). Crop volume estimation is based on a voxel grid with a spatial resolution of 0.04 × 0.04 × 0.001 m. Two different flight patterns are evaluated at an altitude of 6 m to determine the impacts of the mapped LiDAR measurements on crop volume estimations.