Object Detection and Segmentation using LiDAR-Camera Fusion for Autonomous Vehicle

Automatic building extraction is an important topic for many applications such as urban planning, disaster management, 3D building modeling and updating GIS databases. Its approaches mainly depend on two data sources: light detection and ranging (LiDAR) point cloud and aerial imagery both of which have advantages and disadvantages of their own. In this study, in order to benefit from the advantages of each data sources, LiDAR and image data combined together. And then, the building boundaries were extracted with the automated active contour algorithm implemented in MATLAB. Active contour algorithm uses initial contour positions to segment an object in the image. Initial contour positions were detected without user interaction by a series of image enhancements, band ratio and morphological operations. Four test areas with varying building and background levels of detail were selected from ISPRS’s benchmark Vaihingen and Istanbul datasets. Vegetation and shadows were removed from all the datasets by band ratio to improve segmentation quality. Subsequently, LiDAR point cloud data was converted to raster format and added to the aerial imagery as an extra band. Resulting merged image and initial contour positions were given to the active contour algorithm to extract building boundaries. In order to compare the contribution of LiDAR to the proposed method, the boundaries of the buildings were extracted from the input image before and after adding LiDAR data to the image as a layer. Finally extracted building boundaries were smoothed by the Awrangjeb (Int J Remote Sen 37(3): 551–579. https://doi.org/10.1080/01431161.2015.1131868 , 2016) boundary regularization algorithm. Correctness (Corr), completeness (Comp) and accuracy (Q) metrics were used to assess accuracy of segmented building boundaries by comparing extracted building boundaries with manually digitized building boundaries. Proposed approach shows the promising results with over 93% correctness, 92% completeness and 89% quality.

Civil engineering includes a wide range of applications that require three-dimensional (3D) information. With the advent of LiDAR (Light Detection And Ranging) technology, obtaining 3D terrain information accurately in a cost and time effective manner became more popular. In this paper, three case studies demonstrate the roles and applications of airborne LiDAR data including highway, transmission line and hydropower dam development projects. In first case study, the highway alignment was considerably changed after the initial design stage. LiDAR data was able to do the re-alignment survey quickly, allowing for prompt changes in design. In the next case study, a comparison is made between conventional transmission route alignment design and the design using PLS-CADD with LiDAR data. Utilizing the LiDAR data in PLSCADD allows transmission line engineers to model the construction costs of several alignments options effectively and efficiently compared to the conventional method. The last case study uses LiDAR for obtaining the necessary information for proposed dam. Using LiDAR data maximizes topological accuracy and volume measurements while minimizing survey time and costs. Furthermore, utilizing the multi-temporal LiDAR data can reveal valuable information such as topography changes. LiDAR survey technology has proven that it is the best method to capture accurate data from the earth surface which provides maximum density of topographical & digital imagery information for various applications.

The Light detection and ranging (LiDAR) sensor is used for perceiving the environment of an autonomous vehicle. LiDAR data or point cloud is processed to get the obstacles and their speed around an autonomous vehicle. Based on the information retrieved from LiDAR data and the data from other sensors, real-time decisions are taken for the proper navigation. Hence, the time taken in LiDAR data processing should be minimized. One of the important steps for LiDAR data processing is the segmentation of the obstacles. In this paper, we present a quantitative comparison between two different approaches for point cloud segmentation, Euclidean distance-based Cluster Extraction and Cylindrical range image-based method. Based on the simulation performed on ROS (Robot Operating System) platform, we found that the second method is much faster as compared to the first method. In addition to that, the second method can perform proper segmentation at a larger distance from the sensor.

Foresters are increasingly interested in remote sensing data because they provide an overview of landscape conditions, which is impractical with field sample data alone. Light Detection and Ranging (LiDAR) provides exceptional spatial detail of forest structure, but difficulties in processing LiDAR data have limited their application beyond the research community. Another obstacle to operational use of LiDAR data has been the high cost of data collection. Our objectives in this study were to summarize, at the stand level, both LiDAR- and Landsat (satellite)-based predictions of some common structural and volume attributes and to compare the cost of obtaining such summaries with those obtained through traditional stand exams. We found that the accuracy and cost of a LiDAR-based inventory summarized at the stand level was comparable to traditional stand exams for structural attributes. However, the LiDAR data were able to provide information across a much larger area than the stand exams alone.

A Light Detection and Ranging (LiDAR) sensor mounted on an Unmanned Aerial Vehicle (UAV) can map the overflown environment in point clouds. Mapped canopy heights allow for the estimation of crop biomass in agriculture. The work presented in this paper contributes to sensory UAV setup design for mapping and textual analysis of agricultural fields. LiDAR data are combined with data from Global Navigation Satellite System (GNSS) and Inertial Measurement Unit (IMU) sensors to conduct environment mapping for point clouds. The proposed method facilitates LiDAR recordings in an experimental winter wheat field. Crop height estimates ranging from 0.35–0.58 m are correlated to the applied nitrogen treatments of 0–300 . The LiDAR point clouds are recorded, mapped, and analysed using the functionalities of the Robot Operating System (ROS) and the Point Cloud Library (PCL). Crop volume estimation is based on a voxel grid with a spatial resolution of 0.04 × 0.04 × 0.001 m. Two different flight patterns are evaluated at an altitude of 6 m to determine the impacts of the mapped LiDAR measurements on crop volume estimations.

With the increasing number of remote sensing (RS) data sources, the joint utilization of multimodal data in Earth observation tasks has become a crucial research topic. As a typical representative of RS data, hyperspectral images (HSIs) provide accurate spectral information, while rich elevation information can be obtained from light detection and ranging (LiDAR) data. However, due to the significant differences in multimodal heterogeneous features, how to efficiently fuse HSI and LiDAR data remains one of the challenges faced by existing research. In addition, the edge contour information of images is not fully considered by existing methods, which can easily lead to performance bottlenecks. Thus, a joint classification network of HSI and LiDAR data based on Mamba (HLMamba) is proposed. Specifically, a gradient joint algorithm (GJA) is first performed on LiDAR data to obtain the edge contour data of the land distribution. Subsequently, a multimodal feature extraction module (MFEM) was proposed to capture the semantic features of HSI, LiDAR, and edge contour data. Then, to efficiently fuse multimodal features, a novel deep learning (DL) framework called Mamba, was introduced, and a multimodal Mamba fusion module (MMFM) was constructed. By efficiently modeling the long-distance dependencies of multimodal sequences, the MMFM can better explore the internal features of multimodal data and the interrelationships between modalities, thereby enhancing fusion performance. Finally, to validate the effectiveness of HLMamba, a series of experiments were conducted on three common HSI and LiDAR datasets. The results indicate that HLMamba has superior classification performance compared to other state-of-the-art DL methods. The source code of the proposed method will be available publicly at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/Dilingliao/HLMamba</uri>.

Topographical technology by Airborne LIDAR (Light Detection and Ranging) generates a precise points cloud with a density of several points per square meter, LIDAR data processing is a crucial step to be used. Extraction of 3D information in automatic way and especially in urban areas from LIDAR data is one of the most difficult problems in computer vision; it is also a necessary step for implementation of several applications that require a high level interpretation of LASER data. Therefore, there is recently an increased interest in this research field and a vast literature. The problematic discussed in this article lies in the differentiation between the sets of points that represent a specified layer of information (construction, vegetation, roads, lines, etc.). This step is called segmentation. The aim of this study is to provide a set of automatic segmentation techniques tailored to different types of 3D data and proposes a methodology to classify LIDAR data with a maximum degree of automaticity using only point cloud data.

Currently, the worsening impacts of urbanizations have been impelled to the importance of monitoring and management of existing urban trees, securing sustainable use of the available green spaces. Urban tree species identification and evaluation of their roles in atmospheric Carbon Stock (CS) are still among the prime concerns for city planners regarding initiating a convenient and easily adaptive urban green planning and management system. A detailed methodology on the urban tree carbon stock calibration and mapping was conducted in the urban area of Brussels, Belgium. A comparative analysis of the mapping outcomes was assessed to define the convenience and efficiency of two different remote sensing data sources, Light Detection and Ranging (LiDAR) and WorldView-3 (WV-3), in a unique urban area. The mapping results were validated against field estimated carbon stocks. At the initial stage, dominant tree species were identified and classified using the high-resolution WorldView3 image, leading to the final carbon stock mapping based on the dominant species. An object-based image analysis approach was employed to attain an overall accuracy (OA) of 71% during the classification of the dominant species. The field estimations of carbon stock for each plot were done utilizing an allometric model based on the field tree dendrometric data. Later based on the correlation among the field data and the variables (i.e., Normalized Difference Vegetation Index, NDVI and Crown Height Model, CHM) extracted from the available remote sensing data, the carbon stock mapping and validation had been done in a GIS environment. The calibrated NDVI and CHM had been used to compute possible carbon stock in either case of the WV-3 image and LiDAR data, respectively. A comparative discussion has been introduced to bring out the issues, especially for the developing countries, where WV-3 data could be a better solution over the hardly available LiDAR data. This study could assist city planners in understanding and deciding the applicability of remote sensing data sources based on their availability and the level of expediency, ensuring a sustainable urban green management system.

High-value timber species play an important economic role in forest management. The individual tree information for such species is necessary for practical forest management and for conservation purposes. Digital aerial photogrammetry derived from an unmanned aerial vehicle (UAV-DAP) can provide fine spatial and spectral information, as well as information on the three-dimensional (3D) structure of a forest canopy. Light detection and ranging (LiDAR) data enable area-wide 3D tree mapping and provide accurate forest floor terrain information. In this study, we evaluated the potential use of UAV-DAP and LiDAR data for the estimation of individual tree location and diameter at breast height (DBH) values of large-size high-value timber species in northern Japanese mixed-wood forests. We performed multiresolution segmentation of UAV-DAP orthophotographs to derive individual tree crown. We used object-based image analysis and random forest algorithm to classify the forest canopy into five categories: three high-value timber species, other broadleaf species, and conifer species. The UAV-DAP technique produced overall accuracy values of 73% and 63% for classification of the forest canopy in two forest management sub-compartments. In addition, we estimated individual tree DBH Values of high-value timber species through field survey, LiDAR, and UAV-DAP data. The results indicated that UAV-DAP can predict individual tree DBH Values, with comparable accuracy to DBH prediction using field and LiDAR data. The results of this study are useful for forest managers when searching for high-value timber trees and estimating tree size in large mixed-wood forests and can be applied in single-tree management systems for high-value timber species.

We present a remote sensing pipeline that processes LiDAR (Light Detection And Ranging) data through machine & deep learning for the application of archeological feature detection on big geo-spatial data platforms such as e.g. IBM PAIRS Geoscope. Today, archeologists get overwhelmed by the task of visually surveying huge amounts of (raw) LiDAR data in order to identify areas of interest for inspection on the ground. We showcase a software system pipeline that results in significant savings in terms of expert productivity while missing only a small fraction of the artifacts. Our work employs artificial neural networks in conjunction with an efficient spatial segmentation procedure based on domain knowledge. Data processing is constraint by a limited amount of training labels and noisy LiDAR signals due to vegetation cover and decay of ancient structures. We aim at identifying geo-spatial areas with archeological artifacts in a supervised fashion allowing the domain expert to flexibly tune parameters based on her needs.

ABSTRACTLandslides are a pervasive natural disaster, resulting in severe social, environmental and economic impacts worldwide. The tropical, mountainous landscape in South-West Mexico is predisposed to landslides because of frequent hurricanes and earthquakes. The main goal of this study is to compare landslide susceptibility maps in Guerrero derived using high-resolution LIDAR (light detection and ranging) data from both a manual landslide event inventory and an automated landslide inventorying algorithm. The paper also highlights the importance of applying LIDAR data in landslide inventorying and susceptibility mapping.We mapped landslides based on two approaches: (1) manual mapping using satellite images and (2) automatic identification of landslide morphology employing the Contour Connection Method (CCM). We produced a landslide susceptibility map by computing the probability of landslide occurrence from statistical relationships of inventoried landslides detected with LIDAR digital terrain models (DTMs) and derived landslide-causing factors using the logistic regression method.Our results suggest that the automated inventory derived through the CCM algorithm with LIDAR DTMs effectively minimizes the time-consuming and subjective manual inventorying process. The high overall prediction accuracy (up to 0.83) from logistic regression demonstrates the validity and applicability deriving reliable landslide susceptibility maps from an automated inventory; however, LIDAR data are required.

The detailed representation of the topography is an important aspect when dealing with any flood simulation model. A Digital Surface Model (DSM) with fine spatial resolution can capture topographic object boundaries with a high level of detail. Light detection and ranging (LiDAR) is a technology that can rapidly collect elevation data of a relatively large topographic area with fine spatial resolution (< 0.5m ∼1m). Information about physical objects on the DSM is critical for storm water management as the propagation of flood water is influenced by both the terrain and the objects on the terrain. The processing of LiDAR data is an important task for generating an accurate DSM suitable to running a flood simulation model. Filtering is the first step in processing LiDAR data. In the LiDAR community, the term “filtering” means the task of separating LiDAR data into ground (bare earth) and non-ground points (buildings and trees). However, the choice of appropriate filtering algorithm is crucial in order to detect micro objects such as boundary walls, fences, and thin embankments that have significant impact on the surface flow pattern. This study offers an innovative approach for developing a LiDAR filtering algorithm that is flexible in nature and able to detect not only buildings and trees but also the micro objects. The filtering algorithm is applied to two study areas, and the output suggests close correspondence to reality.

Recently, we have lots of LiDAR (light detection and ranging) data, for applications of high-resolution maps including geography, geology, forestry, and others. One of the current research and industrial issues is efficient ways of storing the LiDAR data itself, and also elegant ways of extracting geometric primitives from those LiDAR-scanned 3D point clouds. In this paper, we first analyze the characteristics of LiDAR data and tis storage schemes. Additionally, we present an efficient method to extract geometric primitives from those point clouds. Its implementation and results are also presented.

Obstacle detection on the railway, a crucial operational safety concern, is a complex task that encompasses a multitude of challenges. While Machine Learning (ML) algorithms are commonly employed in analogous applications such as autonomous car driving [1] [2], the railway field faces a significant barrier due to the scarcity of available data (particularly images), rendering conventional ML approaches impractical. In response to this challenge, this study proposes and evaluates a framework which uses LiDAR (Light Detection and Ranging) data for obstacle detection on the railways. The framework aims to address the limitations posed by image data scarcity while enhancing operational safety in railway environments. The developed methodology combines the use of a long-range LiDAR capable of detecting obstacles at distances of up to 500 meters, with the train’s GPS (Global Positioning System) coordinates to accurately determine its position relative to detected obstacles. The LiDAR data is processed using a data fusion approach, where pre-existing knowledge regarding the track topography is combined with a clustering algorithm, specifically DBSCAN (Density-based spatial clustering of applications with noise), to identify and classify potential obstacles at a pre-defined distance. Tests of the proposed framework were conducted within the confines of a moving locomotive, specifically the CP 2600-2620 series, along a designated section of the Contumil-Leixões line. These tests served to validate the effectiveness and feasibility of the approach under real-world operating conditions. Overall, the utilization of LiDAR data coupled with advanced algorithms presents a promising avenue for enhancing obstacle detection capabilities in railway operations. By overcoming the challenges associated with data scarcity, this framework holds the potential to significantly improve operational safety and efficiency within railway networks. Further research and testing are warranted to validate the framework’s performance across diverse railway environments and operating conditions.

&lt;p&gt;LiDAR (Light Detection and Ranging) is a technology that provides three-dimensional point cloud data with high spatial accuracy. It is increasingly used in various applications and disciplines, including engineering. Along with other capabilities, LiDAR systems are able to record reflected backscattered energy as intensity data and measured distances to targets as range data. As the laser signal wavelength that operates in LiDAR sensors is typically in the near-infrared (NIR) spectrum, high spectral reflectance separability can be observed and a number of different materials distinguished. These capabilities have motivated researchers to study the applicability of using LiDAR range and intensity data for pavement crack extraction.&lt;/p&gt; &lt;p&gt;The main goal of the present research is to examine the potential use of LiDAR range and intensity data for automated pavement crack detection. The study explores the following: a) crack detection using the two image classification logics of pixel-based and object-based image classification; b) crack detection using the Maximum Likelihood Classifier (MLC), the Random Trees Classifier (RTC), and the Support Vector Machine (SVM) classifier; c) crack detection by conducting image classification of four scenarios of multi-layer images generated from LiDAR data; and d) crack detection using LiDAR data with different point spacing.&lt;/p&gt; &lt;p&gt;The experimental results indicate that the use of LiDAR data independently is effective for pavement crack detection. In addition, the results show that object-based image classification logic optimizes classification accuracy and crack detection results, and that employing SVM enhances the classification process and improves crack detection accuracy. It is also observed that using range data separately produces high accuracy for the crack detection results while the accuracy is slightly reduced when intensity data are combined with range data compared with using range data only. Furthermore, in the classification process, adding slope and aspect layers to range and intensity data decreases the accuracy of crack detection results, while increasing LiDAR data point spacing reduces crack extraction accuracy.&lt;/p&gt;