SP-Det: Leveraging Saliency Prediction for Voxel-Based 3D Object Detection in Sparse Point Cloud

The 3D object detection is becoming indispensable for environmental perception in autonomous driving. Light detection and ranging (LiDAR) point clouds often fail to distinguish objects with similar structures and are quite sparse for distant or small objects, thereby introducing false and missed detections. To address these issues, LiDAR is often fused with cameras due to the rich textural information provided by images. However, current fusion methods suffer the inefficient data representation and inaccurate alignment of heterogeneous features, leading to poor precision and low efficiency. To this end, we propose a plug-and-play module termed range-image fusion (RI-Fusion) to achieve an effective fusion of LiDAR and camera data, designed to be easily accessible by existing mainstream LiDAR-based algorithms. In this process, we design an image and point cloud alignment method by converting a point cloud into a compact range-view representation through a spherical coordinate transformation. The range image is then integrated with a corresponding camera image utilizing an attention mechanism. The original range image is then concatenated with fusion features to retain point cloud information, and the results are projected onto a spatial point cloud. Finally, the feature-enhanced point cloud can be input into a LiDAR-based 3D object detector. The results of validation experiments involving the KITTI 3D object detection benchmark showed that our proposed fusion method significantly enhanced multiple mainstream LiDAR-based 3D object detectors, PointPillars, SECOND, and Part <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\text{A}{^{2}}$ </tex-math></inline-formula> , improving the 3D mAP (mean Average Precision) by 3.61%, 2.98%, and 1.27%, respectively, particularly for small objects such as pedestrians and cyclists.

Moving object detection and segmentation is an essential task in the Autonomous Driving pipeline. Detecting and isolating static and moving components of a vehicle's surroundings are particularly crucial in path planning and localization tasks. This paper proposes a novel real-time architecture for motion segmentation of Light Detection and Ranging (LiDAR) data. We use three successive scans of LiDAR data in 2D Bird's Eye View (BEV) representation to perform pixel-wise classification as static or moving. Furthermore, we propose a novel data augmentation technique to reduce the significant class imbalance between static and moving objects. We achieve this by artificially synthesizing moving objects by cutting and pasting static vehicles. We demonstrate a low latency of 8 ms on a commonly used automotive embedded platform, namely Nvidia Jetson Xavier. To the best of our knowledge, this is the first work directly performing motion segmentation in LiDAR BEV space. We provide quantitative results on the challenging SemanticKITTI dataset, and qualitative results are provided in https://youtu.be/2aJ-cL8b0LI.

Geometric information constraint 3D object detection from LiDAR point cloud for autonomous vehicles under adverse weather

Accurate object detection is important for the safe navigation of autonomous vehicles (AVs) in dynamic environments. Conventional AVs rely on high-resolution but bulky, topmounted 3D Light Detection And Ranging (LiDAR) sensors, which are impractical for scalable production and computationally expensive to process. To address this, we utilize quasi-2D LiDAR sensors, which offer a compact and scalable solution for AVs. However, point clouds generated by these sensors have a narrow vertical field of view and contain a limited number of points compared to 3D LiDARs, which makes object detection significantly more difficult. To overcome this challenge, we transform the quasi-2D point clouds into Bird's-Eye-View (BEV) images and apply four different input encoding strategies. These encodings enhance the spatial and temporal context of the data, making them suitable for efficient deep-learning-based 2D object detection models to perform real-time detection with minimal computational overhead. In contrast to other approaches for 2D LiDARs, we are able to detect not only vehicles but also pedestrians and cyclists. We use a large real-world dataset to compare our proposed input encodings combined with three different state-of-the-art 2D object detection models. Our results demonstrate that these input encodings enhance object detection performance across all models, significantly reducing false positives, improving heading angle prediction, and boosting the detection accuracy of smaller traffic participants.

Light detection and ranging (LiDAR) is widely used in the automotive industry as it can provide point cloud information containing precise distances. Three-dimensional (3D) object detection based on LiDAR point clouds is significant for environment perception tasks. However, feature learning for point clouds remains challenging. This paper proposes a two-stage voxel-based LiDAR 3D object detector, referred to as density-aware and neighborhood attention (DenNet), that focuses on the neighborhood information of objects. DenNet mainly integrates two modules: voxel density-aware (VDA) and neighborhood attention (NA). VDA introduces density information of the point cloud. Here, point cloud density information was added as voxel features in the voxel-based framework to alleviate the information loss during voxelization. Additionally, to extract neighbor information, the characteristics of 3D objects were analyzed for traffic scenes. The NA mechanism was adopted, which localizes the receptive field for each query to its nearest neighboring points. DenNet yielded competitive results, as compared with state-of-the-art methods, for the KITTI and One Million Scenes (ONCE) datasets; notably, it afforded an improvement of 3.96% relative to the baseline mean average precision on the more challenging ONCE dataset.

Light Detection And Ranging (LiDAR) sensors can generate a number of sequential 3D point clouds, which are widely deployed in many real-world systems. 3D object detection in point clouds, is one of the most fundamental tasks. Unfortunately, the existing 3D object detection methods degrade in snowy weather, because in that situation the annotated samples are difficult to collect. To solve this issue, we propose a novel GAN-based Snowfall Point-cloud AugmentOR (GAN spaor ) to generate high-quality synthetic snowfall point clouds as augmentations. The basic idea of GAN spaor is to transfer annotated point clouds to snowfall versions by simultaneously learning the global style of real snowfall point clouds and the local details of physics-induced ones. Our framework fuses data-driven and physical modeling methods for rapidly generating data in snowy weather. To evaluate the effectiveness of GAN spaor , we employ a number of recent 3D object detection methods and train them by using the synthetic samples of GAN spaor as auxiliary augmentations. Moreover, we conduct a comparative analysis of the characteristics of the data distributions of the snowy point clouds synthesized by GAN spaor . Experimental results demonstrate that GAN spaor can improve the performance of 3D object detection methods compared with other existing snowfall point cloud simulators.

This paper presents a new algorithm for building an indoor map by integrating point clouds of 2D light detection and ranging (LIDAR) scanners in indoor environments. Iterative closest point (ICP) algorithm is one of the well-known methods for such purpose and often used for mobile robot SLAM. However, the algorithm is designed based on dense (or continuous) measurement of the same space with known relative positions of measurement points and angles, and it does not often work efficiently if the measurement is sparse and/or LIDAR locations are unknown. Such situations are seen when some installed LIDARs in a room are used to build a background indoor map for object tracking, or mobile LIDARs are used by technicians to build a digital indoor map, where each space is captured only at a few locations with different angles. To tackle this issue, our method extracts line segments and edge points as features from given LIDAR point clouds and finds shape coincidences commonly contained in a pair of given point clouds to identify positional relationships between LIDARs. By this information, these point clouds can be integrated into common 2D coordinates. Indoor map generation is realized by sequentially applying this integration procedure to every pair of point clouds. Also, the experiments on real data show that our method can identify the relative positions of LIDARs with 10cm-order errors in average, and by sequentially applying the point-cloud integration, the generated maps have only 3% errors.

3D object detection from raw and sparse point clouds has been far less treated to date, compared with its 2D counterpart. In this paper, we propose a novel framework called FVNet for 3D front-view proposal generation and object detection from point clouds. It consists of two stages: generation of front-view proposals and estimation of 3D bounding box parameters. Instead of generating proposals from camera images or bird's-eye-view maps, we first project point clouds onto a cylindrical surface to generate front-view feature maps which retains rich information. We then introduce a proposal generation network to predict 3D region proposals from the generated maps and further extrude objects of interest from the whole point cloud. Finally, we present another network to extract the point-wise features from the extruded object points and regress the final 3D bounding box parameters in the canonical coordinates. Our framework achieves real-time performance with 12ms per point cloud sample. Extensive experiments on the 3D detection benchmark KITTI show that the proposed architecture outperforms state-of-the-art techniques which take either camera images or point clouds as input, in terms of accuracy and inference time.

3D object detection based on LiDAR point clouds is a crucial module in autonomous driving particularly for long range sensing. Most of the research is focused on achieving higher accuracy and these models are not optimized for deployment on embedded systems from the perspective of latency and power efficiency. For high speed driving scenarios, latency is a crucial parameter as it provides more time to react to dangerous situations. Typically a voxel or point-cloud based 3D convolution approach is utilized for this module. Firstly, they are inefficient on embedded platforms as they are not suitable for efficient parallelization. Secondly, they have a variable runtime due to level of sparsity of the scene which is against the determinism needed in a safety system. In this work, we aim to develop a very low latency algorithm with fixed runtime. We propose a novel semantic segmentation architecture as a single unified model for object center detection using key points, box predictions and orientation prediction using binned classification in a simpler Bird's Eye View (BEV) 2D representation. The proposed architecture can be trivially extended to include semantic segmentation classes like road without any additional computation. The proposed model has a latency of 4 ms on the embedded Nvidia Xavier platform. The model is 5X faster than other top accuracy models with a minimal accuracy degradation of 2% in Average Precision at IoU=0.5 on KITTI dataset.

The intelligent road is an important component of the intelligent vehicle infrastructure cooperative system, the latest development of intelligent transportation systems. As an advanced sensor, Light Detection and Ranging (LiDAR) has gradually been used to collect high-resolution micro-traffic data on the roadside of intelligent roads. Furthermore, a fusion of multiple LiDARs has become a current hot spot to extend the data collection range and improve detection accuracy. This paper focuses on point cloud registration in a complex traffic environment and proposes a three-dimensional (3D) registration method based on traffic signs and prior knowledge of traffic scenes. Traffic signs with their reflective films are used as reference targets to register 3D point cloud data from roadside LiDARs. The proposed method consists of a vertical registration and a horizontal registration. For the vertical registration, we propose a panel rotation algorithm to rotate the initial point cloud to register it vertically, converting the 3D point cloud registration into a two-dimensional (2D) rigid body transformation. For the vertical registration, our system registers traffic signs from different LiDARs. Our method has been verified in some actual scenarios. Compared with previous methods, the proposed method is automatic and does not need to search reference targets manually. Furthermore, it is suitable for actual engineering use and can be applied to sparse point cloud data from LiDAR with few beams, realizing point cloud registration of large disparity.

Differential LiDAR (Light Detection and Ranging) from repeated surveys has recently emerged as an effective tool to measure three-dimensional (3D) change for applications, such as quantifying slip and spatially distributed warping associated with earthquake ruptures, and examining the spatial distribution of beach erosion after hurricane impact. Currently, the primary method for determining 3D change from LiDAR is through the use of the iterative closest point (ICP) algorithm and its variants. However, all current studies using ICP have assumed that all LiDAR points in the compared point clouds have uniform accuracy. This assumption is simplistic given that the error for each LiDAR point is variable, and dependent upon time varying factors such as target range, angle of incidence, and aircraft trajectory accuracy. Therefore, to rigorously determine spatial change, it would be ideal to model the random error for every LiDAR observation in the differential point cloud, and use these error estimates as apriori weights in the ICP algorithm. To test this approach, we implemented a rigorous LiDAR observation error propagation method to generate estimated random error for each point in a LiDAR point cloud, and then determine 3D displacements between two point clouds using an anisotropic weighted ICP (A-ICP) algorithm. The algorithm was evaluated by qualitatively and quantitatively comparing point clouds with synthetic fault ruptures between a uniform weight and anistropically weighted ICP algorithm. Then post-earthquake slip is estimated for the 2010 El Mayor-Cucapah Earthquake (EMC), using pre- and post-event LiDAR. Based on the analysis, Moving Window A-ICP is able to better estimate the synthetic surface ruptures, and provides a smoother estimate of actual displacement for the EMC earthquake.

3D Modeling Approach of Building Construction based on Point Cloud Data Using LiDAR Byeongjun Oh, Minju Kim, Chanwoo Lee, Hunhee Cho and Kyung-In Kang Pages 906-912 (2019 Proceedings of the 36th ISARC, Banff, Canada, ISBN 978-952-69524-0-6, ISSN 2413-5844) Abstract: LiDAR (Light Detection and Ranging) emerges as a mapping technology that provides fast, accurate, and reliable data with geometric representation of construction facilities. The technology has received high recognition especially since the importance of digital transformation in construction projects has been emphasized. However, the current state of 3D technologies including LiDAR encounters difficulty in recognizing and extracting accurate information on the as-built status of the buildings and construction sites. As a preliminary study for the development of a 3D architectural geometric representation of building and environment, this paper proposes a 3D modeling approach based on point cloud data obtained by LiDAR technology. This paper suggests a novel method for data acquisition on as-built status of construction sites and buildings during construction-operation phases. Furthermore, a field test has been performed for visualizing and modeling of the indoors of a residential house. The results of this paper are expected to inaugurate adaptation of LiDAR technology during the as-built process, and further implementation of digital technology after construction phase in construction projects. Keywords: LiDAR; 3D modeling; building construction; digital transformation; point cloud data DOI: https://doi.org/10.22260/ISARC2019/0121 Download fulltext Download BibTex Download Endnote (RIS) TeX Import to Mendeley

Accurate object detection is a fundamental requirement for autonomous systems to operate in dynamic urban environments. Considering the intricacy of the environment and the occlusion of objects, various point cloud based three-dimensional (3D) object detection methods have been proposed, such as point-based or voxel-based methods. In this paper, a feature selection mechanism is proposed in a voxel-based method to generate bird’s eye view (BEV) feature maps from the original point cloud. For the 3D Backbone, the U-shaped structure and single scale feature selection module are combined. After combining high-level semantics and low-level fine-grained features, the optimized features after the BEV feature maps are applied to region of interest (RoI) refinement, so that the voxel features can better serve the subsequent 3D object detection. The experimental results on KITTI dataset show higher 3D object detection accuracy compared to the state-of-the-art 3D detection methods, which reflects the effectiveness of the proposed architecture.

Routine bridge inspections require labor intensive and highly subjective visual interpretation to determine bridge deck surface condition. Light Detection and Ranging (LiDAR) a relatively new class of survey instrument has become a popular and increasingly used technology for providing as-built and inventory data in civil applications. While an increasing number of private and governmental agencies possess terrestrial and mobile LiDAR systems, an understanding of the technology’s capabilities and potential applications continues to evolve. LiDAR is a line-of-sight instrument and as such, care must be taken when establishing scan locations and resolution to allow the capture of data at an adequate resolution for defining features that contribute to the analysis of bridge deck surface condition. Information such as the location, area, and volume of spalling on deck surfaces, undersides, and support columns can be derived from properly collected LiDAR point clouds. The LiDAR point clouds contain information that can provide quantitative surface condition information, resulting in more accurate structural health monitoring. LiDAR scans were collected at three study bridges, each of which displayed a varying degree of degradation. A variety of commercially available analysis tools and an independently developed algorithm written in ArcGIS Python (ArcPy) were used to locate and quantify surface defects such as location, volume, and area of spalls. The results were visual and numerically displayed in a user-friendly web-based decision support tool integrating prior bridge condition metrics for comparison. LiDAR data processing procedures along with strengths and limitations of point clouds for defining features useful for assessing bridge deck condition are discussed. Point cloud density and incidence angle are two attributes that must be managed carefully to ensure data collected are of high quality and useful for bridge condition evaluation. When collected properly to ensure effective evaluation of bridge surface condition, LiDAR data can be analyzed to provide a useful data set from which to derive bridge deck condition information.

With the continuous development of the transformation of the “smart mine” in the mineral industry, the use of sensors in autonomous trucks has become very common. However, the driving of trucks causes the point cloud collected by through Light Detection and Ranging (LiDAR) to contain dust points, leading to a significant decline in its detection performance, which makes it easy for vehicles to have failures at the perceptual level. In order to solve this problem, this study proposes a LiDAR point cloud denoising method for the quantitative analysis of laser reflection intensity and spatial structure. This method uses laser reflectivity as the benchmark template, constructs the initial confidence level template and initially screens out the sparse dust point cloud. The results are analyzed through the Euclidean distance of adjacent points, and the confidence level in the corresponding template is reduced for rescreening. The experimental results show that our method can significantly filter dust point cloud particles while retaining the rich environmental information of data. The computational load caused by filtering is far lower than that of other methods, and the overall operation efficiency of the system has no significant delay.