A Weakly Supervised Vehicle Detection Method from LiDAR Point Clouds

Segmentation of vegetation LiDAR point clouds is an important method for obtaining individual tree structure parameters. The current individual tree segmentation methods are mainly for airborne LiDAR point clouds, which use elevation information to form a grid map for segmentation, or use canopy vertices as seed points for clustering. Side-view LiDAR (vehicle LiDAR and hand-held LiDAR) can acquire more information about the lower layer of trees, but it is a challenge to perform the individual tree segmentation because the structure of side-view LiDAR point clouds is more complex. This paper proposes an individual tree segmentation method called Shadow-cut to extract the contours of the street tree point cloud. Firstly, we separated the region of the trees using the binary classifier (e.g., support vector machine) based on point cloud geometric features. Then, the optimal projection of the 3D point clouds to the 2D image is calculated and the optimal projection is the case where the pixels of the individual tree image overlap the least. Finally, after using the image segmentation algorithm to extract the tree edges in the 2D image, the corresponding 3D individual tree point cloud contours are matched with the pixels of individual tree edges in the 2D image. We conducted experiments with the proposed method on LiDAR data of urban street trees, and the correctness, completeness, and quality of the proposed individual tree segmentation method reached 91.67%, 85.33%, and 79.19%, which were superior to the CHM-based method by 2.70%, 6.19%, and 7.12%, respectively. The results show that this method is a practical and effective solution for individual tree segmentation in the LiDAR point clouds of street trees.

The geometry of a point cloud is commonly represented by an octree recursively decomposing a 3D volume into eight child sub-volumes. Said volumes and sub-volumes are associated with nodes and child-nodes of the octree. The geometry is defined by the occupancy information indicating the presence or not of a point in each of the sub-volumes. This naturally leads to an eight-bit occupancy information to be coded for each internal node of the tree. This paper introduces a new binarization scheme to efficiently compress the occupancy information using an optimal set of binary entropy coders. Then, it is shown how using the occupancy information of neighbouring nodes helps to compress the occupancy bits associated with the child nodes of the current node. This information is used to contextualise the binarization scheme by computing, firstly a neighbour configuration, secondly a number of neighbours with occupied child nodes adjacent to the current child node, and thirdly an intra predictor. Objective results show lossless geometry compression gains between 60% and 75% on virtual reality oriented dense point clouds used by MPEG, reaching sub-bit per point bit-rates for the lossless intra coding of such point clouds. Solid gains (between 5% and 25% depending upon the sampling) are also observed on sparse point clouds captured by a LiDAR (Light Detection and Ranging) device attached to a moving vehicle or representing 3D maps.

Point cloud geometry (PCG) is used to precisely represent arbitrary-shaped 3D objects and scenes, is of great interest to vast applications which puts forward the pressing desire of high-efficiency PCG compression for transmission and storage. Existing PCG coding mostly relies on the octree model by which point-wise processing is applied without exploring nonlocal regional geometry similarity across the entire 3D surface. This work, instead, suggests the region-wise processing to leverage the region similarity to exploit inter-region redundancy for efficient lossy point cloud geometry compression. Towards this goal, a given PCG is first segmented into numerous local regions each of which comprises a portion of point cloud surface, and can be represented by a surface vector that describes the geometry shape numerically in a projected principal space. Subsequently, these regions are grouped into several discriminative clusters, assuring that inter-cluster similarity is minimized and intra-cluster similarity is maximized simultaneously, where the similarity is calculated using the regional surface vectors. In each cluster, we set a reference region having the largest similarity score to the others, which enables the non-reference region prediction from the reference one using alignment transform. In the end, we encode the reference regions directly using the lossless mode of the Geometry-based Point Cloud Compression (G-PCC), while corresponding non-reference regions are signaled using associated transform parameters. Compared with the state-of-the-art G-PCC using octree model, our region-wise approach can offer remarkable coding efficiency improvement, e.g., 32.4% and 22.0% Bjontegaard-delta rate (BD-Rate) gains for respective point-to-point ( <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$D1$ </tex-math></inline-formula> ) and point-to-plane ( <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$D2$ </tex-math></inline-formula> ) distortion evaluations, across a variety of common test sequences used in standard committee.

This paper provides a practical procedure for fusing LiDAR and photogrammetric point clouds for the extraction of tree metrics. Aerial photogrammetric point clouds are first generated using the structure-from-motion and dense-matching methods. Registration of the LiDAR and photogrammetric point clouds is then performed using an onboard global positioning system and inertial measurement unit. However, due to systematic deviations, it is impossible to directly merge the two types of point cloud. Therefore, an urban street network obtained from the OpenStreetMap digital mapping system is utilized for point cloud segmentation. After segmentation, each chunk is finely registered and merged based on the iterative closest point algorithm, allowing the two types of data to be accurately co-registered and a fused point cloud obtained. Finally, we conducted experiments to extract stand and individual tree metrics from fused point clouds created for two study plots. The height distributions of the fused point clouds were highly consistent with LiDAR data, with the 5%, 10%, 25%, 50%, 75%, 90%, and 95% height percentiles showing acceptable similarities. The height distribution of individual trees was also consistent with that of field measurements. Furthermore, the fused point clouds contain a high point density and RGB color information, which allow shape delineation and estimation of tree health status. This comprehensive analysis demonstrates that this procedure provides a practical way to inventory tree stands and individuals in urban areas.

A crucial part of safe navigation of autonomous vehicles is the robust detection of surrounding objects. While there are numerous approaches covering object detection in images or LiDAR point clouds, this paper addresses the problem of object detection in radar data. For this purpose, the fully convolutional neural network YOLOv3 is adapted to operate on sparse radar point clouds. In order to apply convolutions, the point cloud is transformed into a grid-like structure. The impact of this representation transformation is shown by comparison with a network based on Frustum PointNets, which directly processes point cloud data. The presented networks are trained and evaluated on the public nuScenes dataset. While experiments show that the point cloud-based network outperforms the grid-based approach in detection accuracy, the latter has a significantly faster inference time neglecting the grid conversion which is crucial for applications like autonomous driving.

Masked autoencoding has become a successful pretraining paradigm for Transformer models for text, images, and, recently, point clouds. Raw automotive datasets are suitable candidates for self-supervised pre-training as they gener-ally are cheap to collect compared to annotations for tasks like 3D object detection (OD). However, the development of masked autoencoders for point clouds has focused solely on synthetic and indoor data. Consequently, existing meth-ods have tailored their representations and models toward small and dense point clouds with homogeneous point den-sities. In this work, we study masked autoencoding for point clouds in an automotive setting, which are sparse and for which the point density can vary drastically among ob-jects in the same scene. To this end, we propose Voxel-MAE, a simple masked autoencoding pre-training scheme designed for voxel representations. We pre-train the back-bone of a Transformer-based 3D object detector to reconstruct masked voxels and to distinguish between empty and non-empty voxels. Our method improves the 3D OD performance by 1.75 mAP points and 1.05 NDS on the challenging nuScenes dataset. Further, we show that by pre-training with Voxel-MAE, we require only 40% of the annotated data to outperform a randomly initialized equivalent. Code is available at https://github.com/georghess/voxel-mae.

The effective representation and feature extraction of 3-D scenes from sparse and unstructured point clouds pose a significant challenge in 3-D object detection. In this article, we propose TransMRE, a network that enables fully sparse multiple observation plane feature fusion using LiDAR point clouds as single-modal input. TransMRE achieves this by sparsely factorizing a 3-D voxel scene into three separate observation planes: XY, XZ, and YZ planes. In addition, we propose Observation Plane Sparse Fusion and Interaction to explore the internal relationship between different observation planes. The Transformer mechanism is employed to realize feature attention within a single observation plane and feature attention across multiple observation planes. This recursive application of attention is done during multiple observation plane projection feature aggregation to effectively model the entire 3-D scene. This approach addresses the limitation of insufficient feature representation ability under a single bird’s-eye view (BEV) constructed by extremely sparse point clouds. Furthermore, TransMRE maintains the full sparsity property of the entire network, eliminating the need to convert sparse feature maps into dense feature maps. As a result, it can be effectively applied to LiDAR point cloud data with large scanning ranges, such as Argoverse 2, while ensuring low computational complexity. Extensive experiments were conducted to evaluate the effectiveness of TransMRE, achieving 64.9 mAP and 70.4 NDS on the nuScenes detection benchmark, and 32.3 mAP on the Argoverse 2 detection benchmark. These results demonstrate that our method outperforms state-of-the-art methods.

Accurate extraction of individual urban objects from aerial LiDAR point clouds presents a complex challenge due to the heterogeneity of object types, occlusions, and overlapping geometries. This research paper introduces a hybrid framework that combines deep learning and geometric algorithms for the detection and classification of both structured and unstructured urban objects within LiDAR point clouds, supported by auxiliary RGB and satellite data. The methodology incorporates a multistage pipeline: initial point cloud classification using neural architectures such as RandLANet and PointPillars, followed by semantic segmentation using YOLOv7, YOLOv11, and SegFormer, and refined object extraction using rule-based and geometric filtering (e.g. DBSCAN, RANSAC). A data set of over 3.1 billion classified points, derived from Estonian LiDAR and OSM data, was prepared and augmented to address the detection of small and overlapping objects. Experimental results demonstrate high accuracy for structured classes (e.g. buildings: 88.3%) and effective segmentation of unstructured objects (e.g. trees: 85.0%). Cascaded networks achieved up to 72.6% classification accuracy depending on neighbourhood size. The 9-channel data fusion (RGB + LiDAR features) significantly improved segmentation performance. Comparative evaluation shows that combining model specialization with rule-based metaclass grouping improves object detection granularity and classification consistency, providing a scalable approach to fine-grained object recognition in smart city and geospatial analysis applications.

Object detection and classification in 3D is a key task in Automated Driving (AD). LiDAR sensors are employed to provide the 3D point cloud reconstruction of the surrounding environment, while the task of 3D object bounding box detection in real time remains a strong algorithmic challenge. In this paper, we build on the success of the one-shot regression meta-architecture in the 2D perspective image space and extend it to generate oriented 3D object bounding boxes from LiDAR point cloud. Our main contribution is in extending the loss function of YOLO v2 to include the yaw angle, the 3D box center in Cartesian coordinates and the height of the box as a direct regression problem. This formulation enables real-time performance, which is essential for automated driving. Our results are showing promising figures on KITTI benchmark, achieving real-time performance (40 fps) on Titan X GPU.

Recently, the advancement of deep learning (DL) in discriminative feature learning from 3-D LiDAR data has led to rapid development in the field of autonomous driving. However, automated processing uneven, unstructured, noisy, and massive 3-D point clouds are a challenging and tedious task. In this article, we provide a systematic review of existing compelling DL architectures applied in LiDAR point clouds, detailing for specific tasks in autonomous driving, such as segmentation, detection, and classification. Although several published research articles focus on specific topics in computer vision for autonomous vehicles, to date, no general survey on DL applied in LiDAR point clouds for autonomous vehicles exists. Thus, the goal of this article is to narrow the gap in this topic. More than 140 key contributions in the recent five years are summarized in this survey, including the milestone 3-D deep architectures, the remarkable DL applications in 3-D semantic segmentation, object detection, and classification; specific data sets, evaluation metrics, and the state-of-the-art performance. Finally, we conclude the remaining challenges and future researches.

Point clouds have been gaining importance as a solution to the problem of efficient representation of 3D geometric and visual information. They are commonly represented by large amounts of data, and compression schemes are important for their manipulation transmission and storing. However, the selection of appropriate compression schemes requires effective quality evaluation. In this work a subjective quality evaluation of point clouds using a surface representation is analyzed. Using a set of point cloud data objects encoded with the popular octree pruning method with different qualities, a subjective evaluation was designed. The point cloud geometry was presented to observers in the form of a movie showing the 3D Poisson reconstructed surface without textural information with the point of view changing in time. Subjective evaluations were performed in three different laboratories. Scores obtained from each test were correlated and no statistical differences were observed. Scores were also correlated with previous subjective tests and a good correlation was obtained when compared with mesh rendering in 2D monitors. Moreover, the results were correlated with state of the art point cloud objective metrics revealing poor correlation. Likewise, the correlation with a subjective test using a different representation of the point cloud data also showed poor correlation. These results suggest the need for more reliable objective quality metrics and further studies on adequate point cloud data representations.

Compressing massive LiDAR point clouds in real-time is critical to autonomous machines such as drones and self-driving cars. While most of the recent prior work has focused on compressing individual point cloud frames, this paper proposes a novel system that effectively compresses a sequence of point clouds. The idea to exploit both the spatial and temporal redundancies in a sequence of point cloud frames. We first identify a key frame in a point cloud sequence and spatially encode the key frame by iterative plane fitting. We then exploit the fact that consecutive point clouds have large overlaps in the physical space, and thus spatially encoded data can be (re-)used to encode the temporal stream. Temporal encoding by reusing spatial encoding data not only improves the compression rate, but also avoids redundant computations, which significantly improves the compression speed. Experiments show that our compression system achieves 40× to 90× compression rate, significantly higher than the MPEG’s LiDAR point cloud compression standard, while retaining high end-to-end application accuracies. Meanwhile, our compression system has a compression speed that matches the point cloud generation rate by today LiDARs and out-performs existing compression systems, enabling real-time point cloud transmission.

Individual tree detection from airborne laser scanning data based on supervoxels and local convexity

It takes much time to do TIN data I/O operation when converting Li DAR points cloud to raster data via TIN. An efficient streaming algorithm based on triangle driver is proposed in this paper. Firstly, a new sub algorithm for converting TIN to GRID based on the straight line positive and negative zone discriminant principle is presented to make the streaming algorithm feasible, for it performs by each triangle rather than by each grid node. Based on the sub algorithm, the streaming algorithm combines constructing triangles, converting triangles to GRID and freeing memory into a pipeline by traversing triangles to simulate the streaming computation. Instead of creating TIN and converting TIN to GRID separately in the CPU, the streaming algorithm integrates them together and creates GRID from points cloud directly. The results show that the sub-algorithm performs efficiently and there is a linear relationship between time consuming and the number of triangles. The streaming algorithm has a better performance in terms of time owing to no TIN data I/O operation and it improves RAM memory utilization. It offers an efficient way to create GRID from a great amount of Li DAR point cloud data and supports parallel computing.

Sensor fusion is very important for collaborative intelligent systems. A regional feature fusion network called ReFuNet for detecting 3D Object is proposed. It is difficult to detect distant or small objects accurately for the sparsity of LiDAR point cloud. The LiDAR point cloud and camera image information to solve the problem of point cloud sparsity is used, which can integrate image‐rich semantic information to enhance point cloud features. Also, the authors’ ReFuNet method segments the possible areas of objects by the results of 2D image detection. A cross‐attention mechanism adaptively fuses image and point cloud features within the areas. Then, the authors’ ReFuNet uses fused features to predict the 3D bounding boxes of objects. Experiments on the KITTI 3D object detection dataset showed that the authors’ proposed fusion method effectively improved the performance of 3D object detection.