Transformers for Object Detection in Large Point Clouds

The use of laser scanning techniques has become a common way to measure the real world. Millions of points could be generated by this system, which brings forth the problem of visualizing and eventually performing analyses on them. In the open source domain, various software packages exist to visualize 3D point clouds. However, most of these software packages do not allow the real-time rendering of large point clouds. Few of them also allow visualization in an immersive manner. The research aims to create an open source viewer for large 3D point clouds, which enables a dynamic and immersive visualization. In order to do so, rendering and point cloud management strategies must be implemented to avoid overloading the computer’s memory. The rendering is done using OpenGL engine by utilizing the graphic card’s memory in order to perform faster visualization. The program consists of a pre-processing stage in which the point cloud files are divided into quadtrees and then subsampled using the random tree sampling method. The visualization itself will calculate the distance between the point of view and the center of nodes generated by the pre-processing stage. The amount of points rendered within each node will depend on this distance; the farther away the node is from the point of view, the fewer points are rendered. A loading and unloading function enables the point cloud to be rendered dynamically. With the point cloud management algorithm implemented, the resulting program is able to load large point clouds generated by a mobile laser scanner using an ordinary computer. The resulting program as well as the source code will be available for the public due to its open source nature.

Topological Data Analysis is a machine learning method that summarizes the topological features of a space. Persistent Homology (PH) can identify these topological features as they persist within a point cloud; persisting in respect to the connectedness of the point cloud at increasing distances. The utility of PH is apparent in several fields including bioinformatics, network security, and object classification. However, the memory complexity of PH limits the application to relatively small point clouds for low-dimensional topological feature identification. For this reason, numerous approaches to optimize and approximate the PH have been introduced for providing results over large point clouds. One solution, Partitioned Persistent Homology (PPH), has shown favorable approximation on a single node with significant performance improvement. However, the single-node approach is limited by the available system memory, leading to the need for a distributed approach for additional (especially memory) resources. This paper studies a distributed version of PPH for use with large point clouds over a high-performance compute cluster. Experimental results of the distributed algorithm against previous studies is presented along with scalability of the distributed library.

Extracting roads from dense point clouds in large scale urban environment

We present a method for detecting and mapping trees in noisy stereo camera point clouds, using a learned 3-D object detector. Inspired by recent advancements in 3-D object detection using a pseudo-lidar representation for stereo data, we train a PointRCNN detector to recognize trees in forest-like environments. We generate detector training data with a novel automatic labeling process that clusters a fused global point cloud. This process annotates large stereo point cloud training data sets with minimal user supervision, and unlike previous pseudo-lidar detection pipelines, requires no 3-D ground truth from other sensors such as lidar. Our mapping system additionally uses a Kalman filter to associate detections and consistently estimate the positions and sizes of trees. We collect a data set for tree detection consisting of 8680 stereo point clouds, and validate our method on an outdoors test sequence. Our results demonstrate robust tree recognition in noisy stereo data at ranges of up to 7 meters, on 720p resolution images from a Stereolabs ZED 2 camera. Code and data are available at https://github.com/brian-h-wang/pseudolidar-tree-detection.

This paper presents a system that can automatically segment objects in large scale 3D point clouds obtained from urban ranging images. The system consists of three steps: The first one involves a ground detection process that can detect relatively complex terrain and separate it from other objects. The second step superpixelizes the remaining objects to speed up the segmentation process. In the final step, a manifold embedded mode seeking method is adopted to segment the point clouds. Even though the segmentation of urban objects is a challenging problem in terms of accuracy and problem scale, our system can efficiently generate very good segmentation results. The proposed manifold learning effectively improves the segmentation performance due to the fact that continuous artificial objects often have manifold-like structures.

A crucial part of safe navigation of autonomous vehicles is the robust detection of surrounding objects. While there are numerous approaches covering object detection in images or LiDAR point clouds, this paper addresses the problem of object detection in radar data. For this purpose, the fully convolutional neural network YOLOv3 is adapted to operate on sparse radar point clouds. In order to apply convolutions, the point cloud is transformed into a grid-like structure. The impact of this representation transformation is shown by comparison with a network based on Frustum PointNets, which directly processes point cloud data. The presented networks are trained and evaluated on the public nuScenes dataset. While experiments show that the point cloud-based network outperforms the grid-based approach in detection accuracy, the latter has a significantly faster inference time neglecting the grid conversion which is crucial for applications like autonomous driving.

In reverse engineering, the point cloud data processing is the key, it will directly affect the quality and efficiency of the subsequent reconstruction of the surface. However, the amount of data obtained by three-dimensional laser scanning is often larger and redundant, which will increase the difficulty of the surface reconstruction, therefore, the point cloud data is needed to streamline. Depth analysis of the efficiency and accuracy of curvature grid method, a secondary streamlined algorithm is proposed, which take use of non-uniform grid streamline law based on octree to streamline to the point cloud, and re-use curvature streamline method based on B-spline surface fitting to streamline it. This method, whose streamline ratio is relatively high, can effectively retain the feature point cloud data, providing a great convenience for the follow-up of reverse engineering.

In recent years, considerable research has been devoted to the efficient representation, modeling, processing and rendering of point-based models (point clouds). Due to their conceptual simplicity and superior flexibility, point-based representations evolved into a valuable alternative to polygonal meshes. This paper explores the problem of extracting direct visibility from point clouds without explicitly reconstructing the surface or estimating normal. The main contribution of this work is a hardware-accelerated hidden point removal operator. This operator is based on the GPU splatting technique combined with a morphological operation over the depth buffer to reduce the silhouette effect in the final image. The motivation to develop a new operator is the low efficiency demonstrated by existing hidden point removal methods in real time applications with large point clouds. We demonstrate that our proposal is useful for visualizing point clouds with interactive frame rates.

This paper introduces a new urban point cloud dataset for automatic segmentation and classification acquired by mobile laser scanning (MLS). We describe how the dataset is obtained from acquisition to post-processing and labeling. This dataset can be used to train pointwise classification algorithms; however, given that a great attention has been paid to the split between the different objects, this dataset can also be used to train the detection and segmentation of objects. The dataset consists of around 2 k m of MLS point cloud acquired in two cities. The number of points and range of classes mean that it can be used to train deep-learning methods. In addition, we show some results of automatic segmentation and classification. The dataset is available at: http://caor-mines-paristech.fr/fr/paris-lille-3d-dataset/.

Inspired by recent improvements in point cloud processing for autonomous navigation, we focus on using hierarchical graph neural networks for processing and feature learning over large-scale outdoor LiDAR point clouds. We observe that existing GNN based methods fail to overcome challenges of scale and irregularity of points in outdoor datasets. Addressing the need to preserve structural details while learning over a larger volume efficiently, we propose Hierarchical Point Graph Neural Network (HPGNN). It learns node features at various levels of graph coarseness to extract information. This enables to learn over a large point cloud while retaining fine details that existing point-level graph networks struggle to achieve. Connections between multiple levels enable a point to learn features in multiple scales, in a few iterations. We design HPGNN as a purely GNN-based approach, so that it offers modular expandability as seen with other point-based and Graph network baselines. To illustrate the improved processing capability, we compare previous point based and GNN models for semantic segmentation with our HPGNN, achieving a significant improvement for GNNs (+36.7 mIoU) on the SemanticKITTI dataset.

With the increase of point cloud scale, the time required by traditional ICP-related point cloud registration methods increases dramatically, which cannot meet the registration requirements of large-scale point clouds. In this paper, a fast registration technique for large scale point clouds based on virtual viewpoint image generation is studied. Firstly, the projection image of color point cloud is generated by virtual viewpoint. Then, the feature is extracted based on ORB and the rotation and translation matrix is calculated. The experimental results show that the registration time of the proposed method is about 1s when the size of the point cloud is from 300,000 to 2 million, which is improved by 17–258 times compared with the traditional ICP registration method, and the registration error is reduced by 80% from ICP 5.0 to 1.0. This paper provides a new idea and method for large-scale color point cloud registration.

Suppose a large and dense point cloud of an object with complex geometry is available that can be approximated by a smooth univariate function. In general, for such point clouds the “best” approximation using the method of least squares is usually hard or sometimes even impossible to compute. In most cases, however, a “near-best” approximation is just as good as the “best”, but usually much easier and faster to calculate. Therefore, a fast approach for the approximation of point clouds using Chebyshev polynomials is described, which is based on an interpolation in the Chebyshev points of the second kind. This allows to calculate the unknown coefficients of the polynomial by means of the Fast Fourier transform (FFT), which can be extremely efficient, especially for high-order polynomials. Thus, the focus of the presented approach is not on sparse point clouds or point clouds which can be approximated by functions with few parameters, but rather on large dense point clouds for whose approximation perhaps even millions of unknown coefficients have to be determined.

Range scanning of building interiors generates very large, partially spurious and unstructured point cloud data. Accurate information extraction from such data sets is a complex task due to the presence of multiple objects, diversity of their shapes, large disparity in the feature sizes, and the spatial uncertainty due to occluded regions. A fast segmentation of such data is necessary for quick understanding of the scanned scene. Unfortunately, traditional range segmentation methodologies are computationally expensive because they rely almost exclusively on shape parameters (normal, curvature) and are highly sensitive to small geometric distortions in the captured data. This paper introduces a quick and effective segmentation technique for large volumes of colorized range scans from unknown building interiors and labelling clusters of points that represent distinct surfaces and objects in the scene. Rather than computing geometric parameters, the proposed technique uses a robust Hue, Saturation and Value (HSV) color model as an effective means of id entifying rough clusters (objects) that are further refined by eliminating spurious and outlier points through region growth an d a fixed distance neighbors (FDNs) analysis. The results demonstrate that the proposed method is effective in identifying continuous clusters and can extract meaningful object clusters, even from geometrically similar regions.

Accurate semantic segmentation of 3D point clouds is a long-standing problem in remote sensing and computer vision. Due to the unstructured nature of point clouds, designing deep neural architectures for point cloud semantic segmentation is often not straightforward. In this work, we circumvent this problem by devising a technique to exploit structured neural architectures for unstructured data. In particular, we employ the popular convolutional neural network (CNN) architectures to perform semantic segmentation of LiDAR data. We propose a projection-based scheme that performs an angle-wise slicing of large 3D point clouds and transforms those slices into 2D grids. Accounting for intensity and reflectivity of the LiDAR input, the 2D grid allows us to construct a pseudo image for the point cloud slice. We enhance this image with low-level image processing techniques of normalization, histogram equalization, and decorrelation stretch to suit our ultimate object of semantic segmentation. A large number of images thus generated are used to induce an encoder-decoder CNN model that learns to compute a segmented 2D projection of the scene, which we finally back project to the 3D point cloud. In addition to a novel method, this article also makes a second major contribution of introducing the enhanced version of our large-scale public PC-Urban outdoor dataset which is captured in a civic setup with an Ouster LiDAR sensor. The updated dataset (PC-Urban_V2) provides nearly 8 billion points including over 100 million points labeled for 25 classes of interest. We provide a thorough evaluation of our technique on PC-Urban_V2 and three other public datasets.

We propose a novel deep architecture for semantic labeling of 3D point clouds referred to as Global and Local Streams Network (GLSNet) which is designed to capture both global and local structures and contextual information for large scale 3D point cloud classification. Our GLSNet tackles a hard problem - large differences of object sizes in large-scale point cloud segmentation including extremely large objects like water, and small objects like buildings and trees, and we design a two-branch deep network architecture to decompose the complex problem to separate processing problems at global and local scales and then fuse their predictions. GLSNet combines the strength of Submanifold Sparse Convolutional Network [1] for learning global structure with the strength of PointNet++ [2] for incorporating local information. The first branch of GLSNet processes a full point cloud in the global stream, and it captures long range information about the geometric structure by using a U-Net structured Submanifold Sparse Convolutional Network (SSCN-U) architecture. The second branch of GLSNet processes a point cloud in the local stream, and it partitions 3D points into slices and processes one slice of the cloud at a time by using the PointNet ++ architecture. The two streams of information are fused by max pooling over their classification prediction vectors. Our results on the IEEE GRSS Data Fusion Contest Urban Semantic 3D, Track 4 (DFT4) [3] [4] [5] point cloud classification dataset have shown that GLSNet achieved performance gains of almost 4% in mIOU and 1% in overall accuracy over the individual streams on the held-back testing dataset.