An Accurate Point Cloud-Based Human Identification Using Micro-Size LiDAR

In recent years, due to the significant advancements in hardware sensors and software technologies, 3D environmental point cloud modeling has gradually been applied in the automation industry, autonomous vehicles, and construction engineering. With the high-precision measurements of 3D LiDAR, its point clouds can clearly reflect the geometric structure and features of the environment, thus enabling the creation of high-density 3D environmental point cloud models. However, due to the enormous quantity of high-density 3D point clouds, storing and processing these 3D data requires a considerable amount of memory and computing time. In light of this, this paper proposes a real-time 3D point cloud environmental contour modeling technique. The study uses the point cloud distribution from the 3D LiDAR body frame point cloud to establish structured edge features, thereby creating a 3D environmental contour point cloud map. Additionally, unstable objects such as vehicles will appear during the mapping process; these specific objects will be regarded as not part of the stable environmental model in this study. To address this issue, the study will further remove these objects from the 3D point cloud through image recognition and LiDAR heterogeneous matching, resulting in a higher quality 3D environmental contour point cloud map. This 3D environmental contour point cloud not only retains the recognizability of the environmental structure but also solves the problems of massive data storage and processing. Moreover, the method proposed in this study can achieve real-time realization without requiring the 3D point cloud to be organized in a structured order, making it applicable to unorganized 3D point cloud LiDAR sensors. Finally, the feasibility of the proposed method in practical applications is also verified through actual experimental data.

Surface defect identification is a crucial task in many manufacturing systems, including automotive, aircraft, steel rolling, and precast concrete. Although image-based surface defect identification methods have been proposed, these methods usually have two limitations: images may lose partial information, such as depths of surface defects, and their precision is vulnerable to many factors, such as the inspection angle, light, color, noise, etc. Given that a three-dimensional (3D) point cloud can precisely represent the multidimensional structure of surface defects, we aim to detect and classify surface defects using a 3D point cloud. This has two major challenges: (i) the defects are often sparsely distributed over the surface, which makes their features prone to be hidden by the normal surface and (ii) different permutations and transformations of 3D point cloud may represent the same surface, so the proposed model needs to be permutation and transformation invariant. In this paper, a two-step surface defect identification approach is developed to investigate the defects’ patterns in 3D point cloud data. The proposed approach consists of an unsupervised method for defect detection and a multi-view deep learning model for defect classification, which can keep track of the features from both defective and non-defective regions. We prove that the proposed approach is invariant to different permutations and transformations. Two case studies are conducted for defect identification on the surfaces of synthetic aircraft fuselage and the real precast concrete specimen, respectively. The results show that our approach receives the best defect detection and classification accuracy compared with other benchmark methods.

Building Information Modeling (BIM) has gained wider acceptance in the Architecture/Engineering/Construction (A/E/C) industries in the U.S. and internationally. Capability of BIM technologies in early detection of spatial conflicts among various building systems and components concurrently designed by different design team members prior to construction has great potential in reducing construction delays, loss of productivity, and overall construction costs. Recent advancements in 3D point cloud technologies, such as 3D laser scanning, can greatly augment BIM technology applications in the A/E/C industry by providing a rapid three-dimensional capture of the built environment in great detail and accuracy. This paper discusses the current approaches of implementing 3D point cloud data in BIM and virtual design and construction (VDC) applications during various stages of a project lifecycle and the challenges associated with processing the huge amount of 3D point cloud data. Conversion from discrete 3D point cloud raster data to geometric/vector BIM data remains to be a labor-intensive process. The needs for intelligent geometric feature detection/reconstruction algorithms for automated point cloud processing and issues related to data management will be discussed. This paper also presents some innovative approaches for integrating 3D point cloud data with BIM to efficiently augment built environment design, construction, and management.

Converting three-dimensional (3D) point cloud data to two-dimensional (2D) image based on virtual structured-light system is a popular method in 3D data compression because the image format is easier to process by the exiting storing and transmitting methods. When this method is used in the 3D point cloud compression, the quantization error is introduced during the coordinate mapping. To solve this problem, a virtual structured-light 3D point cloud compression algorithm based on geometric reshaping is presented. And a least-square system parameter optimization method is proposed to further improve the data decoding accuracy. In the proposed method, the 3D spatial coordinates are reshaped to a 2D matrix first, and then the 2D matrix is stored as a “Holoimage” by using the parameter optimized virtual structured-light system. The quantization error introduced by coordinate mapping between X and Y coordinates of the 3D point cloud and the 2D image pixel coordinates is suppressed, so the decoding accuracy is improved. In addition, the geometric information of the 3D point cloud is hidden synchronously when the 3D point cloud is compressed, which is of great significance in the copyright protection of the point cloud data. Experiments verify the effectiveness of the proposed algorithm. The decoding root mean square error (RMSE) of the proposed method is decreased by 79.86% on average compared with the traditional one under the same compression ratio, and the compression ratio of the proposed method is 1.96 times bigger than the traditional one under the similar decoding accuracy.

In this paper we propose a robust curb detection method which is based on the fusion between semantically labeled camera images and a 3D point cloud coming from LiDAR sensors. The labels from the semantically enhanced cloud are used to reduce the curbs’ searching area. Several spatial cues are next computed on each candidate curb region. Based on these features, a candidate curb region is either rejected or refined for obtaining a precise positioning of the curb points found inside it. A novel local model-based outlier removal algorithm is proposed to filter out the erroneous curb points. Finally, a temporal integration of the detected curb points in multiple consecutive frames is used to densify the detection result. An objective evaluation of the proposed solution is done using a high-resolution digital map containing ground truth curb points. The proposed system has proved capable of detecting curbs of any heights (from 3cm up to 30cm) in complex urban road scenarios (straight roads, curved roads, intersections with traffic isles and roundabouts).

Automated canopy stress classification for field crops has traditionally relied on single-perspective, two-dimensional (2D) photographs, usually obtained through top-view imaging using unmanned aerial vehicles (UAVs). However, this approach may fail to capture the full extent of plant stress symptoms, which can manifest throughout the canopy. Recent advancements in LiDAR technologies have enabled the acquisition of high-resolution 3D point cloud data for the entire canopy, offering new possibilities for more accurate plant stress identification and rating. This study explores the potential of leveraging 3D point cloud data for improved plant stress assessment. We utilized a dataset of RGB 3D point clouds of 700 soybean plants from a diversity panel exposed to iron deficiency chlorosis (IDC) stress. From this unique set of 700 canopies exhibiting varying levels of IDC, we extracted several representations, including (a) handcrafted IDC symptom-specific features, (b) canopy fingerprints, and (c) latent feature-based features. Subsequently, we trained several classification models to predict plant stress severity using these representations. We exhaustively investigated several stress representations and model combinations for the 3-D data. We also compared the performance of these classification models against similar models that are only trained using the associated top-view 2D RGB image for each plant. Among the feature-model combinations tested, the 3D canopy fingerprint features trained with a support vector machine yielded the best performance, achieving higher classification accuracy than the best-performing model based on 2D data built using convolutional neural networks. Our findings demonstrate the utility of color canopy fingerprinting and underscore the importance of considering 3D data to assess plant stress in agricultural applications.

Machine learning methods, in particular those based on deep learning, have gained in importance through the latest development of artificial intelligence and computer hardware. However, the direct application of deep learning methods to improve the results of 3D building reconstruction is often not possible due, for example, to the lack of suitable training data. To address this issue, we present RoofN3D which provides a three-dimensional (3D) point cloud training dataset that can be used to train machine learning models for different tasks in the context of 3D building reconstruction. The details about RoofN3D and the developed framework to automatically derive such training data are described in this paper. Furthermore, we provide an overview of other available 3D point cloud training data and approaches from current literature in which solutions for the application of deep learning to 3D point cloud data are presented. Finally, we exemplarily demonstrate how the provided data can be used to classify building roofs with the PointNet framework.

Accurate semantic segmentation of 3D point clouds is a long-standing problem in remote sensing and computer vision. Due to the unstructured nature of point clouds, designing deep neural architectures for point cloud semantic segmentation is often not straightforward. In this work, we circumvent this problem by devising a technique to exploit structured neural architectures for unstructured data. In particular, we employ the popular convolutional neural network (CNN) architectures to perform semantic segmentation of LiDAR data. We propose a projection-based scheme that performs an angle-wise slicing of large 3D point clouds and transforms those slices into 2D grids. Accounting for intensity and reflectivity of the LiDAR input, the 2D grid allows us to construct a pseudo image for the point cloud slice. We enhance this image with low-level image processing techniques of normalization, histogram equalization, and decorrelation stretch to suit our ultimate object of semantic segmentation. A large number of images thus generated are used to induce an encoder-decoder CNN model that learns to compute a segmented 2D projection of the scene, which we finally back project to the 3D point cloud. In addition to a novel method, this article also makes a second major contribution of introducing the enhanced version of our large-scale public PC-Urban outdoor dataset which is captured in a civic setup with an Ouster LiDAR sensor. The updated dataset (PC-Urban_V2) provides nearly 8 billion points including over 100 million points labeled for 25 classes of interest. We provide a thorough evaluation of our technique on PC-Urban_V2 and three other public datasets.

Three-dimensional (3D) semantic segmentation based on point clouds has become a critical technology in the field of intelligent spatial perception and scene understanding. However, most existing methods cannot fully exploit the spatial feature of 3D point cloud data and the orderliness of two-dimensional (2D) image information, resulting in an inability to effectively improve segmentation accuracy. This study proposes a novel 2D image and 3D point cloud fusion method named shared multi-layer perceptron fusion semantic segmentation (SMFusionSeg) for semantic segmentation. The rich features of 2D image segmentation and spatial information of 3D point clouds are integrated by designing a fusion architecture that facilitates the transfer of 2D semantics to 3D semantics. Specifically, the fusion architecture concatenates the features from both 2D and 3D domains, which are then further integrated through a shared multi-layer perceptron (MLP) and optimized using an attention mechanism to enhance feature effectiveness and distinctiveness. Moreover, a knowledge distillation strategy is introduced to improve the learning capability and segmentation accuracy in parsing complex scenes. Experimental results on the SemanticKITTI dataset demonstrate that the proposed SMFusionSeg achieves a mean Intersection over Union ( mIoU ) of 65.4%, significantly outperforming traditional single-modal methods and many existing segmentation techniques, thereby effectively enhancing the precision and robustness of geographic feature recognition.

LiDAR sensors play a crucial role in autonomous driving and advanced driver assistance systems. By firing high-rate laser beams, a LiDAR device is able to project its surroundings as 2D or 3D point cloud, which can be used for different purposes such as object detection, map generation, localization, and navigation. Autonomous vehicles are often equipped with at least one multi-layer LiDAR sensor with 360-degree coverage to include as much information as possible in the point cloud. Such a device generates enormous amount of data which poses a challenge for data storage, real-time computation, and data transmission, as autonomous vehicles are typically resource-constrained systems. This paper proposes a lightweight and adaptive point cloud data structure to reduce the size of a 3D point cloud. The suggested data structure can be flexibly configured with different parameters to adapt for precision, distance coverage, and reflectivity resolution. The precision of the data structure is evaluated using a 16-layer Velodyne LiDAR sensor (VLP-16) to collect data in the city area of AstaZero proving ground and Gothenburg downtown. Our results show that the adaptive data structure can consume only 1/8th of the original point cloud size and hence, it is particularly suitable for applications with limited hardware resources or certain tolerance to precision of the point cloud. The suggested concept is also generalizable to other types of point cloud providing sensors.

We study an unsupervised domain adaptation problem for the semantic labeling of 3D point clouds, with a particular focus on domain discrepancies induced by different LiDAR sensors. Based on the observation that sparse 3D point clouds are sampled from 3D surfaces, we take a Complete and Label approach to recover the underlying surfaces before passing them to a segmentation network. Specifically, we design a Sparse Voxel Completion Network (SVCN) to complete the 3D surfaces of a sparse point cloud. Unlike semantic labels, to obtain training pairs for SVCN requires no manual labeling. We also introduce local adversarial learning to model the surface prior. The recovered 3D surfaces serve as a canonical domain, from which semantic labels can transfer across different LiDAR sensors. Experiments and ablation studies with our new benchmark for cross-domain semantic labeling of LiDAR data show that the proposed approach provides 6.3-37.6% better performance than previous domain adaptation methods.

In this paper, we show the existence of human radio biometrics and present a human identification system that can discriminate individuals even through the walls in a non-line-of-sight condition. Using commodity Wi-Fi devices, the proposed system captures the channel state information (CSI) and extracts human radio biometric information from Wi-Fi signals using the time-reversal (TR) technique. By leveraging the fact that broadband wireless CSI has a significant number of multipaths, which can be altered by human body interferences, the proposed system can recognize individuals in the TR domain without line-of-sight radio. We built a prototype of the TR human identification system using standard Wi-Fi chipsets with $3 \times 3$ multi-in multi-out (MIMO) transmission. The performance of the proposed system is evaluated and validated through multiple experiments. In general, the TR human identification system achieves an accuracy of 98.78% for identifying about a dozen of individuals using a single transmitter and receiver pair. Thanks to the ubiquitousness of Wi-Fi, the proposed system shows the promise for future low-cost low-complexity reliable human identification applications based on radio biometrics.

Stereo cameras, laser rangers and other time-of-flight ranging devices are utilized with increasing frequency as they can provide information in the 3D plane. The ability to perform real-time registration of the 3D point clouds obtained from these sensors is important in many applications. However, the tasks of locating accurate and dependable correspondences between point clouds and registration can be quite slow. Furthermore, any algorithm must be robust against artifacts in 3D range data as sensor motion, reflection and refraction are commonplace. The SIFT feature detector is a robust algorithm used to locate features, but cannot be extended directly to the 3D range point clouds since it requires dense pixel information, whereas the range voxels are sparsely distributed. This paper proposes an approach which enables SIFT application to locate scale and rotation invariant features in 3D point clouds. The algorithm then utilizes the known point correspondence registration algorithm in order to achieve real-time registration of 3D point clouds.KeywordsPoint CloudScale Invariant Feature TransformIterative Close Point AlgorithmScale Invariant Feature Transform FeatureRANSAC AlgorithmThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

The objective of this work is to develop and apply a spectral graph theoretic approach for differentiating between (classifying) additive manufactured (AM) parts contingent on the severity of their dimensional variation from laser-scanned coordinate measurements (3D point cloud). The novelty of the approach is in invoking spectral graph Laplacian eigenvalues as an extracted feature from the laser-scanned 3D point cloud data in conjunction with various machine learning techniques. The outcome is a new method that classifies the dimensional variation of an AM part by sampling less than 5% of the 2 million 3D point cloud data acquired (per part). This is a practically important result, because it reduces the measurement burden for postprocess quality assurance in AM—parts can be laser-scanned and their dimensional variation quickly assessed on the shop floor. To realize the research objective, the procedure is as follows. Test parts are made using the fused filament fabrication (FFF) polymer AM process. The FFF process conditions are varied per a phased design of experiments plan to produce parts with distinctive dimensional variations. Subsequently, each test part is laser scanned and 3D point cloud data are acquired. To classify the dimensional variation among parts, Laplacian eigenvalues are extracted from the 3D point cloud data and used as features within different machine learning approaches. Six machine learning approaches are juxtaposed: sparse representation, k-nearest neighbors, neural network, naïve Bayes, support vector machine, and decision tree. Of these, the sparse representation technique provides the highest classification accuracy (F-score > 97%).

Object detection in 3D point clouds is essential in fields such as geospatial intelligence and autonomous driving. The common machine learning problem of scarce labeled training data is even more acute with 3D point cloud data. Active learning provides a framework to prioritize the additional effort to manually annotate unlabeled training data. Most active learning methods for deep learning fall into one of two categories: uncertainty methods and diversity methods. Uncertainty methods select data by assessing model outputs for their confidence and consistency and are therefore dependent on the expected output of each deep learning task. These methods tend to select batches of informative yet highly similar samples to label. Diversity-based active learning aims to create a labeled dataset that is both varied and representative of the remaining unlabeled data. Diversity methods operate directly on the feature representations of the inputs and are thus more flexible with respect to the specifics of the deep learning task. Our current work explores applying diversity methods and uncertainty-diversity hybrid methods to 3D object detection. We evaluate various approaches to incorporate diversity, including K-Medoids clustering, core set selection, and furthest nearest neighbors. We address the high dimensionality of the features extracted from a VoxelNet-based object detector by varying the distance metric used in the active learning algorithms. Furthermore, we compare our results to those obtained using only uncertainty methods. We assess the performance and efficiency of each active learning method in addition to the representativeness and diversity of the labeled datasets produced. We find that hybrid uncertainty-diversity methods outperform other methods in terms of object detection AP50 throughout active learning, annotation efficiency, and class balance.