An adaptive normal estimation method for scanned point clouds with sharp features

A 3D object can be recovered from scanned point data, which requires accurate estimating normal directions of the object surface from the cloud data. Many point cloud processing algorithms rely on the accurate normal as input to generate an accurate 3D surface model. The neighborhood of a data point in its smooth region can be well approximated by a plane. However, the neighborhood of a feature point employed for the normal estimation is isotropic which would enclose points belonging to different surface patches across the sharp feature. In this paper, isotropic neighborhoods are segmented to search anisotropic neighborhoods for the accurate normal estimation. Normals and candidate feature points are first estimated by the principal component analysis (PCA) method. Neighborhoods of the feature point are then mapped into a Gaussian image. A k-means clustering algorithm is then used for the Gaussian image to identify an anisotropic sub-neighborhood for the data point. The normal of the candidate feature point is finally estimated by the anisotropic neighborhood with the PCA method. The proposed method can accurately estimate normal directions while preserving sharp features of the object surface. Applications have demonstrated the effectiveness of the proposed method.

A novel sharp features extraction method is proposed in this paper. First, we calculate the displacement between the point and its local weighted average position and we label the point with salient this value as the candidate sharp feature points and we estimate the normal direction of those candidate sharp feature points by means of local PCA methods. Then we refine the normal estimated by inferring the orientation of the points near the candidate sharp feature region and bilateral filtering in the normal field of point clouds. At last we project the displacement between point and its local weighted average position along the direction of normal .We use value of this projection as the criteria of whether a point can be labeled as sharp feature. The extracted discrete sharp feature points are represented in the form of piecewised B-Spline lines. Experiment on both real scanner point clouds and synthesized point clouds show that our method of sharp features extraction are simple to be implemented and efficient for both space and time overhead as well as it robust to the noise ,outlier and un even sample witch are inherent in the point clouds.

The fusion of laser point cloud and visual image depends on the point cloud density and the target framing effect, the traditional laser point cloud processing for sparse point cloud clustering effect is poor, it is difficult to frame small objects as well as medium and long distance objects. Then the subsequent sensor fusion is easy to miss the recognition of obstacles. In this paper, we improve the frame selection method for sparse point clouds, firstly build a deep learning framework pointpillar, use pointpillar to frame the sparse laser point clouds, then spatially calibrate the lidar coordinate system and camera coordinate system, project the lidar point clouds to the camera image, improve the late fusion method, effectively use the detection results of single sensor, and finally The late-fusion is performed with the target detection results of the camera image to output the exact distance as well as the category of the target. Experiments show that compared with the recognition effect of the traditional fusion algorithm, the number of frames is increased by 6 and the missed recognition rate is reduced from 31.41% to 12.31%.

Accurate human activity recognition (HAR) is the key to enable emerging context-aware applications that require an understanding and identification of human behavior, e.g., monitoring disabled or elderly people who live alone. Traditionally, HAR has been implemented either through ambient sensors, e.g., cameras, or through wearable devices, e.g., a smartwatch, with an inertial measurement unit (IMU). The ambient sensing approach is typically more generalizable for different environments as this does not require every user to have a wearable device. However, utilizing a camera in privacy-sensitive areas such as a home may capture superfluous ambient information that a user may not feel comfortable sharing. Radars have been proposed as an alternative modality for coarse-grained activity recognition that captures a minimal subset of the ambient information using micro-Doppler spectrograms. However, training fine-grained, accurate activity classifiers is a challenge as low-cost millimeter-wave (mmWave) radar systems produce sparse and non-uniform point clouds. In this paper, we propose RadHAR, a framework that performs accurate HAR using sparse and non-uniform point clouds. RadHAR utilizes a sliding time window to accumulate point clouds from a mmWave radar and generate a voxelized representation that acts as input to our classifiers. We evaluate RadHAR using a low-cost, commercial, off-the-shelf radar to get sparse point clouds which are less visually compromising. We evaluate and demonstrate our system on a collected human activity dataset with 5 different activities. We compare the accuracy of various classifiers on the dataset and find that the best performing deep learning classifier achieves an accuracy of 90.47%. Our evaluation shows the efficacy of using mmWave radar for accurate HAR detection and we enumerate future research directions in this space.

It is an important task to reconstruct surfaces from 3D point clouds. Current methods are able to reconstruct surfaces by learning Signed Distance Functions (SDFs) from single point clouds without ground truth signed distances or point normals. However, they require the point clouds to be dense, which dramatically limits their performance in real applications. To resolve this issue, we propose to reconstruct highly accurate surfaces from sparse point clouds with an on-surface prior. We train a neural network to learn SDFs via projecting queries onto the surface represented by the sparse point cloud. Our key idea is to infer signed distances by pushing both the query projections to be on the surface and the projection distance to be the minimum. To achieve this, we train a neural network to capture the on-surface prior to determine whether a point is on a sparse point cloud or not, and then leverage it as a differentiable function to learn SDFs from unseen sparse point cloud. Our method can learn SDFs from a single s parse point cloud without ground truth signed distances or point normals. Our numerical evaluation under widely used benchmarks demonstrates that our method achieves state-of-the-art reconstruction accuracy, especially for sparse point clouds. Code and data are available at https://github.com/mabaorui/OnSurfacePrior.

Normal estimation is an essential step in point cloud based geometric processing, such as high quality point based rendering and surface reconstruction. In this paper, we present a clustering based method for normal estimation which preserves sharp features. For a piecewise smooth point cloud, the k-nearest neighbors of one point lie on a union of multiple subspaces. Given the PCA normals as input, we perform a subspace clustering algorithm to segment these subspaces. Normals are estimated by the points lying in the same subspace as the center point. In contrast to the previous method, we exploit the low-rankness of the input data, by seeking the lowest rank representation among all the candidates that can represent one normal as linear combinations of the others. Integration of Low-Rank Representation (LRR) makes our method robust to noise. Moreover, our method can simultaneously produce the estimated normals and the local structures which are especially useful for denoise and segmentation applications. The experimental results show that our approach successfully recovers sharp features and generates more reliable results compared with the state-of-theart.

Robust normal estimation for point clouds with sharp features

Sharp features of 3D point clouds play an important role in many geometric computations and modeling application. In this paper, a novel modified Partition of Unity (PoU) Based Sharp feature extraction algorithm is proposed, which is directly operated on discrete point clouds. For every point in target point cloud, spherical neighborhood with radius δ is acquired with the help of KD-Tree and weighted average position of points within the δ -neighborhood is computed using modified PoU method. Distance which is the projection of the displace between original point and its Weighted average position along normal direction is defined as the criteria for a point belong to sharp feature or crease line. Experiments on both synthetic data and practical scanner point clouds indicate that our algorithm are both efficient and effective to the task of sharp feature extraction from point clouds. Our method is easy to be implemented and more sensitive to sharp features as well as its low computational complexity.

This paper presents a new approach to automatic registration of terrestrial laser scanning (TLS) point clouds utilizing a novel robust estimation method by an efficient BaySAC (BAYes SAmpling Consensus). The proposed method directly generates reflectance images from 3D point clouds, and then using SIFT algorithm extracts keypoints to identify corresponding image points. The 3D corresponding points, from which transformation parameters between point clouds are computed, are acquired by mapping the 2D ones onto the point cloud. To remove false accepted correspondences, we implement a conditional sampling method to select the n data points with the highest inlier probabilities as a hypothesis set and update the inlier probabilities of each data point using simplified Bayes’ rule for the purpose of improving the computation efficiency. The prior probability is estimated by the verification of the distance invariance between correspondences. The proposed approach is tested on four data sets acquired by three different scanners. The results show that, comparing with the performance of RANSAC, BaySAC leads to less iterations and cheaper computation cost when the hypothesis set is contaminated with more outliers. The registration results also indicate that, the proposed algorithm can achieve high registration accuracy on all experimental datasets.

In this paper, we propose a method to estimate normal vectors based on neighborhood clustering segmentation, which improves the accuracy of normal-vector estimation for sharp features. The proposed method adjusts the neighborhood through Gauss mapping and clustering segmentation to solve the problem of inaccurate estimation of the normal vector in the sharp-feature region. First, the normal vectors of the point cloud are initially estimated by principal component analysis (PCA). Next, the neighborhood of points from different patches, which are close to the sharp feature, are mapped to a unit Gauss sphere, and the point cloud on the unit sphere is clustered. All points of the cluster belonging to the target point are divided into sub-neighborhoods. Finally, with this sub-neighborhood as input, the normal vector of the point is accurately calculated by PCA. Experiments show that, even for noise and non-uniform sampling, the method proposed herein outperforms previous methods in terms of quality of results and running time.

Three-dimensional (3D) point clouds have a wide range of applications in the field of 3D vision. The quality of the acquired point cloud data considerably impacts the subsequent work of point cloud processing. Due to the sparsity and irregularity of point cloud data, processing point cloud data has always been challenging. However, existing deep learning-based point cloud dense reconstruction methods suffer from excessive smoothing of reconstruction results and too many outliers. The reason for this is that it is not possible to extract features for local and global features at different scales and provide different levels of attention to different regions in order to obtain long-distance dependence for dense reconstruction. In this paper, we use a parallel multi-scale feature extraction module based on graph convolution and an upsampling method with an added multi-head attention mechanism to process sparse and irregular point cloud data to obtain extended point clouds. Specifically, a point cloud training patch with 256 points is inputted. The PMS module uses three residual connections in the multi-scale feature extraction stage. Each PMS module consists of three parallel DenseGCN modules with different size convolution kernels and different averaging pooling sizes. The local and global feature information of the augmented receptive field is extracted efficiently. The scale information is obtained by averaging the different pooled augmented receptive fields. The scale information was obtained using the different average pooled augmented receptive fields. The upsampling stage uses an upsampling rate of r=4, The self-attentive features with a different focus on different point cloud data regions obtained by fusing different weights make the feature representation more diverse. This operation avoids the bias of one attention, and each focuses on extracting valuable fine-grained feature information. Finally, the coordinate reconstruction module obtains 1024 dense point cloud data. Experiments show that the proposed method demonstrates good evaluation metrics and performance and is able to obtain better visual quality. The problems of over-smoothing and excessive outliers are effectively mitigated, and the obtained sparse point cloud is more dense.

Lidar has received a lot of attention due to its precise ranging accuracy. Ground points filtering is an important task in point clouds processing. It’s a challenge to model the ground surface and filter the point clouds accurately in the case of complex ground undulations, occlusions, and sparse point clouds. A novel ground surface modeling method based on a hybrid regression technique is proposed in this paper. The method integrates Gaussian process regression (GPR) and robust locally weighted regression (RLWR) by dividing the point clouds that are projected on the polar grid map into radial and circumferential filtering processes to form a hybrid regression model, which has the ability to eliminate the influence of outliers and model the ground surface robustly. First, the RLWR combined with gradient filter is applied to fit the sampled points in the radial direction, which will exclude outliers and get the fitting ground line. All radial fitting lines constitute the seed skeleton of the whole plane. Then, based on the seeds in the same circumferential of the skeleton, the GPR is applied to construct the ground surface model. The comparative experiments are implemented quantitatively and qualitatively on the simulated point clouds and measured data. The results show that the proposed method performs well in most real scenarios, even in the cases of ground undulation, occlusion, and sparse point clouds.

Point normals are fundamental geometric attributes of point clouds. To tackle traditional and learning-based technology problems, a normal estimation method is proposed based on geometric prior and deep-learning techniques. First, a multi-scale patch selection module is utilized to preserve surfaces features and details and to lower the learning difficulty of the subsequent network. Afterwards, combining local features and geometric prior, a normal optimization network is proposed to output the refined normals. Finally, when the average angle error metric is used, quantitative and qualitative experiments on synthetic and real- scanned data demonstrate that, compared with the mainstream point cloud normal estimation methods, the proposed method is more effective on both preserving features and robust to noise.

Current 3D object detection methods achieve accurate and efficient results on the standard point cloud dataset. However, in real-world applications, the point cloud samples obtained in the real-time running may be much sparser due to various reasons (occlusion, low reflectivity of objects and fewer laser beams) and existing methods do not consider the limitations of their models on sparse point clouds. To improve the robustness of an object detector to sparser point clouds, we propose PointDrop, which learns to drop the features of some key points in the point clouds to generate challenging sparse samples for data augmentation. Moreover, PointDrop is able to adjust the difficulty of the generated samples based on the capacity of the detector and thus progressively improve the performance of the detector. We create two sparse point clouds datasets from the KITTI dataset to evaluate our method, and the experimental results show that PointDrop significantly improves the robustness of the detector to sparse point clouds.

This document introduces the RadIOCD, which is a dataset that contains sparse point cloud representations of indoor objects, collected by subjects wearing a commercial off-the-shelf mmWave radar. In particular, RadIOCD includes the recordings of 10 volunteers moving towards 5 different objects (i.e., backpack, chair, desk, human, and wall), placed in 3 different environments. RadIOCD includes sparse 3D point cloud data, together with their doppler velocity and intensity provided by the mmWave radar. A total of 5,776 files are available, with each one having an approximate duration of 8s. The scope of RadIOCD is the availability of data for the recognition of objects solely recorded by the mmWave radar, to be used in applications were the vision-based classification is cumbersome though critical (e.g., in search and rescue operation where there is smoke inside a building). Furthermore, we showcase that this dataset after being segmented into 76,821 samples contains enough data to apply Machine Learning-based techniques, ensuring that they could generalize in different environments and “unseen“ subjects.