On the fast approximation of point clouds using Chebyshev polynomials

The fusion of laser point cloud and visual image depends on the point cloud density and the target framing effect, the traditional laser point cloud processing for sparse point cloud clustering effect is poor, it is difficult to frame small objects as well as medium and long distance objects. Then the subsequent sensor fusion is easy to miss the recognition of obstacles. In this paper, we improve the frame selection method for sparse point clouds, firstly build a deep learning framework pointpillar, use pointpillar to frame the sparse laser point clouds, then spatially calibrate the lidar coordinate system and camera coordinate system, project the lidar point clouds to the camera image, improve the late fusion method, effectively use the detection results of single sensor, and finally The late-fusion is performed with the target detection results of the camera image to output the exact distance as well as the category of the target. Experiments show that compared with the recognition effect of the traditional fusion algorithm, the number of frames is increased by 6 and the missed recognition rate is reduced from 31.41% to 12.31%.

In reverse engineering, the point cloud data processing is the key, it will directly affect the quality and efficiency of the subsequent reconstruction of the surface. However, the amount of data obtained by three-dimensional laser scanning is often larger and redundant, which will increase the difficulty of the surface reconstruction, therefore, the point cloud data is needed to streamline. Depth analysis of the efficiency and accuracy of curvature grid method, a secondary streamlined algorithm is proposed, which take use of non-uniform grid streamline law based on octree to streamline to the point cloud, and re-use curvature streamline method based on B-spline surface fitting to streamline it. This method, whose streamline ratio is relatively high, can effectively retain the feature point cloud data, providing a great convenience for the follow-up of reverse engineering.

Extracting roads from dense point clouds in large scale urban environment

Three-dimensional (3D) point clouds have a wide range of applications in the field of 3D vision. The quality of the acquired point cloud data considerably impacts the subsequent work of point cloud processing. Due to the sparsity and irregularity of point cloud data, processing point cloud data has always been challenging. However, existing deep learning-based point cloud dense reconstruction methods suffer from excessive smoothing of reconstruction results and too many outliers. The reason for this is that it is not possible to extract features for local and global features at different scales and provide different levels of attention to different regions in order to obtain long-distance dependence for dense reconstruction. In this paper, we use a parallel multi-scale feature extraction module based on graph convolution and an upsampling method with an added multi-head attention mechanism to process sparse and irregular point cloud data to obtain extended point clouds. Specifically, a point cloud training patch with 256 points is inputted. The PMS module uses three residual connections in the multi-scale feature extraction stage. Each PMS module consists of three parallel DenseGCN modules with different size convolution kernels and different averaging pooling sizes. The local and global feature information of the augmented receptive field is extracted efficiently. The scale information is obtained by averaging the different pooled augmented receptive fields. The scale information was obtained using the different average pooled augmented receptive fields. The upsampling stage uses an upsampling rate of r=4, The self-attentive features with a different focus on different point cloud data regions obtained by fusing different weights make the feature representation more diverse. This operation avoids the bias of one attention, and each focuses on extracting valuable fine-grained feature information. Finally, the coordinate reconstruction module obtains 1024 dense point cloud data. Experiments show that the proposed method demonstrates good evaluation metrics and performance and is able to obtain better visual quality. The problems of over-smoothing and excessive outliers are effectively mitigated, and the obtained sparse point cloud is more dense.

The use of laser scanning techniques has become a common way to measure the real world. Millions of points could be generated by this system, which brings forth the problem of visualizing and eventually performing analyses on them. In the open source domain, various software packages exist to visualize 3D point clouds. However, most of these software packages do not allow the real-time rendering of large point clouds. Few of them also allow visualization in an immersive manner. The research aims to create an open source viewer for large 3D point clouds, which enables a dynamic and immersive visualization. In order to do so, rendering and point cloud management strategies must be implemented to avoid overloading the computer’s memory. The rendering is done using OpenGL engine by utilizing the graphic card’s memory in order to perform faster visualization. The program consists of a pre-processing stage in which the point cloud files are divided into quadtrees and then subsampled using the random tree sampling method. The visualization itself will calculate the distance between the point of view and the center of nodes generated by the pre-processing stage. The amount of points rendered within each node will depend on this distance; the farther away the node is from the point of view, the fewer points are rendered. A loading and unloading function enables the point cloud to be rendered dynamically. With the point cloud management algorithm implemented, the resulting program is able to load large point clouds generated by a mobile laser scanner using an ordinary computer. The resulting program as well as the source code will be available for the public due to its open source nature.

Accurate human activity recognition (HAR) is the key to enable emerging context-aware applications that require an understanding and identification of human behavior, e.g., monitoring disabled or elderly people who live alone. Traditionally, HAR has been implemented either through ambient sensors, e.g., cameras, or through wearable devices, e.g., a smartwatch, with an inertial measurement unit (IMU). The ambient sensing approach is typically more generalizable for different environments as this does not require every user to have a wearable device. However, utilizing a camera in privacy-sensitive areas such as a home may capture superfluous ambient information that a user may not feel comfortable sharing. Radars have been proposed as an alternative modality for coarse-grained activity recognition that captures a minimal subset of the ambient information using micro-Doppler spectrograms. However, training fine-grained, accurate activity classifiers is a challenge as low-cost millimeter-wave (mmWave) radar systems produce sparse and non-uniform point clouds. In this paper, we propose RadHAR, a framework that performs accurate HAR using sparse and non-uniform point clouds. RadHAR utilizes a sliding time window to accumulate point clouds from a mmWave radar and generate a voxelized representation that acts as input to our classifiers. We evaluate RadHAR using a low-cost, commercial, off-the-shelf radar to get sparse point clouds which are less visually compromising. We evaluate and demonstrate our system on a collected human activity dataset with 5 different activities. We compare the accuracy of various classifiers on the dataset and find that the best performing deep learning classifier achieves an accuracy of 90.47%. Our evaluation shows the efficacy of using mmWave radar for accurate HAR detection and we enumerate future research directions in this space.

With the increasing use of commodity RGB-D cameras for computer vision, robotics, mixed and augmented reality and other areas, it is of significant practical interest to calibrate the relative pose between a depth (D) camera and an RGB camera in these typ

Stockpile volume estimation plays a critical role in several industrial/commercial bulk material management applications. LiDAR systems are commonly used for this task. Thanks to Global Navigation Satellite System (GNSS) signal availability in outdoor environments, Uncrewed Aerial Vehicles (UAV) equipped with LiDAR are frequently adopted for the derivation of dense point clouds, which can be used for stockpile volume estimation. For indoor facilities, static LiDAR scanners are usually used for the acquisition of point clouds from multiple locations. Acquired point clouds are then registered to a common reference frame. Registration of such point clouds can be established through the deployment of registration targets, which is not practical for scalable implementation. For scans in facilities bounded by planar walls/roofs, features can be automatically extracted/matched and used for the registration process. However, monitoring stockpiles stored in dome facilities remains to be a challenging task. This study introduces an image-aided fine registration strategy of acquired sparse point clouds in dome facilities, where roof and roof stringers are extracted, matched, and modeled as quadratic surfaces and curves. These features are then used in a Least Squares Adjustment (LSA) procedure to derive well-aligned LiDAR point clouds. Planar features, if available, can also be used in the registration process. Registered point clouds can then be used for accurate volume estimation of stockpiles. The proposed approach is evaluated using datasets acquired by a recently developed camera-assisted LiDAR mapping platform—Stockpile Monitoring and Reporting Technology (SMART). Experimental results from three datasets indicate the capability of the proposed approach in producing well-aligned point clouds acquired inside dome facilities, with a feature fitting error in the 0.03–0.08 m range.

Recently, terrestrial laser scanners have been significantly improved in terms of accuracy, measurement distance, measurement speed, and resolution. They enable us to capture dense 3D point clouds of large-scale objects and fields, such as factories, engineering plants, large equipment, and transport ships. In addition, the mobile mapping system, which is a vehicle equipped with laser scanners and GPSs, can be used for capturing large-scale point clouds from a wide range of roads, buildings, and roadside objects. Large-scale point clouds are useful in a variety of applications, such as renovation and maintenance of facilities, engineering simulation, asset management, and 3D mapping. To realize these applications, new techniques must be developed for processing large-scale point clouds. So far, point processing has been studied mainly for relatively small objects in the field of computer-aided design and computer graphics. However, in recent years, the application areas of point clouds are not limited to conventional domains, but also include manufacturing, civil engineering, construction, transportation, forestry, and so on. This is because the state-of-the-art laser scanner can be used to represent large objects or fields as dense point clouds. We believe that discussing new techniques and applications related to large-scale point clouds beyond the boundaries of traditional academic fields is very important.This special issue addresses the latest research advances in large-scale point cloud processing. This covers a wide area of point processing, including shape reconstruction, geometry processing, object recognition, registration, visualization, and applications. The papers will help readers explore and share their knowledge and experience in technologies and development techniques.All papers were refereed through careful peer reviews. We would like to express our sincere appreciation to the authors for their submissions and to the reviewers for their invaluable efforts for ensuring the success of this special issue.

It is an important task to reconstruct surfaces from 3D point clouds. Current methods are able to reconstruct surfaces by learning Signed Distance Functions (SDFs) from single point clouds without ground truth signed distances or point normals. However, they require the point clouds to be dense, which dramatically limits their performance in real applications. To resolve this issue, we propose to reconstruct highly accurate surfaces from sparse point clouds with an on-surface prior. We train a neural network to learn SDFs via projecting queries onto the surface represented by the sparse point cloud. Our key idea is to infer signed distances by pushing both the query projections to be on the surface and the projection distance to be the minimum. To achieve this, we train a neural network to capture the on-surface prior to determine whether a point is on a sparse point cloud or not, and then leverage it as a differentiable function to learn SDFs from unseen sparse point cloud. Our method can learn SDFs from a single s parse point cloud without ground truth signed distances or point normals. Our numerical evaluation under widely used benchmarks demonstrates that our method achieves state-of-the-art reconstruction accuracy, especially for sparse point clouds. Code and data are available at https://github.com/mabaorui/OnSurfacePrior.

In this paper, we study the semantic segmentation of 3D LiDAR point cloud data in urban environments for autonomous driving, and a method utilizing the surface information of the ground plane was proposed. In practice, the resolution of a LiDAR sensor installed in a self-driving vehicle is relatively low and thus the acquired point cloud is indeed quite sparse. While recent work on dense point cloud segmentation has achieved promising results, the performance is relatively low when directly applied to sparse point clouds. This paper is focusing on semantic segmentation of the sparse point clouds obtained from 32-channel LiDAR sensor with deep neural networks. The main contribution is the integration of the ground information which is used to group ground points far away from each other. Qualitative and quantitative experiments on two large-scale point cloud datasets show that the proposed method outperforms the current state-of-the-art.

Fast automatic three-dimensional road model reconstruction based on mobile laser scanning system

The article explores the creation of digital terrain models (DRMs) based on dense point cloud classification in QGIS, Digitals, and 3D Survey software. In the research process, a dense cloud of points created during 3D scanning of the territory of the botanical garden of NUBiP of Ukraine was used, followed by classification and construction of digital relief models in various software tools. The stages of point cloud classification, creation of classification layers, data export to QGIS, Digitals and 3D Survey software tools for further modeling and drawing of models with isolines were studied. The digital model of the relief and the stages of its creation, "cleaning" of the original cloud of points of different density are considered. On the basis of the same initial data, a dense cloud of points, the formation of digital models of the relief of the territory of the botanical garden of NUBiP of Ukraine was carried out in various software tools. The process of creating a digital relief model was investigated based on a dense point cloud using QGIS, Digitals, 3D Survey programs. The main advantages and disadvantages of building a digital relief model based on a cloud of points formed by the results of 3D scanning are analyzed. Features and capabilities of software tools QGIS, Digitals, 3D Survey and their ability to build 3D surfaces based on exported point clouds are considered.

Lidar has received a lot of attention due to its precise ranging accuracy. Ground points filtering is an important task in point clouds processing. It’s a challenge to model the ground surface and filter the point clouds accurately in the case of complex ground undulations, occlusions, and sparse point clouds. A novel ground surface modeling method based on a hybrid regression technique is proposed in this paper. The method integrates Gaussian process regression (GPR) and robust locally weighted regression (RLWR) by dividing the point clouds that are projected on the polar grid map into radial and circumferential filtering processes to form a hybrid regression model, which has the ability to eliminate the influence of outliers and model the ground surface robustly. First, the RLWR combined with gradient filter is applied to fit the sampled points in the radial direction, which will exclude outliers and get the fitting ground line. All radial fitting lines constitute the seed skeleton of the whole plane. Then, based on the seeds in the same circumferential of the skeleton, the GPR is applied to construct the ground surface model. The comparative experiments are implemented quantitatively and qualitatively on the simulated point clouds and measured data. The results show that the proposed method performs well in most real scenarios, even in the cases of ground undulation, occlusion, and sparse point clouds.

The 3D modeling of urban buildings has become a common research area in various disciplines such as photogrammetry and computer vision, with different applications such as intelligent city management, navigation of self-driving cars and architecture, just to name a few. The objective of this study is to produce a 3D model of the external facade of the buildings with the required precision, accuracy and level of detail according to the user’s requirements, while minimizing time and cost. This research focuses on the production of 3D models for blocks of residential buildings in Tehran, Iran. The Insta 360 One X2 spherical camera is selected to capture the data due to its low cost and 360 × 180° field of view. The camera specifications have facilitated more efficient data collection in terms of both time and cost. The proposed modeling method is based on extracting lines of external walls through the utilization of the point cloud density concept. Initially, photogrammetric point clouds are generated in with a reconstruction precision of 0.24 m from spherical camera images. In the next step, the 3D point cloud is projected into a 2D point cloud by setting the height component to zero. The 2D point cloud is then rotated based on the direction angle determined by the Hough transform so that the perpendicular walls are parallel to the axes of the coordinate system. Next, a 2D point cloud density analysis is performed by voxelizing the point cloud and counting the number of points in each voxel in both the horizontal and vertical directions. By determining the peaks in the density plot, the lines of the external vertical and horizontal walls are extracted. To extract the diagonal external walls, the density analysis is performed in the direction of the first principal component. Finally, by determining the height of each wall in the point cloud, a 3D model is created at the level of detail one. The resulting model has a precision of 0.32 m compared to real sizes, and the 2D plan has a precision of 0.31 m compared to the ground truth map. The use of the spherical camera and point cloud density analysis makes this method efficient and cost-effective, making it a promising approach for future urban modeling projects.