Extended Rigid Multi-Target Tracking in Dense Point Clouds with Probabilistic Occlusion Reasoning

This paper presents an approach named symmetric measurement equation (SME) to track known number of multiple extended targets. The SME approach removes the target-measurement association uncertainty through converting the original observation into pseudo-measurement vector. The work is focused on tracking moving extended target using SMEs which define new measurements through the sums of products of the original measurements. The performance of the SME filter for multiple extended targets tracking is demonstrated by a computer simulation compared with the ETT-PHD filter.

One of the big challenges in multi-target tracking is the track management and correct data association between measurements and tracks. Major reason for tracking errors are detection failures such as merged, split, incomplete or missed detections as well as clutter-based detections (phantom objects). Those effects combined with uncertainties in existence and number of objects in the scene as well as uncertainties in their observability and dynamic object state lead to gross tracking errors. In this contribution we present an algorithm for visual detection and tracking of multiple extended targets which is capable of coping with occlusions and split and merge effects. Unlike most of the state-of-the-art approaches we utilize information about the measurements' composition gained through tracking dedicated feature points in the image and in 3D space, which allows us to reconstruct the desired object characteristics from the data even in the case of detection errors due to above-mentioned reasons. The proposed Feature-Based Probabilistic Data Association approach resolves data association ambiguities in a soft threshold-free decision based not only on target state prediction but also on the existence and observability estimation modeled as two additional Markov chains. A novel measurement reconstruction scheme allows for a correct innovation in case of split, merged and incomplete measurements realizing thus a detection-by-tracking approach. This process is assisted by a grid based object representation which offers a lower abstraction level of targets extent and is used for detailed occlusion analysis.

In the extended target tracking using Gaussian process (GP) or other methods to estimate the extension state, all of them is necessary to obtain the measurement model, and the deviation from the true kinematic state affects the extension result easily. Meanwhile, with the growth of the shape complexity, the corresponding covariance matrix dimension increases, leading to a very high computational burden. To this end, we present an extended target tracking algorithm based on deep learning, which adopts Gaussian mixture (GM) probability hypothesis density (PHD) filter for extended target (ET) tracking, called the ET-GM-PHD filter, to estimate the kinematic state, and uses convolutional conditional neural process (ConvCNP) to estimate the extension state (i.e., the shape of the target) avoiding the complex computation and dependence on priors. Therefore, the proposed algorithm is named as the ConvCNP-GM-PHD filter. In our algorithm, ConvCNP lies on the neural network (NN) and GP, where the former can learn the rules from a large amount of data without any priors, while the latter uses the covariance function to express the distribution over possible functions according to a small number of context observations. Due the proposed algorithm can effectively combine the advantages of both, it is reliable to get a model via training which is able to realize function regression flexibly. Thus, ConvCNP is able to estimate the shape function by putting the angles and radial value between the kinematic state and associated measurements into the pre-trained model. The results show that the accuracy of the extension state estimation of the proposed ConvCNP-GM-PHD filter is improved and its computational complexity is also reduced. It implies that the proposed algorithm is more real-time and effective, and has a good application prospect.

In reverse engineering, the point cloud data processing is the key, it will directly affect the quality and efficiency of the subsequent reconstruction of the surface. However, the amount of data obtained by three-dimensional laser scanning is often larger and redundant, which will increase the difficulty of the surface reconstruction, therefore, the point cloud data is needed to streamline. Depth analysis of the efficiency and accuracy of curvature grid method, a secondary streamlined algorithm is proposed, which take use of non-uniform grid streamline law based on octree to streamline to the point cloud, and re-use curvature streamline method based on B-spline surface fitting to streamline it. This method, whose streamline ratio is relatively high, can effectively retain the feature point cloud data, providing a great convenience for the follow-up of reverse engineering.

Recently, terrestrial laser scanners have been significantly improved in terms of accuracy, measurement distance, measurement speed, and resolution. They enable us to capture dense 3D point clouds of large-scale objects and fields, such as factories, engineering plants, large equipment, and transport ships. In addition, the mobile mapping system, which is a vehicle equipped with laser scanners and GPSs, can be used for capturing large-scale point clouds from a wide range of roads, buildings, and roadside objects. Large-scale point clouds are useful in a variety of applications, such as renovation and maintenance of facilities, engineering simulation, asset management, and 3D mapping. To realize these applications, new techniques must be developed for processing large-scale point clouds. So far, point processing has been studied mainly for relatively small objects in the field of computer-aided design and computer graphics. However, in recent years, the application areas of point clouds are not limited to conventional domains, but also include manufacturing, civil engineering, construction, transportation, forestry, and so on. This is because the state-of-the-art laser scanner can be used to represent large objects or fields as dense point clouds. We believe that discussing new techniques and applications related to large-scale point clouds beyond the boundaries of traditional academic fields is very important.This special issue addresses the latest research advances in large-scale point cloud processing. This covers a wide area of point processing, including shape reconstruction, geometry processing, object recognition, registration, visualization, and applications. The papers will help readers explore and share their knowledge and experience in technologies and development techniques.All papers were refereed through careful peer reviews. We would like to express our sincere appreciation to the authors for their submissions and to the reviewers for their invaluable efforts for ensuring the success of this special issue.

The 3D modeling of urban buildings has become a common research area in various disciplines such as photogrammetry and computer vision, with different applications such as intelligent city management, navigation of self-driving cars and architecture, just to name a few. The objective of this study is to produce a 3D model of the external facade of the buildings with the required precision, accuracy and level of detail according to the user’s requirements, while minimizing time and cost. This research focuses on the production of 3D models for blocks of residential buildings in Tehran, Iran. The Insta 360 One X2 spherical camera is selected to capture the data due to its low cost and 360 × 180° field of view. The camera specifications have facilitated more efficient data collection in terms of both time and cost. The proposed modeling method is based on extracting lines of external walls through the utilization of the point cloud density concept. Initially, photogrammetric point clouds are generated in with a reconstruction precision of 0.24 m from spherical camera images. In the next step, the 3D point cloud is projected into a 2D point cloud by setting the height component to zero. The 2D point cloud is then rotated based on the direction angle determined by the Hough transform so that the perpendicular walls are parallel to the axes of the coordinate system. Next, a 2D point cloud density analysis is performed by voxelizing the point cloud and counting the number of points in each voxel in both the horizontal and vertical directions. By determining the peaks in the density plot, the lines of the external vertical and horizontal walls are extracted. To extract the diagonal external walls, the density analysis is performed in the direction of the first principal component. Finally, by determining the height of each wall in the point cloud, a 3D model is created at the level of detail one. The resulting model has a precision of 0.32 m compared to real sizes, and the 2D plan has a precision of 0.31 m compared to the ground truth map. The use of the spherical camera and point cloud density analysis makes this method efficient and cost-effective, making it a promising approach for future urban modeling projects.

Fast automatic three-dimensional road model reconstruction based on mobile laser scanning system

The article explores the creation of digital terrain models (DRMs) based on dense point cloud classification in QGIS, Digitals, and 3D Survey software. In the research process, a dense cloud of points created during 3D scanning of the territory of the botanical garden of NUBiP of Ukraine was used, followed by classification and construction of digital relief models in various software tools. The stages of point cloud classification, creation of classification layers, data export to QGIS, Digitals and 3D Survey software tools for further modeling and drawing of models with isolines were studied. The digital model of the relief and the stages of its creation, "cleaning" of the original cloud of points of different density are considered. On the basis of the same initial data, a dense cloud of points, the formation of digital models of the relief of the territory of the botanical garden of NUBiP of Ukraine was carried out in various software tools. The process of creating a digital relief model was investigated based on a dense point cloud using QGIS, Digitals, 3D Survey programs. The main advantages and disadvantages of building a digital relief model based on a cloud of points formed by the results of 3D scanning are analyzed. Features and capabilities of software tools QGIS, Digitals, 3D Survey and their ability to build 3D surfaces based on exported point clouds are considered.

Suppose a large and dense point cloud of an object with complex geometry is available that can be approximated by a smooth univariate function. In general, for such point clouds the “best” approximation using the method of least squares is usually hard or sometimes even impossible to compute. In most cases, however, a “near-best” approximation is just as good as the “best”, but usually much easier and faster to calculate. Therefore, a fast approach for the approximation of point clouds using Chebyshev polynomials is described, which is based on an interpolation in the Chebyshev points of the second kind. This allows to calculate the unknown coefficients of the polynomial by means of the Fast Fourier transform (FFT), which can be extremely efficient, especially for high-order polynomials. Thus, the focus of the presented approach is not on sparse point clouds or point clouds which can be approximated by functions with few parameters, but rather on large dense point clouds for whose approximation perhaps even millions of unknown coefficients have to be determined.

The fusion of laser point cloud and visual image depends on the point cloud density and the target framing effect, the traditional laser point cloud processing for sparse point cloud clustering effect is poor, it is difficult to frame small objects as well as medium and long distance objects. Then the subsequent sensor fusion is easy to miss the recognition of obstacles. In this paper, we improve the frame selection method for sparse point clouds, firstly build a deep learning framework pointpillar, use pointpillar to frame the sparse laser point clouds, then spatially calibrate the lidar coordinate system and camera coordinate system, project the lidar point clouds to the camera image, improve the late fusion method, effectively use the detection results of single sensor, and finally The late-fusion is performed with the target detection results of the camera image to output the exact distance as well as the category of the target. Experiments show that compared with the recognition effect of the traditional fusion algorithm, the number of frames is increased by 6 and the missed recognition rate is reduced from 31.41% to 12.31%.

Using Fast Point Feature Histograms feature from point cloud for 3D object recognition or registration, Fast Point Feature Histograms feature descriptor is arbitrarily and inefficiently calculated by subjectively adjusting the neighborhood radius, and the whole process can’t be completed automatically. An adaptive neighborhood-selection Fast Point Feature Histograms point cloud feature extraction algorithm has proposed to solve this problem. Firstly, we estimate the point cloud density of many pairs of point clouds. Secondly, compute the neighborhood radius to extract the Fast Point Feature Histograms features for Sample Consensus Initial Alignment registration, and count the radius and the density when the registration performance is optimal, and then the cubic spline interpolation fitting is used to obtain the function expression of the radius and the density. Finally, the Fast Point Feature Histograms feature extraction algorithm has combined with the function to form adaptive neighborhood-selection Fast Point Feature Histograms feature extraction algorithm. The experimental results have shown that the proposed algorithm can adaptively choose the appropriate neighborhood radius according to the density of point cloud, and improve the Fast Point Feature Histograms feature matching performance. At the same time, it is improved the computing speed to a better value, which is of guiding significance.

In a multitarget tracking (MTT) scenario, the prior knowledge of parameters such as clutter rate and detection probability are usually uncertain or estimated offline from training data. However, significant parameters mismatch in the clutter and detection model will result in biased estimates. In such cases, the ability to adaptively online estimate clutter rate and detection probability is critical in MTT under the random finite sets (RFS) framework. In this article, we propose a robust Poisson multi-Bernoulli mixture (PMBM) filter that can accommodate model mismatch in clutter rate and detection probability for multiple extended target tracking (ETT) problems. Moreover, the closed-form solution to the proposed method is derived by the use of Beta and Gamma Gaussian inverse-Wishart (GGIW) distribution, where Beta is used to describe unknown detection probability and GGIW is used to model extended target extension as an ellipse. Simulation results are presented to verify the effectiveness of the proposed method.

Photogrammetric Point-Cloud Replicability When Documenting Forensic Archaeological Scenes under Variable Lighting Conditions

Point cloud up-sampling network with multi-level spatial local feature aggregation

Sensor miniaturisation, improved battery technology and the availability of low-cost yet advanced Unmanned Aerial Vehicles (UAV) have provided new opportunities for environmental remote sensing. The UAV provides a platform for close-range aerial photography. Detailed imagery captured from micro-UAV can produce dense point clouds using multi-view stereopsis (MVS) techniques combining photogrammetry and computer vision. This study applies MVS techniques to imagery acquired from a multi-rotor micro-UAV of a natural coastal site in southeastern Tasmania, Australia. A very dense point cloud ( < 1–3 cm point spacing) is produced in an arbitrary coordinate system using full resolution imagery, whereas other studies usually downsample the original imagery. The point cloud is sparse in areas of complex vegetation and where surfaces have a homogeneous texture. Ground control points collected with Differential Global Positioning System (DGPS) are identified and used for georeferencing via a Helmert transformation. This study compared georeferenced point clouds to a Total Station survey in order to assess and quantify their geometric accuracy. The results indicate that a georeferenced point cloud accurate to 25–40 mm can be obtained from imagery acquired from ~50 m. UAV-based image capture provides the spatial and temporal resolution required to map and monitor natural landscapes. This paper assesses the accuracy of the generated point clouds based on field survey points. Based on our key findings we conclude that sub-decimetre terrain change (in this case coastal erosion) can be monitored.