MVPCC-Net: Multi-View Based Point Cloud Completion Network for MLS data

In recent years, due to the significant advancements in hardware sensors and software technologies, 3D environmental point cloud modeling has gradually been applied in the automation industry, autonomous vehicles, and construction engineering. With the high-precision measurements of 3D LiDAR, its point clouds can clearly reflect the geometric structure and features of the environment, thus enabling the creation of high-density 3D environmental point cloud models. However, due to the enormous quantity of high-density 3D point clouds, storing and processing these 3D data requires a considerable amount of memory and computing time. In light of this, this paper proposes a real-time 3D point cloud environmental contour modeling technique. The study uses the point cloud distribution from the 3D LiDAR body frame point cloud to establish structured edge features, thereby creating a 3D environmental contour point cloud map. Additionally, unstable objects such as vehicles will appear during the mapping process; these specific objects will be regarded as not part of the stable environmental model in this study. To address this issue, the study will further remove these objects from the 3D point cloud through image recognition and LiDAR heterogeneous matching, resulting in a higher quality 3D environmental contour point cloud map. This 3D environmental contour point cloud not only retains the recognizability of the environmental structure but also solves the problems of massive data storage and processing. Moreover, the method proposed in this study can achieve real-time realization without requiring the 3D point cloud to be organized in a structured order, making it applicable to unorganized 3D point cloud LiDAR sensors. Finally, the feasibility of the proposed method in practical applications is also verified through actual experimental data.

Estimation of plant parameters is prerequisite for fast and precise management of tomato crops on commercial farm level. Destructive estimation methods may be accurate but not suitable when large farm size, spatio-temporal variability or short period of time to obtain information are considered. Therefore, this study was aimed to estimate leaf area (LA) in mature tomato plants from reconstructed geometric plant model based on 3-dimensional (3D) point cloud. A 2D laser scanner was placed on a programable linear conveyor system to move at controlled forward speed. To achieve a high density 3D point cloud of whole plant, close range (< 2 m) scanning was performed at 10 mm/s forward speed. Plant parameters capturing points per plant (PPP), plant height, plant volume, and LA were extracted from the 3D point clouds. Destructive method was utilized in laboratory for reference LA data collection, whereas other required reference data (plant height and canopy area) were measured in-situ. Linear regression models were developed and cross validated to estimate LA using variables from the 3D point clouds and reference LA. Among the models, plant height showed strong correlation with reference LA (R <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> = 0.90, RMSE = 40 cm <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> ) and followed by PPP (R <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> = 0.89, RMSE = 45 cm <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> ), voxel grid approach (R <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> = 0.56, RMSE = 58 cm <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> ) and slicing approach (R <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> = 0.44, RMSE = 70 cm <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> ). Results indicate that mobile laser scanning could be a reliable non-destructive LA estimation tool for tomato and may be adapted by the commercial tomato growers for profitable management decisions.

Abstract. The aim of the research is to evaluate the performance of the point cloud registration methods using mobile laser scanning data. The point cloud registration methods involved in this research are match bounding-box centres and iterative closest point (ICP). The research began with the two epoch’s mobile laser scanning survey using a Phoenix AL-3-32 system. At the same time, the stereo images of the study area were acquired using UAV Photogrammetric method. Both two epoch point cloud datasets were gone through the pre and post-processing stages to produce the cleaned and geo-referenced point clouds data. The data were then gone through the two registration methods and four Cloud-to-Cloud (C2C) distance methods. The 3D surface deviation results obtained from mobile laser scanning data was compared with the 3D surface deviation results from UAV data that undergoes the same registration and C2C distance computation methods. The study area involved in the research is an active landslide area that was located at Kulim Hi-Tech residential area in Kedah state, Malaysia. The study area exposed to the movement of the land which caused cracked to the buildings and drainages. The findings show that the ICP registration becomes the most suitable method to register point clouds dataset that was acquired using mobile laser scanning system. Among the four C2C distance computation methods that was involved in the testing, the least square plane method was the best method to calculate the distance between two sets of point clouds datasets which in turn gave the best results in the process of detecting the movement of the land in the study area.

Semantic segmentation in the context of 3D point clouds for the railway environment holds a significant economic value, but its development is severely hindered by the lack of suitable and specific datasets. Additionally, the models trained on existing urban road point cloud datasets demonstrate poor generalisation on railway data due to a large domain gap caused by non‐overlapping special/rare categories, for example, rail track, track bed etc. To harness the potential of supervised learning methods in the domain of 3D railway semantic segmentation, we introduce RailPC, a new point cloud benchmark. RailPC provides a large‐scale dataset with rich annotations for semantic segmentation in the railway environment. Notably, RailPC contains twice the number of annotated points compared to the largest available mobile laser scanning (MLS) point cloud dataset and is the first railway‐specific 3D dataset for semantic segmentation. It covers a total of nearly 25 km railway in two different scenes (urban and mountain), with 3 billion points that are finely labelled as 16 most typical classes with respect to railway, and the data acquisition process is completed in China by MLS systems. Through extensive experimentation, we evaluate the performance of advanced scene understanding methods on the annotated dataset and present a synthetic analysis of semantic segmentation results. Based on our findings, we establish some critical challenges towards railway‐scale point cloud semantic segmentation. The dataset is available at https://github.com/NNU‐GISA/GISA‐RailPC, and we will continuously update it based on community feedback.

3D vehicle shape completion is a key task in urban environment reconstruction, since the available sensors and 3D scanning procedures (such as Mobile Laser Scanning) in an outdoor city scene cannot usually extract the entire car shapes, which can be necessary for specifying their geometric properties for further analysis or realistic visualization. In this paper, we propose a novel multi-view based 3D object point cloud completion technique. In contrast to existing approaches, our method operates on 2D images formed by projecting the point cloud from several virtual camera positions around the object of interest. Both color and geometrical information is considered during the process, generating dense textured point clouds, displaying realistic patterns in the missing regions from the partial inputs. We present both quantitative and qualitative tests on various synthetic and real laser scanned vehicle point clouds, which demonstrate that our method surpasses existing state-of-the-art approaches. By applying it to vehicles from the Shapenet dataset, our approach outperforms recent techniques in terms of Earth Mover’s Distance (EMD) and Chamfer Distance (CD) by 43.8% and 12.17%, respectively.

The use of laser scanning techniques has become a common way to measure the real world. Millions of points could be generated by this system, which brings forth the problem of visualizing and eventually performing analyses on them. In the open source domain, various software packages exist to visualize 3D point clouds. However, most of these software packages do not allow the real-time rendering of large point clouds. Few of them also allow visualization in an immersive manner. The research aims to create an open source viewer for large 3D point clouds, which enables a dynamic and immersive visualization. In order to do so, rendering and point cloud management strategies must be implemented to avoid overloading the computer’s memory. The rendering is done using OpenGL engine by utilizing the graphic card’s memory in order to perform faster visualization. The program consists of a pre-processing stage in which the point cloud files are divided into quadtrees and then subsampled using the random tree sampling method. The visualization itself will calculate the distance between the point of view and the center of nodes generated by the pre-processing stage. The amount of points rendered within each node will depend on this distance; the farther away the node is from the point of view, the fewer points are rendered. A loading and unloading function enables the point cloud to be rendered dynamically. With the point cloud management algorithm implemented, the resulting program is able to load large point clouds generated by a mobile laser scanner using an ordinary computer. The resulting program as well as the source code will be available for the public due to its open source nature.

Computing multiple aggregation levels and contextual features for road facilities recognition using mobile laser scanning data

Rock bolts are crucial components in the subterranean support systems in underground mines that provide adequate structural reinforcement to the rock mass to prevent unforeseen hazards like rockfalls. This makes frequent assessments of such bolts critical for maintaining rock mass stability and minimising risks in underground mining operations. Where manual surveying of rock bolts is challenging due to the low-light conditions in the underground mines and the time-intensive nature of the process, automated detection of rock bolts serves as a plausible solution. To that end, this study focuses on the automatic identification of rock bolts within medium- to large-scale 3D point clouds obtained from underground mines using mobile laser scanners. Existing techniques for automated rock bolt identification primarily rely on feature engineering and traditional machine learning approaches. However, such techniques lack robustness as these point clouds present several challenges due to data noise, varying environments, and complex surrounding structures. Moreover, the target rock bolts are extremely small objects within large-scale point clouds and are often partially obscured due to the application of reinforcement shotcrete. Addressing these challenges, this paper proposes an approach termed DeepBolt, which employs a novel two-stage deep learning architecture specifically designed for handling severe class imbalance for the automatic and efficient identification of rock bolts in complex 3D point clouds. The proposed method surpasses state-of-the-art semantic segmentation models by up to 42.5% in Intersection over Union (IoU) for rock bolt points. Additionally, it outperforms existing rock bolt identification techniques, achieving a 96.41% precision and 96.96% recall in classifying rock bolts, demonstrating its robustness and effectiveness in complex underground environments.

Comparing Mobile and Aerial Laser Scanner point cloud data sets for automating the detection and delimitation procedure of safety-critical near-road slopes

In the paper the authors decided to present the capabilities of the application of the technology of mobile and airborne laser scanning both in surveying and modeling of railway infrastructure. First of all, they presented the possibility of using the laser scanning technology and in particular mobile laser scanning in terms of data collection and analysis for the tasks associated with the elaboration and diagnostics of railway infrastructure. For the mobile laser scanning the object of the study was the railway station in Slomniki, on a section of the railway line No. 8, and length of about 550 meters. The cloud of points together with photographic documentation was obtained using Riegl VMX - 450 system. Data from the airborne laser scanning included two different fragments of the railway line. The first one is located in Strzalkowo in the Wielkopolskie Voivodship (railway tracks along with the station building and the second one in Bochnia in the Malopolskie Voivodship. The section in Strzalkowo was about 470m and in Bochnia 302m. For example, the density of point cloud in Strzalkowo was 10 points/m2, while in Bochnia 17 points/m2.

Aerial data collection is well known as an efficient method to study the impact following extreme events. While datasets predominately include images for post-disaster remote sensing analyses, images alone cannot provide detailed geometric information due to a lack of depth or the complexity required to extract geometric details. However, geometric and color information can easily be mined from three-dimensional (3D) point clouds. Scene classification is commonly studied within the field of machine learning, where a workflow follows a pipeline operation to compute a series of engineered features for each point and then points are classified based on these features using a learning algorithm. However, these workflows cannot be directly applied to an aerial 3D point cloud due to a large number of points, density variation, and object appearance. In this study, the point cloud datasets are transferred into a volumetric grid model to be used in the training and testing of 3D fully convolutional network models. The goal of these models is to semantically segment two areas that sustained damage after Hurricane Harvey, which occurred in 2017, into six classes, including damaged structures, undamaged structures, debris, roadways, terrain, and vehicles. These classes are selected to understand the distribution and intensity of the damage. The point clouds consist of two distinct areas assembled using aerial Structure-from-Motion from a camera mounted on an unmanned aerial system. The two datasets contain approximately 5000 and 8000 unique instances, and the developed methods are assessed quantitatively using precision, accuracy, recall, and intersection over union metrics.

Converting three-dimensional (3D) point cloud data to two-dimensional (2D) image based on virtual structured-light system is a popular method in 3D data compression because the image format is easier to process by the exiting storing and transmitting methods. When this method is used in the 3D point cloud compression, the quantization error is introduced during the coordinate mapping. To solve this problem, a virtual structured-light 3D point cloud compression algorithm based on geometric reshaping is presented. And a least-square system parameter optimization method is proposed to further improve the data decoding accuracy. In the proposed method, the 3D spatial coordinates are reshaped to a 2D matrix first, and then the 2D matrix is stored as a “Holoimage” by using the parameter optimized virtual structured-light system. The quantization error introduced by coordinate mapping between X and Y coordinates of the 3D point cloud and the 2D image pixel coordinates is suppressed, so the decoding accuracy is improved. In addition, the geometric information of the 3D point cloud is hidden synchronously when the 3D point cloud is compressed, which is of great significance in the copyright protection of the point cloud data. Experiments verify the effectiveness of the proposed algorithm. The decoding root mean square error (RMSE) of the proposed method is decreased by 79.86% on average compared with the traditional one under the same compression ratio, and the compression ratio of the proposed method is 1.96 times bigger than the traditional one under the similar decoding accuracy.

Along with increasingly popular virtual reality applications, the three-dimensional (3D) point cloud has become a fundamental data structure to characterize 3D objects and surroundings. To process 3D point clouds efficiently, a suitable model for the underlying structure and outlier noises is always critical. In this work, we propose a hypergraph-based new point cloud model that is amenable to efficient analysis and processing. We introduce tensor-based methods to estimate hypergraph spectrum components and frequency coefficients of point clouds in both ideal and noisy settings. We establish an analytical connection between hypergraph frequencies and structural features. We further evaluate the efficacy of hypergraph spectrum estimation in two common applications of sampling and denoising of point clouds for which we provide specific hypergraph filter design and spectral properties. Experimental results demonstrate the strength of hypergraph signal processing as a tool in characterizing the underlying properties of 3D point clouds.

ABSTRACTGeneration of indoor networks for navigation will normally be done out of standard floor plans that are only 2D and is more often manual than automatic. These floor plans are drawn at a specific time and do not correspond to the reality, moreover some of those buildings were built already differently than designed. Then in due course the building will change both externally and internally. Also objects like furniture will be moved around in the building. If these changes are not updated in the map of the building, it becomes out of date and cannot be used for the creation of indoor navigable models anymore. To enable correct indoor navigation, we will need to have the current data of the indoor environment. This article concentrates on providing a new approach to generate up to date floor plans by using a mobile (and hand held) laser scanner in the fastest way. This device creates a point cloud and the corresponding trajectory at the same time. Because the mobile laser scanner device is operated by a walking human, the trajectory contains information about the surface the person is walking on. In this article, a method is explained for the detection of walkable spaces based on the analysis of the point cloud and its corresponding trajectory provided by the mobile laser scanner. Three steps will be used: voxelization, trajectory analysis and the identification of floor regions. Dynamic objects, doors, and furniture objects are also used to identify the surfaces which are available for navigation purposes. Three types of surfaces are considered: horizontal, slopes, and stairs.

The inclusion of thermal infrared (TIR) data in point clouds derived from unmanned aircraft system (UAS) imagery can benefit a variety of applications in which surface temperature and 3D geometry are both important discriminants of feature type and condition. Low resolution and narrow fields of view (FOV) of current consumer-grade TIR cameras on UAS, combined with the lack of sharpness and texture in many image regions, may cause failure or poor results from structure from motion (SfM) photogrammetric software, which has gained widespread use for generating point clouds from UAS imagery. This paper proposes a photogrammetric approach for generating 3D multispectral point clouds utilizing coacquired TIR-RGB images. A 3D point cloud is first generated from the RGB imagery using standard SfM procedures. Then the TIR attributes are assigned to points, where the image coordinates of the points in TIR images are estimated using transformation parameters obtained from co-registration procedures. To obtain RGB-to-TIR transformation parameters, this study tests 3D and 2D co-registration approaches. The latter produces better results due to the challenge of calibrating the TIR camera as required for the 3D approach. This proposed approach is advantageous for generating TIR point clouds without loss of photogrammetric precision compared with solely TIR-based SfM, as the 3D accuracy, point density, and reliability are greatly enhanced.