Indoor Scene Point Cloud Registration Algorithm Based on RGB-D Camera Calibration

Abstract. In order to reconstruct the three-dimensional model of the soybean rapidly and efficiently, this paper analyzes the registration effect of point clouds of soybean plants from multiple angles. First, the Kinect V2 camera is used to obtain the point cloud information of soybean plants at 24 different angles, which means that, the images are taken at intervals of 15 degree angles. Second, the obtained point cloud information is subjected to preprocessing operations such as straight through filtering and outlier filtering. Thirdly, two representative point cloud registration algorithms are investigated, which include local features based FPFH (Fast point feature histograms) and probability distribution based NDT (Normal distributions transform). Meanwhile, the FPFH-ICP and NDT-ICP are used to perform pairwise registration for point cloud data at different perspectives. Finally, time cost and accuracy of the above two registration strategies are compared and analyzed. The experimental results show that the NDT-ICP soybean plant point cloud registration algorithm is superior to that of FPFH-ICP in terms of time-consuming, registration accuracy, and adaptation at different angles. At the same time, it is found that when the image acquisition interval between two point clouds is less than or equal to 45 degrees, the adaptability to different point cloud registration algorithms is more robust, and the registration error is less.

Indoor scene understanding is an active research topic that has recently gained increasing attention in both computer graphics and computer vision community. This paper presents a novel automatic point cloud instance abstraction and segmentation of indoor scenes using learned pairwise planar patch relations. Planar patches have sufficient representative power to abstract the semantics in mostly man-made 3D indoor scenes. To exploit the planar patch relationships, a new indoor point clouds dataset is automatically derived from the public dataset ScanNet, which can be adopted for training a pairwise patch relation classifier. Given the pre-processed indoor scene point clouds, a set of planar patches are generated to provide robust and compact representation of indoor objects and parts. For each extracted planar patch, a feature descriptor in view of its geometric properties and appearance is calculated. Owing to the proposed feature descriptors, the relations of pairwise planar patch are measured and can be fed into the pre-trained patch relation classifier. To yield the instance segmentation of indoor scenes, a robust graph-based algorithm is adopted to group these planar patches which incorporate the learned patch relations. Experimental results demonstrate that our proposed approach can produce instance semantic segmentation of various cluttered indoor scenes robustly and efficiently.

Understanding of indoor scenes based on projection of spatial rectangles

Measuring characteristics of animals in the wild is not always possible, due to their demeanour and lack of human contact. Remote capture and processing methods, including the segmentation of animal data into relevant body parts, are required. Existing solutions are either costly or too cumbersome to use in the wild. This study explores the use of RGB depth (RGB-D) cameras for data capture of a target animal from a distance. In addition, this study explores the extraction and segmentation of the resulting animal data into point clouds, and the creation of machine learning models for the automated segmentation of this data. Results of this study, including an experimental evaluation, demonstrate the feasibility of utilizing RGB-D cameras for animal data capture, and that classification outperformed clustering for automated animal data segmentation.

From object tracking to 3D reconstruction, RGB-Depth (RGB-D) camera networks play an increasingly important role in many vision and graphics applications. Practical applications often use sparsely-placed cameras to maximize visibility, while using as few cameras as possible to minimize cost. In general, it is challenging to calibrate sparse camera networks due to the lack of shared scene features across different camera views. In this paper, we propose a novel algorithm that can accurately and rapidly calibrate the geometric relationships across an arbitrary number of RGB-D cameras on a network. Our work has a number of novel features. First, to cope with the wide separation between different cameras, we establish view correspondences by using a spherical calibration object. We show that this approach outperforms other techniques based on planar calibration objects. Second, instead of modeling camera extrinsic calibration using rigid transformation, which is optimal only for pinhole cameras, we systematically test different view transformation functions including rigid transformation, polynomial transformation and manifold regression to determine the most robust mapping that generalizes well to unseen data. Third, we reformulate the celebrated bundle adjustment procedure to minimize the global 3D reprojection error so as to fine-tune the initial estimates. Finally, our scalable client-server architecture is computationally efficient: the calibration of a five-camera system, including data capture, can be done in minutes using only commodity PCs. Our proposed framework is compared with other state-of-the-arts systems using both quantitative measurements and visual alignment results of the merged point clouds.

Recent advancements in 3D scanning technology have spurred extensive research on indoor scene point clouds. However, point clouds captured by 3D sensors are often sparse, noisy, and irregular, posing significant challenges for downstream tasks. While learning-based upsampling methods have achieved promising results, they often struggle with large-scale indoor scenes due to the high computational cost of K-Nearest Neighbor (KNN) search and insufficient reconstruction of fine-grained structures. To address these issues, we propose S 2 PU-Net, a sparse semantic-guided progressive point cloud upsampling network for indoor scenes. Specifically, we introduce a progressive upsampling mechanism based on sparse tensors, where a progressive Point Generation Block is designed to generate and prune intermediate point clouds iteratively. By employing multiple generative deconvolutions and adaptive pruning with different coefficients, our method progressively reconstructs the target point cloud while preserving fine-grained local structures. Furthermore, we integrate semantic information into each layer of the hourglass encoder through a Sparse Semantic Embedding (SSE) module, enhancing point feature extraction. To further strengthen feature representation, we propose a Multi-Scale Sparse Semantic Embedding (MSSE) module, which refines the fused features across spatial and channel dimensions. Extensive experiments on multiple indoor scene datasets demonstrate that our method consistently outperforms state-of-the-art approaches across most evaluation metrics.

We present a generative neural network that enables us to generate plausible 3D indoor scenes in large quantities and varieties, easily and highly efficiently. Our key observation is that indoor scene structures are inherently hierarchical . Hence, our network is not convolutional; it is a recursive neural network, or RvNN. Using a dataset of annotated scene hierarchies, we train a variational recursive autoencoder , or RvNN-VAE, which performs scene object grouping during its encoding phase and scene generation during decoding. Specifically, a set of encoders are recursively applied to group 3D objects based on support, surround, and co-occurrence relations in a scene, encoding information about objects’ spatial properties, semantics , and relative positioning with respect to other objects in the hierarchy. By training a variational autoencoder (VAE), the resulting fixed-length codes roughly follow a Gaussian distribution. A novel 3D scene can be generated hierarchically by the decoder from a randomly sampled code from the learned distribution. We coin our method GRAINS, for Generative Recursive Autoencoders for INdoor Scenes. We demonstrate the capability of GRAINS to generate plausible and diverse 3D indoor scenes and compare with existing methods for 3D scene synthesis. We show applications of GRAINS including 3D scene modeling from 2D layouts, scene editing, and semantic scene segmentation via PointNet whose performance is boosted by the large quantity and variety of 3D scenes generated by our method.

Navigation system for orchard spraying robot based on 3D LiDAR SLAM with NDT_ICP point cloud registration

Although 3D Gaussian Splatting (3DGS) has greatly advanced the development of novel view synthesis (NVS) and surface reconstruction tasks, it still faces serious challenges in indoor scenes with only sparse views due to the lack of initialization point cloud. Existing GS-like methods are geometrically initialized using point clouds from Structure from Motion (SfM), and their performance is heavily limited by the point cloud quality. However, in indoor scenes containing large areas of textureless or weakly textured regions, the SfM algorithm struggles to generate point clouds in these areas, especially when only a limited number of input views are available. This situation can cause the GS optimization process to overfit the photometric error, resulting in severe geometric degradation in regions lacking proper initialization. To address the above problem, we propose a novel sparse-view 3DGS method using global scale-aligned monocular depth to initialize Gaussian primitives and optimize them in view space, named VGA-GS. First, we design a global scale-aligned monocular depth initialization strategy, which provides geometric priors in textureless or weakly textured regions while avoiding the local geometric distortions in vanilla alignments. Second, we propose a novel Gaussian primitive representation based on view space and a refinement strategy based on pixel errors, which can constrain each primitive to always be within the field of view where it is observed to be fully optimized. Finally, we extract the surface prior from monocular depth to regularize the rendering depth and normal, and further propagate the constraints of the training view to the neighboring pseudo-views via image warping. Our method achieves state-of-the-art reconstruction performance in both textureless regions and local details for indoor scenes with sparse views. Moreover, it is highly scalable, supporting integration with various 3DGS rasterizers and depth estimation methods. We release the code at https://github.com/XT5un/VGA-GS.

In response to the issues of slow convergence and the tendency to fall into local optima in traditional iterative closest point (ICP) point cloud registration algorithms, this study presents a fast registration algorithm for laser point clouds based on 3D scale-invariant feature transform (3D-SIFT) feature extraction. First, feature points are preliminarily extracted using a normal vector threshold; then, more high-quality feature points are extracted using the 3D-SIFT algorithm, effectively reducing the number of point cloud registrations. Based on the extracted feature points, a coarse registration of the point cloud is performed using the fast point feature histogram (FPFH) descriptor combined with the sample consensus initial alignment (SAC-IA) algorithm, followed by fine registration using the point-to-plane ICP algorithm with a symmetric target function. The experimental results show that this algorithm significantly improved the registration efficiency. Compared with the traditional SAC-IA+ICP algorithm, the registration accuracy of this algorithm increased by 29.55% in experiments on a public dataset, and the registration time was reduced by 81.01%. In experiments on actual collected data, the registration accuracy increased by 41.72%, and the registration time was reduced by 67.65%. The algorithm presented in this paper maintains a high registration accuracy while greatly reducing the registration speed.

3D shape modeling using a self-developed hand-held 3D laser scanner and an efficient HT-ICP point cloud registration algorithm

Abstract. Remote sensing techniques are an important tool in fluvial transport monitoring, since they allow for effective evaluation of the volume of transported material. Nevertheless, there is no methodology for automatic calculation of movement parameters of individual rocks. These parameters can be determined by point cloud registration. Hence, the goal of this study is to develop a robust algorithm for terrestrial laser scanning point cloud registration. The registration is based on Iterative Closest Point algorithm, which requires well established initial parameters of transformation. Thus, we propose to calculate the initial parameters based on key points representing the maximum of Gaussian curvature. For each key point the set of geometrical features is calculated. The key points are then matched between two point clouds as a nearest neighbor in feature domain. Different combinations of neighborhood sizes, feature subsets, metrics and number of nearest neighbors were tested to obtain the highest ratio between properly and improperly matched key points. Finally, RANSAC algorithm was used to calculate the initial transformation parameters between the point clouds and the ICP algorithm was used for calculation of final transformation parameters. The investigations carried out on sample point clouds representing rocks enabled the adjustment of parameters of the algorithm and showed that the Gaussian curvature can be used as a 3-dimentional key point detector for such objects. The proposed algorithm enabled to register point clouds with the mean distance between point clouds equal to 3 mm.

In nuclear measurement, radiation area reconstruction is a hot topic. To reconstruct a radiation environment, we proposed an information fusion method based on images from a gamma camera and an RGB-D (Red Green Blue-Depth) camera. First, on a mobile base, the reconstruction platform is presented to integrate gamma imaging with RGB-D information. Secondly, the dense point cloud map of the environment is formatted with VSLAM (Visual Simultaneous Localization and Mapping) based on an RGB-D camera. Thirdly, from the gamma camera images, the radioactive contour feature is extracted to localize the source right ahead cameras, and the radiation area point cloud is obtained using the relative position of the two cameras. Lastly, the whole radiation environment map is generated with point cloud fusion, and the source position is estimated with the minimum bounding box. To validate the efficiency of this method, a laboratory room with one <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">152</sup> Eu source is reconstructed under above fusion platform. In the experiments, three feature objects are selected, and their size are calculated based on the cloud point. Also, in two scenes with the same size in released literature, the sources are localized. Although there exist some errors, comparisons show that this method performs well and keeps at low deviation level.

This paper introduces a simultaneous estimation method of the extrinsic parameters of multiple fish-eye cameras using simple calibration markers. Precise extrinsic parameters of cameras mounted on a car are important, for example, to provide a seamless overhead view image to the driver. Calibration markers are set in the area that are observable from adjacent two cameras. Extrinsic parameters of each camera are estimated individually and then combined and refined using a geometric constraint. Cube markers are chosen as the calibration markers. The method is evaluated by simulation and experiments using a real car. It is shown that extrinsic parameters are obtained by the proposed method and suggested that the errors of intrinsic parameters affect the estimation of extrinsic parameters.

Simultaneous Localization and Mapping (SLAM) refers to the common requirement for autonomous platforms to estimate their pose and map their surroundings. There are many robust and real-time methods available for solving the SLAM problem. Most are divided into a front-end, which performs incremental pose estimation, and a back-end, which smooths and corrects the results. A low-drift front-end odometry solution is needed for robust and accurate back-end performance. Front-end methods employ various techniques, such as point cloud-to-point cloud (PC2PC) registration, key feature extraction and matching, and deep learning-based approaches. The front-end algorithms have become increasingly complex in the search for low-drift solutions and many now have large configuration parameter sets. It is desirable that the front-end algorithm should be inherently robust so that it does not need to be tuned by several, perhaps many, configuration parameters to achieve low drift in various environments. To address this issue, we propose Simple Mapping and Localization Estimation (SiMpLE), a front-end LiDAR-only odometry method that requires five low-sensitivity configurable parameters. SiMpLE is a scan-to-map point cloud registration algorithm that is straightforward to understand, configure, and implement. We evaluate SiMpLE using the KITTI, MulRan, UrbanNav, and a dataset created at the University of Queensland. SiMpLE performs among the top-ranked algorithms in the KITTI dataset and outperformed all prominent open-source approaches in the MulRan dataset whilst having the smallest configuration set. The UQ dataset also demonstrated accurate odometry with low-density point clouds using Velodyne VLP-16 and Livox Horizon LiDARs. SiMpLE is a front-end odometry solution that can be integrated with other sensing modalities and pose graph-based back-end methods for increased accuracy and long-term mapping. The lightweight and portable code for SiMpLE is available at: https://github.com/vb44/SiMpLE.