Optical Flow Based Pose Estimation

This paper presents a method for pose estimation of a rigid body using unit dual quaternions where pose measurements from point clouds are filtered with a multiplicative extended Kalman filter (MEKF). The point clouds come from a 3D camera fixed to the moving rigid body, and then consecutive point clouds are aligned with the Iterative Closest Point (ICP) algorithm to obtain pose measurements. The unit constraint of the dual quaternion is ensured in the filtering process with the Dual Quaternion MEKF (DQ-MEKF), where the measurement updates are performed using the dual quaternion product so that the result is a unit dual quaternion. In addition, we use the Cayley transform for the discrete time propagation of the DQ-MEKF estimate, eliminating the need for normalization and projection of the resulting unit dual quaternion. The ICP algorithm is initialized with the time propagated state of the filter to give faster and more accurate pose measurements. To further improve the accuracy of the initialization, angular velocity measurements from a gyroscope fixed to the camera are included in the filter. The proposed method has been tested in experiments using a Kinect v2 3D camera mounted rigidly on a KUKA KR6 robotic arm, while a customized ICP algorithm was successfully implemented on a Graphical Processing Unit (GPU) system.

The iterative closest point (ICP) algorithm is widely used in three-dimensional (3D) point cloud registration, and it is very stable and robust. However, its biggest drawback is being easily trapped in a local optimal solution, which results in the incorrect registration result. Currently, there is neither a clear effective range to define whether the ICP algorithm will fall into a local optimum nor a study providing a comprehensive evaluation of the ICP algorithm. In this paper, we take the overlap ratio, angle, distance, and noise as the influencing factors of ICP and evaluate the validity, robustness, accuracy, and efficiency of point-to-point and point-to-plane ICP by using four datasets. We first analyze the effective ranges of the two ICP algorithms with respect to overlap ratio, angle, and distance and then propose a universal effective range for the three factors. Next, the effect of Gaussian noise on the validity and accuracy of two ICPs is evaluated. We also analyze the factors influencing ICP accuracy and explain their changing rules. We finally study the effect of different parameters on the efficiency. All results are compared by using point-to-point and point-to-plane ICP algorithms. The results show that the overlap ratio has no effect on validity, but it has a significant influence on accuracy. The angle has a great impact on the validity and efficiency of ICP but has no effect on accuracy. The distance only affects validity, has a limited effect on efficiency and no effect on accuracy. Meanwhile, Gaussian noise has a little effect on the validity. In addition, the general effective range of point-to-point ICP is larger than that of point-to-plane ICP, but the point-to-plane ICP algorithm presents a better efficiency. The point-to-point ICP is more robust to Gaussian noise with respect to validity, while the point-to-plane is more resistant in terms of accuracy.

As 3D point cloud acquisition sensors become increasingly prevalent in urban environments (e.g., LiDAR sensors for autonomous vehicles), the need arises to find efficient ways to align large amounts of such 3D data, often in real-time. In this work, we propose a novel method for 3D point cloud registration refinement in an urban environment (e.g., between Terrestrial LiDAR Scans - TLS - and Airborne LiDAR Scans - ALS), assuming an initial coarse registration is available. The proposed method is based on estimation of the direction of gravity, wall detection, projection of the point clouds on a perpendicular horizontal plane, and conversion into 2D edge-maps. Then, two methods are considered for alignment between the 2D edge-maps: a 2D variant of the well-known ICP (Iterative Closest Point) algorithm, and Edge-Map Phase-Correlation (EMPC). We demonstrate the usefulness of the proposed methods for registration in this challenging task, where the 2D variant of ICP achieves a meaningful advantage over 3D ICP in terms of runtime, while maintaining comparable registration accuracy.

Localization and mapping are key requirements for autonomous mobile systems to perform navigation and interaction tasks. Iterative Closest Point (ICP) is widely applied for LiDAR scan-matching in the robotic community. In addition, the standard ICP algorithm only considers geometric information when iteratively searching for the nearest point. However, ICP individually cannot achieve accurate point-cloud registration performance in challenging environments such as dynamic environments and highways. Moreover, the computation of searching for the closest points is an expensive step in the ICP algorithm, which is limited to meet real-time requirements, especially when dealing with large-scale point-cloud data. In this paper, we propose a segment-based scan-matching framework for six degree-of-freedom pose estimation and mapping. The LiDAR generates a large number of ground points when scanning, but many of these points are useless and increase the burden of subsequent processing. To address this problem, we first apply an image-based ground-point extraction method to filter out noise and ground points. The point cloud after removing the ground points is then segmented into disjoint sets. After this step, a standard point-to-point ICP is applied into to calculate the six degree-of-freedom transformation between consecutive scans. Furthermore, once closed loops are detected in the environment, a 6D graph-optimization algorithm for global relaxation (6D simultaneous localization and mapping (SLAM)) is employed. Experiments based on publicly available KITTI datasets show that our method requires less runtime while at the same time achieves higher pose estimation accuracy compared with the standard ICP method and its variants.

In this paper we present a new variant of ICP (iterative closest point) algorithm based on local feature correlation. Our approach combines global and local feature information to find better correspondence sets and to use them to compute the 3D pose of the object model even for the case of large displacements between model and image data. For such cases, we propose a 2D alignment in the image plane (rotation plus translation) before the feature extraction process. This has some advantages over the classical methods like better convergence and robustness. Furthermore, it avoids the need of a normal pre-alignment step in 3D. Our approach was tested on synthetical and real-world data to compare the convergence behavior and performance against other versions of the ICP algorithm combined with a classical pre-alignment approach.

Bin picking is an important task in smart manufacturing and intelligent robotics. For a robot to pick or grip an object with a human-like gripping action, it needs to know the accurate 3D pose of the object. In this paper, we propose a method for estimating the 3D pose of a small ring-shaped object using infrared and depth images generated by a depth camera. The proposed method consists of two algorithm modules, the first to recognize an object in a 2D infrared image and the second to estimate the 3D pose by applying the ICP (iterative closest point) algorithm to 3D depth data. In the first module, we propose a method to generate a three-channel integrated image with features from the depth and infrared images. Next, we introduce a method for training an object detector based on deep-learning. Because the bin-picking test object in this paper is small and ring-shaped, it is difficult to detect and find 3D poses of individual objects when many such objects are piled up. We solved this problem with a depth-based filtering method. Using the filtered image, each object region is separated by the deep learning approach. In the second module, the ICP algorithm is employed to estimate the 3D pose of the ring object. We match a 3D reference model of the object and the real object using the point-to-point ICP algorithm. Performance of the proposed method is evaluated by using two different types of depth camera in the experiments.

Point set augmentation through fitting for enhanced ICP registration of point clouds in multisensor coordinate metrology

The 3D reconstruction and modeling of face, objects and surrounding environment etc., has gained significant attention from the research community in recent days. Availability of depth data from various active and passive sensors have accelerated the work in 3D domain. It finds various applications in robotics, Human Computer Interaction (HCI), medical analysis etc. The major challenge in this area is accurate and real time 3D pose estimation which forms the basis for alignment of multi-view point cloud data. Noise in the depth data due to sensor irregularities significantly affects the accuracy of pose estimation. The main idea of this paper is to compute 2D A-KAZE feature correspondence and map to 3D for obtaining more reliable 3D sparse points. The 2D-3D correspondence is used to estimate accurate pose by formulating a graph optimization problem. The experimental results of this approach is evaluated against well known Iterative Closest Point (ICP) and Singular Value Decomposition (SVD) methods for point cloud registration on face and object data obtained using stereo camera. The results shows that the algorithm is able to estimate the accurate pose with least alignment error as compared with ICP and SVD methods.

Fine registration of 3D point clouds fusing structural and photometric information using an RGB-D camera

This paper deals with the problem of 3D alignment of faces in the presence of some facial expression changes. The data are three dimensional and obtained using a laser scanner. Our approach is based on the differential geometry of the surface and computes the intrinsic local fiducial points on the surface and on curves that reside on the surface. Because these fiducial points are local, they allow partial alignment, where part of the face is viewed. Moreover, since the fiducial points are relatively affine‐invariant to local affine transformations, they allow matching and alignment when facial expression changes affect part of the face. A fast, noniterative alignment procedure is presented in this paper that establishes reliable correspondences between fiducial points without any prior knowledge of the overall nonlinear global transformation that takes place after the changes in facial expressions. This is achieved through the construction of a set of ordered novel absolute local affine invariants. With enough fiducial points set as correspondents, the overall nonlinear transformation is computed and the face before and after the transformation are aligned. For comparison, we also compare the alignment performance of our method with that of the iterative closest point (ICP) method and the coherent point drift (CPD) method which is based on the Gaussian mixture model by looking at the between‐to‐within variation (signal‐to‐noise ratio, SNR) between these two classes (match vs nonmatch) based on the average alignment errors for the genuine matches and nonmatches, normalized by their respective within variation, and by running it on the 3D face database GavabDB. The separation between the true match and nonmatch is best for our zero‐torsion method (SNR of 1.2), followed by the ICP method (SNR of 0.37)and the CPD method (SNR of 0.15). It is interesting to observe here that, although the alignment error for the true match cases is lowest for the CPD method, it is also extremely low for the nonmatch cases, which would mean that the method would force a query into a wrong face, yielding poor specificity (lots of false alarms). © 2013 Institute of Electrical Engineers of Japan. Published by John Wiley & Sons, Inc.

The accuracy of the iterative closest point (ICP) algorithm, which is widely employed in image registration, depends on the complexity of the shape of the object under registration. Objects with complex features yield higher reliability in estimating registration parameters. For objects with rotation symmetry, a cylinder for example, rotation along the center axis can not be distinguished. We derive the sensitivity of the rotation error of the ICP algorithm from the curvature of the error function near the minimum error position. We approximate the defined error function to a second order polynomial and show that the coefficient of the second-order term is related to the reliability of the estimated rotation angle. Also the coefficient is related to the shape of the object. In the known correspondence case, the reliability can be expressed by the second moment of the input image. Finally, we apply the sensitivity formula to a simple synthetic object and ellipses, and verify that the predicted orientation variance of the ICP algorithm is in good agreement with computer simulations.

Differential LiDAR (Light Detection and Ranging) from repeated surveys has recently emerged as an effective tool to measure three-dimensional (3D) change for applications, such as quantifying slip and spatially distributed warping associated with earthquake ruptures, and examining the spatial distribution of beach erosion after hurricane impact. Currently, the primary method for determining 3D change from LiDAR is through the use of the iterative closest point (ICP) algorithm and its variants. However, all current studies using ICP have assumed that all LiDAR points in the compared point clouds have uniform accuracy. This assumption is simplistic given that the error for each LiDAR point is variable, and dependent upon time varying factors such as target range, angle of incidence, and aircraft trajectory accuracy. Therefore, to rigorously determine spatial change, it would be ideal to model the random error for every LiDAR observation in the differential point cloud, and use these error estimates as apriori weights in the ICP algorithm. To test this approach, we implemented a rigorous LiDAR observation error propagation method to generate estimated random error for each point in a LiDAR point cloud, and then determine 3D displacements between two point clouds using an anisotropic weighted ICP (A-ICP) algorithm. The algorithm was evaluated by qualitatively and quantitatively comparing point clouds with synthetic fault ruptures between a uniform weight and anistropically weighted ICP algorithm. Then post-earthquake slip is estimated for the 2010 El Mayor-Cucapah Earthquake (EMC), using pre- and post-event LiDAR. Based on the analysis, Moving Window A-ICP is able to better estimate the synthetic surface ruptures, and provides a smoother estimate of actual displacement for the EMC earthquake.

The problem of aligning of 3D point data is the known registration task. The most popular registration algorithm is the Iterative Closest Point (ICP) algorithm. The traditional ICP algorithm is a fast and accurate approach for rigid registration between two point clouds but it is unable to handle affine case. Recently, extension of the ICP algorithm for composition of scaling, rotation, and translation is proposed. A generalized ICP version for an arbitrary affine transformation is also suggested. In this paper, a new iterative algorithm for registration of point clouds based on the point-to-plane ICP algorithm with affine transformations is proposed. At each iteration, a closed-form solution to the affine transformation is derived. This approach allows us to get a precise solution for transformations such as rotation, translation, and scaling. With the help of computer simulation, the proposed algorithm is compared with common registration algorithms.

The traditional affine iterative closest point (ICP) algorithm is fast and accurate for affine registration between two point sets, but it is easy to fall into local minimum. This paper proposes a robust Affine ICP algorithm based on corner points. First, an objective function is established under the guidance of corner points, where the corner points as the shape control point guides the affine registration of 2D point sets. Then, at each step of the algorithm, the affine transformation obtained by the last iterative step is used to establish the correspondence of these two point sets. Next, the new affine transformation is solved by using the objective function under the guidance of the corner points. Experimental results demonstrate that the robustness and convergence of our algorithm are greatly improved compared with the traditional affine registration algorithm.

A bounded iterative closest point method for minimally invasive registration of the femur