Robot Hand-Eye Calibration Method Based on Intermediate Measurement System

Augmented reality (AR) allows the surgeon to represent holographic patient-specific anatomical information and surgical instruments in the physical world. To correctly superimpose virtual and physical objects, a hand-eye (HE) calibration method for mapping the virtual and physical spaces was proposed. Mathematical relationships between the virtual camera and the physical space were derived. Finally, the accuracy and robustness of the proposed HE calibration method were qualitatively and quantitatively evaluated. The proposed calibration method allows us to determine an optimal invariant spatiotemporal mapping between the virtual camera and the physical space. Qualitatively and quantitatively reliable and accurate estimates for the physical-virtual mapping transformation were verified. Consequently, imaging data and surgical instruments holograms can be precisely represented in the physical space.

Hand-eye calibration, a fundamental task in vision-based robotic systems, is commonly equipped with collaborative robots, especially for robotic applications in small and medium-sized enterprises (SMEs). Most approaches to hand-eye calibration rely on external markers or human assistance. We proposed a novel methodology that addresses the hand-eye calibration problem using the robot base as a reference, eliminating the need for external calibration objects or human intervention. Using point clouds of the robot base, a transformation matrix from the coordinate frame of the camera to the robot base is established as “I=AXB.” To this end, we exploit learning-based 3D detection and registration algorithms to estimate the location and orientation of the robot base. The robustness and accuracy of the method are quantified by ground-truth-based evaluation, and the accuracy result is compared with other 3D vision-based calibration methods. To assess the feasibility of our methodology, we carried out experiments utilizing a low-cost structured light scanner across varying joint configurations and groups of experiments. The proposed hand-eye calibration method achieved a translation deviation of 0.930 mm and a rotation deviation of 0.265 degrees according to the experimental results. Additionally, the 3D reconstruction experiments demonstrated a rotation error of 0.994 degrees and a position error of 1.697 mm. Moreover, our method offers the potential to be completed in 1 second, which is the fastest compared to other 3D hand-eye calibration methods. We conduct indoor 3D reconstruction and robotic grasping experiments based on our hand-eye calibration method. Related code is released at https://github.com/leihui6/LRBO.

The traditional hand-eye calibration needs to take the calibration target image acquired by the vision system and the corresponding robot pose information as input, the process is complicated, and there is a risk of calibration failure when the data are not corresponding. To solve these problems, a fast hand-eye calibration method is proposed. According to the principle that a certain point in space has the same three-dimensional (3D) coordinates in its coordinate system after the motion of the coordinate system, to control the end of the robot to carry out multiple translation and rotational motion, the traditional hand-eye calibration model AX=XB is simplified into AX=B. The hand-eye calibration matrix can be quickly solved only by the transformation between the camera coordinate system and the calibration target coordinate system. The experimental results show that the proposed method is simple, fast and high accuracy. It can be applied to eye-in-hand system and eye-to-hand system.

A new one-time registration method was developed in this research for hand-eye calibration of a surgical robot to simplify the operation process and reduce the preparation time. And a new and practical method is introduced in this research to optimize the end-tool parameters of the surgical robot based on analysis of the error sources in this registration method. In the process with one-time registration method, firstly a marker on the end-tool of the robot was recognized by a fixed binocular camera, and then the orientation and position of the marker were calculated based on the joint parameters of the robot. Secondly the relationship between the camera coordinate system and the robot base coordinate system could be established to complete the hand-eye calibration. Because of manufacturing and assembly errors of robot end-tool, an error equation was established with the transformation matrix between the robot end coordinate system and the robot end-tool coordinate system as the variable. Numerical optimization was employed to optimize end-tool parameters of the robot. The experimental results showed that the one-time registration method could significantly improve the efficiency of the robot hand-eye calibration compared with the existing methods. The parameter optimization method could significantly improve the absolute positioning accuracy of the one-time registration method. The absolute positioning accuracy of the one-time registration method can meet the requirements of the clinical surgery.

With the promotion of Industry 4.0 reform, the trend of intelligent and precise production in the production workshop is gradually highlighted. This directly leads to higher requirements for robot hand eye coordination accuracy in automated workshops. In order to achieve more precise robot hand eye coordination control, this study designed a new mean calculation method based on the probability density theory, and designed a new mean robot hand eye calibration algorithm based on this. After the test, it is found that the translation error and rotation error calculated by the new mean algorithm are 0.26 and 0.92 respectively, which are significantly lower than other comparison algorithms when using all test samples of normal distribution. And the calculation time of the algorithm when using all the test samples is 2115 ms, which is also significantly lower than the comparison algorithm. The simulation results show that the new mean hand eye calibration method designed in this study can achieve more accurate hand eye coordination control of robots, and has certain application potential in high-precision industrial production scenarios.

To solve the problem of inflexibility of offline hand–eye calibration in “eye-in-hand” modes, an online hand–eye calibration method based on the ChArUco board is proposed in this paper. Firstly, a hand–eye calibration model based on the ChArUco board is established, by analyzing the mathematical model of hand–eye calibration, and the image features of the ChArUco board. According to the advantages of the ChArUco board, with both the checkerboard and the ArUco marker, an online hand–eye calibration algorithm based on the ChArUco board is designed. Then, the online hand–eye calibration algorithm, based on the ChArUco board, is used to realize the dynamic adjustment of the hand–eye position relationship. Finally, the hand–eye calibration experiment is carried out to verify the accuracy of the hand–eye calibration based on the ChArUco board. The robustness and accuracy of the proposed method are verified by online hand–eye calibration experiments. The experimental results show that the accuracy of the online hand–eye calibration method proposed in this paper is between 0.4 mm and 0.6 mm, which is almost the same as the offline hand–eye calibration accuracy. The method in this paper utilizes the advantages of the ChArUco board to realize online hand–eye calibration, which improves the flexibility and robustness of hand–eye calibration.

An accurate and reliable estimation of the transformation matrix between an optical sensor and a robot is a key aspect of the hand–eye system calibration process in vision-guided robotic applications. This paper presents a novel approach to markerless hand–eye calibration that achieves streamlined, flexible, and highly accurate results, even without error compensation. The calibration procedure is mainly based on using the robot’s tool center point (TCP) as the reference point. The TCP coordinate estimation is based on the robot’s flange point cloud, considering its geometrical features. A mathematical model streamlining the conventional marker-based hand–eye calibration is derived. Furthermore, a novel algorithm for the automatic estimation of the flange’s geometric features from its point cloud, based on a 3D circle fitting, the least square method, and a nearest neighbor (NN) approach, is proposed. The accuracy of the proposed algorithm is validated using a calibration setting ring as the ground truth. Furthermore, to establish the minimal required number and configuration of calibration points, the impact of the number and the selection of the unique robot’s flange positions on the calibration accuracy is investigated and validated by real-world experiments. Our experimental findings strongly indicate that our hand–eye system, employing the proposed algorithm, enables the estimation of the transformation between the robot and the 3D scanner with submillimeter accuracy, even when using the minimum of four non-coplanar points for calibration. Our approach improves the calibration accuracy by approximately four times compared to the state of the art, while eliminating the need for error compensation. Moreover, our calibration approach reduces the required number of the robot’s flange positions by approximately 40%, and even more if the calibration procedure utilizes just four properly selected flange positions. The presented findings introduce a more efficient hand–eye calibration procedure, offering a superior simplicity of implementation and increased precision in various robotic applications.

This letter describes a novel hand–eye calibration technique based on reprojection error minimization. In contrast to traditional hand–eye calibration methods, the proposed method directly takes images of the calibration pattern and does not require to explicitly estimate the camera pose for each input image. The proposed method is implemented as a pose graph optimization problem, so that it can solve the estimation problem efficiently and robustly, and it can be easily extended for different projection models. It can deal with different camera models (e.g, X-ray cameras with a source-detector projection model) by changing the projection model. Through simulations, we validated that the proposed method shows a good estimation accuracy, and it can be applied to hand–eye calibration with a source-detector camera model. The experimental results with real robots show that the proposed method is applicable to real environments, and it improves the quality of a task that requires accurate hand–eye estimation, such as three-dimensional reconstruction.

Purpose – The purpose of this paper is to propose a method that calibrates the hand-eye relationship for eye-to-hand configuration and afterwards a rectification to improve the accuracy of general calibration. Design/methodology/approach – The hand-eye calibration of eye-to-hand configuration is summarized as a equation AX = XB which is the same as in eye-in-hand calibration. A closed-form solution is derived. To abate the impact of noise, a rectification is conducted after the general calibration. Findings – Simulation and actual experiments confirm that the accuracy of calibration is obviously improved. Originality/value – Only a calibration plane is required for the hand-eye calibration. Taking the impact of noise into account, a rectification is carried out after the general calibration and, as a result, that the accuracy is obviously improved. The method can be applied in many actual applications.

We present an algorithm in the Conformal Geometric Algebra framework which computes the transformation (rotation and translation) relating the coordinate system of an endoscopic camera, with the coordinate system of the markers placed on it. Such markers are placed in order to track the camera’s position in real time by using an optical tracking system. The problem is an adaptation of the so-called hand-eye calibration, well known in robotics; however we name it the endoscope-tracking system calibration. By this way, we can relate the preoperative data (3D model and planing of the surgery), with the data acquired in real time during the surgery by means of the optical tracking system and the endoscope. Results obtained with our approach are compared with other approach which computes separately (in a two-step algorithm) the rotation and translation, and they are promising.

Received 16 March 2015; received in revised form 14 April 2015; accepted 14 April 2015ABSTRACTIn this paper, we propose calibration methods that can be applied to the markerless surgicalrobotic system for Intracerebral Hematoma (ICH) Surgery. This surgical robotic system does notrequire additional process of patient imaging but only uses CT images that are initially taken fora diagnosis purpose. Furthermore, the system applies markerless registration method other thanusing stereotactic frames. Thus, in overall, our system has many advantages when compared toother conventional ICH surgeries in that they are non-invasive, much less exposed to radiationexposure, and most importantly reduces a total operation time. In the paper, we specificallyfocus on the application of calibration methods and their verification which is one of the mostcritical factors that determine the accuracy of the system. We implemented three applications ofcalibration methods between the coordinates of robot’s end-effector and the coordinates of 3Dfacial surface scanner, based on the hand-eye calibration method. Phantom tests were conductedto validate the feasibility and accuracy of our proposed calibration methods and the surgicalrobotic system.Key Words:Hand-eye calibration, ICH surgery, Markerless registration, Stereotactic device,Surgical robot

In this paper, a new calibration method is proposed, which combines the extrinsic camera calibration and the robot hand-eye calibration. The advantage of this method is that it eliminates the errors and difficulties by solving complex equations such as AX=XB or AX=YB in traditional calibration methods. Error resources analysis and comparison with traditional methods are described in this paper. The experiment results on actual robot vision measurement system prove the validity and feasibility of this new method.

Online implementation of robotic hand-eye calibration consists in determining the relative pose between the robot gripper/end-effector and the sensors mounted on it, as the robot makes unplanned movement. With noisy measurements, inevitable in real applications, the calibration is sensitive to small rotations. Moreover, degenerate cases such as pure translations are of no effect in hand-eye calibration. This paper proposes an algorithm of motion selection for hand-eye calibration. Using this method, not only can we avoid the degenerate cases, but also the small rotations to decrease the calibration error. Thus, the procedure lends itself to an online implementation of hand-eye calibration, where degenerate cases and small rotations frequently occur in the sampled motions. Simulation and real experiments validate our method.KeywordsDegenerate CaseGolden RuleCalibration ErrorSmall RotationNoisy MeasurementThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

The explosive ordnance disposal (EOD) robots work in a special environment, which requires dual robotic arms to work together for removing the bomb. Therefore, machine vision is very important to locating bombs, especially the accuracy of the hand-eye calibration technology. A basic problem encountered with the collaborative work of dual robotic arms is to solve the unknown homogeneous transformation matrix, including: hand-eye of robotic arm 1, base-base, and camera-end effector’s robotic arm 2. In this article, the hand-eye calibration problem of the dual robotic arm system is expressed as two matrix equations. A new method of simultaneously solving the unknowns in the matrix equation is proposed. This method consists of a closed-form method based on the Kronecker product and an iterative method that transforms a nonlinear problem with a convex function optimization problem. The closed-form method is used to quickly obtain the initial value of the iteration method to improve the efficiency and accuracy of the iteration. In addition, we propose a hand-eye calibration method based on the re-projection error of the RGB-D camera. In order to prove the feasibility and superiority in the proposed iterative method, we conducted simulation and actual experiments and compared them with the other two calibration methods. The comparison results verify the superiority in the proposed method in terms of accuracy.

Objective To evaluate the effect of setup errors on the 2D image projection and image registration, and then propose an improved registration method based on mutual information. Methods An anthropomorphic head phantom was used to simulate the rotational and translational setup errors. The geometric disparities were reflected by the changes of mutual information. Known setup errors were intentionally introduced to twenty cases divided into two groups demarcated by 3 mm translation error and 3° rotation error: ten cases with larger errors and ten with smaller errors. Then the anterior-posterior and lateral portal images were captured by the electronic portal imaging device (EPID), based on which the setup errors were calculated using two mutual information registration method respectively: the vender provided one, and the improved method as proposed. The calculated errors were compared with the actual setup errors to evaluate robustness of the method. Results For the ten cases with smaller setup errors, the average translational registration disparities using the conventional method were 0.3, 0.4, and 0.3 mm in x, y and z directions respectively. The rotational disagreements were 0.4° in both x and z directions. The average time consumption was 28.7 s. The corresponding discrepancies analyzed using the improved method were 0.3, 0.4, 0.3 mm, 0.5° and 0.4°, respectively. On average, 31.1 s was needed for registration. For the ten cases with larger setup errors, the mean disparities of the conventional method were 0.9, 0.7, 0.8 mm, 0.9° and 0.8°, 29.9 s taken on average. The corresponding result of the improved method was 0.5, 0.4, 0.5 mm, 0.6° and 0.5°, 33.2 s taken on average. Conclusions Regarding smaller setup errors, the two methods showed little difference and both had good performance in image registration accuracy. For larger setup errors, however, the improved mutual information registration method exhibited significantly higher accuracy than the conventional method, at cost of clinically acceptable registration time. Key words: Setup errors; Electronic portal imaging device; Portal image; Image registration