LiDAR and Camera Calibration Using Near-Far Dual Targets

Camera calibration using one-dimensional information and its applications in both controlled and uncontrolled environments

An improved outdoor camera calibration method based on Extended Kalman Filter (EKF) is proposed in this paper. GPS coordinate system is introduced to establish world coordinate system in the calibration procedure. According to the characteristics of GPS coordinate converted to plane coordinate and the camera calibration model, we establish the mapping relations of reference points between image plane coordinate and the world coordinate. Further, we consider the world coordinate error, set the world coordinates as the measure quantities, use the method of EKF to get the optimal estimation of intrinsic and extrinsic camera parameters, in EKF the camera's intrinsic and extrinsic parameters are set as state variables, the transformation equations of image coordinate to the world coordinate are set as the measurement equations, and we let the observed feature points' image coordinates and corresponding world coordinates as the input of the filter. Experimental results confirmed that the proposed method with high precision. Index Terms - camera calibration, GPS coordinate, EKF, world coordinate error

Camera calibration plays a crucial role in 3D measurement tasks of machine vision. In typical calibration processes, camera parameters are iteratively optimized in the forward imaging process (FIP). However, the results can only guarantee the minimum of 2D projection errors on the image plane, but not the minimum of 3D reconstruction errors. In this paper, we propose a universal method for camera calibration, which uses the back projection process (BPP). In our method, a forward projection model is used to obtain initial intrinsic and extrinsic parameters with a popular planar checkerboard pattern. Then, the extracted image points are projected back into 3D space and compared with the ideal point coordinates. Finally, the estimation of the camera parameters is refined by a non-linear function minimization process. The proposed method can obtain a more accurate calibration result, which is more physically useful. Simulation and practical data are given to demonstrate the accuracy of the proposed method.

Based on analyzing and partially testing the current methods for camera calibration in computer vision, inspired by Zhang’s technique, we introduced a new method for binocular stereovision camera calibration. In this method, a planar object with a series of circles was used as calibration template, and the centers of circles were regarded as control points. The proposed method requires the two cameras with relatively fixed position to simultaneously observe the calibration template at a few (at least two) different orientations by moving the cameras or planar object freely, then extracts the centers’ coordinates of circles or ellipses by image processing technique. On this basis, calculate every camera’ intrinsic and extrinsic parameters, then calculate the position parameters of the two cameras. The main point of this method is that it can match the points in the model plane and their image points easily; compared with Zhang’s calibration methods, it can reduce the errors of extracting control points. Experimental results and contrast tests show proposed method is accurate and very applicative to camera calibration of binocular stereovision system.

Many approaches for camera and LiDAR calibration are presented in literature but none of them estimates all intrinsic and extrinsic parameters simultaneously and therefore optimally in a probabilistic sense.In this work, we present a method to simultaneously estimate intrinsic and extrinsic parameters of cameras and LiDARs in a unified problem. We derive a probabilistic formulation that enables flawless integration of different measurement types without hand-tuned weights. An arbitrary number of cameras and LiDARs can be calibrated simultaneously. Measurements are not required to be time-synchronized. The method is designed to work with any camera model.In evaluation, we show that additional LiDAR measurements significantly improve intrinsic camera calibration. Further, we show on real data that our method achieves state-of-the-art calibration precision with high reliability.

The reliability of the extrinsic parameters of 3D Light Detection and Ranging (LiDAR) and camera is a prerequisite to ensure the common operation of intelligent perception systems. Compared with structured light cameras, monocular or binocular cameras commonly used in intelligent perception systems cannot provide rich and high-precision environmental information. Therefore, the extrinsic parameter calibration between the structured light camera and the lidar is very necessary for the information fusion between both. Here we propose a novel method for extrinsic parameters calibration between structured light camera and 3D lidar to enhance the intelligent perception capacity. The hemispherical surface on the calibration board can provide enough effective point clouds, so that the accurate coordinates of the center of the sphere can be stably fitted. And the extrinsic parameters can be solved by considering the centers of the sphere as reference points. To demonstrate the performance of this method, we value it by rotation error and translation error, using the ground truth value obtained from the simulation environment established by gazebo. The experimental results show that our method can get more reliable and accuracy extrinsic parameters between the structured light camera and lidar comparing toward the original method when the transformation is not pure translation.

The external parameter calibration of LiDAR and camera is the core link of multimodal perception fusion technology, and its accuracy directly determines the effectiveness of cross modal information fusion. This article proposes a method for high-precision calibration of laser radar and camera extrinsic parameters using a checkerboard calibration board based on spatial registration theory. In response to the difficulty of feature extraction caused by the sparsity of LiDAR point clouds, this method enhances feature information by accumulating multi view point clouds and combines filtering algorithms to improve feature saliency; A standardized experimental platform was built based on ROS and Matlab, utilizing the reflection intensity distribution pattern of the checkerboard point cloud to achieve accurate extraction of its three-dimensional corner coordinates. At the same time, the corresponding two-dimensional checkerboard corner points are detected in the camera image, and the relative pose transformation matrix between the LiDAR and the camera is solved by establishing an accurate matching relationship between the two-dimensional and three-dimensional corner points. The experimental results show that this method has high-precision calibration results and good repeatability, which can provide a reliable extrinsic basis for multi-sensor fusion perception systems.

This paper focuses on the study of 2D calibration method, involving a new planar pattern based on equilateral triangles, and a new camera calibration algorithm using the new pattern. Firstly, three triangles with same size and same color on the same plane, which side length can be any size, are spliced together to form a large equilateral triangle. Secondly, more than three pattern images are captured from different orientation. Thirdly, sub-pixel coordinates of six corners are extracted from each pattern image. Then, using perspective geometry theory and the geometric characteristics of triangle template, more than ten vanishing points are calculated out, vanishing line is fitted out according to them, and the image coordinates of circular points are deduced. Finally, all the five intrinsic parameters can be determined linearly according the coordinates of circular points. Experiments with simulated data as well as with real images show that the new algorithm is robust and accurate. The integral calibration precision of this method can reach about 2% comparing with Zhang's method. Equilateral triangle pattern the method used is easy to access, and easy to build in the industrial field, and its measurement also need not know in the calibration process. So this method with higher precision has a good practical value.

Extrinsic parameter calibration is the foundation and prerequisite for LiDAR and camera data fusion of the autonomous system. This technology is widely used in fields such as autonomous driving, mobile robots, intelligent surveillance, and visual measurement. The learning-based method is one of the targetless calibrating methods in LiDAR and camera calibration. Due to its advantages of fast speed, high accuracy, and robustness under complex conditions, it has gradually been applied in practice from a simple theoretical model in just a few years, becoming an indispensable and important method. This paper systematically summarizes the research and development of this type of method in recent years. According to the principle of calibration parameter estimation, learning-based calibration algorithms are divided into two categories: accurate calibrating estimation and relative calibrating prediction. The evolution routes and algorithm frameworks of these two types of algorithms are elaborated, and the methods used in the algorithms’ steps are summarized. The algorithm mechanism, advantages, limitations, and applicable scenarios are discussed. Finally, we make a summary, pointing out existing research issues and trends for future development.

This paper proposes an improved method for the calibration of a 2D LiDAR or a planar scanning laser rangefinder with respect to a camera, which uses the same checkerboard target as in the camera calibration. To address one of the drawbacks in the LiDAR calibration with a checkerboard, the sensitivity to checkerboard poses, laser points on the checkerboard are obtained from both the originally positioned LiDAR and the same LiDAR vertically rotated from the original position, in the proposed calibration. They construct the additional constraints with the normal vector of the checkerboard, and strengthen the basic point-on-plane constraints in the LiDAR calibration with a checkerboard. However, in practice, the LiDAR rotation can be imprecise. Thus, we further develop the calibration procedure and examine its calibration performance, in the presence of such impreciseness in the LiDAR rotation. A performance analysis is conducted with synthetic noisy data, and it is confirmed that the proposed calibration yields smaller error in the parameter estimation than a conventional baseline calibration method. Subsequently, we examine the calibration performance with real data, where the proposed calibration is carried out by building a calibration system. It is shown that the proposed calibration also has more consistency in the parameter estimation. For further study, we compare the proposed calibration based on point-on-plane constraints with a calibration based on point-to-point constraints, and show that the proposed calibration, with easier data acquisition, yields comparable results.

The calibration problem of binocular stereo vision rig is critical for its practical application. However, most existing calibration methods are based on manual off-line algorithms for specific reference targets or patterns. In this paper, we propose a novel simultaneous localization and mapping (SLAM)-based self-calibration method designed to achieve real-time, automatic and accurate calibration of the binocular stereo vision (BSV) rig’s extrinsic parameters in a short period without auxiliary equipment and special calibration markers, assuming the intrinsic parameters of the left and right cameras are known in advance. The main contribution of this paper is to use the SLAM algorithm as our main tool for the calibration method. The method mainly consists of two parts: SLAM-based construction of 3D scene point map and extrinsic parameter calibration. In the first part, the SLAM mainly constructs a 3D feature point map of the natural environment, which is used as a calibration area map. To improve the efficiency of calibration, a lightweight, real-time visual SLAM is built. In the second part, extrinsic parameters are calibrated through the 3D scene point map created by the SLAM. Ultimately, field experiments are performed to evaluate the feasibility, repeatability, and efficiency of our self-calibration method. The experimental data shows that the average absolute error of the Euler angles and translation vectors obtained by our method relative to the reference values obtained by Zhang’s calibration method does not exceed 0.5˚ and 2 mm, respectively. The distribution range of the most widely spread parameter in Euler angles is less than 0.2˚ while that in translation vectors does not exceed 2.15 mm. Under the general texture scene and the normal driving speed of the mobile robot, the calibration time can be generally maintained within 10 s. The above results prove that our proposed method is reliable and has practical value.

Removal of BCG artefact from concurrent fMRI-EEG recordings based on EMD and PCA

The genomic prediction of phenotypes and breeding values in animals and plants has developed rapidly into its own research field. Results of genomic prediction studies are often difficult to compare because data simulation varies, real or simulated data are not fully described, and not all relevant results are reported. In addition, some new methods have been compared only in limited genetic architectures, leading to potentially misleading conclusions. In this article we review simulation procedures, discuss validation and reporting of results, and apply benchmark procedures for a variety of genomic prediction methods in simulated and real example data. Plant and animal breeding programs are being transformed by the use of genomic data, which are becoming widely available and cost-effective to predict genetic merit. A large number of genomic prediction studies have been published using both simulated and real data. The relative novelty of this area of research has made the development of scientific conventions difficult with regard to description of the real data, simulation of genomes, validation and reporting of results, and forward in time methods. In this review article we discuss the generation of simulated genotype and phenotype data, using approaches such as the coalescent and forward in time simulation. We outline ways to validate simulated data and genomic prediction results, including cross-validation. The accuracy and bias of genomic prediction are highlighted as performance indicators that should be reported. We suggest that a measure of relatedness between the reference and validation individuals be reported, as its impact on the accuracy of genomic prediction is substantial. A large number of methods were compared in example simulated and real (pine and wheat) data sets, all of which are publicly available. In our limited simulations, most methods performed similarly in traits with a large number of quantitative trait loci (QTL), whereas in traits with fewer QTL variable selection did have some advantages. In the real data sets examined here all methods had very similar accuracies. We conclude that no single method can serve as a benchmark for genomic prediction. We recommend comparing accuracy and bias of new methods to results from genomic best linear prediction and a variable selection approach (e.g., BayesB), because, together, these methods are appropriate for a range of genetic architectures. An accompanying article in this issue provides a comprehensive review of genomic prediction methods and discusses a selection of topics related to application of genomic prediction in plants and animals.

In this paper we present an easy method for multiple camera calibration with common field of view only from vertical lines. The locations of the vertical lines are known in advance. Compared to other calibration objects, the vertical lines have some good properties, since they can be easily built and can be visible by cameras in any direction simultaneously. Given 5 fixed vertical lines, an image containing them taken by a camera may provide 2 constraints in the intrinsic parameters of the camera, and extrinsic parameters can then be recovered. The calibration procedure consists of three main steps: Firstly, the image is rectified by a homography, which makes the projections of vertical lines parallel to u-axis in the rectified image. Secondly, for any vertical scan line in the rectified image, if we consider the scan line is taken by a virtual 1D camera, then we can calibrate the 1D camera. Finally, the intrinsic parameters of the original camera can be determined from the intrinsic parameters of the virtual 1D camera. By evaluating on both simulated and real data we demonstrate that our method is efficient and robust.

Camera calibration is a necessity in various tasks including 3D reconstruction, hand-eye coordination for a robotic interaction, autonomous driving, etc. In this work we propose a novel method to predict extrinsic (baseline, pitch, and translation), intrinsic (focal length and principal point offset) parameters using an image pair. Unlike existing methods, instead of designing an end-to-end solution, we proposed a new representation that incorporates camera model equations as a neural network in a multi-task learning framework. We estimate the desired parameters via novel camera projection loss (CPL) that uses the camera model neural network to reconstruct the 3D points and uses the reconstruction loss to estimate the camera parameters. To the best of our knowledge, ours is the first method to jointly estimate both the intrinsic and extrinsic parameters via a multi-task learning methodology that combines analytical equations in learning framework for the estimation of camera parameters. We also proposed a novel CVGL Camera Calibration dataset using CARLA Simulator [1]. Empirically, we demonstrate that our proposed approach achieves better performance with respect to both deep learning-based and traditional methods on 8 out of 10 parameters evaluated using both synthetic and real data. Our code and generated dataset are available at https://github.com/thanif/Camera-Calibration-through-Camera-Projection-Loss.