Motion-error-free calibration of event camera systems using a flashing target.

With the continuous development and popularization of sensor-fusion technology for mobile robots, the application of camera and light detection and ranging (LiDAR) fusion perception has become particularly important. Moreover, the calibration of extrinsic parameters between the camera and LiDAR is a crucial prerequisite for fusion. Although traditional target-based calibration methods have been widely adopted, their cumbersome operation and high costs necessitate the development of more efficient and flexible calibration methods. To address this problem, this study proposed a two-stage calibration method based on motion and edge matching. In the first stage, the preliminary estimation of the extrinsic parameters between the camera and LiDAR was performed by matching visual odometry and LiDAR odometry using a hand–eye target method. In the second stage, the calibration results from the first stage were further refined by matching the image edges and discontinuous depth point clouds. The calibration system was then tested in both simulated and actual environments. The experimental results showed that this method, which did not require specially structured targets, could achieve highly flexible and robust automated calibration. Compared to other advanced methods, the accuracy of the proposed method was higher.

A newly developed flexible calibration algorithm for telecentric 3D measurement systems is presented in this paper. We theoretically analyzed the similarities and differences between the telecentric and entocentric system. The telecentric system can be calibrated with the aid of the traditional 2D planar calibration method. An additional two-step refining process is proposed to improve the calibration accuracy effectively. With the calibration and refining algorithm, an affine camera can be calibrated with a reprojection error of 0.07 pixel. A projector with small field of view (FOV) is applied to achieve a full 3D reconstruction in our profilometry system. Experiments with a prototype demonstrate the validation and accuracy of the proposed calibration algorithm and system configuration. The reconstruction accuracy can achieve 5 µm with a measurement FOV of 28.43 mm×21.33 mm and a working distance of 110 mm.

Depth estimation and camera calibration of a focused plenoptic camera for visual odometry

The basis of visual measurement is camera calibration. In the traditional calibration method, the radial and tangential distortion models usually are adopted. Because of the randomness of lens distortion, the above fixed form distortion model cannot accurately express the distortion distribution. To solve the above problems, a calibration method with a new distortion model is presented in the paper. First, an exact linear model is obtained, using only the corner coordinates of the image center region; then, using this model, the projection deviation of all corner points in the pixel plane can be obtained, that is, the point cloud of projection deviation distribution; finally, the Kriging interpolation method is used to obtain a continuous projection deviation distribution function which can accurately express lens distortion in the pixel plane. Using this function and the corresponding linear model, all two-dimensional image points can be accurately projected into three-dimensional space. To compare with the traditional method, the mean error of projection and measurement error are calculated in the experiment, and the experimental results show that the calibration method is more accurate and more suitable for measuring requirements.

With the rapid development of autonomous driving and SLAM technology, the perception system of a vehicle heavily relies on laser and image sensors to capture the real-world scenario and avoid obstacles autonomously. To achieve accurate and robust multi-sensor fusion computation, high-precision extrinsic calibration of camera and laser scanner is a necessary requirement. Traditional multi-sensor calibration methods based on manual features rely on specific scenarios and may not provide feature information over long distances. In this paper, we present a novel approach for robustly calibrating the extrinsic parameters of a solid-state(SS) lidar-camera system in a natural environment. Our proposed method begins with obtaining robust line feature information. we first innovatively employ a super-voxel clustering method to extract global 3D line features from the complete point cloud and then back-project these 3D line features into 2D space. Afterward, a transformer-based edge detection network, EDTER, is used to detect the edge features and estimate the probability pixel-by-pixel. To consider the uncertainty of two-dimensional line features and the inconsistency of residuals at different distances, we construct a line feature weight model for line feature residual calculation. Finally, we minimize the residual errors using least squares optimization to recover the relative pose of the camera and the lidar sensor. We conducted a performance study to compare our proposed method against existing targetless calibration methods on various natural scenarios. The experimental results demonstrate that our proposed method achieves higher robustness, accuracy, and consistency, making it suitable for real-world applications.

Binocular camera is widely used in high accuracy 3D measurement and the systematic error is critically determined by the calibration of camera parameters. This paper proposes a binocular camera calibration method based on planar target with different orientations translating along single axis. The translation of planar target composes a calibration grid, which is localized in an affine coordinate system (ACS). Additionally, non-homogeneous projection equations of binocular camera indicate the relationship between ACS and image projection coordinate system (IPCS). Therefore, a linear estimation of binocular camera parameters is obtained. In addition, we collect multiple calibration grids with different orientations to improve the estimation accuracy of principal points. Finally, we minimize reprojection error to optimize these camera parameters. Simulation and application experiments demonstrate that our method achieves less focal length error (0.063 pixel) than traditional calibration method using arbitrary placed planar target (0.123 pixel), and less principal point error (0.220 pixel) than pure translation method (0.271 pixel).

Robot calibration plays an increasingly important role in manufacturing. For robot calibration on the manufacturing floor, it is desirable that the calibration technique be easy and convenient to implement. This paper presents a new self-calibration method to calibrate and compensate for robot system kinematic errors. Compared with the traditional calibration methods, this calibration method has several unique features. First, it is not necessary to apply an external measurement system to measure the robot end-effector position for the purpose of kinematic identification since the robot measurement system has a sensor as its integral part. Second, this self-calibration is based on distance measurement rather than absolute position measurement for kinematic identification; therefore the calibration of the transformation from the world coordinate system to the robot base coordinate system, known as base calibration, is not necessary. These features not only greatly facilitate the robot system calibration, but also shorten the error propagation chain, therefore, increase the accuracy of parameter estimation. An integrated calibration system is designed to validate the effectiveness of this calibration method. Experimental results show that after calibration there is a significant improvement of robot accuracy over a typical robot workspace. [S1087-1357(00)01301-0]

For traditional camera calibration methods, the calibration accuracy of camera parameters is highly dependent on the feature extraction of the control points of the calibration target. However, due to problems such as perspective and lens distortion, the commonly used checkerboards, circular dots, and other calibration patterns will inevitably undergo large deformation, resulting in a serious impact on the accuracy of the extraction of control points. To solve this problem, a camera calibration method based on active phase calibration targets is proposed. The method uses the MI Pad that can automatically display the structured light patterns as the calibration target. Compared with the method of using a checkerboard calibration target, this method does not require pattern detection and avoids the process of manual marking. Also, based on the phase information of the structured light gratings, a large number of dense, high-precision control point coordinates can be obtained even in areas where the image edge distortion is severe. Not only is it highly automated, but it is also suitable for defocused cameras. Experimental results show that the reprojection error of our method is only onetenth of the comparison method, and it has better robustness and accuracy.

Cameras are appealing sensors for mobile robots because they are small, passive, inexpensive and provide rich information about the environment. While cameras have been used successfully on a plenitude of robots, such as autonomous cars or drones, serious challenges remain: power consumption, latency, dynamic range, and frame rate, among others. The sequences of images acquired by a camera are highly redundant (both in space and time), and both acquiring and processing such an amount of data consumes significant power. This limits the operation time of mobile robots and, moreover, defines a fundamental power-latency tradeoff. Specialized cameras designed for high-speed or high-dynamic-range scenarios are expensive, heavy, and require additional power, which prevents their use in agile mobile robots. In this thesis, we investigate event cameras as a biologically-inspired alternative to overcome the limitations of standard cameras. These neuromorphic vision sensors work in a completely different way: instead of providing a sequence of images (i.e., frames) at a constant rate, event cameras transmit only information from those pixels that undergo a significant brightness change. These pixel-level brightness changes, called events, are timestamped with micro-second resolution and transmitted asynchronously at the time they occur. Hence, event cameras are power efficient because they convey only non-redundant information, and are able to capture very high-speed motions, thus they directly address the power-latency tradeoff. Additionally, event cameras achieve a dynamic range of more than 140dB, compared to about 60dB of standard cameras, because each pixel is autonomous and operates at its own set-point. However, since the output of an event camera is fundamentally different from that of standard cameras for which computer-vision algorithms have been developed during the past fifty years, new algorithms that can deal with the asynchronous nature of the sensor and exploit its high temporal resolution are required to unlock its potential. This thesis presents algorithms for using event cameras in the context of robotics. Since event cameras are novel sensors that are being intensively prototyped and have been commercially available only recently (ca. 2008), the literature on event-based algorithms is scarce. This poses some operational challenges as well as uncountable opportunities to explore in research. This thesis focuses on exploring the possibilities that event cameras bring to some fundamental problems in robotics and computer vision, such as localization and actuation. Amongst others, this thesis provides contributions to solving the localization problem, i.e., for a robot equipped with an event camera to be able to infer its location with respect to a given map of the environment. Classical approaches for robot localization build upon lower-level vision algorithms, and so, this thesis also presents contributions in the topics of detection, extraction, and tracking of salient visual features with an event camera, whose applicability expands far beyond the localization problem. This thesis also presents contributions in the use of event cameras for actuation and closed-loop control, i.e., in endowing the robot with the capabilities to interact with the environment to fulfill a given task. Additionally, this thesis also presents the infrastructure developed to work with event cameras in a de-facto standard robotics platform. The following is a list of contributions: * Software infrastructure, consisting of publicly available drivers, calibration tools, sensor delay characterization, and the first event camera dataset and simulator tailored for 6-DOF (degrees of freedom) camera pose estimation and SLAM (Simultaneous localization and mapping). * We introduce the concept of event and provide an algorithm to compute it. The lifetime endows the events with a finite temporal extent for a proper continuous representation of events in time. * The first method to extract FAST-like visual features (i.e., interest points or corners) from the output of an event camera. The detector operates an order of magnitude faster than previous corner detectors. * The first method to extract and track features from the output of a DAVIS camera (an event camera that also outputs standard frames from the same pixel array). Using these feature tracks, we developed the first sparse, feature-based visual-odometry pipeline. * The first two methods to track the 6-DOF pose of an event camera in a known map.While the first method minimizes the reprojection error of the events and only works on black-and-white scenes consisting of line segments, the second method uses a probabilistic filtering framework that allows tracking at high speeds on natural scenes. * The first application of a continuous-time framework to estimate the trajectory of an event camera, possibly incorporating inertial measurements, showing superior performance over pose-tracking-only methods. * An application of event cameras to collision avoidance of a quadrotor, showing how event cameras can be used to control a robot with very low latency. * An application of the use of an event camera for human-vs-machine slot-car racing, showing that event-driven algorithms are power efficient and can outperform human control.

Calibration is vital to improving the accuracy of the strapdown inertial navigation system (SINS). Calibration is one of the crucial phases before the operation of SINS and requires high accuracy and speed. Traditional system-level calibration methods have the disadvantages of long calibration times and complicated path design. A fast calibration method for the lever arm and other error parameters is proposed in this article. To shorten the calibration time and improve the calibration accuracy, a backtracking calibration method in inertial frames based on the reduced-order Kalman filter is proposed. Simulations and experiments are given to illustrate the effectiveness of the novel calibration method. The calibration process can be completed in 10 min, which is greatly shortened compared with traditional methods. The calibration time is shortened, and the calibration accuracy is improved. Furthermore, navigation experiments show that the velocity and position accuracies of the SINS are improved significantly after compensation with the calibration results, fully illustrating the significance of the proposed calibration method in improving the navigation performance.

Event cameras can show better performance than frame cameras in challenging scenarios, such as fast-moving environments or high-dynamic-range scenes. However, it is still difficult for event cameras to replace frame cameras in non-challenging normal scenarios. In order to leverage the advantages of both cameras, we conduct a study for the heterogeneous stereo camera system which employs both an event and a frame camera. The proposed system estimates the semi-dense disparity in real-time by matching heterogeneous data of an event and a frame camera in stereo. We propose an accurate, intuitive and efficient way to align events with 6-DOF camera motion, by suggesting the maximum shift distance method. The aligned event image shows high similarity to the edge image of the frame camera. The proposed method can estimate poses of an event camera and depth of events in a few frames, which can speed up the initialization of the event camera system. We verified our algorithm in the DSEC dataset. The proposed hetero-stereo matching outperformed other methods. For real-time operation, we implemented our code using parallel computation with CUDA and release our code open source:

In the system of distributed radar array system using phase interference angle measurement, the phase center coordinate error of arrays and the phase difference error have relatively large influence on the angle measurement. The phase center position is often inconsistent with physical center position. Thus it is necessary to compensate these errors precisely. Far field radiation sources are often used to calibrate radar in traditional calibration methods. However, it is usually hard to achieve far field radiation sources for distributed radar array with large space between units surveilling space targets. In this paper, a calibration method based on the precise ephemeris of refined orbit satellites without measuring with special instruments is proposed. The phase error caused by coordinate error can be whitened by the precise ephemeris of multiple arcs, and the coordinate and phase difference will be searched out by matching the minimum variance. This method can get the errors easily. The simulation results and actual data verify that angle measurement accuracy gets large improvement by the method.

Purpose– The purpose of this paper is to present a distance accuracy-based industrial robot kinematic calibration model. Nowadays, the repeatability of the industrial robot is high, while the absolute positioning accuracy and distance accuracy are low. Many factors affect the absolute positioning accuracy and distance accuracy, and the calibration method of the industrial robot is an important factor. When the traditional calibration methods are applied on the industrial robot, the accumulative error will be involved according to the transformation between the measurement coordinate and the robot base coordinate.Design/methodology/approach– In this manuscript, a distance accuracy-based industrial robot kinematic calibration model is proposed. First, a simplified kinematic model of the robot by using the modified Denavit–Hartenberg (MDH) method is introduced, then the proposed distance error-based calibration model is presented; the experiment is set up in the next section.Findings– The experimental results show that the proposed calibration model based on MDH and distance error can improve the distance accuracy and absolute position accuracy dramatically.Originality/value– The proposed calibration model based on MDH and distance error can improve the distance accuracy and absolute position accuracy dramatically.

It is necessary to calibrate interferometric system parameters in order to achieve high accuracy of DEM. The traditional calibration method use a lot of ground artificial calibrators as ground control points. However, the number of calibrators is limited and the distribution is difficult in some mountainous areas. The paper presents a calibration method based on an external DEM without any artificial calibrators. A simulated SAR image was generated with the aid of this reference DEM and a number of tie points were selected to be ground control points. This calibration method can improve the DEM accuracy, and this paper also illustrates its efficient application to airborne interferometric SAR data.

The process of determining camera settings to deduce geometric attributes from recorded sequences is known as camera calibration. This process is essential in the fields of robotics and computer vision, encompassing both two-dimensional and three-dimensional applications. Traditional calibration methods, however, are time-consuming and require specific expertise. Recent endeavors have demonstrated that learning-based systems can replace the monotonous tasks associated with manual calibrations. Responses have been examined through a range of learning techniques, networks, geometric assumptions, and datasets. A thorough examination of camera calibration systems that rely on learning algorithms is offered in this paper, assessing their advantages and disadvantages. The primary categories of calibration presented are the regular pinhole camera model, distortion camera model, cross-sensor model, and cross-view model. These categories align with current research trends and have diverse applications. As there is no existing standard in this field, a large dataset of calibration has been created, which can be used as a public platform to assess the effectiveness of current methods. This collection consists of both artificially generated and genuine data, including images and videos obtained from various cameras in different locations. The difficulties faced will be analyzed, and alternative avenues for further research will be suggested in the next stage of this project. This survey represents the initial attempt to perform camera calibration using learning-based methods spanning a period of eight years. Our findings indicate that learning-based methods significantly reduce the time and expertise required for calibration while maintaining or improving accuracy compared to traditional methods. Specifically, our research demonstrates a calibration error reduction of up to 20% and speed improvements by a factor of three compared to traditional methods, as well as better adaptability to different camera types and environments.