Tracking-based depth recovery for virtual reality applications

An analytical method to calibrate a camera's extrinsic parameters and effective focal lengths is presented. We used a single image with four coplanar control lines. Unique solutions of the camera's extrinsic parameters and the two effective focal lengths are derived linearly and analytically. If the effective focal lengths are known, unique solutions of the extrinsic parameters can be derived more easily. Our method can easily be implemented and has better precision than the analogous method by use of control points. The results of simulated experiments are presented.

The paper presents a new closed-form solution to estimate focal length and extrinsic parameters from a single calibration image. Firstly, a new backward camera model which is based on plane equation of calibration panel in 3D space is introduced, and three indirect extrinsic parameters are used to determine the final extrinsic parameter. Based on this model and distance between neighboring control points in calibration panel, the focal length and indirect extrinsic parameters are compute with closed-form solution and numerical post-processing. The estimated results of simulated camera and actual camera show that our method has higher accuracy and less computational complexity. The method can be used for on-line camera self-calibration where relative low calibration accuracy can be accepted and low algorithm complexity is needed, or as the initial values of complex optimization algorithm.

Ground target tracking using electro-optical and infrared video sensors onboard unmanned aerial vehicles has drawn a great deal of interest in recent years due to the evolution of inexpensive video sensors and platforms. We present algorithms for geolocation using pixel location measurements which are based on the perspective transformation and includes radial and tangential lens distortions. The covariance of geolocation error takes into account the errors in pixel location, intrinsic and extrinsic camera parameters, and terrain height. Pixel coordinates of the optical center, focal distances along the X and Y axes, lens distortion parameters, and the skew parameter constitute the intrinsic camera parameters. The extrinsic camera parameters include the sensor position and sensor attitude relative a local east-north-up coordinate frame. Numerical results are presented using simulated data. Our results show that the errors in the Euler angles used to represent the sensor attitude is the dominant source of geolocation error.

In this paper, we present a methodology for extrinsic calibration of a camera attached to a long-reach manipulator in an unknown environment. The methodology comprises coarse frame alignment and fine matching based on probabilistic point set registration. The coarse frame alignment is based on the known initial pose and assists in the fine matching step, which is based on robust generalized point set registration that utilizes position and orientation data. Comparison with other methods utilizing only position data is provided. The first 6 DOF point set is obtained using a SLAM algorithm running on a camera attached near the tip of a manipulator, whereas the second point set is obtained using a kinematic model and joint encoders. Real-time experiments and a use case are presented. The results demonstrate that the proposed methodology is suited for the application, and that it can be useful in operations requiring precise visual measurements obtained near the tip of the manipulator.

Recently, autonomous driving based on multiple sensors has been studied actively in the field of automobiles and unmanned robots. Extrinsic parameter calibration is the first step to integrate the camera with Light Detection And Ranging (LiDAR). This paper proposes an extrinsic parameter calibration method using camera images and single 2D LiDAR points. The removal of infrared cut filter makes the line of laser scan points visible in the camera image. The scan line of the laser points on the calibration target is detected using edge matching in camera images, and the laser points are mapped to the coordinates of image with the initial value of the extrinsic parameters. We estimate the extrinsic parameters using edge matching and top-down method. The proposed method is verified by experiment according to the distance of the target.

3D Face Modeling from Perspective-Views and Contour-Based Generic-Model

We present a camera calibration technique to determine the focal length and the extrinsic parameters of a camera merely by using one perspective view of two coplanar circles with arbitrary radii and topological configuration. We believe our effort is valuable, in that the position of a camera and its focal length are frequently adjusted in many vision applications. In contrast to the iterative optimization technique in previous work, whose convergence relies heavily on proper initialization, we propose a closed-form solution on the basis of simple matrix operations, which turns out to be computationally efficient. It can also be used to initialize existing algorithms for higher accuracy and speed. Our method is based on the projective equation of a circle, first presented here, which naturally encodes the intrinsic and extrinsic camera parameters and the degenerate conic envelope spanned by the image of circular points. Extensive experiments with both synthetic data and real images verify the efficiency and robustness of our technique.

We consider the problem of self-calibration of a camera whose intrinsic parameters are known, besides the focal length. In the past, algorithms were mainly proposed for estimating two values of the focal length, for two images taken with different zoom. While there exist closed form solutions based on the fundamental matrix, their applicability is limited due to a generic singularity that occurs whenever the optical axes intersect, leading to numerical instabilities in most practical situations. Here, we consider the case of two views with identical focal length. We derive closed form solutions (one quadratic and two linear equations). Their respective singularity conditions are studied analytically and experimental results on their stability are given.

This paper introduces a simultaneous estimation method of the extrinsic parameters of multiple fish-eye cameras using simple calibration markers. Precise extrinsic parameters of cameras mounted on a car are important, for example, to provide a seamless overhead view image to the driver. Calibration markers are set in the area that are observable from adjacent two cameras. Extrinsic parameters of each camera are estimated individually and then combined and refined using a geometric constraint. Cube markers are chosen as the calibration markers. The method is evaluated by simulation and experiments using a real car. It is shown that extrinsic parameters are obtained by the proposed method and suggested that the errors of intrinsic parameters affect the estimation of extrinsic parameters.

The process of solving camera intrinsic and extrinsic parameters has been presented by using geometric techniques. In contrast with traditional method, which is a pure algebra process, the geometric technique accomplishes the camera parameters more intuitively and easily. If the perspective projection matrix is given, according to the geometric meaning of perspective projection camera model, the coordinates of optical centre can be directly estimated in the world coordinate system. Then rotation matrix and translation vector can be obtained from the matrix. Finally, we can derive camera intrinsic parameters. During the process of derivation, for any given 3 by 4 matrix, the constraint conditions have been discussed in another way when the matrix could be described as a perspective projection transformation. Although it has the same result as pure algebra progress, it's of great significance to understand the geometric meaning among the intrinsic and extrinsic camera parameters

Camera calibration using one-dimensional information and its applications in both controlled and uncontrolled environments

A new technique is proposed for calibrating a 3D modeling system with variable zoom based on multi-view stereo image analysis. The 3D modeling system uses a stereo camera with variable zoom setting and a turntable for rotating an object. Given an object whose complete 3D model (mesh and texture-map) needs to be generated, the object is placed on the turntable and stereo images of the object are captured from multiple views by rotating the turntable. Partial 3D models generated from different views are integrated to obtain a complete 3D model of the object. Changing the zoom to accommodate objects of different sizes and at different distances from the stereo camera changes several internal camera parameters such as focal length and image center. Also, the parameters of the rotation axis of the turntable changes. We present camera calibration techniques for estimating the camera parameters and the rotation axis for different zoom settings. The Perspective Projection Matrices (PPM) of the cameras are calibrated at a selected set of zoom settings. The PPM is decomposed into intrinsic parameters, orientation angles, and translation vectors. Camera parameters at an arbitrary intermediate zoom setting are estimated from the nearest calibrated zoom positions through interpolation. A performance evaluation of this technique is presented with experimental results. We also present a refinement technique for stereo rectification that improves partial shape recovery. And the rotation axis of multi-view at different zoom setting is estimated without further calibration. Complete 3D models obtained with our techniques are presented.

We present CineMPC, an algorithm to autonomously control a UAV-borne video camera in a nonlinear Model Predicted Control (MPC) loop. CineMPC controls both the position and orientation of the camera-the camera extrinsics-as well as the lens focal length, focal distance, and aperture-the camera intrinsics. While some existing solutions autonomously control the position and orientation of the camera, no existing solutions also control the intrinsic parameters, which are essential tools for rich cinematographic expression. The intrinsic parameters control the parts of the scene that are focused or blurred, the viewers' perception of depth in the scene and the position of the targets in the image. CineMPC closes the loop from camera images to UAV trajectory and lens parameters in order to follow the desired relative trajectory and image composition as the targets move through the scene. Experiments using a photo-realistic environment demon-strate the capabilities of the proposed control framework to successfully achieve a full array of cinematographic effects not possible without full camera control.

Computer vision plays a key role in measuring the relative posture and position between spacecrafts, especially in various close-range space tasks. As one of the essential steps for computer vision, camera calibration is important for obtaining precise three-dimensional contours of a space target. The focal length of on-orbit zoom cameras constantly changes. Thus, it is practical to calibrate the focal length rather than other intrinsic camera parameters. However, traditional calibration targets, such as checkerboards, cannot be used to calibrate a space camera in orbit. To address this problem, we propose a two-step process for focal length calibration. In the first step, the initial estimate of the camera focal length was generated with vanishing points obtained from the solar panels of satellites. In the second step, the initial solution was optimized by the particle swarm optimization algorithm. The results of the simulations and laboratory experiments confirmed the accuracy, flexibility, and good antinoise interference performance of the proposed method. Thus, the proposed method has practical significance for space tasks, such as space rendezvous-docking and on-orbit maintenance.

This paper describes a new approach that generates a higher resolution curved surface of an object from fewer three-dimensional (3D) measurement points compared with conventional approaches. In computer graphics (CG) field, an object is often expressed as the cluster of the polygons, which are converted from the 3D position data. However, if the 3D position data of an object is small in number, it is difficult to accurately reconstruct the object. In this paper, we propose a new approach of reconstruction of an object using both the position data and the orientation data. The advantage of this approach is to accurately reconstruct with a small number of the 3D position data compared with the conventional approach. The main idea of this approach is both to introduce the orientation data of an object and to use Hermite interpolation for accurate reconstruction of the object