Calibration of a hybrid camera network

Vanishing points and vanishing lines are classical geometrical concepts in perspective cameras that have a lineage dating back to 3 centuries. A vanishing point is a point on the image plane where parallel lines in 3D space appear to converge, whereas a vanishing line passes through 2 or more vanishing points. While such concepts are simple and intuitive in perspective cameras, their counterparts in catadioptric cameras (obtained using mirrors and lenses) are more involved. For example, lines in the 3D space map to higher degree curves in catadioptric cameras. The projection of a set of 3D parallel lines converges on a single point in perspective images, whereas they converge to more than one point in catadioptric cameras. To the best of our knowledge, we are not aware of any systematic development of analytical models for vanishing points and vanishing curves in different types of catadioptric cameras. In this paper, we derive parametric equations for vanishing points and vanishing curves using the calibration parameters, mirror shape coefficients, and direction vectors of parallel lines in 3D space. We show compelling experimental results on vanishing point estimation and absolute pose estimation for a wide range of catadioptric cameras in both simulations and real experiments.

Vision algorithms utilizing camera networks with a common field of view are becoming increasingly feasible and important. Calibration of such camera networks is a challenging and cumbersome task. The current approaches for calibration using planes or a known 3D target may not be feasible as these objects may not be simultaneously visible in all the cameras. In this paper, we present a new algorithm to calibrate cameras using occluding contours of spheres. In general, an occluding contour of a sphere projects to an ellipse in the image. Our algorithm uses the projection of the occluding contours of three spheres and solves for the intrinsic parameters and the locations of the spheres. The problem is formulated in the dual space and the parameters are solved for optimally and efficiently using semidefinite programming. The technique is flexible, accurate and easy to use. In addition, since the contour of a sphere is simultaneously visible in all the cameras, our approach can greatly simplify calibration of multiple cameras with a common field of view. Experimental results from computer simulated data and real world data, both for a single camera and multiple cameras, are presented.

In the robot feedback control system of visual feedback, the calibration of the hand-eye system is very important, because it directly affects the accuracy of the robot. Aiming at the small operating space and compact structure of the micro-assembly robot system, a fixed-vision hand-eye calibration method is proposed. The method first obtains the coordinates of 15 spatial points in the robot base coordinate system, and then uses the camera pose measurement principle to obtain the coordinates of the corresponding spatial points in the camera coordinate system, and then establishes the calibration equations. Based on the least squares method, the coordinate transformation matrix between the hand-eye systems is calibrated. The experimental results show that the constructed hand-eye system coordinate transformation matrix is effective and reliable. The calibration accuracy and effectiveness of the method are verified by relevant grabbing experiments.

This paper addresses the problem of rigid motion estimation and 3D reconstruction in vision systems where it is possible to recover the incident light ray direction from the image points. Such systems include pinhole cameras and catadioptric cameras. Given two images of the same scene acquired from two different positions, the transformation is estimated by means of an iterative process. The estimation process aims at having corresponding incident rays intersecting at the same 3D point. Geometrical relationships are derived to support the estimation method. Furthermore, this paper also addresses the problem of the mapping from 3D points to image points, for non-central catadioptric cameras with mirror surfaces given by quadrics. The projection model presented can be expressed in a non-linear equation of only one variable, being more stable and easier to solve than the classical Snell's law. Experiments with real images are presented, by using simulated annealing as estimation method.

A network of co-operative cameras for the visual surveillance of parking lots is presented. Such a network employs multiple subnets able to manage static and active cameras in a hierarchical framework. The system is able to track multiple targets simultaneously and in real-time throughout the controlled areas. The positions of detected objects, computed from different sensors, are fused considering a dynamic reliability factor for each sensor reading. Close-up recordings of suspicious events are obtained by tasking the active camera systems (ACSs). The co-operation is performed through a multicast communication system studied to transmit useful data both intra and inter networks. In particular, information about the position of the object to track, sent by a static camera system (SCS), is used by an ACS to operate an initial repositioning. The ACS compensates background changes owing to the camera motion, detects mobile objects in the scene and autonomously tracks the object of interest. Tracking results are presented in the context of a video surveillance application for a parking lot.

In order to meet the accuracy requirements of target geo-location of a wide-area reconnaissance camera, it is necessary to calibrate the extrinsic parameters of the camera. A novel calibration method is proposed for the orientation relationship between the camera coordinate system (CCS) and the frame coordinate system (FCS). First, the calibration between the roll axis of the FCS and the CCS is carried out based on the method of the extended Kalman filter. Second, the calibration between the pitch axis of the FCS and the CCS is deduced based on the least mean square combined with the particle swarm optimization method. Then, the calibration accuracy of the proposed method is quantitatively analyzed by numerical simulation. Finally, a calibration experiment is conducted on verifying the effectiveness of the method.

In the agriculture sector, multispectral camera near-ground observations are currently in demand. Altum multispectral camera is a new type of agricultural multispectral camera, which can be installed on ground observation equipment to obtain near-ground remote sensing images with high resolution and multispectral information. However, the optical system of the Altum camera adopts a multi-lens beam splitting design, and the channel registration problem used near-ground remains to be solved. Therefore, a near-ground image registration method is proposed based on a combination of a channel transformation model and a mapping relationship of the homologous image points. The way first corrects the internal lens parameters using the distortion coefficient of each lens. It then constructs a channel transformation model in the pixel coordinate system based on the transformation relationship between the camera coordinate system and the pixel coordinate system. The following calculates the transformation parameters from the corrected internal parameters and the relative position parameters between the lenses to complete the channel transformation. Finally, the homography model is estimated according to the mapping relationship of the homologous image points to perform inter-channel fine registration. The results show that the error in each band after registration is less than 0.5 pixels when the observation distance is greater than 7 metres. The method can offer a pre-processing service for channel registration for the Altum multispectral camera near-ground applications.

In this paper we present an easy method for multiple camera calibration with common field of view only from vertical lines. The locations of the vertical lines are known in advance. Compared to other calibration objects, the vertical lines have some good properties, since they can be easily built and can be visible by cameras in any direction simultaneously. Given 5 fixed vertical lines, an image containing them taken by a camera may provide 2 constraints in the intrinsic parameters of the camera, and extrinsic parameters can then be recovered. The calibration procedure consists of three main steps: Firstly, the image is rectified by a homography, which makes the projections of vertical lines parallel to u-axis in the rectified image. Secondly, for any vertical scan line in the rectified image, if we consider the scan line is taken by a virtual 1D camera, then we can calibrate the 1D camera. Finally, the intrinsic parameters of the original camera can be determined from the intrinsic parameters of the virtual 1D camera. By evaluating on both simulated and real data we demonstrate that our method is efficient and robust.

This paper proposes human tracking and recognition method in a camera network. Human matching in a multi-camera surveillance system is a fundamental issue for increasing the accuracy of recognition in multiple views of cameras. In camera network, videos have different characteristics such as pose, scale and illumination. Therefore it is necessary to use a hybrid scheme of scale invariant feature transform to detection and recognition human's behaviors. The main focus of this paper is to analyze activities for tracking and recognition humans to extract trajectories. Extracting the trajectories help to detect abnormal behavior which may be occluded in single- camera surveillance. KEYWORDS: Camera network, Multi-camera surveillance, Human's behavior, Trajectories extraction. I. INTRODUCTION Tracking and behavior recognition are two fundamental tasks in video surveillance systems which are widely employed in commercial applications for purposes of statistics gathering and processing. The number of cameras and complexity of surveillance systems have been continuously increasing to have better coverage and accuracy. Multi-camera systems become increasingly attractive in machine vision. Applications include multi view object tracking, event detection, occlusion handling and etc. In this paper, we develop method for tracking and recognition by a traffic video surveillance system of two cameras with a partially overlapping field of view. This paper is organized as follows: an overview of the past works in section2. Our proposed architecture and algorithm is presented in section3. Results of subjective evaluations and objective performance measurements with respect to Ground-truth are presented in section4. Section5 contains the conclusion.

This work addresses the problem of rigid motion estimation and 3D reconstruction in vision systems where it is possible to recover the incident direction from image points. Such systems include pinhole cameras and catadioptric cameras. Given two images of the same scene acquired from two positions, the transformation is estimated by means of an iterative process. The estimation process aims at having corresponding incident rays intersecting at the same 3D point. Geometrical relationships are derived to support the estimation method. Experiments with real images are presented.

A new one-time registration method was developed in this research for hand-eye calibration of a surgical robot to simplify the operation process and reduce the preparation time. And a new and practical method is introduced in this research to optimize the end-tool parameters of the surgical robot based on analysis of the error sources in this registration method. In the process with one-time registration method, firstly a marker on the end-tool of the robot was recognized by a fixed binocular camera, and then the orientation and position of the marker were calculated based on the joint parameters of the robot. Secondly the relationship between the camera coordinate system and the robot base coordinate system could be established to complete the hand-eye calibration. Because of manufacturing and assembly errors of robot end-tool, an error equation was established with the transformation matrix between the robot end coordinate system and the robot end-tool coordinate system as the variable. Numerical optimization was employed to optimize end-tool parameters of the robot. The experimental results showed that the one-time registration method could significantly improve the efficiency of the robot hand-eye calibration compared with the existing methods. The parameter optimization method could significantly improve the absolute positioning accuracy of the one-time registration method. The absolute positioning accuracy of the one-time registration method can meet the requirements of the clinical surgery.

Accurate pose (position and attitude) measurement is a key for automatic docking of cabins. In this paper, a stereo vision based cabin's six-dimensional (6D: three translational positions plus three rotational angles) pose measurement method has been proposed. First, the feature points provided by the visual marker pasted on each cabin's outer surfaces are extracted. The three-dimensional translational coordinates of the feature points in the camera coordinate system are obtained by the stereo vision system. Then the coordinate systems of two markers on two cabins are established through the feature points. The two cabins' 6D poses are denoted by two pose matrices calculated from the two markers' coordinate systems. Relative pose between two cabins is obtained through the two pose matrices. The pose measurement method has been verified using a six-degree-of-freedom (DOF) motion platform. Experimental results show that the errors in three translations and three rotations are ±0.06 mm out of ±5 mm and ±0.07° out of ±3°, respectively.

The traditional hand-eye calibration needs to take the calibration target image acquired by the vision system and the corresponding robot pose information as input, the process is complicated, and there is a risk of calibration failure when the data are not corresponding. To solve these problems, a fast hand-eye calibration method is proposed. According to the principle that a certain point in space has the same three-dimensional (3D) coordinates in its coordinate system after the motion of the coordinate system, to control the end of the robot to carry out multiple translation and rotational motion, the traditional hand-eye calibration model AX=XB is simplified into AX=B. The hand-eye calibration matrix can be quickly solved only by the transformation between the camera coordinate system and the calibration target coordinate system. The experimental results show that the proposed method is simple, fast and high accuracy. It can be applied to eye-in-hand system and eye-to-hand system.

Utilizing the optical characteristics of the target for detection and localization does not require actively emitting signals and has the advantage of strong concealment. Once the optoelectronic platform mounted on the unmanned aerial vehicle (UAV) detects the target, the vector pointing to the target in the camera coordinate system can estimate the angle of arrival (AOA) of the target relative to the UAV in the Earth-centered Earth-fixed (ECEF) coordinate system through a series of rotation transformations. By employing two UAVs and the corresponding AOA measurements, passive localization of an unknown target is possible. To achieve high-precision target localization, this paper investigates the following three aspects. Firstly, two error transfer models are established to estimate the noise distributions of the AOA and the UAV position in the ECEF coordinate system. Next, to reduce estimation errors, a weighted least squares (WLS) estimator is designed. Theoretical analysis proves that the mean squared error (MSE) of the target position estimation can reach the Cramér–Rao lower bound (CRLB) under the condition of small noise. Finally, we study the optimal placement problem of two coplanar UAVs relative to the target based on the D-optimality criterion and provide explicit conclusions. Simulation experiments validate the effectiveness of the localization method.

Nowadays internet video surveillance cameras are widely use in security monitoring. The quantities of installations of these cameras also become more and more. This paper reports that the internet video surveillance cameras can be applied as a remote sensor for monitoring the concentrations of particulate matter less than 10 micron (PM10), so that real time air quality can be monitored at multi location simultaneously. An algorithm was developed based on the regression analysis of relationship between the measured reflectance components from a surface material and the atmosphere. This algorithm converts multispectral image pixel values acquired from these cameras into quantitative values of the concentrations of PM10. These computed PM10 values were compared to other standard values measured by a DustTrak^TM meter. The correlation results showed that the newly develop algorithm produced a high degree of accuracy as indicated by high correlation coefficient (R²) and low root-mean-square-error (RMS) values. The preliminary results showed that the accuracy produced by this internet video surveillance camera is slightly better than that from the internet protocol (IP) camera. Basically the spatial resolution of images acquired by the IP camera was poorer compared to the internet video surveillance camera. This is because the images acquired by IP camera had been compressed and there was no compression for the images from the internet video surveillance camera.