Blur Compensation Weighted Calibration Algorithm for Stereo Vision

Stereo vision is an important computer vision technique which estimates depth information from two images - similar to how human vision works. However, the computational intensity of stereo vision deters implementation in real-time systems. Recently, massively parallel processors called Graphics Processing Units or GPUs are gaining traction as an acceleration platform to offload compute-intensive tasks to. This paper presents the development of a GPU-accelerated stereo vision system called DepthStream. The DepthStream algorithm uses a combined absolute difference and census cost, cross-based cost aggregation and bitwise fast region voting to achieve error rates up to 3.7× lower than Block-Matching based algorithms in mainstream libraries. Leveraging the parallel processing power that GPUs offer enables the DepthStream algorithm to achieve processing times up to 3.12× faster than Semi-Global Matching based implementations in the same libraries. The study also includes measurements of the accuracy of the estimated distances of the developed stereo vision system. Results show that the estimated depths of the DepthStream stereo vision system are within 4% of the true value for distances less than 6m. Statistical analysis also indicate that there is no significant difference between the stereo vision systems estimated distances and the actual value.

NASA scenarios for lunar and planetary missions include robotic vehicles that function in both teleoperated and semi-autonomous modes. Under teleoperation, on-board stereo cameras may provide 3-D scene information to human operators via stereographic displays; likewise, under semi-autonomy, machine stereo vision may provide 3-D information for obstacle avoidance. In the past, the slow speed of machine stereo vision systems has posed a hurdle to the semi- autonomous scenario; however, recent work at JPL and other laboratories has produced stereo vision systems with high reliability and near real-time performance for low-resolution image pairs. In particular, JPL has taken a significant step by achieving the first autonomous, cross- country robotic traverses (of up to 100 meters to use stereo vision, with all computing on- board the vehicle. This paper describes the stereo vision system, including the underlying statistical model and the details of the implementation. The statistical and algorithmic aspects employ random field models of the disparity map, Bayesian formulations of single-scale matching, and area-based image comparisons. The implementations builds Laplacian image pyramids and produces disparity maps from the 60X64 level of the pyramids at rates of up to two seconds per image pair. All vision processing is done on one 68020 augmented with Datacube image processing boards. The author argues that the overall approach provides a unifying paradigm for practical, domain-independent stereo ranging.

Depth estimation for dynamic scenes is a challenging and relevant problem in computer vision. Although this problem can be tackled by means of ToF cameras or stereo vision systems, each of the two systems alone has its own limitations. In this paper a framework for the fusion of 3D data produced by a ToF camera and a stereo vision system is proposed. Initially, depth data acquired by the ToF camera are up-sampled to the spatial resolution of the stereo vision images by a novel up-sampling algorithm based on image segmentation and bilateral filtering. In parallel a dense disparity field is obtained by a stereo vision algorithm. Finally, the up-sampled ToF depth data and the disparity field provided by stereo vision are synergically fused by enforcing the local consistency of depth data. The depth information obtained with the proposed framework is characterized by the high resolution of the stereo vision system and by an improved accuracy with respect to the one produced by both subsystems. Experimental results clearly show how the proposed method is able to outperform the compared fusion algorithms.

Computer stereo vision has been paid high attention in areas such as industry measurement and automatic control. Calibration of stereo vision system is a key step to implement vision measurement. Nonlinear calibration method involves too many parameters and too complex function, so the processing of optimization is difficult and slow. In this paper, two digital cameras are used to set up stereo vision system which provides convenient experimental operation. The binocular stereo vision system is calibrated using linear calibration model. The results indicate that the method is stable and fast with high precision and practicability.

Self-supervised learning as an enabling technology for future space exploration robots: ISS experiments on monocular distance learning

In this paper we propose a new technique for calibrating a camera with very high accuracy and low computational cost. The geometric camera parameters considered include camera position, orientation, focal length, radial lens distortion, pixel size, and optical axis piercing point. With our method, the camera parameters to be estimated are divided into two parts: the radial lens distortion coefficient (kappa) and a composite parameter vector c composed of all the above geometric camera parameters other than (kappa) . Instead of using nonlinear optimization techniques, the estimation of (kappa) is transformed into an eigenvalue problem of a 8 X 8 matrix. Our method is fast since it requires only linear computation. It is accurate since the effect of the lens distortion is considered and all the information contained in the calibration points is used. Computer simulation and real experiment have shown that the performance of our calibration method is better than that of the well-known method proposed by Tsai.

Significant obstacle location with ultra-wide FOV LWIR stereo vision system

NASA scenarios for lunar and planetary missions include robotic vehicles that function in both teleoperated and semi-autonomous modes. Under teleoperation, on-board stereo cameras may provide 3-D scene information to human operators via stereographic displays; likewise, under semi-autonomy, machine stereo vision may provide 3-D information for obstacle avoidance. In the past, the slow speed of machine stereo vision systems has posed a hurdle to the semi-autonomous scenario; however, recent work at JPL and other laboratories has produced stereo systems with high reliability and near real-time performance for low-resolution image pairs. In particular, JPL has taken a significant step by achieving the first autonomous, cross-country robotic traverses (of up to 100 meters) to use stereo vision, with all computing on-board the vehicle. Here, we describe the stereo vision system, including the underlying statistical model and the details of the implementation. The statistical and algorithmic aspects employ random field models of the disparity map, Bayesian formulations of single-scale matching, and area-based image comparisons. The implementation builds bandpass image pyramids and produces disparity maps from the 60×64 level of the pyramids at rates of up to two seconds per image pair. All vision processing is done in one 68020 augmented with Datacube image processing boards. We argue that the overall approach provides a unifying paradigm for practical, domain-independent stereo ranging. We close with a discussion of practical and theoretical issues involved in evaluating and extending the performance of the stereo system.

Camera calibration is a crucial technique which significantly influences the performance of many robotic systems. Robustness and high precision have always been the pursuit of diverse calibration methods. State-of-the-art calibration techniques, however, still suffer from inexact corner detection, radial lens distortion and unstable parameter estimation. Therefore, in this paper, we improve the precision and robustness of calibration by widening these bottlenecks. In particular, effective distortion correction is performed by a learning-based method. Then, accurate sub-pixel feature location is achieved by the combination of robust learning detection, exact refinement and complete post-processing. To obtain stable parameter estimation, an image-level RANSAC-based calibration procedure is proposed. Ultimately, we assemble these methods into a novel and practical calibration framework. Compared with state-of-art methods, experiment results on both real and synthetic datasets under noise, bad lighting and distortion manifest the better robustness and higher precision of the proposed framework.

Machine vision systems become to play an important role in robot based factory automation, because they enable robots to recognize the environment around the workspace like human visual systems and make the manufacturing process flexible. So a large number of 2-dimensional (2D) vision systems have been successfully applied for industrial robots, but there are increasing needs for 3-dimensional (3D) vision systems providing accurate 3D position of work-pieces. For the 3D vision system, the stereo vision based position measurement systems have been applied in various fields. However the stereo vision systems which have been applied were specific for their particular tasks only. This paper introduces a general-purpose stereo vision system applicable to wide range of tasks with industrial robots.

This article presents a technique for registration and segmentation of dense depth maps provided by a stereo vision system. The vision system uses inertial sensors to give a reference for camera pose. The maps are registered using a modified version of the ICP - iterative closet point algorithm to register dense depth maps obtained from a stereo vision system. The proposed technique explores the integration of inertial sensor data for dense map registration. Depth maps obtained by vision systems, are very point of view dependent, providing discrete layers of detected depth aligned with the camera. In this work we use inertial sensors to recover camera pose, and rectify the maps to a reference ground plane, enabling the segmentation of vertical and horizontal geometric features and map registration. We propose a real-time methodology segmentation of structures, object recognition, robot navigation or any other task that requires a three-dimensional representation of the physical environment. The aim of this work is a fast real-time system, which can be applied to autonomous robotic systems or to automated car driving systems, for modelling the road, identifying obstacles and roadside features in real-time.

In this paper, the stereo vision technique with two cameras has been employed to locate object's position in 3D space. It is necessary to use many algorithms to derive object positions, including stereo calibration, rectification, correspondence and triangulation. Two tracking algorithms have been applied to identify and track objects alternately: template matching and Camshift. To evaluate the proposed approach, the stereo vision system is installed on a cubic space which is the working space of a tiny robot arm. The real positions in this coordinate system are then compared to the received positions from the cameras. The end-effector of the robot arm will track the object depending on received positions. Experiment results show that the robot arm can effectively track the indicated objects, and thus the tracking errors between actual and received positions are less than 3% which are reasonably acceptable.

Using stereo vision and marks pasted on wall to solve the guidance and the environment learning problem for a mobile robot is proposed. A circular window is used to detect corner points as features in the scene. From the perspective transformation matrices of the two cameras and matching point-pairs in images the 3-D coordinates can be determined. With ground projections of these space points and the constraint of minimum diameter of the mobile robot, a few separated polygons are used to represent the occupied space of the surrounding obstacles. With the corner points of the mark the location of the mobile robot can be determined not only by the stereo vision system but also by each of the two cameras with plane-motion constraint. Experimental results show that the depth measurement error is less than 2%, the location determination of the mobile robot working on the ground plane is less than 5%, and the map of the occupied space can be used in further path-planning or collision avoidance.

When designing a multi-vision stereo vision network, the camera’s positional information has a critical impact on the measurement accuracy. In order to solve the problem of optimizing the camera pose during network design, in this paper, we split the multi-vision stereo vision system into binocular stereo vision system. Based on the mathematical model of binocular stereo vision system, the error function of the positional parameters is constructed. By simulation analyzing the function models the field of view angle, the angle between the optical axis and the baseline, the baseline length, the law of the influence of each parameter on the measurement accuracy and the optimal range of values are obtained. Then we compare the actual measurement results with the simulation analysis results. The results show that as the field of view angle, optical axis and baseline angle, baseline length increases, the measurement error first decreases and then increases, and the increase is gradually accelerated. When each parameter is located in the optimal range, the measurement error meets the precision engineering measurement accuracy requirement of 0.05mm, and the minimum error is reduced by 0.114mm, 0.120mm and 0.061mm compared to the maximum error. In the network design, the optimization of the pose parameters can effectively obtain better pose information of the camera and thus improve the accuracy of the measurement.

The stereo vision system is one of the most important sensors for 3D reconstruction of the environment and scene understanding. It plays an essential role for the automated driving and ADAS applications. Although recently great progress has been made in this field both in the density and accuracy, the stereo vision in the challenging environment, such as rain, snow and low light conditions, are still open problems and need to be improved. To address this issue, we propose a novel real-time stereo vision algorithm, we transform the rough confidence score of the matching cost to the outlier probability rate and use it to guide the multi-path Viterbi method to estimate the disparity. Our proposed algorithm costs less than 40ms and improves the accuracy of stereo results in difficult scenarios. Real-world experiments show that the proposed method can significantly improve the disparity results in the various challenging environment.