Locally Consistent ToF and Stereo Data Fusion

Extracting a computer model of a real scene from a sequence of views, is one of the most challenging and fundamental problems in computer vision. Stereo vision algorithms allow us to extract from the images a sparse 3D point cloud on the scene surfaces. However, computing an accurate mesh of the scene based on such poor quality data points (noise, sparsity) is very difficult. Here we describe a simple yet original approach that uses both the stereo vision extracted point cloud and the calibrated images. Our method is a three-stage process in which the first stage merges, filters and smoothes the input 3D points. The second stage builds for each calibrated image a triangular depth-map and fuses the set of depth-maps into a triangle soup that minimize violations of size and visibility constraints. Finally, a mesh is computed from the triangle soup using a reconstruction method that combines restricted Delaunay triangulation and Delaunay refinement.

Recovery of depth information for 3-D modeling of surfaces and objects is a central problem in low-level computer vision. A number of applications related to reconstruction, representation, and recognition of scenes and objects in the three-dimensional geometry of the real world rely on the correct estimation of range data. In stereo vision, depth information is obtained from triangulation of corresponding points in a pair of stereo images. The automatic determination of local similarity between two data sets or collections of pixels is fundamental in a depth-algorithm from stereo algorithm for numerical evaluation of disparities. In this paper, the use of the Cepstrum transformation as a disparity measurement technique between corresponding windows of different block sizes is described. This measurement process is embedded within a coarse to fine stereopsis algorithm, providing an initial depth map with the depth information encoded as gray levels. In real stereo imagery the features to be used as information in the matching procedure are usually sparsely and irregularly distributed over the images. For that reason, obtention of a dense array of depth data is a difficult task in passive methods. This paper makes emphasis in surface recovery of smooth curved surfaces with application to depth estimation of medical images. Examples and experimental results using random dot stereograms and real imagery are presented.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

Stereo vision is an important computer vision technique which estimates depth information from two images - similar to how human vision works. However, the computational intensity of stereo vision deters implementation in real-time systems. Recently, massively parallel processors called Graphics Processing Units or GPUs are gaining traction as an acceleration platform to offload compute-intensive tasks to. This paper presents the development of a GPU-accelerated stereo vision system called DepthStream. The DepthStream algorithm uses a combined absolute difference and census cost, cross-based cost aggregation and bitwise fast region voting to achieve error rates up to 3.7× lower than Block-Matching based algorithms in mainstream libraries. Leveraging the parallel processing power that GPUs offer enables the DepthStream algorithm to achieve processing times up to 3.12× faster than Semi-Global Matching based implementations in the same libraries. The study also includes measurements of the accuracy of the estimated distances of the developed stereo vision system. Results show that the estimated depths of the DepthStream stereo vision system are within 4% of the true value for distances less than 6m. Statistical analysis also indicate that there is no significant difference between the stereo vision systems estimated distances and the actual value.

Current developments in the field of automated assembly systems show an increasing interest in systems that are flexible in both CAD based product design and CAD based assembly. For the application addressed in this paper, coupling the vision system and a CAD database is of prime importance in order to achieve the required automatic reconfiguration of the assembly cell when new parts are defined. This paper presents a 3D CAD-based vision system for obtaining 3D data about the scene. After the images are acquired, edge detection is preformed and the detected edges are stored as chaincodes. Following that, a stereo vision algorithm is applied for finding the recognition features. The output are lists of features that are combined into a 3D wireframe representing the scene. The recognition algorithm takes the observed wireframe outputs from the stereo vision system, and compares them with a set of model wireframes derived from previous models, in order to select the 'best match', where the previous models used for recognition are derived from a product data model (PDM). The PDM is an interface between the CAD database and the recognition system, which allows the automatic generation of new models when new parts are introduced into the system. The vision system described in this paper is part of an intelligent robotic assembly cell, where the aim is to build a flexible intelligent robotic assembly cell, such that robots would be able to automatically assemble a random variety of small- batch products.

Abstract. Nowadays time-of-flight (ToF) cameras and multiple RGB cameras are being embedded in an increasing number of high-end smartphones: despite their integration in mobile devices is mostly motivated by photographic applications, their availability can be exploited to enable 3D reconstructions directly on smartphones. Furthermore, even when a ToF camera is not embedded in a smartphone, low cost solutions are available on the market in order to easily provide standard mobile devices with a lightweight and extremely portable ToF camera. This work deals with the assessment of a low cost ToF camera, namely Pico Zense DCAM710, which perfectly fits with the above description. According to the results obtained in the considered tests, the ranging error (precision) of the DCAM710 camera increases linearly approximately up to the nominal maximum range in the considered working mode, up to approximately 1 cm. Despite the device allows to acquire measurements also at larger ranges, the measurement quality significantly worsen. After assessing the main characteristics of such ToF camera, this paper aims at comparing its 3D reconstruction ability with that of a smartphone stereo vision system. In particular, the comparison of a 3D reconstruction obtained with stereo vision from images acquired with an LG G6 shows that the stereo reconstruction leads to a much larger point cloud. However, points generated by the ToF camera are more homogeneously distributed, and they seem to slightly better describe the real geometry of the reconstructed object. The combination of such two technologies, which will be investigated in our future work, can potentially lead to a denser cloud with respect to the ToF camera, while preserving a reasonable accuracy.

NASA scenarios for lunar and planetary missions include robotic vehicles that function in both teleoperated and semi-autonomous modes. Under teleoperation, on-board stereo cameras may provide 3-D scene information to human operators via stereographic displays; likewise, under semi-autonomy, machine stereo vision may provide 3-D information for obstacle avoidance. In the past, the slow speed of machine stereo vision systems has posed a hurdle to the semi- autonomous scenario; however, recent work at JPL and other laboratories has produced stereo vision systems with high reliability and near real-time performance for low-resolution image pairs. In particular, JPL has taken a significant step by achieving the first autonomous, cross- country robotic traverses (of up to 100 meters to use stereo vision, with all computing on- board the vehicle. This paper describes the stereo vision system, including the underlying statistical model and the details of the implementation. The statistical and algorithmic aspects employ random field models of the disparity map, Bayesian formulations of single-scale matching, and area-based image comparisons. The implementations builds Laplacian image pyramids and produces disparity maps from the 60X64 level of the pyramids at rates of up to two seconds per image pair. All vision processing is done on one 68020 augmented with Datacube image processing boards. The author argues that the overall approach provides a unifying paradigm for practical, domain-independent stereo ranging.

This paper presents a multiple point hierarchical approach to the stereo correspondence problem in computer vision. The low-level processing employs an area-and-token hybrid method to obtain, for distinctive points in one image, a set of points in the other image that are candidates for correspondence. The refinement of the set of low-level correspondences obtained is performed by a high-level N-point simultaneous correspondence process, a new constraint introduced for this problem. The high-level processing uses a genetic algorithm approach for searching the solution space. Experimental results show the effectiveness of the method on real world scenes.

Since 3D data has additional depth information than 2D image, it can be used various researches and applications such as robot vision, car navigation and human computer interaction. Especially, 3D face recognition is robust against facial pose and light variation since it can use 3D face shape information such as facial profile, curvature and depth map. However, it is difficult to acquire 3D face data in real-time since conventional 3D data acquisition systems like laser scanner or structured light system are very slow for the data capturing. Recently, Time-of-Flight (ToF) camera provides full range distance data at high framerates and thus comes into the spotlight as an alternative to conventional 3D acquisition systems. ToF camera provides both distance images and gray-scale images simultaneously in realtime. However, it has drawbacks which are low resolution and gray-scale image, not color image. In this paper, a registration method of color and ToF cameras is presented to find precise corresponding color pixel information with range distance data from ToF camera.

The focus settings and physical characteristics of lenses significantly affect distortion correction, particularly under blurred imaging conditions, thereby limiting the measurement accuracy of existing stereo vision systems. Moreover, traditional calibration processes for stereo vision systems are complex due to the need for precise focus distance adjustments and careful calibration board selection. In this paper, we propose a camera parametric model based on weighted radial constraints that accounts for both radial and non-central lens distortion. This model aims to improve calibration accuracy while simplifying the calibration workflow. Additionally, we introduce a stereo vision correction method that utilizes speckle patterns as calibration images and eliminates the requirement for precise camera focusing. This approach is more user-friendly than conventional planar chessboard techniques, as it avoids frequent lens adjustments for optimal focus. Experimental results demonstrate improvements in accuracy of 17.2% and 15.6% over Zhang’s and Tsai’s calibration methods, respectively.

Self-supervised learning as an enabling technology for future space exploration robots: ISS experiments on monocular distance learning

Computer stereo vision has been paid high attention in areas such as industry measurement and automatic control. Calibration of stereo vision system is a key step to implement vision measurement. Nonlinear calibration method involves too many parameters and too complex function, so the processing of optimization is difficult and slow. In this paper, two digital cameras are used to set up stereo vision system which provides convenient experimental operation. The binocular stereo vision system is calibrated using linear calibration model. The results indicate that the method is stable and fast with high precision and practicability.

Significant obstacle location with ultra-wide FOV LWIR stereo vision system

The stereo vision system is one of the most important sensors for 3D reconstruction of the environment and scene understanding. It plays an essential role for the automated driving and ADAS applications. Although recently great progress has been made in this field both in the density and accuracy, the stereo vision in the challenging environment, such as rain, snow and low light conditions, are still open problems and need to be improved. To address this issue, we propose a novel real-time stereo vision algorithm, we transform the rough confidence score of the matching cost to the outlier probability rate and use it to guide the multi-path Viterbi method to estimate the disparity. Our proposed algorithm costs less than 40ms and improves the accuracy of stereo results in difficult scenarios. Real-world experiments show that the proposed method can significantly improve the disparity results in the various challenging environment.

This paper describes the development of a practical stereo vision system which is intended for use in visual tracking of man-made targets from a hovering subsea vehicle. Blob tracking, centroid computation, camera calibration, and stereo vision algorithms are combined to form a robust real-time implementation requiring minimal human intervention. The cameras are calibrated using images of a manipulator-mounted target. Using a less-than-state-of-the-art workstation (Sun IPX) without any special-purpose hardware, targets within a 400/spl times/150/spl times/400 millimeter viewing volume are tracked at rates exceeding 1 Hz (including the acquisition of 3 512/spl times/512 images) with a worst-case accuracy of better than 3 mm for both two- and three-camera configurations.

Verification of computer vision theories is facilitated by the development and implementation of computer simulation systems. Computer simulation avoids the necessity of building actual camera systems; they are fast, flexible, and can be easily duplicated for use by others. In our previous work, we proposed a useful computational model to explore the image sensing process. This model decouples the photometric information and the geometric information of objects in the scene. In this paper, we further extend the proposed image sensing model to simulate the image formation of moving objects (motion) and stereo vision system. The simulation algorithms for curved objects, moving objects, and stereo imaging are presented. Based on the proposed model and algorithms, a computer simulation system called Active Vision Simulator (AVS) has been implemented. AVS can be used to simulate image formation process in a monocular (MONO mode) or binocular (STEREO mode) camera system to synthesize the images. It is useful for research on image restoration, motion analysis, depth from defocus, and algorithms for solving the correspondence problem in stereo vision. The implementation of AVS is efficient, modular, extensible, and user-friendly.© (1994) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.