Optimal optical flow based disparity map estimation for lossless stereo image coding

Disparity refinement is an essential step of local stereo matching methods to produce fine dense disparity maps. The inherent defect of local stereo methods results in erroneous disparity in occluded areas. In this paper, we present a novel post processing method which can effectively improve the accuracy of dense disparity maps by rectifying disparity errors iteratively. Invalid disparities are first detected by left-right consistency check and color-disparity consistency check. For each invalid pixel, supports from valid pixels in the neighborhood are collected to determine the plane parameters of the local window. An iterative strategy is adopted to gradually propagate disparity information from valid pixels to invalid areas. We apply the proposed method to disparity maps produced by two recent stereo matching methods, and compare the refining results with other post processing methods. Experimental results show the effectiveness of our method in improving dense disparity maps.

The stereo matching algorithm is a technique that analyses two or more images captured at diverse view points in order to find positions in real 3D space for the pixels of 2D image. The stereo matching methods have been used in various fields such as drawing the topographical map from aerial photograph and finding the depth information of objects in machine vision system. Nowdays, optical motion capture techniques using stereo matching algorithms are being developed for visual applications such as virtual reality or 3D graphics. Stereo matching algorithms can be generally classified into two methods: feature-based and area-based ones. The feature-based method matches the feature elements between two images, and uses the interpolation to obtain the disparity information for the pixels other than the feature elements, while the area-based method performs matching between pixels in two images by calculating the correlations of the pixels residing in the search window. Area-based method cannot match feature element with more accuracy than feature-based method even though it can make more dense disparity map. Moreover, it has more possibility of error in the area of insufficient texture information or depth discontinuities. In this paper we present a novel technique for area-based stereo matching algorithm which provides more accurate and error-prone matching capabilities by using adaptive search range and window size. We propose two new strategies (1) for determining search range adaptively from the disparity map and multiresolutional images of region segments obtained by applying feature-based algorithm, and (2) for changing the window size adaptively according to the edge information derived from the wavelet transform such that the combination of two adaptive methods in search range and window size greatly enhance accuracy while reducing errors. We test our matching algorithms for various types of images, and shall show the outperformance of our stereo matching algorithm.

Stereo matching is to obtain disparity map from two or more images whose pixels existing disparity between each other. Aim at the difficulty of meeting both accuracy and speed when using the adaptive support-weight algorithm, an improved adaptive support-weight sparse region-based algorithm accorded with HVS is proposed. First, improve the traditional support-weight formula, and the support-weight of the points can be calculated according to the improved support-weight formula. Second, the dense disparity map is obtained by sparse region-based matching. Third, left-right consistency check and blocking filling is performed for the obtained disparity map. Finally, median filter is used to remove isolated mismatching points and noise points. Experiment results show that, using the presented method, the matching efficiency is above 90 times faster than that of the adaptive support-weight algorithm proposed by Yoon; and the matching accuracy is 12.34% higher than that of SSD. So the improved algorithm is verified for obtaining accurate disparity map fast and meeting the requirements of system practicability.

It is an important problem for computer vision to match the stereo image pair. Only the problems of stereo matching are solved, the accurate location or measurement of object can be realized. In this paper, an integrated stereo matching approach is presented. Unlike most stereo matching approach, it integrates area-based and feature-based primitives. This allows it to take advantages of the unique attributes of each of these techniques. The feature-based process is used to match the image feature. It can provide a more precise sparse disparity map and accurate location of discontinuities. The area-based process is used to match the continuous surfaces.It can provide a dense disparity map. The techniques of stereo matching with adaptive window are adopted in the area-based process. It can make the results of area-based process get high precise. An integrated process is also used in this approach. It can integrate the results of feature-based process and area-based process, so that the approach can provide not only a dense disparity map but also an accurate location of discontinuities. The approach has been tested by some synthetic and nature images. From the results of matched wedding cake and matched aircraft model, we can see that the surfaces and configuration are well reconstructed. The integrated stereo matching approach can be used in 3D part recognition in intelligent assembly system and computer vision.

This paper presents a new algorithm of disparity map estimation. The originality of this method lies in the process of dense disparity map estimation using dynamic programming constrained by interest points and using the higher order statistics (HOS) criteria for matching noisy images. Experiments with noisy real images have validated our method and have clearly shown the improvement over the existing ones. The dense disparity map obtained is more reliable when compared to the similar second-order statistics (SOS)-based dynamic programming and HOS-based correlation methods.

Many research works have been developed for stereo image compression purpose where most of them aim at encoding a reference image, a residual one and a disparity map. While the disparity field is often losslessly encoded, we are mainly interested in this paper in the bit allocation problem between the reference and residual images. Generally, the bit allocation is expressed as an optimization problem which involves the computation of the operational rate-distortion (RD) functions for all the wavelet subbands and for different quantization steps. However, this strategy is computationally intensive. To solve this problem, we consider the uniform scalar quantization of the wavelet subbands of both images modeled by a Generalized Gaussian distribution. Thanks to recent approximations of the entropy and distortion functions, we develop an optimal and fast bit allocation method. The obtained results confirm the efficiency of the proposed bit allocation method in the context of stereo image coding.

Three-dimensional reconstruction of internal organ surfaces provides useful information for better control and guidance of the operations of surgical tools for minimally invasive surgery (MIS). The current reconstruction techniques using stereo cameras are still challenging due to the difficulties in correspondence matching in MIS, since there is very limited texture but significant specular reflection on organ surfaces. This paper proposes a new approach to overcome the problem by introducing weakly structured light actively casting surgical tool shadows on organ surfaces. The contribution of this paper is twofold: first, we propose a robust approach to extract shadow edges from a sequence of shadowed images; second, we develop a novel field surface interpolation (FSI) approach to obtain an accurate and dense disparity map. Our approach does not rely on texture information and is able to reconstruct accurate 3-D information by exploiting shadows from surgical tools. One advantage is that the point correspondences are directly calculated and no explicit stereo matching is required, which ensures the efficiency of the method. Another advantage is the minimum hardware requirement because only stereo cameras and a separated single-point light source are required. We evaluated the proposed approach using both phantom models and ex vivo images. Based on the experimental results, we achieved the precision of the recovered 3-D surfaces within 0.7 mm for phantom models and 1.2 mm for ex vivo images. The comparison of disparity maps indicates that with the addition of shadows, the proposed method significantly outperforms the state-of-the-art stereo algorithms for MIS.

3D reconstruction is the base of mobile robot navigation. And it is necessary to provide the environment information for robots effectively and rapidly. The traditional 3D reconstruction method relies on the dense disparity map which requires high computing ability, and there is large redundant data in the reconstructed 3D data. Unlike traditional methods, the proposed method needn't obtain the disparity map. We partitioned the scene scope uniformly into grids, and introduced virtual height lines(VHLs) which do not exist in the scene. Each VHL is projected in left image and right image. The intersection of VHL and the scene is only existent on projection line. So the point has maximum similarity metric. We used dynamic programming(DP) to find the matched lines in left image corresponding to the projections of VHLs(PVHLs) in right image. The intersection point of the matched line and the PVHL in left image is the answer. The experiments to reconstruct 3D scene are performed and the results show the proposed method is reliable and effective.

An algorithm for disparity estimation with energy-based variational regularization using three image views is presented. In the new algorithm, dense disparity maps, including the disparity values for feature points, are first estimated by phase-based methods. Then a set of coupled partial differential equations (PDEs) are solved to refine these disparity maps together with the feature information. The visual appearance of the refined disparity is better than that of the results using similar variational regularization involving only one stereo pair of images. In addition, a new effective occlusion detection scheme is presented with good results.

Dense stereo image matching in remotely sensed images is a challenging problem, though it has been studied for more than two decades, due to occlusions, discontinuities, geometric, and radiometric distortions. A novel multiobjective fitness function-based dense stereo matching approach using genetic algorithms (GAs) is proposed in this paper. The proposed method is useful for estimating dense disparity map with an improved number of inliers for a stereo image pair, despite the constraint of finding correct disparity at depth discontinuities. In this paper, the steps of GA, such as initialization of the population, fitness function, and crossover and mutation operation, are designed and implemented to effectively deal with the problem of dense stereo image matching. To initialize the population, a Scale Invariant Feature Transform (SIFT) descriptor is computed for each pixel and multiple-size window-based matching is performed, using the similarity measures: 1) Euclidean distance and 2) spectral angle mapper. The generated disparity maps are pruned to choose a suitable subset using the designed fitness functions, considering the constraints related to stereo image pair, such as epipolar constraint, which encodes the epipolar geometry and the similarity measure that is useful to decide accuracy of the correspondences. The two objective functions are the number of inliers computed using the fundamental matrix and an energy minimization function, considering discontinuities and occlusions. The usefulness of this approach for remotely sensed stereo image pairs is demonstrated by improving the number of inliers and favorably comparing with state-of-the-art dense stereo image matching methods.

Unsupervised learning of depth estimation from imperfect rectified stereo laparoscopic images

A binocular stereo system for images coded by visual pattern image coding (VPIC) is presented and evaluated. The use of VPIC for stereo vision applications is motivated by the capability of its coding primitives to reflect meaningful physical properties of projected real-scene surfaces: high-information edge regions and uniform regions. An algorithm for spatial matching of VPIC primitives is proposed. It involves the solution of a complex optimization problem satisfying VPIC-related constraints. A dense disparity map is obtained and then used, together with one of the VPIC coded images of the stereo pair, to predict the second stereo image. The proposed stereo system allows a very high compression of the overall stereo information. Experimental results are presented to illustrate the performance of both stereo VPIC matching and predicting algorithms in natural images.

A binocular stereo system for images coded by visual pattern image coding (VPIC) is presented and evaluated. The use of VPIC for stereo vision applications is motivated by the capability of its coding primitives to reflect meaningful physical properties of projected real-scene surfaces: high-information edge regions and uniform regions. An algorithm for spatial matching of VPIC primitives is proposed. A dense disparity map is obtained and then used, together with one of the VPIC coded images of the stereo pair, to predict the second stereo image. The proposed stereo system allows a very high compression of the overall stereo information.

Stereo vision is a fundamental method for measuring 3D structures by observing them from two cameras placed on different positions. In order to reconstruct 3D structures, it is necessary to create a disparity map from a pair of stereo images. To create a disparity map we compute the matching cost for each point correspondence and compute the disparity that minimizes the sum of the whole matching costs. In this paper, we propose a method to estimate a dense disparity map using region segmentation. We segment each scanline using region homogeneity properties. Using the segmented regions, we prohibit false matches in the stereo matching process. Disparities for pixels that failed in matching are filled by interpolating neighborhood disparities. We applied the proposed method to various stereo images of real environments. Experimental results showed that the proposed method is stable and potentially viable in practical applications.

Binocular stereo vision processes estimate 3D surfaces using a pair of images taken from different points of view. Three-dimensional surface characteristics are estimated by matching 2D image areas or features corresponding to the projections of the same 3D points. The most classic area-based methods used cross correlation with a fixed window-size, but this technique presents a major drawback: the computation of depth is generally prone to errors close to surface discontinuities. In this paper, we present our current work on aerial stereo images of urban areas. A correlation-based algorithm, with an adaptive window-size constrained by an edge map extracted from the images, is presented. This technique is not used to refine an initial disparity map but to create one with good discontinuities location. The algorithm follows three steps: first, window sizes for each pixel are computed; second, a disparity map is created; third, map completion is performed and a final dense disparity map with subpixel precision is produced by Kanade correction. Experimental results on real aerial images are presented.