An Improved Adaptive Support-weight Sparse Region-based Algorithm for Stereo Matching

The stereo matching algorithm is a technique that analyses two or more images captured at diverse view points in order to find positions in real 3D space for the pixels of 2D image. The stereo matching methods have been used in various fields such as drawing the topographical map from aerial photograph and finding the depth information of objects in machine vision system. Nowdays, optical motion capture techniques using stereo matching algorithms are being developed for visual applications such as virtual reality or 3D graphics. Stereo matching algorithms can be generally classified into two methods: feature-based and area-based ones. The feature-based method matches the feature elements between two images, and uses the interpolation to obtain the disparity information for the pixels other than the feature elements, while the area-based method performs matching between pixels in two images by calculating the correlations of the pixels residing in the search window. Area-based method cannot match feature element with more accuracy than feature-based method even though it can make more dense disparity map. Moreover, it has more possibility of error in the area of insufficient texture information or depth discontinuities. In this paper we present a novel technique for area-based stereo matching algorithm which provides more accurate and error-prone matching capabilities by using adaptive search range and window size. We propose two new strategies (1) for determining search range adaptively from the disparity map and multiresolutional images of region segments obtained by applying feature-based algorithm, and (2) for changing the window size adaptively according to the edge information derived from the wavelet transform such that the combination of two adaptive methods in search range and window size greatly enhance accuracy while reducing errors. We test our matching algorithms for various types of images, and shall show the outperformance of our stereo matching algorithm.

Disparity refinement is an essential step of local stereo matching methods to produce fine dense disparity maps. The inherent defect of local stereo methods results in erroneous disparity in occluded areas. In this paper, we present a novel post processing method which can effectively improve the accuracy of dense disparity maps by rectifying disparity errors iteratively. Invalid disparities are first detected by left-right consistency check and color-disparity consistency check. For each invalid pixel, supports from valid pixels in the neighborhood are collected to determine the plane parameters of the local window. An iterative strategy is adopted to gradually propagate disparity information from valid pixels to invalid areas. We apply the proposed method to disparity maps produced by two recent stereo matching methods, and compare the refining results with other post processing methods. Experimental results show the effectiveness of our method in improving dense disparity maps.

In dependent stereo image compression, the aim is to minimize the bitrate of disparity map and that of residual image. Traditionally, focus has been paid on either disparity map or residual image. In this paper, we compute an optimal disparity map (in terms of bitrates) by jointly exploiting the trade-off between the disparity map and the residual image. Firstly, the dense disparity map is obtained using existing optical flow technique. Secondly, the dense disparity map is quantized using a RD framework. Consequently, the resulting bitrate of the disparity map decreases significantly at the cost of a slight increase of the bitrate of the residual image. As a result, the overall bitrate attains minimum value. The proposed scheme is compatible and can be integrated in JPEG2000 framework.

It is an important problem for computer vision to match the stereo image pair. Only the problems of stereo matching are solved, the accurate location or measurement of object can be realized. In this paper, an integrated stereo matching approach is presented. Unlike most stereo matching approach, it integrates area-based and feature-based primitives. This allows it to take advantages of the unique attributes of each of these techniques. The feature-based process is used to match the image feature. It can provide a more precise sparse disparity map and accurate location of discontinuities. The area-based process is used to match the continuous surfaces.It can provide a dense disparity map. The techniques of stereo matching with adaptive window are adopted in the area-based process. It can make the results of area-based process get high precise. An integrated process is also used in this approach. It can integrate the results of feature-based process and area-based process, so that the approach can provide not only a dense disparity map but also an accurate location of discontinuities. The approach has been tested by some synthetic and nature images. From the results of matched wedding cake and matched aircraft model, we can see that the surfaces and configuration are well reconstructed. The integrated stereo matching approach can be used in 3D part recognition in intelligent assembly system and computer vision.

The stereo matching <span>process is one of the key areas that impact the stereo vision technologies which are commonly used in the application of three-dimensional reconstructions. The accuracy of the depth information used in three-dimensional reconstruction is directly proportional to the accuracy of the disparity obtained from stereo matching. The challenging issue in the stereo matching process is to determine the accurate corresponding point between the left image and right image, especially for image pairs that have different exposure such as different illumination and image pair with less texture region. In order to increase the accuracy of disparity value, a new stereo matching algorithm is proposed based on the combination of Sum of absolute different and census transform at matching cost computation. guided filter was used in the matching cost aggregation in order to remove noise and preserve the edge of the image. In the optimization step, the winner take all strategy is used to select the minimum matching cost. Finally, a median filter is applied to the initial disparity map for refinement purposes. The experimental results show that the algorithm is effective in reducing the error and improving the accuracy of the disparity map in different illumination regions, less textured regions and different environmental exposure.</span>

Aiming at the problem that the similarity measures in the overwhelming majority of stereo matching approaches rely heavily on the statistical characteristics of image gray values,a stereo matching algorithm insensitive to differentiations in illumination is proposed.Firstly,the Census non-parametric transform is investigated and its limitations are analyzed. Secondly,in order to take the pixels' spatial location information into consideration when searching stereo correspondences, gray values of the neighborhood pixels,whose relative positions are one unit greater than the center pixel in the transformation window,are replaced by gray values' interpolation with the four pixels adjacent to them.Finally,the Census non-parametric transform of images and stereo matching are implemented,and then,dense disparity map is obtained.The experiment results show that the proposed method has simple mechanism,high computational efficiency and strong robustness;satisfactory stereo matching results also can be obtained even under non-ideal illumination conditions.

Three-dimensional reconstruction of internal organ surfaces provides useful information for better control and guidance of the operations of surgical tools for minimally invasive surgery (MIS). The current reconstruction techniques using stereo cameras are still challenging due to the difficulties in correspondence matching in MIS, since there is very limited texture but significant specular reflection on organ surfaces. This paper proposes a new approach to overcome the problem by introducing weakly structured light actively casting surgical tool shadows on organ surfaces. The contribution of this paper is twofold: first, we propose a robust approach to extract shadow edges from a sequence of shadowed images; second, we develop a novel field surface interpolation (FSI) approach to obtain an accurate and dense disparity map. Our approach does not rely on texture information and is able to reconstruct accurate 3-D information by exploiting shadows from surgical tools. One advantage is that the point correspondences are directly calculated and no explicit stereo matching is required, which ensures the efficiency of the method. Another advantage is the minimum hardware requirement because only stereo cameras and a separated single-point light source are required. We evaluated the proposed approach using both phantom models and ex vivo images. Based on the experimental results, we achieved the precision of the recovered 3-D surfaces within 0.7 mm for phantom models and 1.2 mm for ex vivo images. The comparison of disparity maps indicates that with the addition of shadows, the proposed method significantly outperforms the state-of-the-art stereo algorithms for MIS.

Stereo matching is one of the research directions of binocular stereo vision system. In this paper, a novel adaptive local stereo matching algorithm is proposed. Firstly, the cost function calculation is redefined according to the gray similarity and the spatial similarity. Then, the data smoothing term is introduced and the penalty coefficient is determined by the local entropy of the image. Finally, the initial disparity map of the cost aggregation is analyzed by using the left-right consistency check and weight median filter to perform disparity refinement. The algorithm uses four international standard stereo image pairs provided by Middlebury website. The experiment shows that the proposed algorithm can get more accurate disparity map.

In stereo vision, disparity map after stereo matching using projection model can get the depth information and 3D information of the original image. In order to improve the accuracy and speed of stereo matching, improving the quality of disparity map becomes the core problem of stereo matching. Firstly, the paper extracted edge information of binocular images using Canny, Scharr and Sobel edge detection algorithm. On this basis, the paper use SGBM algorithm(Semi-global block Matching) and DP algorithm(Dynamic Programming)and CSCA algorithm(Cross-scale Cost Aggregation) respectively calculated the disparity map and its error rate. After analysis, the combination of Canny detection algorithm and CSCA algorithm can eliminate a large number of number of irrelevant information and reduce the amount of data processing. The real scene image is selected to validate the method .Based on the above research results, the method improved the quality of disparity map.

In this paper, we propose a stereo matching method based on adaptive plane-fitting. In the proposed method, we use the disparity map adjustment with adaptive weight filter to preserve the boundary region. To estimate the initial disparity map, we use two cost aggregation functions, and in addition to that, we combined color and gradient for cost computation. In addition, the error value of the initial disparity map is calibrated by applying plane-fitting using mean-shift segmentation. The disparity map is adjusted by using adaptive weights to improve the accuracy of the disparity map. Experimental results show that the proposed method generates a high-quality disparity map which has the distinct boundary between objects compared to other methods.

This paper presents a new algorithm of disparity map estimation. The originality of this method lies in the process of dense disparity map estimation using dynamic programming constrained by interest points and using the higher order statistics (HOS) criteria for matching noisy images. Experiments with noisy real images have validated our method and have clearly shown the improvement over the existing ones. The dense disparity map obtained is more reliable when compared to the similar second-order statistics (SOS)-based dynamic programming and HOS-based correlation methods.

As-planar-as-possible depth map estimation

This paper describes a trinocular stereo vision system using a single chip of FPGA to generate the composite color (RGB) and disparity data stream at video rate, called the RGBD imager. The system uses the triangular configuration of three cameras for synchronous image capture and the trinocular adaptive cooperative algorithm based on local aggregation for smooth and accurate dense disparity mapping. We design a fine-grain parallel and pipelining architecture in FPGA for implementation to achieve a high computational and real-time throughput. A binary floating-point format is customized for data representation to satisfy the wide data range and high computation precision demands in the disparity calculation. Memory management and data bit-width control are applied in the system to reduce the hardware resource consumption and accelerate the processing speed. The system is able to produce dense disparity maps with 320 × 240 pixels in a disparity search range of 64 pixels at the rate of 30 frames per second.

This paper proposes a new hybrid method between the learning-based and handcrafted methods for a stereo matching algorithm. The main purpose of the stereo matching algorithm is to produce a disparity map. This map is essential for many applications, including three-dimensional (3D) reconstruction. The raw disparity map computed by a convolutional neural network (CNN) is still prone to errors in the low texture region. The algorithm is set to improve the matching cost computation stage with hybrid CNN-based combined with truncated directional intensity computation. The difference in truncated directional intensity value is employed to decrease radiometric errors. The proposed method’s raw matching cost went through the cost aggregation step using the bilateral filter (BF) to improve accuracy. The winner-take-all (WTA) optimization uses the aggregated cost volume to produce an initial disparity map. Finally, a series of refinement processes enhance the initial disparity map for a more accurate final disparity map. This paper verified the performance of the algorithm using the Middlebury online stereo benchmarking system. The proposed algorithm achieves the objective of generating a more accurate and smooth disparity map with different depths at low texture regions through better matching cost quality.

In this paper, we propose a disparity refinement method that directly refines the winner-take-all (WTA) disparity map by exploring its statistical significance. According to the primary steps of the segment-based stereo matching, the reference image is over-segmented into superpixels and a disparity plane is fitted for each superpixel by an improved random sample consensus (RANSAC). We design a two-layer optimization to refine the disparity plane. In the global optimization, mean disparities of superpixels are estimated by Markov random field (MRF) inference, and then, a 3D neighborhood system is derived from the mean disparities for occlusion handling. In the local optimization, a probability model exploiting Bayesian inference and Bayesian prediction is adopted and achieves second-order smoothness implicitly among 3D neighbors. The two-layer optimization is a pure disparity refinement method because no correlation information between stereo image pairs is demanded during the refinement. Experimental results on the Middlebury and KITTI datasets demonstrate that the proposed method can perform accurate stereo matching with a faster speed and handle the occlusion effectively. It can be indicated that the "matching cost computation + disparity refinement" framework is a possible solution to produce accurate disparity map at low computational cost.