A hierarchical stereo matching algorithm based on adaptive support region aggregation method

With the wide application of artificial intelligence in intelligent driving, visual SLAM technology has become one of the popular research areas. Binocular SLAM, as one of the cheapest cost technologies in SLAM, has become a research hotspot for many scholars. For the problems of high time consumption, poor accuracy and high mis-matching rate of traditional binocular stereo matching algorithm, an adaptive weight cost aggregation D* algorithm based on Dijkstra's idea is proposed. The algorithm improves on the local stereo matching Census algorithm, and fuses the Census algorithm and grayscale histogram in the cost matching stage to make it compatible with computational efficiency and accuracy, which can achieve real-time effect when combined with hardware acceleration. The experimental results show that the improved Census algorithm greatly improves the stereo matching accuracy, and computational efficiency. In comparison with other algorithms on the Middlebury 2.0 testbed, the improved algorithm is found to be 42% more efficient and 1.2% better in terms of mis-matching rate.

The human visual perception uses structural information to recognize stereo correspondences in natural scenes. Therefore, structural information is important to build an efficient stereo matching algorithm. In this paper, we demonstrate that incorporating the structural information similarity, extracted either from image intensity (SSIM) directly or from image gradients (GSSIM), between two patches can accurately describe the patch structures and, thus, provides more reliable initial cost values. We also address one of the major phenomenons faced in stereo matching for real world scenes, radiometric changes. The performance of the proposed cost functions was evaluated within two stages: the first one considers these costs without aggregation process while the second stage uses the fast adaptive aggregation technique. The experiments were conducted on the real road traffic scenes KITTI 2012 and KITTI 2015 benchmarks. The obtained results demonstrate the potential merits of the proposed stereo similarity measurements under radiometric changes.

Human beings process stereoscopic correspondence across multiple purposes like robot navigation, automatic driving, and virtual or augmented reality. However, this bioinspiration is ignored by state-of-the-art dense stereo correspondence matching methods. Cost aggregation is one of the critical steps in the stereo matching method. In this article, we propose an optimized cross-scale cost aggregation scheme with coarse-to-fine strategy for stereo matching. This scheme implements cross-scale cost aggregation with the smoothness constraint on neighborhood cost, which essentially extends the idea of the inter-scale and intra-scale consistency constraints to increase the matching accuracy. The neighborhood costs are not only used in the intra-scale consistency to ensure that the regularized costs vary smoothly in an eight-connected neighbors region but also incorporated with inter-scale consistency to optimize the disparity estimation. Additionally, the improved method introduces an adaptive scheme in each scale with different aggregation methods. Finally, experimental results evaluated both on classic Middlebury and Middlebury 2014 data sets show that the proposed method performs better than the cross-scale cost aggregation. The whole stereo correspondence algorithm achieves competitive performance in terms of both matching accuracy and computational efficiency. An extensive comparison, including the KITTI benchmark, illustrates the better performance of the proposed method also.

Depth information can be obtained using stereo matching algorithms, which compute the horizontal displacement (disparity) of the corresponding points and convert to depth information using the triangular relation. However, the matching process is challenging with presence of textureless regions. This paper proposes a novel disparity refinement method for stereo matching based on Semi-global Matching (SGM) algorithm for texture less images. To be brief, the SGM algorithm is a high-performance matching algorithm of a stereo image pair and it reaches a trade off between matching quality and computing complexity. The main contribution comes from the improvement of matching quality on texture less regions. At the end of SGM, a right-to-left consistency check is performed to remove the invalid pixels in the disparity map. The proposed method is added after the right-to-left consistent check. We assume that the texture less regions in the original stereo pairs are planar. We employ edge detection to extract the textureless regions. The fitting method is employed in horizontal and vertical directions respectively. If the distance of two adjacent edge pixels is sufficiently large, the corresponding line segment will be considered as texturelss. For every texturelss line segment, the RANSAC algorithm is performed on the corresponding line segment in the disparity map. We use the well-known Middlebury dataset to compare our method with the normal SGM and other matching algorithms. It shows that our method performs well for most texture less stereo pairs.

Stereo matching is the key problem in many stereo vison based 3D applications. One of the factors make local stereo matching time-consuming is that every pixel has the same disparity range as the pre-set one, which should be larger than all possible disparities. An improper pre-set disparity range may lead to redundant computation for some pixels and inadequate computation for some others. In this paper, we propose an accurate and fast local stereo matching method, which employs fine disparity estimation and adaptive cost aggregation. The main contributions of our work include two parts. Firstly, we use phase-based correlation to estimate an initial disparity range for block center pixels. Secondly, we estimate a more limited disparity range for every in-block pixel according to its support weight to the block center pixel. Our disparity estimation and block matching techniques can not only reduce the disparity searching range for every pixel, but also can eliminate some pseudo match pairs. Four standard Middlebury stereo image pairs are tested to evaluate the performance of the proposed algorithm. Experimental results show that the proposed algorithm can reduce the matching time by 37.4% on average with relatively higher accuracy compared to traditional method.

In order to reduce the mismatch rate of binocular stereo matching algorithm, a stereo matching algorithm based on improved Census transform is proposed in this paper. Firstly, the classical Census transform is improved, the mean value rather than the center pixel is used as the reference value, and the three states based on the pixels' distances are used to describe the relative relationship between each pixel in the transform window and the reference value; Secondly, the sparse transform window is used to reduce the computational complexity of the algorithm, and the transformed Hamming distance is calculated as the matching cost; Finally, in the maximum disparity range, the points with the least matching cost are selected as the matching points by WTA algorithm, and the disparity is refined by sub-pixel fitting. Experimental results show that the improved Census transformed image has more detailed information in the depth discontinuous region. Compared with the algorithm that directly uses the gray feature as the matching base elements, the proposed algorithm has better operation speed and matching accuracy.

Stereo matching is one of the most active research fields in computer vision. The paper introduces the categories and the performance index of stereo matching and introduces three high-speed and state-of-the-art stereo matching algorithms with different cost aggregation: fast bilateral stereo (FBS), binary stereo matching (BSM), and a non-local cost aggregation method (NLCA). By comparing the performance in terms of both quality and speed, we concluded that FSB deals with the effects of noise well; BSM is suitable for embedded devices and has a good performance with radiometric differences; NLCA combines the efficiency with the accuracy of state-of-the-art algorithms.

Abstract. To automate high-throughput phenotyping for infield biomass sorghum morphological traits characterization, a capable 3D vision system that can overcome challenges imposed by field conditions including variable lighting, strong wind and extreme plant height is needed. Among all available 3D sensors, traditional stereo cameras offer a viable solution to obtaining high-resolution 3D point-cloud data with the use of high-accuracy (sub-pixel) stereo matching algorithms, which, however, are inevitably highly computational. This paper reports a GPU-based parallelized implementation of the PatchMatch Stereo algorithm which reconstructs highly slanted leaf and stalk surfaces of sorghum at high speed from high-resolution stereo image pairs. Our algorithm enhanced accuracy and smoothness by using L2 norm for color distance calculation instead of L1 norm and speeded up convergence by testing the plane of the lowest cost within a local window in addition to the original spatial propagation. To better handle textureless regions, after left-right consistency check, the disparity of an occluded pixel is assigned to that of a nearby non-occluded pixel with the most similar pattern. Some of these occluded pixels in textureless region would survive a following left-right consistency check. Therefore more valid pixels would exist in textureless regions for occlusion filling. Accuracy and performance were evaluated on Middlebury datasets as well as our sorghum datasets. It achieved a high ranking in Middlebury table of subpixel precision and revealed subtle details on leaf and stalk surfaces. The output disparity maps were used to estimate stalk diameters of different varieties and growth stages. The results showed high correlation to hand measurement.

In this paper, we propose a hybrid approach for accurate depth acquisition by using a structured light-based method with a stereo matching. By projecting additional light patterns onto a scene, a structured light-based method works well on a textureless region where a stereo matching shows poor performance. In contrast, the patterns projected onto a rich textured region obstruct in estimating reliable depth information in the structured light-based method, while a stereo matching excels. We exploit these complementary characteristics by combining the results from both methods that outperform the one from either alone. In our fusion framework, a hybrid stereo matching is introduced, in which the disparity search range for each pixel is limited based on the initial depth map obtained by the structured light-based method. In addition, we introduce a confidence-based fusion method which combines the depth maps, while incorporating the advantages of each method. The experimental results show that the proposed method achieves to estimate accurate depth information, while other methods fail.

The stereo matching algorithm is a technique that analyses two or more images captured at diverse view points in order to find positions in real 3D space for the pixels of 2D image. The stereo matching methods have been used in various fields such as drawing the topographical map from aerial photograph and finding the depth information of objects in machine vision system. Nowdays, optical motion capture techniques using stereo matching algorithms are being developed for visual applications such as virtual reality or 3D graphics. Stereo matching algorithms can be generally classified into two methods: feature-based and area-based ones. The feature-based method matches the feature elements between two images, and uses the interpolation to obtain the disparity information for the pixels other than the feature elements, while the area-based method performs matching between pixels in two images by calculating the correlations of the pixels residing in the search window. Area-based method cannot match feature element with more accuracy than feature-based method even though it can make more dense disparity map. Moreover, it has more possibility of error in the area of insufficient texture information or depth discontinuities. In this paper we present a novel technique for area-based stereo matching algorithm which provides more accurate and error-prone matching capabilities by using adaptive search range and window size. We propose two new strategies (1) for determining search range adaptively from the disparity map and multiresolutional images of region segments obtained by applying feature-based algorithm, and (2) for changing the window size adaptively according to the edge information derived from the wavelet transform such that the combination of two adaptive methods in search range and window size greatly enhance accuracy while reducing errors. We test our matching algorithms for various types of images, and shall show the outperformance of our stereo matching algorithm.

This article presents a new stereo matching algorithm based on local method. The absolute difference (AD) algorithm works fast and precise at low texture area, however this algorithm is sensitive to radiometric distortions (i.e., contrast or brightness) and low texture areas. To overcome these problems, the proposed algorithm utilizes a combination of AD, gradient difference (GM) and census transform (CT). The GM algorithm is robust against the radiometric errors and CT algorithm works well on the low texture regions. The proposed algorithm performs much better based on the experimental results of the Middlebury dataset. The disparity map from the result consists of depth information which requires for the three-dimensional (3D) surface reconstruction.

A fast stereo matching algorithm suitable for embedded real-time systems

Efficient local stereo matching algorithm based on fast gradient domain guided image filtering

Stereo matching is the process of finding disparity map between a pair of stereo images. Markov Random Field (MRF) model has been used in stereo matching algorithms in which the goal is to minimize total data, smoothness and occlusion cost. Recent MRF based stereo matching algorithms assign the same smoothness cost to adjacent pixels which is not effective at objects boundaries. In this paper, we propose a method to constrain the smoothness cost such that the cost applied to disparity discontinuity within an object is higher than that of the object's boundary. Data cost function is also normalized before computing its probability distribution function (PDF) to enforce an uniform PDF for the entire image. Experimental results have demonstrated that the proposed method is competitive to the state-of-the-art stereo matching algorithms.

Stacking learning with coalesced cost filtering for accurate stereo matching