Improving stereo matching algorithm with adaptive cross-scale cost aggregation

Computational complexity of the local stereo matching method is affected by disparity ranges. In case of depth estimation in sequential stereo images, high computational complexity is a problem in terms of real-time processing. In this paper, we propose a temporal correlation based stereo matching method in sequential images. Using temporal information in a sequential stereo matching method provides inaccurate disparity ranges, since the estimated depth map accuracy is gradually degraded. To preserve the depth map quality in temporal stereo matching procedure, we adopt the guided image filtering for matching cost aggregation. Since the guided image filtering has a similar structure with bilateral filter, it preserves an object boundary region even in restricted disparity search ranges. Inaccurately estimated disparity values from the temporal correlation are compensated by filtering based cost aggregation method. From the experiment results, we check that the proposed depth map acquisition method preserves the depth map quality in temporal domain stereo matching.

Stereo matching is the key problem in many stereo vison based 3D applications. One of the factors make local stereo matching time-consuming is that every pixel has the same disparity range as the pre-set one, which should be larger than all possible disparities. An improper pre-set disparity range may lead to redundant computation for some pixels and inadequate computation for some others. In this paper, we propose an accurate and fast local stereo matching method, which employs fine disparity estimation and adaptive cost aggregation. The main contributions of our work include two parts. Firstly, we use phase-based correlation to estimate an initial disparity range for block center pixels. Secondly, we estimate a more limited disparity range for every in-block pixel according to its support weight to the block center pixel. Our disparity estimation and block matching techniques can not only reduce the disparity searching range for every pixel, but also can eliminate some pseudo match pairs. Four standard Middlebury stereo image pairs are tested to evaluate the performance of the proposed algorithm. Experimental results show that the proposed algorithm can reduce the matching time by 37.4% on average with relatively higher accuracy compared to traditional method.

Hardware platforms with limited processing power are often incapable of running dense stereo analysis algorithms at acceptable speed. Sparse algorithms provide an alternative but generally lack in accuracy. To overcome this predicament, we present an efficient sparse stereo analysis algorithm that applies a dense consistency check, leading to accurate matching results. We further improve matching accuracy by introducing a new feature detector based on FAST, which exhibits a less clustered feature distribution. The new feature detector leads to a superior performance of our stereo analysis algorithm. Performance evaluation shows that the proposed stereo matching system achieves processing rates above 200 frames per second on a commodity dual core CPU, and faster than video frame-rate processing on a low-performance embedded platform. The stereo matching results prove to be superior to those obtained with ordinary sparse matching algorithms.

It's a difficult problem in three-dimensional visual measurement to improve the precision and accuracy of stereo matching. In this paper, a thin line power increase (TLPI) stereo matching method used in high-precision three-dimensional visual measurement is designed. It's a coding and stereo matching method which combines the second power increases method, the step moving method, the difference shadow method, the progressive decrease method and the stereo matching method. Through the combination of the second power increases method and the step moving method, the projectile optical information can cover all the measured object space. Through the difference shadow method, the background interference of the measured space can be wiped off. Through the progressive decrease method, the searching range of the optical information can be narrowed; the speed and the precision of optical information matching will be improved. Through the stereo matching method, the three-dimensional coordinate of object can be calculated. The thin line power increase method designed by this paper can solve the problem of misjudgment and improve the precision and reliability of three-dimensional measurement.

An asymmetric post-processing for correspondence problem

Although convolutional neural network (CNN)-based stereo matching methods have become increasingly popular thanks to their robustness, they primarily have been focused on the matching cost computation. By leveraging CNNs, we present a novel method for matching cost aggregation to boost the stereo matching performance. Our insight is to learn the convolution kernel within CNN architecture for cost aggregation in a fully convolutional manner. Tailored to cost aggregation problem, our method differs from handcrafted methods in terms of its convolutional aggregation through optimally learned CNNs. First, the matching cost is aggregated with cost volume unary network, and then optimized with explicit disparity boundary, estimated through disparity boundary pairwise network, within a global energy minimization. Experiments demonstrate that our method outperforms conventional hand-crafted aggregation methods.

Stereo matching is a branch of 3D vision and has a wide range of applications in 3D reconstructionand autonomous driving. Recently, stereo matching methods leverage all the information of the stereo image to calculate disparity map. However, these methods still have difficulties in texture-less areas and occlusion areas, and post-processing improves accuracy. Therefore, there is a high computational cost in feature extraction and post-processing. In this paper, we propose a stereo matching method based on features of image patches to predict the disparity of non-occlusion areas instead of full image features. And aggregation methods are performed to modify all kinds of mismatching pixels based on the correct disparity in the non-occlusion areas. Furthermore, we evaluated the proposed method on the Middlebury dataset. The result shows that the proposed method performs well in all areas.

With the wide application of artificial intelligence in intelligent driving, visual SLAM technology has become one of the popular research areas. Binocular SLAM, as one of the cheapest cost technologies in SLAM, has become a research hotspot for many scholars. For the problems of high time consumption, poor accuracy and high mis-matching rate of traditional binocular stereo matching algorithm, an adaptive weight cost aggregation D* algorithm based on Dijkstra's idea is proposed. The algorithm improves on the local stereo matching Census algorithm, and fuses the Census algorithm and grayscale histogram in the cost matching stage to make it compatible with computational efficiency and accuracy, which can achieve real-time effect when combined with hardware acceleration. The experimental results show that the improved Census algorithm greatly improves the stereo matching accuracy, and computational efficiency. In comparison with other algorithms on the Middlebury 2.0 testbed, the improved algorithm is found to be 42% more efficient and 1.2% better in terms of mis-matching rate.

In the past three years, the stereo matching method based on 3D CNNs has achieved surprising results and has received more and more attention. However, most stereo matching approaches aim to improve prediction accuracy by constructing and aggregating cost volumes through extensive 3D convolutions, which not only does not fully utilize the geometric information, but also overlooks the computational speed. Thus, achieving high-accuracy, high-efficiency stereo matching has become challenging. In this paper, we present a rapid and precise stereo matching network named CGW based on 3D CNNs, which simultaneously achieves real-time functioning, considerable accuracy, and a strong generalization capability. The network is divided into two parts. The first part constructs the geometric attention cube through a lightweight feature extraction network and a lightweight 3D regularization network. The second part filters the context features using the geometric attention cube to obtain the context geometric cube, and finally, the disparity is predicted and refined to obtain the final disparity. We adopted MobileNetV3 as an efficient backbone for feature extraction and designed 3D depthwise separable convolutions with residual structures to replace traditional 3D convolutions for constructing the cost volume and performing cost aggregation, aiming to reduce the model size and improve the computational speed. Additionally, we designed the context geometric attention (CGA) module and embedded it into the lightweight 3D regularization network, as well as designed the Warped Disparity Refinement (WDR) network to further improve the disparity prediction accuracy. CGA effectively guides cost aggregation by integrating rich contextual and geometric information, while also providing feedback for feature learning to guide more efficient context feature extraction. WDR constructs a warping cost volume using the obtained initial disparity, combined with image features, the initial disparity map, and reconstruction errors, to optimize the disparity. According to the initial disparity, it searches for the accurate disparity within a refined range. By narrowing the search range, WDR simplifies the task for the network to locate the correct disparity (residual), while simultaneously improving the computational efficiency. Experiments conducted on multiple benchmark datasets showed that, compared to other fast methods, CGW has advantages in both speed and accuracy and exhibits better generalization performance.

This paper presents a novel method for stereo matching with occlusion handling. In order to estimate optimal cost, we define an energy function and solve the iterative equation with the numerical method. We improve performance and convergence rate by using several acceleration techniques. The proposed method is computationally efficient since it does not use color segmentation or any global optimization techniques. For occlusion handling, which has not been performed effectively by any conventional cost aggregation approaches, we combine the occlusion problem with the proposed minimization scheme. Asymmetric information is used so that few additional computational loads are necessary. Experimental results show that performance is comparable to that of many state-of-the-art methods.

A hierarchical stereo matching algorithm based on adaptive support region aggregation method

Dense matching plays an important role in many fields, such as DEM (digital evaluation model) producing, robot navigation and 3D environment reconstruction. Traditional approaches may meet the demand of accuracy. But the calculation time and out puts density is hardly be accepted. Focus on the matching efficiency and complex terrain surface matching feasibility an aerial image dense matching method based on optical flow field is proposed in this paper. First, some high accurate and uniformed control points are extracted by using the feature based matching method. Then the optical flow is calculated by using these control points, so as to determine the similar region between two images. Second, the optical flow field is interpolated by using the multi-level B-spline interpolation in the similar region and accomplished the pixel by pixel coarse matching. Final, the results related to the coarse matching refinement based on the combined constraint, which recognizes the same points between images. The experimental results have shown that our method can achieve per-pixel dense matching points, the matching accuracy achieves sub-pixel level, and fully meet the three-dimensional reconstruction and automatic generation of DSM-intensive matching’s requirements. The comparison experiments demonstrated that our approach’s matching efficiency is higher than semi-global matching (SGM) and Patch-based multi-view stereo matching (PMVS) which verifies the feasibility and effectiveness of the algorithm.

A method of stereo matching which is based on color image segmentation and cross adaptive window has been used to solve the problems of matching imprecision in depth discontinuity regions and low texture regions. This article first produces the matching cost based on color segmentation region by segmenting color image and the one based on cross adaptive window which makes use of color similarity, and then integrates the two to form the combined matching cost. At last it uses the fast searching method of optical parallax to narrow the matching range and then raise the efficiency rate. The result of this experiment can improve the matching accuracy in depth discontinuity regions and low texture regions and raise the matching speed.

Stereo matching is an essential basis for various applications, but most stereo matching methods have poor generalization performance and require a fixed disparity search range. Moreover, current stereo matching methods focus on the scenes that only have positive disparities, but ignore the scenes that contain both positive and negative disparities, such as 3D movies. In this paper, we present a new stereo matching pipeline that first computes semi-dense disparity maps based on binocular disparity, and then completes the rest depending on monocular cues. The new stereo matching pipeline have the following advantages: It 1) has better generalization performance than most of the current stereo matching methods; 2) relaxes the limitation of a fixed disparity search range; 3) can handle the scenes that involve both positive and negative disparities, which has more potential applications, such as view synthesis in 3D multimedia and VR/AR. Experimental results demonstrate the effectiveness of our new stereo matching pipeline.

Stereo matching is a difficult and challenging task due to many uncontrollable factors that affect the results. These factors include the non-ideal radiometric conditions and the presence of weak-textured regions. A stereo matching algorithm including cost computation, cost aggregation, disparity computation, and disparity refinement is proposed to overcome these limitations. First, census, gradient, and HSL measures are combined as the pixel-wise matching cost to reduce the radiometric distortions. Second, a second-order smoothness constraint based on angle direction priors is utilized to improve the matching accuracy in the weak-textured regions. Then, the matching cost and smoothness constraint are applied to semi-global matching. Third, a winner-takes-all strategy is adopted to calculate the initial disparity maps. Finally, a coarse-to-fine simple linear iterative clustering superpixel segmentation is proposed to split the image into some regions with similar colors, adjacent pixels, and similar disparity planes. Through the segmentation result, boundary lines are classified into occluded, hinge, and coplanar lines. Slanted plane iterative optimization is performed to obtain the optimized disparity maps according to the segmentation result and boundary lines classification. The algorithm is evaluated in the Middlebury and KITTI datasets. The experimental results show that the method has good performance under non-ideal radiometric conditions and in the weak-textured regions. The disparity maps obtained by the method have a lower mismatching rate compared with other algorithms.