An asymmetric post-processing for correspondence problem

Depth information can be obtained using stereo matching algorithms, which compute the horizontal displacement (disparity) of the corresponding points and convert to depth information using the triangular relation. However, the matching process is challenging with presence of textureless regions. This paper proposes a novel disparity refinement method for stereo matching based on Semi-global Matching (SGM) algorithm for texture less images. To be brief, the SGM algorithm is a high-performance matching algorithm of a stereo image pair and it reaches a trade off between matching quality and computing complexity. The main contribution comes from the improvement of matching quality on texture less regions. At the end of SGM, a right-to-left consistency check is performed to remove the invalid pixels in the disparity map. The proposed method is added after the right-to-left consistent check. We assume that the texture less regions in the original stereo pairs are planar. We employ edge detection to extract the textureless regions. The fitting method is employed in horizontal and vertical directions respectively. If the distance of two adjacent edge pixels is sufficiently large, the corresponding line segment will be considered as texturelss. For every texturelss line segment, the RANSAC algorithm is performed on the corresponding line segment in the disparity map. We use the well-known Middlebury dataset to compare our method with the normal SGM and other matching algorithms. It shows that our method performs well for most texture less stereo pairs.

Disparity refinement is an essential step of local stereo matching methods to produce fine dense disparity maps. The inherent defect of local stereo methods results in erroneous disparity in occluded areas. In this paper, we present a novel post processing method which can effectively improve the accuracy of dense disparity maps by rectifying disparity errors iteratively. Invalid disparities are first detected by left-right consistency check and color-disparity consistency check. For each invalid pixel, supports from valid pixels in the neighborhood are collected to determine the plane parameters of the local window. An iterative strategy is adopted to gradually propagate disparity information from valid pixels to invalid areas. We apply the proposed method to disparity maps produced by two recent stereo matching methods, and compare the refining results with other post processing methods. Experimental results show the effectiveness of our method in improving dense disparity maps.

Initializing and accelerating Stereo-DIC computation using semi-global matching with geometric constraints

It's a difficult problem in three-dimensional visual measurement to improve the precision and accuracy of stereo matching. In this paper, a thin line power increase (TLPI) stereo matching method used in high-precision three-dimensional visual measurement is designed. It's a coding and stereo matching method which combines the second power increases method, the step moving method, the difference shadow method, the progressive decrease method and the stereo matching method. Through the combination of the second power increases method and the step moving method, the projectile optical information can cover all the measured object space. Through the difference shadow method, the background interference of the measured space can be wiped off. Through the progressive decrease method, the searching range of the optical information can be narrowed; the speed and the precision of optical information matching will be improved. Through the stereo matching method, the three-dimensional coordinate of object can be calculated. The thin line power increase method designed by this paper can solve the problem of misjudgment and improve the precision and reliability of three-dimensional measurement.

This paper presents a novel method for stereo matching with occlusion handling. In order to estimate optimal cost, we define an energy function and solve the iterative equation with the numerical method. We improve performance and convergence rate by using several acceleration techniques. The proposed method is computationally efficient since it does not use color segmentation or any global optimization techniques. For occlusion handling, which has not been performed effectively by any conventional cost aggregation approaches, we combine the occlusion problem with the proposed minimization scheme. Asymmetric information is used so that few additional computational loads are necessary. Experimental results show that performance is comparable to that of many state-of-the-art methods.

Stereo matching is an essential basis for various applications, but most stereo matching methods have poor generalization performance and require a fixed disparity search range. Moreover, current stereo matching methods focus on the scenes that only have positive disparities, but ignore the scenes that contain both positive and negative disparities, such as 3D movies. In this paper, we present a new stereo matching pipeline that first computes semi-dense disparity maps based on binocular disparity, and then completes the rest depending on monocular cues. The new stereo matching pipeline have the following advantages: It 1) has better generalization performance than most of the current stereo matching methods; 2) relaxes the limitation of a fixed disparity search range; 3) can handle the scenes that involve both positive and negative disparities, which has more potential applications, such as view synthesis in 3D multimedia and VR/AR. Experimental results demonstrate the effectiveness of our new stereo matching pipeline.

Computational complexity of the local stereo matching method is affected by disparity ranges. In case of depth estimation in sequential stereo images, high computational complexity is a problem in terms of real-time processing. In this paper, we propose a temporal correlation based stereo matching method in sequential images. Using temporal information in a sequential stereo matching method provides inaccurate disparity ranges, since the estimated depth map accuracy is gradually degraded. To preserve the depth map quality in temporal stereo matching procedure, we adopt the guided image filtering for matching cost aggregation. Since the guided image filtering has a similar structure with bilateral filter, it preserves an object boundary region even in restricted disparity search ranges. Inaccurately estimated disparity values from the temporal correlation are compensated by filtering based cost aggregation method. From the experiment results, we check that the proposed depth map acquisition method preserves the depth map quality in temporal domain stereo matching.

The pantograph is an essential component in modern electrified trains. Due to the characters of contactless and high precision, the 3-D stereo vision, which can effectively monitor the abrasion, is of great significance for ensuring train safety. However, illumination conditions, such as overbrightness and insufficient light, have a great impact on semiglobal matching (SGM) stereo matching, inevitably affecting the integrity and accuracy of the abrasion measurement of the pantograph slipper. In this regard, a robust SGM stereo matching is proposed in this article for the 3-D abrasion measurement. First, the combination of the neighborhood information-aware census cost and multiscale fusion for cost aggregation is put forward. While insensitive to illumination condition changes, the improved census cost can quantify the texture information of the neighborhood. Moreover, multiscale cost aggregation further fuses the neighborhood information of different scales. As a result, the performance of SGM in weakly textured areas is greatly improved, especially in complex illumination conditions. Eventually, with the help of the subpixel disparity refinement and local points fitting, an accurate 3-D abrasion calculation is achieved. Practical experiments show the robustness of stereo matching. The precision verification and computation time analysis confirm the measurement accuracy and applicability.

This paper presents a stereo image coding method. We first use a new stereo matching method with difference-based adaptive searching windows to get the fine disparity. Then, we encode the disparity instead of one channel of the stereo image while the other channel is encoded normally. The stereo matching employs the continuity of difference of stereo images to compute the size of adaptive searching windows, which is superior to the approaches based on fixed searching windows. Experimental results show that the proposed scheme can provide better performance for stereo image coding.

Human beings process stereoscopic correspondence across multiple purposes like robot navigation, automatic driving, and virtual or augmented reality. However, this bioinspiration is ignored by state-of-the-art dense stereo correspondence matching methods. Cost aggregation is one of the critical steps in the stereo matching method. In this article, we propose an optimized cross-scale cost aggregation scheme with coarse-to-fine strategy for stereo matching. This scheme implements cross-scale cost aggregation with the smoothness constraint on neighborhood cost, which essentially extends the idea of the inter-scale and intra-scale consistency constraints to increase the matching accuracy. The neighborhood costs are not only used in the intra-scale consistency to ensure that the regularized costs vary smoothly in an eight-connected neighbors region but also incorporated with inter-scale consistency to optimize the disparity estimation. Additionally, the improved method introduces an adaptive scheme in each scale with different aggregation methods. Finally, experimental results evaluated both on classic Middlebury and Middlebury 2014 data sets show that the proposed method performs better than the cross-scale cost aggregation. The whole stereo correspondence algorithm achieves competitive performance in terms of both matching accuracy and computational efficiency. An extensive comparison, including the KITTI benchmark, illustrates the better performance of the proposed method also.

Stereo matching is a branch of 3D vision and has a wide range of applications in 3D reconstructionand autonomous driving. Recently, stereo matching methods leverage all the information of the stereo image to calculate disparity map. However, these methods still have difficulties in texture-less areas and occlusion areas, and post-processing improves accuracy. Therefore, there is a high computational cost in feature extraction and post-processing. In this paper, we propose a stereo matching method based on features of image patches to predict the disparity of non-occlusion areas instead of full image features. And aggregation methods are performed to modify all kinds of mismatching pixels based on the correct disparity in the non-occlusion areas. Furthermore, we evaluated the proposed method on the Middlebury dataset. The result shows that the proposed method performs well in all areas.

Hardware platforms with limited processing power are often incapable of running dense stereo analysis algorithms at acceptable speed. Sparse algorithms provide an alternative but generally lack in accuracy. To overcome this predicament, we present an efficient sparse stereo analysis algorithm that applies a dense consistency check, leading to accurate matching results. We further improve matching accuracy by introducing a new feature detector based on FAST, which exhibits a less clustered feature distribution. The new feature detector leads to a superior performance of our stereo analysis algorithm. Performance evaluation shows that the proposed stereo matching system achieves processing rates above 200 frames per second on a commodity dual core CPU, and faster than video frame-rate processing on a low-performance embedded platform. The stereo matching results prove to be superior to those obtained with ordinary sparse matching algorithms.

Mobile stereo matching systems are becoming more important in many applications such as auto-driving and autonomous robots. However, to maintain its low power consumption, mobile platforms have only limited hardware resources. Accurate stereo matching methods require a high computational complexity, and it is difficult to maintain both acceptable accuracy and processing speed on the mobile platforms. To solve this trade-off, in this paper, we propose a novel acceleration approach for a well-known matching algorithm Zero-means Normalized Cross Correlation (ZNCC), and show its effectiveness on a Jetson TX2 embedded GPU. By combining our new approach, Z2- ZNCC, with the Semi-Global Matching (SGM) algorithm, our system achieves a low error rate of 7.76% while keeping 28 fps for 1242×375 pixels images with the maximum disparity of 128 on the KITTI 2015 dataset. This performance is higher than previous state-of-the-art system on the same hardware platform.

A method of stereo matching which is based on color image segmentation and cross adaptive window has been used to solve the problems of matching imprecision in depth discontinuity regions and low texture regions. This article first produces the matching cost based on color segmentation region by segmenting color image and the one based on cross adaptive window which makes use of color similarity, and then integrates the two to form the combined matching cost. At last it uses the fast searching method of optical parallax to narrow the matching range and then raise the efficiency rate. The result of this experiment can improve the matching accuracy in depth discontinuity regions and low texture regions and raise the matching speed.

In many cases, the Digital Surface Models (DSMs) and Digital Elevation Models (DEMs) are obtained with Light Detection and Ranging (LiDAR) or stereo matching. As an active method, LiDAR is very accurate but expensive, thus often limiting its use in small-scale acquisition. Stereo matching is suitable for large-scale acquisition of terrain information as the increase of satellite stereo sensors. However, underperformance of stereo matching easily occurs in textureless areas. Accordingly, this study proposed a Shading Aware DSM GEneration Method (SADGE) with high resolution multi-view satellite images. Considering the complementarity of stereo matching and Shape from Shading (SfS), SADGE combines the advantage of stereo matching and SfS technique. First, an improved Semi-Global Matching (SGM) technique is used to generate an initial surface expressed by a DSM; then, it is refined by optimizing the objective function which modeled the imaging process with the illumination, surface albedo, and normal object surface. Different from the existing shading-based DEM refinement or generation method, no information about the illumination or the viewing angle is needed while concave/convex ambiguity can be avoided as multi-view images are utilized. Experiments with ZiYuan-3 and GaoFen-7 images show that the proposed method can generate higher accuracy DSM (12.5–56.3% improvement) with sound overall shape and temporarily detailed surface compared with a software solution (SURE) for multi-view stereo.