Stereo matching algorithm based on adaptive weight and local entropy

Innovative applications in rapidly evolving domains such as robotic navigation and autonomous (driverless) vehicles rely on binocular computer vision systems that meet stringent response time and accuracy requirements. A key problem in these vision systems is stereo matching, which involves matching pixels from two input images in order to construct the output, a 3D map. Building upon the existing local stereo matching algorithms, this paper proposes a novel stereo matching algorithm that is based on a weighted guided filtering foundation. The proposed algorithm consists of three main steps; each step is designed with the goal of improving accuracy. First, the matching costs are computed using a unique combination of complementary methods (absolute difference, Census, and gradient algorithms) to reduce errors. Second, the costs are aggregated using an adaptive weighted guided image filtering method. Here, the regularization parameters are adjusted adaptively using the Canny method, further reducing errors. Third, a disparity map is generated using the winner-take-all strategy; this map is subsequently refined using a densification method to reduce errors. Our experimental results indicate that the proposed algorithm provides a higher level of accuracy in comparison to a collection of the existing state-of-the-art local algorithms.

In this paper, we propose a stereo matching method based on adaptive plane-fitting. In the proposed method, we use the disparity map adjustment with adaptive weight filter to preserve the boundary region. To estimate the initial disparity map, we use two cost aggregation functions, and in addition to that, we combined color and gradient for cost computation. In addition, the error value of the initial disparity map is calibrated by applying plane-fitting using mean-shift segmentation. The disparity map is adjusted by using adaptive weights to improve the accuracy of the disparity map. Experimental results show that the proposed method generates a high-quality disparity map which has the distinct boundary between objects compared to other methods.

The stereo matching algorithm is a technique that analyses two or more images captured at diverse view points in order to find positions in real 3D space for the pixels of 2D image. The stereo matching methods have been used in various fields such as drawing the topographical map from aerial photograph and finding the depth information of objects in machine vision system. Nowdays, optical motion capture techniques using stereo matching algorithms are being developed for visual applications such as virtual reality or 3D graphics. Stereo matching algorithms can be generally classified into two methods: feature-based and area-based ones. The feature-based method matches the feature elements between two images, and uses the interpolation to obtain the disparity information for the pixels other than the feature elements, while the area-based method performs matching between pixels in two images by calculating the correlations of the pixels residing in the search window. Area-based method cannot match feature element with more accuracy than feature-based method even though it can make more dense disparity map. Moreover, it has more possibility of error in the area of insufficient texture information or depth discontinuities. In this paper we present a novel technique for area-based stereo matching algorithm which provides more accurate and error-prone matching capabilities by using adaptive search range and window size. We propose two new strategies (1) for determining search range adaptively from the disparity map and multiresolutional images of region segments obtained by applying feature-based algorithm, and (2) for changing the window size adaptively according to the edge information derived from the wavelet transform such that the combination of two adaptive methods in search range and window size greatly enhance accuracy while reducing errors. We test our matching algorithms for various types of images, and shall show the outperformance of our stereo matching algorithm.

In this paper, we propose an adaptive stereo matching algorithm to encompass stereo matching problems in projective distortion region. Since the projective distortion region can not be estimated in terms of fixed-size block matching algorithm, we tried to use adaptive window warping method in hierarchical matching process to compensate the perspective distortions. In addition, probability theory was adopted to encompass uncertainty of disparity of points over the window in this study. The proposed stereo matching algorithm has been tested on both disparity map and 3D model view. The experimental results show that remarkable improvement is obtained in the projective distortion region.

As-planar-as-possible depth map estimation

In this paper, a novel local stereo matching algorithm is proposed. As is known, finding the adaptively appropriate support region is the key for accurate disparity map. In this algorithm edges are first detected by Canny edge detector and then are used to construct support windows. With left and right consistency check, reliable disparity seeds are detected. And then the disparities of these seeds are propagated to the neighbor pixels with color difference and occlusion constraint. Finally the false matches in the disparity map will be eliminated by the disparity refinement process. Experiments show that accurate disparity maps are obtained from Middlebury data sets and the algorithm is among the best local stereo algorithms. To accelerate the speed of cost aggregation, an applied method is proposed.

Stereo vision finds a wide range of applications in automotive, object detection, robot navigation, agriculture mapping, and others. Stereo matching is a stereo algorithm targeted to identify the corresponding pixels from two or more images. This study shows the implementation of stereo matching using the Sum of Absolute Difference (SAD) algorithm to extract the object's depth or disparity from stereo images. Our key objective revolves around the implementation of the stereo matching algorithm for a small field-programmable gate array (FPGA)—requiring relatively few resources while maintaining the processing speed as well as disparity map. For meeting this requirement, we used small window buffers to compute the stereo matching. The occluded pixels were reduced by introducing secondary consistency checking implementation. From the results of experiments performed using the Zynq UltraScale+ ZCU102 FPGA board with SDSoC compiler, the processing speed for computing the stereo matching algorithm with a 4×4 window buffer was 0.038 s for an image size of 486×720 pixels and 0.051 s for 375×1242 pixels resolution. The proposed design needed 1% each of BRAM and FF and 7% of LUT. An 18% reduction in the pixel errors has been observed when employing the secondary consistency matching on the post-processing.

The stereo matching <span>process is one of the key areas that impact the stereo vision technologies which are commonly used in the application of three-dimensional reconstructions. The accuracy of the depth information used in three-dimensional reconstruction is directly proportional to the accuracy of the disparity obtained from stereo matching. The challenging issue in the stereo matching process is to determine the accurate corresponding point between the left image and right image, especially for image pairs that have different exposure such as different illumination and image pair with less texture region. In order to increase the accuracy of disparity value, a new stereo matching algorithm is proposed based on the combination of Sum of absolute different and census transform at matching cost computation. guided filter was used in the matching cost aggregation in order to remove noise and preserve the edge of the image. In the optimization step, the winner take all strategy is used to select the minimum matching cost. Finally, a median filter is applied to the initial disparity map for refinement purposes. The experimental results show that the algorithm is effective in reducing the error and improving the accuracy of the disparity map in different illumination regions, less textured regions and different environmental exposure.</span>

Stereo matching is a difficult and challenging task due to many uncontrollable factors that affect the results. These factors include the non-ideal radiometric conditions and the presence of weak-textured regions. A stereo matching algorithm including cost computation, cost aggregation, disparity computation, and disparity refinement is proposed to overcome these limitations. First, census, gradient, and HSL measures are combined as the pixel-wise matching cost to reduce the radiometric distortions. Second, a second-order smoothness constraint based on angle direction priors is utilized to improve the matching accuracy in the weak-textured regions. Then, the matching cost and smoothness constraint are applied to semi-global matching. Third, a winner-takes-all strategy is adopted to calculate the initial disparity maps. Finally, a coarse-to-fine simple linear iterative clustering superpixel segmentation is proposed to split the image into some regions with similar colors, adjacent pixels, and similar disparity planes. Through the segmentation result, boundary lines are classified into occluded, hinge, and coplanar lines. Slanted plane iterative optimization is performed to obtain the optimized disparity maps according to the segmentation result and boundary lines classification. The algorithm is evaluated in the Middlebury and KITTI datasets. The experimental results show that the method has good performance under non-ideal radiometric conditions and in the weak-textured regions. The disparity maps obtained by the method have a lower mismatching rate compared with other algorithms.

Depth information can be obtained using stereo matching algorithms, which compute the horizontal displacement (disparity) of the corresponding points and convert to depth information using the triangular relation. However, the matching process is challenging with presence of textureless regions. This paper proposes a novel disparity refinement method for stereo matching based on Semi-global Matching (SGM) algorithm for texture less images. To be brief, the SGM algorithm is a high-performance matching algorithm of a stereo image pair and it reaches a trade off between matching quality and computing complexity. The main contribution comes from the improvement of matching quality on texture less regions. At the end of SGM, a right-to-left consistency check is performed to remove the invalid pixels in the disparity map. The proposed method is added after the right-to-left consistent check. We assume that the texture less regions in the original stereo pairs are planar. We employ edge detection to extract the textureless regions. The fitting method is employed in horizontal and vertical directions respectively. If the distance of two adjacent edge pixels is sufficiently large, the corresponding line segment will be considered as texturelss. For every texturelss line segment, the RANSAC algorithm is performed on the corresponding line segment in the disparity map. We use the well-known Middlebury dataset to compare our method with the normal SGM and other matching algorithms. It shows that our method performs well for most texture less stereo pairs.

Stereo vision is used in many application areas, such as robot-assisted manufacturing processes. Recently, many different efficient stereo matching algorithms based on deep learning have been developed to solve the limitations of traditional correspondence point analysis, among others. The challenges include texture-poor objects or non-cooperative objects. One of these end-to-end learning algorithms is the Adaptive Aggregation Network (AANet/AANet+), which is divided into five steps: feature extraction, cost volume construction, cost aggregation, disparity computation and disparity refinement. By combining different components, it is easy to create an individual stereo matching model. Our goal is to develop efficient learning methods for robot-assisted manufacturing processes for cross-domain data streams. The aim is to improve recognition tasks and process optimisation. To achieve this, we have investigated the AANet+ in terms of usability and efficiency on our own test-dataset with different measurement setups (passive stereo system). Input of the AANet+ are rectified stereo pairs of the test-dataset and a pre-trained model. Instead of generating our own training dataset, we used two pre-trained models based on the KITTI-2015 and SceneFlow datasets. Our research has shown that the pretrained model based on the Scene Flow dataset predicts disparities with better object delimination. Due to the Out-of-Distribution inputs, only reliable disparity predictions of the AANet are possible for test data sets with parallel measurement setup. We compared the results with two traditional stereo matching algorithms (SemiGlobal block matching and DAISY). Compared to the traditionally computed disparity maps, the AANet+ is able to robustly detect texture-poor objects and optically non-cooperative objects.

Efficient local stereo matching algorithm based on fast gradient domain guided image filtering

In this study, novel techniques are presented addressing the challenges of stereo matching algorithms for surveillance and vehicle control. For this purpose, one of the most efficient local stereo matching techniques, namely permeability filter, is modified in terms of road plane geometry and temporal consistency in order to take the major challenges of such a scenario into account. Relaxing smoothness assumption of the permeability filter along vertical axis enables extraction of road geometry with high accuracy, even for the cases where ground plane does not contain sufficient textural information. On the other hand, temporal smoothness is enforced by transferring reliable depth assignments against illumination changes, reflections and instant occlusions. According to the extensive experiments on a recent challenging stereo video dataset, the proposed modifications provide reliable disparity maps under severe challenges and low texture distribution, improving scene analyses for surveillance related applications. Although improvements are illustrated for a specific local stereo matching algorithm, the presented specifications and modifications can be applied for the other similar stereo algorithms as well.

With the wide application of artificial intelligence in intelligent driving, visual SLAM technology has become one of the popular research areas. Binocular SLAM, as one of the cheapest cost technologies in SLAM, has become a research hotspot for many scholars. For the problems of high time consumption, poor accuracy and high mis-matching rate of traditional binocular stereo matching algorithm, an adaptive weight cost aggregation D* algorithm based on Dijkstra's idea is proposed. The algorithm improves on the local stereo matching Census algorithm, and fuses the Census algorithm and grayscale histogram in the cost matching stage to make it compatible with computational efficiency and accuracy, which can achieve real-time effect when combined with hardware acceleration. The experimental results show that the improved Census algorithm greatly improves the stereo matching accuracy, and computational efficiency. In comparison with other algorithms on the Middlebury 2.0 testbed, the improved algorithm is found to be 42% more efficient and 1.2% better in terms of mis-matching rate.

In this paper, a new stereo matching algorithm is proposed that separately estimates high and low-confidence region disparities. First, a mutually complementary tree structure decides on the high-confidence region, estimating its disparity map using dynamic programming optimization. Later, a disparity fitting algorithm restores low-confidence region disparity using high-confidence disparity and the information from one view color, fulfilling global optimization. The confidence stereo matching algorithm enhances both the occlusion areas and difficult-to-estimate such as thin objects, resulting in a high-quality disparity map.