Stereo Matching Based on Visual Sensitive Information

A fast stereo matching algorithm suitable for embedded real-time systems

With the wide application of artificial intelligence in intelligent driving, visual SLAM technology has become one of the popular research areas. Binocular SLAM, as one of the cheapest cost technologies in SLAM, has become a research hotspot for many scholars. For the problems of high time consumption, poor accuracy and high mis-matching rate of traditional binocular stereo matching algorithm, an adaptive weight cost aggregation D* algorithm based on Dijkstra's idea is proposed. The algorithm improves on the local stereo matching Census algorithm, and fuses the Census algorithm and grayscale histogram in the cost matching stage to make it compatible with computational efficiency and accuracy, which can achieve real-time effect when combined with hardware acceleration. The experimental results show that the improved Census algorithm greatly improves the stereo matching accuracy, and computational efficiency. In comparison with other algorithms on the Middlebury 2.0 testbed, the improved algorithm is found to be 42% more efficient and 1.2% better in terms of mis-matching rate.

Stereo vision is used in many application areas, such as robot-assisted manufacturing processes. Recently, many different efficient stereo matching algorithms based on deep learning have been developed to solve the limitations of traditional correspondence point analysis, among others. The challenges include texture-poor objects or non-cooperative objects. One of these end-to-end learning algorithms is the Adaptive Aggregation Network (AANet/AANet+), which is divided into five steps: feature extraction, cost volume construction, cost aggregation, disparity computation and disparity refinement. By combining different components, it is easy to create an individual stereo matching model. Our goal is to develop efficient learning methods for robot-assisted manufacturing processes for cross-domain data streams. The aim is to improve recognition tasks and process optimisation. To achieve this, we have investigated the AANet+ in terms of usability and efficiency on our own test-dataset with different measurement setups (passive stereo system). Input of the AANet+ are rectified stereo pairs of the test-dataset and a pre-trained model. Instead of generating our own training dataset, we used two pre-trained models based on the KITTI-2015 and SceneFlow datasets. Our research has shown that the pretrained model based on the Scene Flow dataset predicts disparities with better object delimination. Due to the Out-of-Distribution inputs, only reliable disparity predictions of the AANet are possible for test data sets with parallel measurement setup. We compared the results with two traditional stereo matching algorithms (SemiGlobal block matching and DAISY). Compared to the traditionally computed disparity maps, the AANet+ is able to robustly detect texture-poor objects and optically non-cooperative objects.

Stacking learning with coalesced cost filtering for accurate stereo matching

Although many algorithms perform very well on certain datasets, existing stereo matching algorithms still fail to obtain ideal disparity images with high precision in practical robotic applications with weak or untextured objects. This greatly limits the application of binocular vision for robotic arm guidance. Traditional stereo matching algorithms suffer from disparity loss, dilation and other problems, and deep learning algorithms have weakly generalization ability, making high‐accuracy results impossible with non‐training images. We propose an algorithm that uses segments and edges as matching units. We find the mapping relationship between two‐dimensional images and three‐dimensional scenes using segments. The algorithm obtains highly accurate results in industrial robotic applications with mainly planar facets. We combine it with a deep learning algorithm to obtain very good high‐accuracy results in both general scenes and applications of industrial robots. The algorithm effectively improves the non‐linear optimization ability of traditional algorithms and generalization ability of deep learning, and provides an effective method for the binocular vision guidance of industrial robotic scenes. We used the algorithm to guide the robot arm for threading with a success rate of 70%.

Stereo matching is one of the most active research fields in computer vision. The paper introduces the categories and the performance index of stereo matching and introduces three high-speed and state-of-the-art stereo matching algorithms with different cost aggregation: fast bilateral stereo (FBS), binary stereo matching (BSM), and a non-local cost aggregation method (NLCA). By comparing the performance in terms of both quality and speed, we concluded that FSB deals with the effects of noise well; BSM is suitable for embedded devices and has a good performance with radiometric differences; NLCA combines the efficiency with the accuracy of state-of-the-art algorithms.

This paper presents a novel area-based stereo matching algorithm based on improved census transform under changing illumination. For the traditional census-based stereo matching, the result is not robust under variant illumination, because the intensity value of center pixel in the mask is affected by the noise to cause distortion. In order to solve this problem, we propose the improved census transform method, which takes the standard deviation of census mask as the base point instead of the center pixel, comparing with the difference of per neighborhood pixel and the mean intensity of the mask to build the sparse census transform. The experiments show that the stereo matching algorithm is robust even if the illumination changes.

The stereo matching algorithm is a technique that analyses two or more images captured at diverse view points in order to find positions in real 3D space for the pixels of 2D image. The stereo matching methods have been used in various fields such as drawing the topographical map from aerial photograph and finding the depth information of objects in machine vision system. Nowdays, optical motion capture techniques using stereo matching algorithms are being developed for visual applications such as virtual reality or 3D graphics. Stereo matching algorithms can be generally classified into two methods: feature-based and area-based ones. The feature-based method matches the feature elements between two images, and uses the interpolation to obtain the disparity information for the pixels other than the feature elements, while the area-based method performs matching between pixels in two images by calculating the correlations of the pixels residing in the search window. Area-based method cannot match feature element with more accuracy than feature-based method even though it can make more dense disparity map. Moreover, it has more possibility of error in the area of insufficient texture information or depth discontinuities. In this paper we present a novel technique for area-based stereo matching algorithm which provides more accurate and error-prone matching capabilities by using adaptive search range and window size. We propose two new strategies (1) for determining search range adaptively from the disparity map and multiresolutional images of region segments obtained by applying feature-based algorithm, and (2) for changing the window size adaptively according to the edge information derived from the wavelet transform such that the combination of two adaptive methods in search range and window size greatly enhance accuracy while reducing errors. We test our matching algorithms for various types of images, and shall show the outperformance of our stereo matching algorithm.

Efficient local stereo matching algorithm based on fast gradient domain guided image filtering

Stereo matching is the process of finding disparity map between a pair of stereo images. Markov Random Field (MRF) model has been used in stereo matching algorithms in which the goal is to minimize total data, smoothness and occlusion cost. Recent MRF based stereo matching algorithms assign the same smoothness cost to adjacent pixels which is not effective at objects boundaries. In this paper, we propose a method to constrain the smoothness cost such that the cost applied to disparity discontinuity within an object is higher than that of the object's boundary. Data cost function is also normalized before computing its probability distribution function (PDF) to enforce an uniform PDF for the entire image. Experimental results have demonstrated that the proposed method is competitive to the state-of-the-art stereo matching algorithms.

Stereo matching using weighted dynamic programming on a single-direction four-connected tree

The key and difficult issue in the research of binocular vision-based 3D measurement is the accurate calibration of internal and external parameters of the camera and stereo matching. Matlab calibration is more efficient and accurate compared with manual or OpenCV calibration. In this paper, binocular camera is calibrated by Matlab calibration toolbox, and calibration parameters imported in OpenCV for follow-up image correction and stereo matching. By studying and comparing Block Matching (BM) and Graph Cut (GC) stereo matching algorithms, a disparity image of the object is obtained, thus laying foundation for follow-up 3D data information acquisition and reconstruction.

Stereo vision finds a wide range of applications in automotive, object detection, robot navigation, agriculture mapping, and others. Stereo matching is a stereo algorithm targeted to identify the corresponding pixels from two or more images. This study shows the implementation of stereo matching using the Sum of Absolute Difference (SAD) algorithm to extract the object's depth or disparity from stereo images. Our key objective revolves around the implementation of the stereo matching algorithm for a small field-programmable gate array (FPGA)—requiring relatively few resources while maintaining the processing speed as well as disparity map. For meeting this requirement, we used small window buffers to compute the stereo matching. The occluded pixels were reduced by introducing secondary consistency checking implementation. From the results of experiments performed using the Zynq UltraScale+ ZCU102 FPGA board with SDSoC compiler, the processing speed for computing the stereo matching algorithm with a 4×4 window buffer was 0.038 s for an image size of 486×720 pixels and 0.051 s for 375×1242 pixels resolution. The proposed design needed 1% each of BRAM and FF and 7% of LUT. An 18% reduction in the pixel errors has been observed when employing the secondary consistency matching on the post-processing.

The stereo matching <span>process is one of the key areas that impact the stereo vision technologies which are commonly used in the application of three-dimensional reconstructions. The accuracy of the depth information used in three-dimensional reconstruction is directly proportional to the accuracy of the disparity obtained from stereo matching. The challenging issue in the stereo matching process is to determine the accurate corresponding point between the left image and right image, especially for image pairs that have different exposure such as different illumination and image pair with less texture region. In order to increase the accuracy of disparity value, a new stereo matching algorithm is proposed based on the combination of Sum of absolute different and census transform at matching cost computation. guided filter was used in the matching cost aggregation in order to remove noise and preserve the edge of the image. In the optimization step, the winner take all strategy is used to select the minimum matching cost. Finally, a median filter is applied to the initial disparity map for refinement purposes. The experimental results show that the algorithm is effective in reducing the error and improving the accuracy of the disparity map in different illumination regions, less textured regions and different environmental exposure.</span>

To solve the problem that traditional local stereo matching algorithm is susceptible to illumination and environmental noise, ADCensus transformation is used to calculate the initial matching cost, and adaptive window algorithm is used to construct different aggregation windows for regions with different texture degrees to improve the matching accuracy. A regional voting optimization algorithm based on confidence map is proposed. Combined with multiple optimization steps, the mis-matching point and the matching error rate of the disparity map is effectively reduced. The proposed algorithm is tested using the Middlebury standard image set. The results show that the algorithm can calculate the disparity value accurately.