A Modified Census Transform Based on the Neighborhood Information for Stereo Matching Algorithm

Aiming at the problem that the similarity measures in the overwhelming majority of stereo matching approaches rely heavily on the statistical characteristics of image gray values,a stereo matching algorithm insensitive to differentiations in illumination is proposed.Firstly,the Census non-parametric transform is investigated and its limitations are analyzed. Secondly,in order to take the pixels' spatial location information into consideration when searching stereo correspondences, gray values of the neighborhood pixels,whose relative positions are one unit greater than the center pixel in the transformation window,are replaced by gray values' interpolation with the four pixels adjacent to them.Finally,the Census non-parametric transform of images and stereo matching are implemented,and then,dense disparity map is obtained.The experiment results show that the proposed method has simple mechanism,high computational efficiency and strong robustness;satisfactory stereo matching results also can be obtained even under non-ideal illumination conditions.

The stereo matching algorithm is a technique that analyses two or more images captured at diverse view points in order to find positions in real 3D space for the pixels of 2D image. The stereo matching methods have been used in various fields such as drawing the topographical map from aerial photograph and finding the depth information of objects in machine vision system. Nowdays, optical motion capture techniques using stereo matching algorithms are being developed for visual applications such as virtual reality or 3D graphics. Stereo matching algorithms can be generally classified into two methods: feature-based and area-based ones. The feature-based method matches the feature elements between two images, and uses the interpolation to obtain the disparity information for the pixels other than the feature elements, while the area-based method performs matching between pixels in two images by calculating the correlations of the pixels residing in the search window. Area-based method cannot match feature element with more accuracy than feature-based method even though it can make more dense disparity map. Moreover, it has more possibility of error in the area of insufficient texture information or depth discontinuities. In this paper we present a novel technique for area-based stereo matching algorithm which provides more accurate and error-prone matching capabilities by using adaptive search range and window size. We propose two new strategies (1) for determining search range adaptively from the disparity map and multiresolutional images of region segments obtained by applying feature-based algorithm, and (2) for changing the window size adaptively according to the edge information derived from the wavelet transform such that the combination of two adaptive methods in search range and window size greatly enhance accuracy while reducing errors. We test our matching algorithms for various types of images, and shall show the outperformance of our stereo matching algorithm.

In order to reduce the mismatch rate of binocular stereo matching algorithm, a stereo matching algorithm based on improved Census transform is proposed in this paper. Firstly, the classical Census transform is improved, the mean value rather than the center pixel is used as the reference value, and the three states based on the pixels' distances are used to describe the relative relationship between each pixel in the transform window and the reference value; Secondly, the sparse transform window is used to reduce the computational complexity of the algorithm, and the transformed Hamming distance is calculated as the matching cost; Finally, in the maximum disparity range, the points with the least matching cost are selected as the matching points by WTA algorithm, and the disparity is refined by sub-pixel fitting. Experimental results show that the improved Census transformed image has more detailed information in the depth discontinuous region. Compared with the algorithm that directly uses the gray feature as the matching base elements, the proposed algorithm has better operation speed and matching accuracy.

Disparity refinement is an essential step of local stereo matching methods to produce fine dense disparity maps. The inherent defect of local stereo methods results in erroneous disparity in occluded areas. In this paper, we present a novel post processing method which can effectively improve the accuracy of dense disparity maps by rectifying disparity errors iteratively. Invalid disparities are first detected by left-right consistency check and color-disparity consistency check. For each invalid pixel, supports from valid pixels in the neighborhood are collected to determine the plane parameters of the local window. An iterative strategy is adopted to gradually propagate disparity information from valid pixels to invalid areas. We apply the proposed method to disparity maps produced by two recent stereo matching methods, and compare the refining results with other post processing methods. Experimental results show the effectiveness of our method in improving dense disparity maps.

Unsupervised learning of depth estimation from imperfect rectified stereo laparoscopic images

Traditionally, sub-pixel interpolation in stereo-vision systems has been used for the block-matching algorithm. In this paper, Census transform algorithm which has been on area-based matching algorithm is improved and it’s compared with existing census transform algorithm. Two algorithms are compared using Tsukuba stereo images provided by Middlebury web site. As a result, disparity map error rate is decreased from 16.3 to 11.8 %.

In dependent stereo image compression, the aim is to minimize the bitrate of disparity map and that of residual image. Traditionally, focus has been paid on either disparity map or residual image. In this paper, we compute an optimal disparity map (in terms of bitrates) by jointly exploiting the trade-off between the disparity map and the residual image. Firstly, the dense disparity map is obtained using existing optical flow technique. Secondly, the dense disparity map is quantized using a RD framework. Consequently, the resulting bitrate of the disparity map decreases significantly at the cost of a slight increase of the bitrate of the residual image. As a result, the overall bitrate attains minimum value. The proposed scheme is compatible and can be integrated in JPEG2000 framework.

This work presents the composition of a new algorithm for a stereo vision system to acquire accurate depth measurement from stereo correspondence. Stereo correspondence produced by matching is commonly affected by image noise such as illumination variation, blurry boundaries, and radiometric differences. The proposed algorithm introduces a pre-processing step based on the combination of Contrast Limited Adaptive Histogram Equalization (CLAHE) and Adaptive Gamma Correction Weighted Distribution (AGCWD) with a guided filter (GF). The cost value of the pre-processing step is determined in the matching cost step using the census transform (CT), which is followed by aggregation using the fixed-window and GF technique. A winner-takes-all (WTA) approach is employed to select the minimum disparity map value and final refinement using left-right consistency checking (LR) along with a weighted median filter (WMF) to remove outliers. The algorithm improved the accuracy 31.65% for all pixel errors and 23.35% for pixel errors in nonoccluded regions compared to several established algorithms on a Middlebury dataset.

Dense stereo image matching in remotely sensed images is a challenging problem, though it has been studied for more than two decades, due to occlusions, discontinuities, geometric, and radiometric distortions. A novel multiobjective fitness function-based dense stereo matching approach using genetic algorithms (GAs) is proposed in this paper. The proposed method is useful for estimating dense disparity map with an improved number of inliers for a stereo image pair, despite the constraint of finding correct disparity at depth discontinuities. In this paper, the steps of GA, such as initialization of the population, fitness function, and crossover and mutation operation, are designed and implemented to effectively deal with the problem of dense stereo image matching. To initialize the population, a Scale Invariant Feature Transform (SIFT) descriptor is computed for each pixel and multiple-size window-based matching is performed, using the similarity measures: 1) Euclidean distance and 2) spectral angle mapper. The generated disparity maps are pruned to choose a suitable subset using the designed fitness functions, considering the constraints related to stereo image pair, such as epipolar constraint, which encodes the epipolar geometry and the similarity measure that is useful to decide accuracy of the correspondences. The two objective functions are the number of inliers computed using the fundamental matrix and an energy minimization function, considering discontinuities and occlusions. The usefulness of this approach for remotely sensed stereo image pairs is demonstrated by improving the number of inliers and favorably comparing with state-of-the-art dense stereo image matching methods.

This paper introduces a novel system for facial landmark detection using a modified Active Appearance Model (AAM). Traditional AAMs operate directly on the pixel values of the image, leading to problems with inhomogeneously illuminated scenes. Instead of using the gray-level image to detect the facial landmark directly, the Modified Census Transformation (MCT) is performed on the region of interest (ROI) being analyzed, making the system invariant to illumination variations and nonlinear camera characteristics. To achieve efficient and robust fitting with regard to occluded or invisible parts of the face, parameter constraints, coarse to fine fitting and occlusion handling are introduced. The result shows that the new system yields good results even if some areas of the face are occluded or unrecognizable in the image.

This paper proposes a robust real-time method for hand tracking and hand posture recognition. Dealing with complex background, scale-invariance and rotation-invariance are the difficulties for hand posture recognition. To solve these difficulties, we firstly detect the specific posture using the method based on Modified Census Transform, in order to trigger hand tracking and hand posture recognition. For the complex background particularly with large skin-color alike objects, a multi-cue method, based on velocity weighted features and color cue, is proposed to deal with the hand tracking. Then we segment the hand using both Bayesian skin-color model and the hand tracking result. Finally, we use a novel method based on density distribution feature to recognize hand posture. It largely enforces the robustness of hand posture recognition because of scale-invariance and rotation-invariance. Experiment results and applications demonstrate the effectiveness of our method.

It is an important problem for computer vision to match the stereo image pair. Only the problems of stereo matching are solved, the accurate location or measurement of object can be realized. In this paper, an integrated stereo matching approach is presented. Unlike most stereo matching approach, it integrates area-based and feature-based primitives. This allows it to take advantages of the unique attributes of each of these techniques. The feature-based process is used to match the image feature. It can provide a more precise sparse disparity map and accurate location of discontinuities. The area-based process is used to match the continuous surfaces.It can provide a dense disparity map. The techniques of stereo matching with adaptive window are adopted in the area-based process. It can make the results of area-based process get high precise. An integrated process is also used in this approach. It can integrate the results of feature-based process and area-based process, so that the approach can provide not only a dense disparity map but also an accurate location of discontinuities. The approach has been tested by some synthetic and nature images. From the results of matched wedding cake and matched aircraft model, we can see that the surfaces and configuration are well reconstructed. The integrated stereo matching approach can be used in 3D part recognition in intelligent assembly system and computer vision.

This paper presents a new algorithm of disparity map estimation. The originality of this method lies in the process of dense disparity map estimation using dynamic programming constrained by interest points and using the higher order statistics (HOS) criteria for matching noisy images. Experiments with noisy real images have validated our method and have clearly shown the improvement over the existing ones. The dense disparity map obtained is more reliable when compared to the similar second-order statistics (SOS)-based dynamic programming and HOS-based correlation methods.

Moving objects segmentation is the foundation of intelligent moving body information collection. In the monocular vision system, it is too difficult to segment the foreground from background when their grayscale and color are similar. Compared with the grayscale and color, the scenery depth are less susceptible to external environment, if depth information can be got in stereo vision, it would be much easier to segment the foreground. Unfortunately, scenery accurate dense disparity map is often hard to get. In the paper, an optimizing method of calculating the dense disparity based on graph cut is used, and based on the dense disparity, the features of color and disparity are combined by graph cut to improve the accuracy of the image segmentation. The experimental results show that the method provides a better dense disparity map and also reduces the effect of light and strengthens the stability of segmentation by combining the features of color and disparity in graph cut optimization algorithm.

This article presents a new stereo matching algorithm based on local method. The absolute difference (AD) algorithm works fast and precise at low texture area, however this algorithm is sensitive to radiometric distortions (i.e., contrast or brightness) and low texture areas. To overcome these problems, the proposed algorithm utilizes a combination of AD, gradient difference (GM) and census transform (CT). The GM algorithm is robust against the radiometric errors and CT algorithm works well on the low texture regions. The proposed algorithm performs much better based on the experimental results of the Middlebury dataset. The disparity map from the result consists of depth information which requires for the three-dimensional (3D) surface reconstruction.