An FPGA-based RGBD imager

Disparity refinement is an essential step of local stereo matching methods to produce fine dense disparity maps. The inherent defect of local stereo methods results in erroneous disparity in occluded areas. In this paper, we present a novel post processing method which can effectively improve the accuracy of dense disparity maps by rectifying disparity errors iteratively. Invalid disparities are first detected by left-right consistency check and color-disparity consistency check. For each invalid pixel, supports from valid pixels in the neighborhood are collected to determine the plane parameters of the local window. An iterative strategy is adopted to gradually propagate disparity information from valid pixels to invalid areas. We apply the proposed method to disparity maps produced by two recent stereo matching methods, and compare the refining results with other post processing methods. Experimental results show the effectiveness of our method in improving dense disparity maps.

In dependent stereo image compression, the aim is to minimize the bitrate of disparity map and that of residual image. Traditionally, focus has been paid on either disparity map or residual image. In this paper, we compute an optimal disparity map (in terms of bitrates) by jointly exploiting the trade-off between the disparity map and the residual image. Firstly, the dense disparity map is obtained using existing optical flow technique. Secondly, the dense disparity map is quantized using a RD framework. Consequently, the resulting bitrate of the disparity map decreases significantly at the cost of a slight increase of the bitrate of the residual image. As a result, the overall bitrate attains minimum value. The proposed scheme is compatible and can be integrated in JPEG2000 framework.

Deep learning, big data, and the internet of things (IoT) have changed the world entirely. As an embedded computer, Raspberry Pi is playing a dynamic and prominent role in the era of ubiquitous computing. In ubiquitous computing, performance and real-time throughput are still an area of focus for Raspberry Pi. Indeed, process scheduling, page swapping, and process synchronization techniques are essential and crucial parameters of the operating system for Raspberry Pi. The key study objectives were; (i) explore the recent trend of applications of Raspberry Pi and (ii) comparison of process scheduling, page swapping and process synchronization techniques with different Raspberry Pi operating systems. The study concluded that Linux-based operating systems are offering an optimized and efficient computing environment for Raspberry Pi.

It is an important problem for computer vision to match the stereo image pair. Only the problems of stereo matching are solved, the accurate location or measurement of object can be realized. In this paper, an integrated stereo matching approach is presented. Unlike most stereo matching approach, it integrates area-based and feature-based primitives. This allows it to take advantages of the unique attributes of each of these techniques. The feature-based process is used to match the image feature. It can provide a more precise sparse disparity map and accurate location of discontinuities. The area-based process is used to match the continuous surfaces.It can provide a dense disparity map. The techniques of stereo matching with adaptive window are adopted in the area-based process. It can make the results of area-based process get high precise. An integrated process is also used in this approach. It can integrate the results of feature-based process and area-based process, so that the approach can provide not only a dense disparity map but also an accurate location of discontinuities. The approach has been tested by some synthetic and nature images. From the results of matched wedding cake and matched aircraft model, we can see that the surfaces and configuration are well reconstructed. The integrated stereo matching approach can be used in 3D part recognition in intelligent assembly system and computer vision.

This paper presents a new algorithm of disparity map estimation. The originality of this method lies in the process of dense disparity map estimation using dynamic programming constrained by interest points and using the higher order statistics (HOS) criteria for matching noisy images. Experiments with noisy real images have validated our method and have clearly shown the improvement over the existing ones. The dense disparity map obtained is more reliable when compared to the similar second-order statistics (SOS)-based dynamic programming and HOS-based correlation methods.

Moving objects segmentation is the foundation of intelligent moving body information collection. In the monocular vision system, it is too difficult to segment the foreground from background when their grayscale and color are similar. Compared with the grayscale and color, the scenery depth are less susceptible to external environment, if depth information can be got in stereo vision, it would be much easier to segment the foreground. Unfortunately, scenery accurate dense disparity map is often hard to get. In the paper, an optimizing method of calculating the dense disparity based on graph cut is used, and based on the dense disparity, the features of color and disparity are combined by graph cut to improve the accuracy of the image segmentation. The experimental results show that the method provides a better dense disparity map and also reduces the effect of light and strengthens the stability of segmentation by combining the features of color and disparity in graph cut optimization algorithm.

Three-dimensional reconstruction of internal organ surfaces provides useful information for better control and guidance of the operations of surgical tools for minimally invasive surgery (MIS). The current reconstruction techniques using stereo cameras are still challenging due to the difficulties in correspondence matching in MIS, since there is very limited texture but significant specular reflection on organ surfaces. This paper proposes a new approach to overcome the problem by introducing weakly structured light actively casting surgical tool shadows on organ surfaces. The contribution of this paper is twofold: first, we propose a robust approach to extract shadow edges from a sequence of shadowed images; second, we develop a novel field surface interpolation (FSI) approach to obtain an accurate and dense disparity map. Our approach does not rely on texture information and is able to reconstruct accurate 3-D information by exploiting shadows from surgical tools. One advantage is that the point correspondences are directly calculated and no explicit stereo matching is required, which ensures the efficiency of the method. Another advantage is the minimum hardware requirement because only stereo cameras and a separated single-point light source are required. We evaluated the proposed approach using both phantom models and ex vivo images. Based on the experimental results, we achieved the precision of the recovered 3-D surfaces within 0.7 mm for phantom models and 1.2 mm for ex vivo images. The comparison of disparity maps indicates that with the addition of shadows, the proposed method significantly outperforms the state-of-the-art stereo algorithms for MIS.

A geometry constraint applicable to natural terrain is derived based on the analysis of lunar surface. With this constraint a multi-stage stereo matching algorithm is presented to provide the fairly accurate and dense disparity map needed for lunar rover autonomous navigation. Points with larger intensity gradient are matched with a multi-feature matching strategy in earlier stages. Then the information of matched points is used to limit search region in the matching of other points in later stages within the help of several geometry constraints. Finally the unmatched points are dealt with to resolve the occlusion caused by large raised objects. Experimental results with real images of various types of natural terrain indicate that the algorithm improves the accuracy of dense disparity maps effectively, which lays the foundation for the reconstruction of 3D terrain map.

3D reconstruction is the base of mobile robot navigation. And it is necessary to provide the environment information for robots effectively and rapidly. The traditional 3D reconstruction method relies on the dense disparity map which requires high computing ability, and there is large redundant data in the reconstructed 3D data. Unlike traditional methods, the proposed method needn't obtain the disparity map. We partitioned the scene scope uniformly into grids, and introduced virtual height lines(VHLs) which do not exist in the scene. Each VHL is projected in left image and right image. The intersection of VHL and the scene is only existent on projection line. So the point has maximum similarity metric. We used dynamic programming(DP) to find the matched lines in left image corresponding to the projections of VHLs(PVHLs) in right image. The intersection point of the matched line and the PVHL in left image is the answer. The experiments to reconstruct 3D scene are performed and the results show the proposed method is reliable and effective.

In this paper, we propose the dense disparity map-based pedestrian detection method for intelligent vehicle. The dense disparity map is utilized to improve the pedestrian detection performance. Our method consists of several steps namely, obstacle area detection using road feature information and column detection, pedestrian area detection using dense disparity map-based segmentation, and pedestrian detection using optimal feature. The first step is to detect all obstacle areas using column detection and pedestrian height information. However, there are many objects in single obstacle area. Thus each obstacle area needs to be separated into single object for improving pedestrian detection performance. Thus, the second step is performed to segment the detected obstacle area. The last step is to detect only pedestrian using classifier trained by optimal feature. The optimal feature is extracted by positive and negative training images. ETH database is utilized to evaluate our proposed pedestrian detection method.

Census transform is a non-parametric local transform. Its weakness is that the results relied on the center pixel too much. This paper proposes a modified Census transform based on the neighborhood information for stereo matching. By improving the classic Census transform, the new technique utilizes more bits to represent the differences between the pixel and its neighborhood information. The result image of the modified Census transform has more detailed information at depth discontinuity. After stereo correspondence, sub-pixel interpolation and the disparity refinement, a better dense disparity map can be obtained. The experiments present that the proposed algorithm has simple mechanism and strong robustness. It can improve the accuracy of matching and is applicable to hardware systems.

The stereo matching algorithm is a technique that analyses two or more images captured at diverse view points in order to find positions in real 3D space for the pixels of 2D image. The stereo matching methods have been used in various fields such as drawing the topographical map from aerial photograph and finding the depth information of objects in machine vision system. Nowdays, optical motion capture techniques using stereo matching algorithms are being developed for visual applications such as virtual reality or 3D graphics. Stereo matching algorithms can be generally classified into two methods: feature-based and area-based ones. The feature-based method matches the feature elements between two images, and uses the interpolation to obtain the disparity information for the pixels other than the feature elements, while the area-based method performs matching between pixels in two images by calculating the correlations of the pixels residing in the search window. Area-based method cannot match feature element with more accuracy than feature-based method even though it can make more dense disparity map. Moreover, it has more possibility of error in the area of insufficient texture information or depth discontinuities. In this paper we present a novel technique for area-based stereo matching algorithm which provides more accurate and error-prone matching capabilities by using adaptive search range and window size. We propose two new strategies (1) for determining search range adaptively from the disparity map and multiresolutional images of region segments obtained by applying feature-based algorithm, and (2) for changing the window size adaptively according to the edge information derived from the wavelet transform such that the combination of two adaptive methods in search range and window size greatly enhance accuracy while reducing errors. We test our matching algorithms for various types of images, and shall show the outperformance of our stereo matching algorithm.

Stereo matching is to obtain disparity map from two or more images whose pixels existing disparity between each other. Aim at the difficulty of meeting both accuracy and speed when using the adaptive support-weight algorithm, an improved adaptive support-weight sparse region-based algorithm accorded with HVS is proposed. First, improve the traditional support-weight formula, and the support-weight of the points can be calculated according to the improved support-weight formula. Second, the dense disparity map is obtained by sparse region-based matching. Third, left-right consistency check and blocking filling is performed for the obtained disparity map. Finally, median filter is used to remove isolated mismatching points and noise points. Experiment results show that, using the presented method, the matching efficiency is above 90 times faster than that of the adaptive support-weight algorithm proposed by Yoon; and the matching accuracy is 12.34% higher than that of SSD. So the improved algorithm is verified for obtaining accurate disparity map fast and meeting the requirements of system practicability.

Aiming at the problem that the similarity measures in the overwhelming majority of stereo matching approaches rely heavily on the statistical characteristics of image gray values,a stereo matching algorithm insensitive to differentiations in illumination is proposed.Firstly,the Census non-parametric transform is investigated and its limitations are analyzed. Secondly,in order to take the pixels' spatial location information into consideration when searching stereo correspondences, gray values of the neighborhood pixels,whose relative positions are one unit greater than the center pixel in the transformation window,are replaced by gray values' interpolation with the four pixels adjacent to them.Finally,the Census non-parametric transform of images and stereo matching are implemented,and then,dense disparity map is obtained.The experiment results show that the proposed method has simple mechanism,high computational efficiency and strong robustness;satisfactory stereo matching results also can be obtained even under non-ideal illumination conditions.

An algorithm for disparity estimation with energy-based variational regularization using three image views is presented. In the new algorithm, dense disparity maps, including the disparity values for feature points, are first estimated by phase-based methods. Then a set of coupled partial differential equations (PDEs) are solved to refine these disparity maps together with the feature information. The visual appearance of the refined disparity is better than that of the results using similar variational regularization involving only one stereo pair of images. In addition, a new effective occlusion detection scheme is presented with good results.