Stereo image compression using VPIC

A binocular stereo system for images coded by visual pattern image coding (VPIC) is presented and evaluated. The use of VPIC for stereo vision applications is motivated by the capability of its coding primitives to reflect meaningful physical properties of projected real-scene surfaces: high-information edge regions and uniform regions. An algorithm for spatial matching of VPIC primitives is proposed. It involves the solution of a complex optimization problem satisfying VPIC-related constraints. A dense disparity map is obtained and then used, together with one of the VPIC coded images of the stereo pair, to predict the second stereo image. The proposed stereo system allows a very high compression of the overall stereo information. Experimental results are presented to illustrate the performance of both stereo VPIC matching and predicting algorithms in natural images.

In this paper, we propose a Kansei retrieval method based on the design pattern of traditional Japanese crafting object to provide a user with the desired presentation space in digital traditional Japanese crafting system. The visual quantitative feature values are extracted by using Visual Pattern Image Coding (VPIC). These values include the total number, the frequency, the dispersion rate and the deviation rate for different edges. The quantitative feature values for traditional Japanese crafting objects are registered in the multimedia database and the relation between Kansei words and the visual feature of traditional Japanese crafting objects are analyzed by using the questionnaire. Then, the visual features are compared with the quantitative feature values. Through the above process, we can find the relation between the design pattern components and edge types using VPIC. By finding this relation, the Kansei retrieval method can be realized.

In this series of two papers, a high-level overview of the Tabular Nearest Neighbor Encoding (TNE) algorithm is presented. The performance of TNE is analyzed using training images having different size, statistical properties, and noise level than the source image. TNE is compared with several published algorithms such as visual pattern image coding, JPEG, and EBLAST. The latter is a relatively new, high-compression image transform that has compression ratio CR approximately equals 200:1 that can be consistently achieved with low MSE. Analysis focuses on the ability of TNE to provide low to moderate compression ratios at high computational efficiency on small- to large-format text and surveillance images.

The increased use of power- and space-constrained embedded processors in a wide variety of autonomous imaging and surveillance applications implies increased speed of computational resources that follow image acquisition in the processing stream. In such early vision applications, one typically processes an entire image data stream prior to spatial downselection operations such as focus-of-attention involving area-of-interest (AOI) selection. Downselection is especially useful in the emerging technologies of spatiotemporal adaptive processing (STA) and biomimetic automated target recognition (ATR). Here, progressive data reduction employs operations or sub-algorithms that process fewer data but in a more involved manner at each step of an algorithm. Implementationally, the STAP approach is amenable to embedded hardware using processors with deep pipelines and fine-grained parallelism. In part 1 of this series of two papers, we showed how compression of an image or sequence of images could facilitated more efficient image computation or ATR by processing fewer (compressed) data. This technique, called compressive processing or compressive computation, typically utilizes fewer operations than the corresponding ATR or image processing operation over noncompressed data, and can produce space, time, error, and power (STEP) savings on the order of the compression ratio (CR). Part 1 featured algorithms for edge detection in images compressed by vector quantization (VQ), visual pattern image coding (VPIC), and EBLAST, which is a recently-reported block-oriented high- compression transform. In this paper, we continue presentation of theory for compressive computation of early processing operations such as morphological erosion and dilation, as well as higher- level operations such as connected component labeling. We also discuss the algorithm and hardware modeling technique that supports analysis and verification of compressive processing efficiency. This methodology emphasizes 1) translation of each image processing algorithm or operation to a prespecified compressive format, 2) determination of the operation mix M for each algorithms produced in Step 1), and 3) simulation of M on various architectural models, to estimate performance. As in part 1, algorithms are expressed in image algebra, a rigorous, concise notation that unified linear and nonlinear mathematics in the image domain.© (2000) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.

In dependent stereo image compression, the aim is to minimize the bitrate of disparity map and that of residual image. Traditionally, focus has been paid on either disparity map or residual image. In this paper, we compute an optimal disparity map (in terms of bitrates) by jointly exploiting the trade-off between the disparity map and the residual image. Firstly, the dense disparity map is obtained using existing optical flow technique. Secondly, the dense disparity map is quantized using a RD framework. Consequently, the resulting bitrate of the disparity map decreases significantly at the cost of a slight increase of the bitrate of the residual image. As a result, the overall bitrate attains minimum value. The proposed scheme is compatible and can be integrated in JPEG2000 framework.

Unsupervised learning of depth estimation from imperfect rectified stereo laparoscopic images

Disparity refinement is an essential step of local stereo matching methods to produce fine dense disparity maps. The inherent defect of local stereo methods results in erroneous disparity in occluded areas. In this paper, we present a novel post processing method which can effectively improve the accuracy of dense disparity maps by rectifying disparity errors iteratively. Invalid disparities are first detected by left-right consistency check and color-disparity consistency check. For each invalid pixel, supports from valid pixels in the neighborhood are collected to determine the plane parameters of the local window. An iterative strategy is adopted to gradually propagate disparity information from valid pixels to invalid areas. We apply the proposed method to disparity maps produced by two recent stereo matching methods, and compare the refining results with other post processing methods. Experimental results show the effectiveness of our method in improving dense disparity maps.

The stereo matching algorithm is a technique that analyses two or more images captured at diverse view points in order to find positions in real 3D space for the pixels of 2D image. The stereo matching methods have been used in various fields such as drawing the topographical map from aerial photograph and finding the depth information of objects in machine vision system. Nowdays, optical motion capture techniques using stereo matching algorithms are being developed for visual applications such as virtual reality or 3D graphics. Stereo matching algorithms can be generally classified into two methods: feature-based and area-based ones. The feature-based method matches the feature elements between two images, and uses the interpolation to obtain the disparity information for the pixels other than the feature elements, while the area-based method performs matching between pixels in two images by calculating the correlations of the pixels residing in the search window. Area-based method cannot match feature element with more accuracy than feature-based method even though it can make more dense disparity map. Moreover, it has more possibility of error in the area of insufficient texture information or depth discontinuities. In this paper we present a novel technique for area-based stereo matching algorithm which provides more accurate and error-prone matching capabilities by using adaptive search range and window size. We propose two new strategies (1) for determining search range adaptively from the disparity map and multiresolutional images of region segments obtained by applying feature-based algorithm, and (2) for changing the window size adaptively according to the edge information derived from the wavelet transform such that the combination of two adaptive methods in search range and window size greatly enhance accuracy while reducing errors. We test our matching algorithms for various types of images, and shall show the outperformance of our stereo matching algorithm.

It is an important problem for computer vision to match the stereo image pair. Only the problems of stereo matching are solved, the accurate location or measurement of object can be realized. In this paper, an integrated stereo matching approach is presented. Unlike most stereo matching approach, it integrates area-based and feature-based primitives. This allows it to take advantages of the unique attributes of each of these techniques. The feature-based process is used to match the image feature. It can provide a more precise sparse disparity map and accurate location of discontinuities. The area-based process is used to match the continuous surfaces.It can provide a dense disparity map. The techniques of stereo matching with adaptive window are adopted in the area-based process. It can make the results of area-based process get high precise. An integrated process is also used in this approach. It can integrate the results of feature-based process and area-based process, so that the approach can provide not only a dense disparity map but also an accurate location of discontinuities. The approach has been tested by some synthetic and nature images. From the results of matched wedding cake and matched aircraft model, we can see that the surfaces and configuration are well reconstructed. The integrated stereo matching approach can be used in 3D part recognition in intelligent assembly system and computer vision.

Previous work has shown that independent component analysis (ICA) applied to feature extraction from natural image data yields features resembling Gabor functions and simple-cell receptive fields. This article considers the effects of including chromatic and stereo information. The inclusion of colour leads to features divided into separate red/green, blue/yellow, and bright/dark channels. Stereo image data, on the other hand, leads to binocular receptive fields which are tuned to various disparities. The similarities between these results and the observed properties of simple cells in the primary visual cortex are further evidence for the hypothesis that visual cortical neurons perform some type of redundancy reduction, which was one of the original motivations for ICA in the first place. In addition, ICA provides a principled method for feature extraction from colour and stereo images; such features could be used in image processing operations such as denoising and compression, as well as in pattern recognition.

Dense stereo image matching in remotely sensed images is a challenging problem, though it has been studied for more than two decades, due to occlusions, discontinuities, geometric, and radiometric distortions. A novel multiobjective fitness function-based dense stereo matching approach using genetic algorithms (GAs) is proposed in this paper. The proposed method is useful for estimating dense disparity map with an improved number of inliers for a stereo image pair, despite the constraint of finding correct disparity at depth discontinuities. In this paper, the steps of GA, such as initialization of the population, fitness function, and crossover and mutation operation, are designed and implemented to effectively deal with the problem of dense stereo image matching. To initialize the population, a Scale Invariant Feature Transform (SIFT) descriptor is computed for each pixel and multiple-size window-based matching is performed, using the similarity measures: 1) Euclidean distance and 2) spectral angle mapper. The generated disparity maps are pruned to choose a suitable subset using the designed fitness functions, considering the constraints related to stereo image pair, such as epipolar constraint, which encodes the epipolar geometry and the similarity measure that is useful to decide accuracy of the correspondences. The two objective functions are the number of inliers computed using the fundamental matrix and an energy minimization function, considering discontinuities and occlusions. The usefulness of this approach for remotely sensed stereo image pairs is demonstrated by improving the number of inliers and favorably comparing with state-of-the-art dense stereo image matching methods.

This paper presents a new algorithm of disparity map estimation. The originality of this method lies in the process of dense disparity map estimation using dynamic programming constrained by interest points and using the higher order statistics (HOS) criteria for matching noisy images. Experiments with noisy real images have validated our method and have clearly shown the improvement over the existing ones. The dense disparity map obtained is more reliable when compared to the similar second-order statistics (SOS)-based dynamic programming and HOS-based correlation methods.

Moving objects segmentation is the foundation of intelligent moving body information collection. In the monocular vision system, it is too difficult to segment the foreground from background when their grayscale and color are similar. Compared with the grayscale and color, the scenery depth are less susceptible to external environment, if depth information can be got in stereo vision, it would be much easier to segment the foreground. Unfortunately, scenery accurate dense disparity map is often hard to get. In the paper, an optimizing method of calculating the dense disparity based on graph cut is used, and based on the dense disparity, the features of color and disparity are combined by graph cut to improve the accuracy of the image segmentation. The experimental results show that the method provides a better dense disparity map and also reduces the effect of light and strengthens the stability of segmentation by combining the features of color and disparity in graph cut optimization algorithm.

A geometry constraint applicable to natural terrain is derived based on the analysis of lunar surface. With this constraint a multi-stage stereo matching algorithm is presented to provide the fairly accurate and dense disparity map needed for lunar rover autonomous navigation. Points with larger intensity gradient are matched with a multi-feature matching strategy in earlier stages. Then the information of matched points is used to limit search region in the matching of other points in later stages within the help of several geometry constraints. Finally the unmatched points are dealt with to resolve the occlusion caused by large raised objects. Experimental results with real images of various types of natural terrain indicate that the algorithm improves the accuracy of dense disparity maps effectively, which lays the foundation for the reconstruction of 3D terrain map.

Census transform is a non-parametric local transform. Its weakness is that the results relied on the center pixel too much. This paper proposes a modified Census transform based on the neighborhood information for stereo matching. By improving the classic Census transform, the new technique utilizes more bits to represent the differences between the pixel and its neighborhood information. The result image of the modified Census transform has more detailed information at depth discontinuity. After stereo correspondence, sub-pixel interpolation and the disparity refinement, a better dense disparity map can be obtained. The experiments present that the proposed algorithm has simple mechanism and strong robustness. It can improve the accuracy of matching and is applicable to hardware systems.