Approximate Labeling via Graph Cuts Based on Linear Programming

Recent work on early vision such as image segmentation, image restoration, stereo matching, and optical flow models these problems using Markov Random Fields. Although this formulation yields an NP-hard energy minimization problem, good heuristics have been developed based on graph cuts and belief propagation. Nevertheless both approaches still require tens of seconds to solve stereo problems on recent PCs. Such running times are impractical for optical flow and many image segmentation and restoration problems. We show how to reduce the computational complexity of belief propagation by applying the Four Color Theorem to limit the maximum number of labels in the underlying image segmentation to at most four. We show that this provides substantial speed improvements for large inputs to a variety of vision problems, while maintaining competitive result quality.KeywordsImage SegmentationBelief PropagationMarkov Random FieldsImage RestorationStereo MatchThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

We introduce a new approach for image filtering in a Bayesian framework, in this case the probability density function (pdf) of the likelihood function is approximated using the concept of non-parametric or kernel estimation. The method is also based on generalized Gaussian Markov random fields (GGMRF), a class of Markov random fields which are used as prior information into the Bayesian rule, which principal objective is to eliminate those effects caused by the excessive smoothness on the reconstruction process of images which are rich in contours or edges. Accordingly to the hypothesis made for the present work, it is assumed a limited knowledge of the noise pdf, so the idea is to use a non-parametric estimator to estimate such a pdf and then apply the entropy to construct the cost function for the likelihood term. The previous idea leads to the construction of Maximum a posteriori (MAP) robust estimators, since the real systems are always exposed to continuous perturbations of unknown nature. Some promising results of three new MAP entropy estimators (MAPEE) for image filtering are presented, together with some partial concluding remarks.

With advancements in robust stereo matching and optical flow estimation networks, models pre-trained on synthetic data demonstrate strong robustness to unseen domains. However, their robustness can be seriously degraded when fine-tuning them in real-world scenarios. This paper investigates fine-tuning stereo matching and optical flow estimation networks without compromising their robustness to unseen domains. Specifically, we divide the pixels into consistent and inconsistent regions by comparing Ground Truth (GT) with Pseudo Label (PL) and demonstrate that the imbalance learning of consistent and inconsistent regions in GT causes robustness degradation. Based on our analysis, we propose the DKT framework, which utilizes PL to balance the learning of different regions in GT. The core idea is to utilize an exponential moving average (EMA) teacher to measure what the student network has learned and dynamically adjust the learning regions. We further propose the DKT++ framework, which improves target-domain performances and network robustness by applying slow-fast update teachers to generate more accurate PL, introducing the unlabeled data and synthetic data. We integrate our frameworks with state-of-the-art networks and evaluate their effectiveness on several real-world datasets. Extensive experiments show that our method effectively preserves the robustness of stereo matching and optical flow networks during fine-tuning.

Stereo matching is the process of finding disparity map between a pair of stereo images. Markov Random Field (MRF) model has been used in stereo matching algorithms in which the goal is to minimize total data, smoothness and occlusion cost. Recent MRF based stereo matching algorithms assign the same smoothness cost to adjacent pixels which is not effective at objects boundaries. In this paper, we propose a method to constrain the smoothness cost such that the cost applied to disparity discontinuity within an object is higher than that of the object's boundary. Data cost function is also normalized before computing its probability distribution function (PDF) to enforce an uniform PDF for the entire image. Experimental results have demonstrated that the proposed method is competitive to the state-of-the-art stereo matching algorithms.

We consider approximation algorithms for the metric labeling problem. This problem was introduced in a recent paper by Kleinberg and Tardos [20], and captures many classification problems that arise in computer vision and related fields. They gave an O(log k log log k) approximation for the general case where k is the number of labels and a 2-approximation for the uniform metric case. More recently, Gupta and Tardos [15] gave a 4-approximation for the truncated linear metric, a natural non-uniform metric motivated by practical applications to image restoration and visual correspondence. In this paper we introduce a natural integer programming formulation and show that the integrality gap of its linear relaxation either matches or improves the ratios known for several cases of the metric labeling problem studied until now, providing a unified approach to solving them. In particular, we show that the integrality gap of our LP is bounded by O(log k log log k) for general metric and 2 for the uniform metric thus matching the ratios in [20]. We also develop an algorithm based on our LP that achieves a ratio of 2 + √2 s 3.414 for the truncated linear metric improving the ratio given in [15]. Our algorithm uses the fact that the integrality gap of our LP is 1 on a linear metric. We believe that our formulation has the potential to provide improved approximation algorithms for the general case and other useful special cases. Finally, our formulation admits general non-metric distance functions. This leads to a non-trivial approximation guarantee for a non-metric case that arises in practice [21], namely the truncated quadratic distance function. We note here that there are non-metric distance functions for which no bounded approximation ratio is achievable.

The maximum a posteriori assignment for general structure Markov random fields is computationally intractable. In this paper, we exploit tree-based methods to efficiently address this problem. Our novel method, named Tree-based Iterated Local Search (T-ILS), takes advantage of the tractability of tree-structures embedded within MRFs to derive strong local search in an ILS framework. The method efficiently explores exponentially large neighborhoods using a limited memory without any requirement on the cost functions. We evaluate the T-ILS on a simulated Ising model and two real-world vision problems: stereo matching and image denoising. Experimental results demonstrate that our methods are competitive against state-of-the-art rivals with significant computational gain.

We present a framework based on modular integration and multiresolution for restoring images. We model the image as a Markov random field (MRF) and propose a restoration algorithm. In essence, the problem of image restoration requires learning of the MRF model and noise parameters which are used to restore degraded images. In the developed scheme, there exists interaction between the model learning module and the image restoration module. A method based on homotopy continuation is used for unsupervised model learning and the restoration is achieved through the minimization of an energy function.

For medical application, we are interested in estimation of optical flow on the face and particularly on area around the eyes. Among the methods of optical flow estimation, gradient estimation and block matching are the main ones. However the gradient based approach can only be applied for small displacement. Generally the process of block matching gives only good result if the searching strategy is judiciously selected. Our approach is based on a Markov random field model combined with an algorithm of block matching in a multiresolution scheme. The multiresolution approach leads to detect a large range of displacement amplitude. The large displacements are detected on the coarse scales and the small ones will be detected successively on finer scales in a coarse scales and the small ones will be detected successively on finer scales. The tracking of motion is achieved by a block matching algorithm. This method gives the otpical flow whatever the amplitude of the motion is, if included in the range defined by the multiresolution approach. The result show clearly the complementary of Markov random fields estimation and block matching across the scales.

Bayesian estimation of two-dimensional stratified structures is described. The major point addressed is the derivation of a statistical prior model that adequately describes such layered media. This problem is of interest in applications in which the data are generally processed in one dimension only. In order to take local interactions into account, a Markovian description is used. The model is derived so as to fulfill a set of constraints that summarize physical and geometrical characteristics of the problem as well as practical requirements. The resulting class of Markov random fields presents a unilateral structure on a nonrectangular lattice and a hierarchical organization which involves a line process. In addition, it is shown to be an extension of one-dimensional models already in use. The properties of the model are investigated, and its practicality is demonstrated by an application to seismic deconvolution. Simulation results show significant improvements with respect to the usual one-dimensional methods. >

Stationary reciprocal processes defined on a finite interval of the integer line can be seen as a special class of Markov random fields restricted to one dimension. Nonstationary reciprocal processes have been extensively studied in the past especially by Jamison et al. The specialization of the nonstationary theory to the stationary case, however, does not seem to have been pursued in sufficient depth in the literature. Stationary reciprocal processes (and reciprocal stochastic models) are potentially useful for describing signals which naturally live in a finite region of the time (or space) line. Estimation or identification of these models starting from observed data seems still to be an open problem which can lead to many interesting applications in signal and image processing. In this paper, we discuss a class of reciprocal processes which is the acausal analog of auto-regressive (AR) processes, familiar in control and signal processing. We show that maximum likelihood identification of these processes leads to a covariance extension problem for block-circulant covariance matrices. This generalizes the famous covariance band extension problem for stationary processes on the integer line. As in the usual stationary setting on the integer line, the covariance extension problem turns out to be a basic conceptual and practical step in solving the identification problem. We show that the maximum entropy principle leads to a complete solution of the problem.

Sparse matrix decompositions and graph characterizations

In this article, we analyse right censored survival data by modelling their common hazard function nonparametrically. The hazard rate is assumed to be a stochastic process, with sample paths taking the form of step functions. This process jumps at times that form a time-homogeneous Poisson process, and a class of Markov random fields is used to model the values of these sample paths. Features of the posterior distribution, such as the mean hazard rate and survival probabilities, are evaluated using the Metropolis--Hastings--Green algorithm. We illustrate our methodology by simulation examples.

Mammography associated with clinical breast examination is the only effective method for mass breast screening. Microcalcifications are one of the primary signs for early detection of breast cancer. In this paper 1 we propose a new kernel method for classification of difficult-to-diagnose regions in mammographic images. It consists of a novel class of Markov Random Fields, using techniques developed within the context of statistical mechanics. This method is used for the classification of positive Region of Interest (ROI's) containing clustered microcalcifications and negative ROI's containing normal tissue. The obtained results show that the proposed approach can be successfully employed for detection of microcalcifications.

Many vision applications have been formulated as Markov Random Field (MRF) problems. Although many of them are discrete labeling problems, continuous formulation often achieves great improvement on the qualities of the solutions in some applications such as stereo matching and optical flow. In continuous formulation, however, it is much more difficult to optimize the target functions. In this paper, we propose a new method called fusion move driven Markov Chain Monte Carlo method (MCMC-F) that combines the Markov Chain Monte Carlo method and the fusion move to solve continuous MRF problems effectively. This algorithm exploits powerful fusion move while it fully explore the whole solution space. We evaluate it using the stereo matching problem. We empirically demonstrate that the proposed algorithm is more stable and always finds lower energy states than the state-of-the art optimization techniques.

This paper proposes a stochastic approach to estimate the occlusion and disparity fields of stereoscopic images. The fields are estimated by the Bayesian maximum a posteriori (MAP) framework and Markov random field (MRF) models. The occlusion field model is based on the stochastic observation that the probability distribution in MAP estimator is relatively unstable and uniform at occluded regions. The occlusion is explicitly modelled as MRF, and is estimated in an energy optimization method called stochastic diffusion. The detected occluded region is compensated for the re-estimated disparity field in the same stochastic diffusion and the dynamic programming approach. Experimental results show good occlusion detection and disparity estimation. These results show the novel stochastic approach is suitable for occlusion detection and disparity estimation.