Blurred Image Deconvolution Using Gaussian Scale Mixtures Model in Wavelet Domain

A statistical model for images decomposed in an overcomplete wavelet pyramid is described. Each neighborhood of pyramid coefficients is modeled as the product of a Gaussian vector of known covariance, and an independent hidden positive scalar random variable. We propose an efficient Bayesian estimator for the pyramid coefficients of an image degraded by linear distortion (e.g., blur) and additive Gaussian noise. We demonstrate the quality of our results in simulations over a wide range of blur and noise levels.

Image deconvolution is formulated in the wavelet domain under the Bayesian framework. The well-known sparsity of the wavelet coefficients of real-world images is modeled by heavy-tailed priors belonging to the Gaussian scale mixture (GSM) class; i.e., priors given by a linear (finite of infinite) combination of Gaussian densities. This class includes, among others, the generalized Gaussian, the Jeffreys, and the Gaussian mixture priors. Necessary and sufficient conditions are stated under which the prior induced by a thresholding/shrinking denoising rule is a GSM. This result is then used to show that the prior induced by the "nonnegative garrote" thresholding/shrinking rule, herein termed the garrote prior, is a GSM. To compute the maximum a posteriori estimate, we propose a new generalized expectation maximization (GEM) algorithm, where the missing variables are the scale factors of the GSM densities. The maximization step of the underlying expectation maximization algorithm is replaced with a linear stationary second-order iterative method. The result is a GEM algorithm of O(N log N) computational complexity. In a series of benchmark tests, the proposed approach outperforms or performs similarly to state-of-the art methods, demanding comparable (in some cases, much less) computational complexity.

We describe an efficient generalized expectation maximization algorithm for estimating the spectral features of a noise source corrupting an observed image. We use a statistical model for images decomposed in an overcomplete oriented pyramid. Each neighborhood of clean pyramid coefficients is modeled as a Gaussian scale mixture, whereas the noise is assumed Gaussian. Combining this GEM technique with a previous Bayesian denoise estimator, we obtain a full blind denoising algorithm, able to deal with homogeneous, Gaussian or mesokurtotic, noise sources of arbitrary covariance. Results demonstrate the high performance of the method for a wide range of corruption sources.

Wavelet-based demosaicing techniques have the advantage of being computationally relatively fast, while having a reconstruction performance that is similar to state-of-the-art techniques. Because the demosaicing rules are linear, it is fairly simple to integrate denoising into the demosaicing. In this paper, we present a method that performs joint denoising and demosaicing, using a Gaussian Scale Mixture (GSM) prior model, thereby modeling the local edge direction as a hidden variable. The results indicate that this technique offers a better reconstruction performance (in PSNR sense and visually) than sequential demosaicing and denoising. On a recent GPU, our algorithm takes 3.5 s for reconstructing a 12 megapixel RAW digital camera image.

A wavelet domain, nonlinear inverse scattering approach is presented for imaging subsurface defects in a material sample, given observations of scattered thermal waves. Unlike methods using the Born linearization, our inversion scheme is based on the full wave-field model describing the propagation of thermal waves. Multiresolution techniques are employed to regularize and to lower the computational burden of this ill-posed imaging problem. We use newly developed wavelet-based regularization methods to resolve better the edge structures of defects relative to reconstructions obtained with smoothness-type regularizers. A nonlinear approximation to the exact forward-scattering model is introduced to simplify the inversion with little loss in accuracy. We demonstrate this approach on cross-section imaging problems by using synthetically generated scattering data from transmission and backprojection geometries.

Sparse acoustical holography from iterated Bayesian focusing

EM (expectation-maximization) algorithm is a classical method for parameter estimation of HMM (Hidden Markov model ). Concerning that EM algorithm is easily affected by initial parameter values, we proposed a mixture splitting algorithm based on decision boundary confusion (DBC) to describe more about boundary distribution. The algorithm mainly includes three aspects: firstly the number of incremented mixtures for every decision boundary could be determined according to decision boundary confusion; secondly the mixtures which are the closest to the decision boundary are chosen to split, thirdly the split mean of mixture is in the direction of decision boundary. Our experiments show that our proposed algorithm is more effective for classification using HMM.

In this work, an expectation maximization (EM) algorithm is derived for maximum likelihood estimation of the autocorrelation function of a linear distortion channel as well as the level of additive noise, under the assumption that the source signal comes from a mixture of Gaussian sources. To facilitate parameter initialization in the EM algorithm, a correlation-matching based estimation algorithm is developed for the channel autocorrelation function. The proposed EM algorithm was evaluated on speech-derived simulated data of multiple autoregressive Gaussian sources and real speech of isolated digits under signal-to-noise ratios (SNRs) of 20 dB down to 0 dB. The algorithm is shown to produce convergent estimation results as well as estimates of signal statistics that lead to significantly improved classification accuracy under additive and convolutive noise conditions.

In this paper, a computationally efficient algorithm is presented for joint maximum likelihood (ML) timing and carrier phase estimation of OFDM systems employing M-PSK modulation scheme with additive Gaussian noise, based on the expectation-maximization (EM) algorithm. A nondata-aided (NDA) scheme is considered for the joint timing and phase synchronizer which maximizes the low SNR limit of the likelihood function averaged over the M-PSK signal constellation. For this, an EM algorithm is derived which estimates the timing offset and the phase rotations of each subcarrier iteratively and which converges to the true ML estimation of the unknown timing and phase. It is shown that the algorithm becomes independent of the signal-to-noise ratio for both low and high SNR cases. The algorithm is applied to the QPSK modulated OFDM systems and it is concluded that for SNR values greater than 10 dB the convergence is achieved in first iteration and for SNR values less than 10 dB, at most in three iterations. It is also concluded that the convergence is independent of the initial starting points.

Image segmentation plays an important role in medical image processing. The aim of conventional hard segmentation methods is to assign a unique label to each voxel. However, due to the limited spatial resolution of medical imaging equipment and the complex anatomic structure of soft tissues, a single voxel in a medical image may be composed of several tissue types, which is called partial volume (PV) effect. Using the hard segmentation methods, the PV effect can substantially decrease the accuracy of quantitative measurements and the quality of visualizing different tissues. In this paper, instead of labeling each voxel with a unique label or tissue type, the percentage of different tissues within each voxel, which we call a mixture, was considered in establishing an image segmentation framework of maximum a posterior (MAP) probability. A new Markov random field (MRF) model was used to reflect the spatial information for the tissue mixture. Parameters of each tissue class were estimated through the expectation-maximization (EM) algorithm during the MAP tissue mixture segmentation. The MAP-EM mixture segmentation methodology was tested by digital phantom MR and patient CT images with PV effect evaluation. Results demonstrated that a hard segmentation method would lose a significant amount of details along the tissue boundaries, while the presented new PV segmentation method can dramatically improve the performance of preserving the details.

The Probabilistic RBF (PRBF) network constitutes an adaptation of the RBF network for classification. Moreover it extends the typical mixture model by allowing the sharing of mixture components among all classes, in contrast to the conventional approach that suggests mixture components describing only one class. The typical learning method of PRBF for a classification task employs the Expectation – Maximization (EM) algorithm. This widely used method depends strongly on the initial parameter values. The Greedy EM algorithm is a recently proposed method that tries to overcome this drawback, in the case of the density estimation problem using mixture models. In this work we propose a similar approach for incremental training of the PRBF network for classification. The proposed algorithm starts with a single component and incrementally adds more components. After convergence the algorithm splits all the components of the network. The addition of a new component is based on criteria for detecting a region in the data space that is crucial for the classification task. Experimental results using several well-known classification datasets indicate that the incremental method provides solutions of superior classification performance.

Segmentation of MRI brain images plays a critical role in medical image processing and analysis. In this paper, a new method based on Markov Random Field (MRF) is proposed for segmentation of MR brain images. We consider the low-level MRF as a linear combination of Gaussians (LCG) with positive and negative component, and we use the modified Expectation-maximization (MEM) algorithm to estimate the mean, variance and proportion for each distribution.The MEM algorithm is sensitive to initial parameters, so we improve the method of initialization. In high-level MRF, we use Potts model to describe the label image. By using the Bayesian maximum a posterior (MAP) rule, the segmentation problem is converted to precisely identify the models parameters. The MAP estimation is obtained using the Metroplis algorithm to search the optimization. The experimental results show that the proposed method is effective for segmentation of MR brain images.

The expectation-maximization (EM) algorithm can compute the maximum-likelihood (ML) or maximum a posterior (MAP) point estimate of the mixture models or latent variable models such as latent Dirichlet allocation (LDA), which has been one of the most popular probabilistic topic modeling methods in the past decade. However, batch EM has high time and space complexities to learn big LDA models from big data streams. In this paper, we present a fast online EM (FOEM) algorithm that infers the topic distribution from the previously unseen documents incrementally with constant memory requirements. Within the stochastic approximation framework, we show that FOEM can converge to the local stationary point of the LDA's likelihood function. By dynamic scheduling for the fast speed and parameter streaming for the low memory usage, FOEM is more efficient for some lifelong topic modeling tasks than the state-of-the-art online LDA algorithms to handle both big data and big models (aka, big topic modeling) on just a PC.

This paper introduces an expectation-maximization (EM) algorithm for image restoration (deconvolution) based on a penalized likelihood formulated in the wavelet domain. Regularization is achieved by promoting a reconstruction with low-complexity, expressed in the wavelet coefficients, taking advantage of the well known sparsity of wavelet representations. Previous works have investigated wavelet-based restoration but, except for certain special cases, the resulting criteria are solved approximately or require demanding optimization methods. The EM algorithm herein proposed combines the efficient image representation offered by the discrete wavelet transform (DWT) with the diagonalization of the convolution operator obtained in the Fourier domain. Thus, it is a general-purpose approach to wavelet-based image restoration with computational complexity comparable to that of standard wavelet denoising schemes or of frequency domain deconvolution methods. The algorithm alternates between an E-step based on the fast Fourier transform (FFT) and a DWT-based M-step, resulting in an efficient iterative process requiring O(N log N) operations per iteration. The convergence behavior of the algorithm is investigated, and it is shown that under mild conditions the algorithm converges to a globally optimal restoration. Moreover, our new approach performs competitively with, in some cases better than, the best existing methods in benchmark tests.

We present an automatic segmentation method using the Maximum a posterior (MAP)-Markov random field (MRF) framework that possesses regional adaptive capability for the segmentation of MR brain images with the presence of noise and the intensity inhomogeneity. A spatial-varying Gaussian mixture (SVGM) is used to model the conditional probability distribution of each of the three brain tissues (WM, GM and CSF), and the MRF is used to represent prior probabilities. A three-component random vector consisting of spatial and intensity information is used in SVGM such that the model can represent more complex image characteristics, such as intensity inhomogeneity. The regional adaptive capability is achieved by imposing a similarity criterion that minimizes the intensity variation of the segmented tissues in local regions. Initial parameters of SVGM are estimated from a brain atlas using either the expectation maximization (EM) algorithm or modified k-mean (MKM) algorithm, and the iterated conditional modes (ICM) algorithm is used to perform the segmentation. The parameters estimation and the ICM algorithm are alternatively operated until a stopping criterion is reached. The implement of the method is validated by quantitatively comparing with other published classification methods on fifteen simulated and twenty in vivo brain volumes using the same evaluation criteria. Our algorithm demonstrates superior performance in all cases other than two, and is especially suitable for the volumes with higher noise and inhomogeneity. The complete experiments illustrate that the incorporations of the spatial information and the regional adaptive processing into MAP-MRF framework provide better solution to the segmentation of MR brain images.