Identification of a Linear System with Switching Parameters by the EM Algorithm

The ability to theoretically model the propagation of photon noise through PET and SPECT tomographic reconstruction algorithms is crucial in evaluating the reconstructed image quality as a function of parameters of the algorithm. In a previous approach for the important case of the iterative ML - EM (maximum-likelihood - expectation-maximization) algorithm, judicious linearizations were used to model theoretically the propagation of a mean image and a covariance matrix from one iteration to the next. Our analysis extends this approach to the case of MAP (maximum a posteriori) - EM algorithms, where the EM approach incorporates prior terms. We analyse in detail two cases: a MAP - EM algorithm incorporating an independent gamma prior, and a one-step-late (OSL) version of a MAP - EM algorithm incorporating a multivariate Gaussian prior, for which familiar smoothing priors are special cases. To validate our theoretical analyses, we use a Monte Carlo methodology to compare, at each iteration, theoretical estimates of mean and covariance with sample estimates, and show that the theory works well in practical situations where the noise and bias in the reconstructed images do not assume extreme values.

This paper discusses the problem of Non Data Aided (NDA) Signal to Noise Ratio (SNR) estimation of Binary Phase Shift keying (BPSK) modulated signals using the Expectation Maximization (EM) Algorithm. In addition, the Cramer-Rao Lower Bounds (CRLB) for the estimation of Data Aided (DA) and Non Data Aided (NDA) Signal to Noise Ratio (SNR) estimation is derived. Multiple Input Single Output (MISO) channels with Space Time Block Codes (STBC) is used. The EM algorithm is a method that finds the Maximum Likelihood (ML) solution iteratively when there are unobserved (hidden or missing) data. Extension of the proposed approach to other types of linearly modulated signals in estimating SNR is straight forward. The performance of the estimator is assessed using the NDA CRLBs. Alamouti coding technique is used in this paper with two transmit antennas and one receive antenna. The authors’ assumption is that the received signal is corrupted by additive white Gaussian noise (AWGN) with unknown variance, and scaled by fixed unknown complex channel gain. Monte Carlo simulations are used to show that the proposed estimator offers a substantial improvement over the conventional Single Input Single Output (SISO) NDA SNR estimator due to the use of the statistical dependences in space and time. Moreover, the proposed NDA SNR estimator works close to the NDA SNR estimator over Single Input Multiple Output (SIMO) channels.

Many diseases show dichotomous phenotypic variation but do not follow a simple Mendelian pattern of inheritance. Variances of these binary diseases are presumably controlled by multiple loci and environmental variants. A least-squares method has been developed for mapping such complex disease loci by treating the binary phenotypes (0 and 1) as if they were continuous. However, the least-squares method is not recommended because of its ad hoc nature. Maximum Likelihood (ML) and Bayesian methods have also been developed for binary disease mapping by incorporating the discrete nature of the phenotypic distribution. In the ML analysis, the likelihood function is usually maximized using some complicated maximization algorithms (e.g. the Newton-Raphson or the simplex algorithm). Under the threshold model of binary disease, we develop an Expectation Maximization (EM) algorithm to solve for the maximum likelihood estimates (MLEs). The new EM algorithm is developed by treating both the unobserved genotype and the disease liability as missing values. As a result, the EM iteration equations have the same form as the normal equation system in linear regression. The EM algorithm is further modified to take into account sexual dimorphism in the linkage maps. Applying the EM-implemented ML method to a four-way-cross mouse family, we detected two regions on the fourth chromosome that have evidence of QTLs controlling the segregation of fibrosarcoma, a form of connective tissue cancer. The two QTLs explain 50-60% of the variance in the disease liability. We also applied a Bayesian method previously developed (modified to take into account sex-specific maps) to this data set and detected one additional QTL on chromosome 13 that explains another 26% of the variance of the disease liability. All the QTLs detected primarily show dominance effects.

An optimal maximal ratio combiner (MRC) based on the expectation-maximization (EM) algorithm is devel- oped for noisy constant envelope signals transmitted over a Rayleigh fading channel. Instead of using a transmitted pilot signal with the data to estimate the combiner gains, the EM algorithm is used to perform this estimation. In the developed MRC, estimation of the transmitted data sequence is performed also by the EM algorithm. Estimation using the EM algo- rithm provides an iterative solution to the maximum likelihood (ML) approach. Therefore, the resulting receiver is opti- mum and does not suffer from the difficulties resulted from direct application of the ML procedure. One of these difficul- ties is the computational complexity which depends exponentially on the data sequence length. Introducing an iterative structure in the developed MRC achieves a linear computational complexity and enables efficient data extraction by the Viterbi algorithm when trellis coding is used.

Joint modeling of survival and longitudinal data has been studied extensively in the recent literature. The likelihood approach is one of the most popular estimation methods employed within the joint modeling framework. Typically, the parameters are estimated using maximum likelihood, with computation performed by the expectation maximization (EM) algorithm. However, one drawback of this approach is that standard error (SE) estimates are not automatically produced when using the EM algorithm. Many different procedures have been proposed to obtain the asymptotic covariance matrix for the parameters when the number of parameters is typically small. In the joint modeling context, however, there may be an infinite-dimensional parameter, the baseline hazard function, which greatly complicates the problem, so that the existing methods cannot be readily applied. The profile likelihood and the bootstrap methods overcome the difficulty to some extent; however, they can be computationally intensive. In this paper, we propose two new methods for SE estimation using the EM algorithm that allow for more efficient computation of the SE of a subset of parametric components in a semiparametric or high-dimensional parametric model. The precision and computation time are evaluated through a thorough simulation study. We conclude with an application of our SE estimation method to analyze an HIV clinical trial dataset.

In this paper, a computationally efficient algorithm is presented for joint maximum likelihood (ML) timing and carrier phase estimation of OFDM systems employing M-PSK modulation scheme with additive Gaussian noise, based on the expectation-maximization (EM) algorithm. A nondata-aided (NDA) scheme is considered for the joint timing and phase synchronizer which maximizes the low SNR limit of the likelihood function averaged over the M-PSK signal constellation. For this, an EM algorithm is derived which estimates the timing offset and the phase rotations of each subcarrier iteratively and which converges to the true ML estimation of the unknown timing and phase. It is shown that the algorithm becomes independent of the signal-to-noise ratio for both low and high SNR cases. The algorithm is applied to the QPSK modulated OFDM systems and it is concluded that for SNR values greater than 10 dB the convergence is achieved in first iteration and for SNR values less than 10 dB, at most in three iterations. It is also concluded that the convergence is independent of the initial starting points.

This paper introduces structural identification using expectation maximization (STRIDE), a novel application of the expectation maximization (EM) algorithm and approach for output-only modal identification. The EM algorithm can be used to estimate the maximum likelihood parameters of a state-space model. In this context, the state-space model represents the equation of motion for a linear dynamic system. STRIDE is an iterative procedure that uses Kalman filtering and Rauch-Tung-Striebel (RTS) smoothing equations to produce estimates of the unobserved states; these calculations are based on the observed data and prior estimates of the state-space parameters. With this information, the conditional likelihood of the model is maximized and the state-space parameters are updated at each iteration. Once an iteration meets user-prescribed convergence criterion, the algorithm ends—yielding maximum likelihood estimates (MLE) for the state-space model parameters. The modal properties of the structure are th...

Motivated by the many applications associated with estimation of sparse multivariate models, the estimation of sparse directional connectivity between the imperfectly measured nodes of a network is studied. Node dynamics and interactions are assumed to follow a multivariate autoregressive model driven by noise, and the observations are a noisy linear combination of the underlying node activities. The corresponding maximum a posteriori (MAP) problem is derived to estimate system parameters. Due to the intractability of the MAP problem, the expectation maximization (EM) framework is used to iteratively implement the MAP estimation. To impose sparsity, the EM algorithm is augmented with an $\ell _1$ regularization of the connectivity matrix. Multiple techniques have been used to lower the computational complexity. Importantly, an efficient coordinate descent algorithm utilizing a closed-form solution is designed to solve the $\ell _1$ -regularized sub-problem of the EM. Cholesky factors of the unknown covariance matrices are used directly in the optimization process in order to impose positive definiteness and guarantee the functionality of the $\ell _1$ optimization. The algorithm is first applied to synthetic data to evaluate the estimation accuracy. A comparison with the previous work over an extensive set of configurations shows that our method is superior under moderate to high sparsity. Finally, the algorithm is applied to real temperature data obtained from 98 stations across the U.S. mainland to identify the predictive interactions between the time series.

To overcome the curse of dimensionality in joint probability learning, recent work has proposed to recover the joint probability mass function (PMF) of an arbitrary number of random variables (RVs) from three-dimensional marginals, exploiting the uniqueness of tensor decomposition and the (unknown) dependence among the RVs. Nonetheless, accurately estimating three-dimensional marginals is still costly in terms of sample complexity. Tensor decomposition also poses a computationally intensive optimization problem. This work puts forth a new framework that learns the joint PMF using pairwise marginals that are relatively easy to acquire. The method is built upon nonnegative matrix factorization (NMF) theory, and features a Gram–Schmidt-like economical algorithm that works provably well under realistic conditions. Theoretical analysis of a recently proposed expectation maximization (EM) algorithm for joint PMF recovery is also presented. In particular, the EM algorithm is shown to provably improve upon the proposed pairwise marginal-based approach. Synthetic and real-data experiments are employed to showcase the effectiveness of the proposed approach.

Maximum likelihood based identification for nonlinear multichannel communications systems

A classical technique for reconstruction of emission tomography (ET) images from measured projections is based on the maximum likelihood (ML) estimation, achieved with the expectation maximization (EM) algorithm. We incorporate the wavelet transform (WT) and total variation (TV) based penalties into the ML framework, and compare performance of the EM algorithm and the previously proposed conjugate barrier (CB) algorithm. Using the WT- and TV-based penalties allows one to embed regularization procedures into the iterative process. In the case of the WT-based penalty, we impose a subset of wavelet coefficients with a desired resolution on the objective function. It appears that the CB algorithm outperforms substantially the EM algorithm in penalized reconstruction. The properties of the optimization algorithms along with WTand TV-based regularization are demonstrated on image reconstructions of a synthetic brain phantom, and the quality of reconstruction is compared with standard methods.

In this paper we propose an identification and restoration algorithm for image degraded by noise as well as blurring. In our proposed method, the original images are modeled as Markov Random Fields(MRF) and the blurred image is presented by the gaussian blurred model. To estimate the parameters involved in the image model and to restore the degraded images, we use Generalized Maximum Likelihood(GML) estimation. The GML consists of the two procedures which is a Maximum Likelihood(ML) estimation and a Maximum A Posteriori(MAP) estimation. These procedures are implemented by using the Expectation-Maximization(EM) algorithm and mean field annealing(MFA) based on the mean field theory.

ABSTRACTWe consider a Bayesian framework for estimating a high-dimensional sparse precision matrix, in which adaptive shrinkage and sparsity are induced by a mixture of Laplace priors. Besides discussing our formulation from the Bayesian standpoint, we investigate the MAP (maximum a posteriori) estimator from a penalized likelihood perspective that gives rise to a new nonconvex penalty approximating the ℓ0 penalty. Optimal error rates for estimation consistency in terms of various matrix norms along with selection consistency for sparse structure recovery are shown for the unique MAP estimator under mild conditions. For fast and efficient computation, an EM algorithm is proposed to compute the MAP estimator of the precision matrix and (approximate) posterior probabilities on the edges of the underlying sparse structure. Through extensive simulation studies and a real application to a call center data, we have demonstrated the fine performance of our method compared with existing alternatives. Supplementary materials for this article are available online.

Random effects in ordinal regression models

In this work, an expectation maximization (EM) algorithm is derived for maximum likelihood estimation of the autocorrelation function of a linear distortion channel as well as the level of additive noise, under the assumption that the source signal comes from a mixture of Gaussian sources. To facilitate parameter initialization in the EM algorithm, a correlation-matching based estimation algorithm is developed for the channel autocorrelation function. The proposed EM algorithm was evaluated on speech-derived simulated data of multiple autoregressive Gaussian sources and real speech of isolated digits under signal-to-noise ratios (SNRs) of 20 dB down to 0 dB. The algorithm is shown to produce convergent estimation results as well as estimates of signal statistics that lead to significantly improved classification accuracy under additive and convolutive noise conditions.