Unsupervised texture segmentation/classification using 2-D autoregressive modeling and the stochastic expectation-maximization algorithm

The stochastic approximation EM (SAEM) algorithm is a simulation-based alternative to the expectation/maximization (EM) algorithm for situations when the E-step is hard or impossible. One of the appeals of SAEM is that, unlike other Monte Carlo versions of EM, it converges with a fixed (and typically small) simulation size. Another appeal is that, in practice, the only decision that has to be made is the choice of the step size which is a one-time decision and which is usually done before starting the method. The downside of SAEM is that there exist no data-driven and/or model-driven recommendations as to the magnitude of this step size. We argue in this article that a challenging model/data combination coupled with an unlucky step size can lead to very poor algorithmic performance and, in particular, to a premature stop of the method. This article proposes a new heuristic for SAEM's step size selection based on the underlying EM rate of convergence. We also use the much-appreciated EM likelihood-ascent property to derive a new and flexible way of monitoring the progress of the SAEM algorithm. The method is applied to a challenging geostatistical model of online retailing.

Locus heterogeneity is a major problem plaguing the mapping of disease genes responsible for complex genetic traits via linkage analysis. A common feature of several available methods to account for heterogeneity is that they involve maximizing a multidimensional likelihood to obtain maximum likelihood estimates. The high dimensionality of the likelihood surface may be due to multiple heterogeneity (mixing) parameters, linkage parameters, and/or regression coefficients corresponding to multiple covariates. Here, we focus on this nontrivial computational aspect of incorporating heterogeneity by considering several likelihood maximization procedures, including the expectation maximization (EM) algorithm and the stochastic expectation maximization (SEM) algorithm. The wide applicability of these procedures is demonstrated first through a general formulation of accounting for heterogeneity, and then by applying them to two specific formulations. Furthermore, our simulation studies as well as an application to the Genetic Analysis Workshop 12 asthma datasets show that, among other observations, SEM performs better than EM. As an aside, we illustrate a limitation of the popular admixture approach for incorporating heterogeneity, proved elsewhere. We also show how to obtain standard errors (SEs) for EM and SEM estimates, using methods available in the literature. These SEs can then be combined with the corresponding estimates to provide confidence intervals of the parameters.

The majority of anomaly detection processes used for hyperspectral image data are based on pixel-by-pixel whitening and thresholding operations using local area statistics. This paper discusses an alternative approach to anomaly detection in which a mixture model is fitted to the whole of the image. This mixture model may be used to segment the image into component memberships and these may, in turn, be used for anomaly detection. In this study the mixture model is generated for the whole scene using the stochastic expectation maximization (SEM) algorithm. This is parameterized such that mixture components consisting of small numbers of pixels are eliminated. The maximum a-posteriori probability (MAP) mixture component for each pixel is then determined. The pixel may then be examined using a conventional statistical hypothesis test to see whether it is plausible that it was drawn from the distribution of the identified component, at a given significance level. This anomaly detection process has been examined using both synthetic and real hyperspectral imagery and results are presented here for real data containing no known military targets and for synthesized imagery which includes military target pixels. A range of results is presented for different parameterizations of the SEM algorithm and significance test. These results include the component map of the imagery and anomalous pixel maps at given significance levels.

A maximum likelihood estimation procedure is presented for the frailty model. The procedure is based on a stochastic Expectation Maximization algorithm which converges quickly to the maximum likelihood estimate. The usual expectation step is replaced by a stochastic approximation of the complete log-likelihood using simulated values of unobserved frailties whereas the maximization step follows the same lines as those of the Expectation Maximization algorithm. The procedure allows to obtain at the same time estimations of the marginal likelihood and of the observed Fisher information matrix. Moreover, this stochastic Expectation Maximization algorithm requires less computation time. A wide variety of multivariate frailty models without any assumption on the covariance structure can be studied. To illustrate this procedure, a Gaussian frailty model with two frailty terms is introduced. The numerical results based on simulated data and on real bladder cancer data are more accurate than those obtained by using the Expectation Maximization Laplace algorithm and the Monte-Carlo Expectation Maximization one. Finally, since frailty models are used in many fields such as ecology, biology, economy, ?, the proposed algorithm has a wide spectrum of applications.

Synthetic Aperture Radar (SAR) data presents a great potential for environmental monitoring applications and natural disaster management thanks to their insensitivity to atmospheric and Sun-illumination conditions. However, the automatic generation of change maps from multichannel SAR images acquired on the same geographic area at different times is still an open issue in the remote-sensing literature. In the present paper an automatic unsupervised contextual change-detection method is proposed for two-date multichannel SAR images, by integrating a SAR-specific extension of the Fisher transform with the Expectation-Maximization (EM) algorithm or with some of its variants (the Landgrebe-Jackson EM and the Stochastic EM algorithms), applied according to a Markov random field model for the image data. The method is validated by experiments on SIR-C/XSAR data.

The standard errors (SE) of the maximum likelihood estimates (MLE) of the population parameter vector in nonlinear mixed effect models (NLMEM) are usually estimated using the inverse of the Fisher information matrix (FIM). However, at a finite distance, i.e. far from the asymptotic, the FIM can underestimate the SE of NLMEM parameters. Alternatively, the standard deviation of the posterior distribution, obtained in Stan via the Hamiltonian Monte Carlo algorithm, has been shown to be a proxy for the SE, since, under some regularity conditions on the prior, the limiting distributions of the MLE and of the maximum a posterior estimator in a Bayesian framework are equivalent. In this work, we develop a similar method using the Metropolis-Hastings (MH) algorithm in parallel to the stochastic approximation expectation maximisation (SAEM) algorithm, implemented in the saemix R package. We assess this method on different simulation scenarios and data from a real case study, comparing it to other SE computation methods. The simulation study shows that our method improves the results obtained with frequentist methods at finite distance. However, it performed poorly in a scenario with the high variability and correlations observed in the real case study, stressing the need for calibration.

This paper presents a Field Programmable Gate Array (FPGA) based embedded system which is used to achieve high speed segmentation of 3D images. Segmentation is performed using Expectation-Maximization (EM) with Maximization of Posterior Marginals (MPM) Bayesian algorithm. This algorithm segments the 3D image using neighboring pixels based on a Markov Random Field (MRF) model. In this system, the embedded processor controls a custom circuit which performs the MPM and portions of the EM algorithm. The embedded processor completes the EM algorithm and also controls image data transmission between host computer and on-board memory. The whole system has been implemented on Xilinx Virtex 6 FPGA and achieved over 100 times processing improvement compared to standard desktop computer. Three new techniques were the key to achieve this speed: Pipelined computational cores, sixteen parallel data paths and a novel memory interface for maximizing the external memory bandwidth.

Brain vessel segmentation is a fundamental component of cerebral disease screening systems. However, detecting vessels is still a challenging task owing to their complex appearance and thinning geometry as well as the contrast decrease from the root of the vessel to its thin branches. We present a method for segmentation of the vasculature in Magnetic Resonance Angiography (MRA) images. First, we apply volume projection, 2D segmentation, and back-projection procedures for first stage of background subtraction and vessel reservation. Those labeled as background or vessel voxels are excluded from consideration in later computation. Second, stochastic expectation maximization algorithm (SEM) is used to estimate the probability density function (PDF) of the remaining voxels, which are assumed to be mixture of one Rayleigh and two Gaussian distributions. These voxels can then be classified into background, middle region, or vascular structure. Third, we adapt the K-means method which is based on the gradient of remaining voxels to effectively detect true positives around boundaries of vessels. Experimental results on clinical cerebral data demonstrate that using gradient information as a further step improves the mixture model based segmentation of cerebral vasculature, in particular segmentation of the low contrast vasculature.

Adaptive Mixture Estimation and Unsupervised Local Bayesian Image Segmentation

ABSTRACTFactor analysis (FA) can be used to identify key biomarkers in biological processes by assuming that latent biological pathways (statistically, “latent factors”) drive the activity of measurable biomarkers (“observed variables”). However, biological pathways often interact, meaning that the classical FA assumption of independence between factors is questionable. Motivated by sparsely and irregularly collected longitudinal measurements of metabolites in a COVID‐19 study, we propose a dynamic factor analysis model that accounts for cross‐correlations between pathways via a multi‐output Gaussian processes (MOGP) prior on the factor trajectories. To mitigate against overfitting caused by sparsity of longitudinal measurements, we introduce a roughness penalty upon MOGP hyperparameters and allow for non‐zero mean functions. We also propose a scalable stochastic expectation maximization (StEM) algorithm that, in simulations, is both 20 times faster and provides more accurate and stable MOGP hyperparameter estimates than a previously‐proposed Monte Carlo Expectation Maximization algorithm. In the motivating COVID‐19 study, our methodology identifies a kynurenine pathway that affects the clinical severity of patients with COVID‐19 disease and uncovers the role of the biomarker taurine. Our R package DFA4SIL implements the proposed method.

Unsupervised Bayesian image segmentation using orthogonal series

Maximum likelihood autoregressive model parameter estimation with noise corrupted independent snapshots

In earlier research the asymptotic distribution of a test statistic, that uses the algebraic representation of Hotelling’s \(T^2\) and is pertaining to a process generated from the Stochastic EM (SEM) algorithm, was established in order to assess the performance of the EM algorithm in the estimation of the number of components in finite mixtures; theory concerning the distribution of \(T^2\) was based on a regularity assumption stating that the vector random process generated from SEM is normally distributed. In the present paper a central limit theorem and some theory concerning second order moments are used in order to investigate corresponding results obtained in case the process is generated from the stationary state of SEM without making any assumption of normality. A comparison between our findings and usual asymptotic theory for independently distributed vector random variables is also provided.

This paper examines three Bayesian statistical segmentation techniques with an innovative attenuation compensation on synthetic data and breast ultrasound medical images. All use expectation maximization for estimating the Gaussian model parameters and segment the data using a three-dimensional (3-D) Markov random field pixel neighborhood. This paper compares three Bayesian segmentation techniques: maximum a posteriori simulated annealing (MAP-SA), MAP iterated conditional modes (MAP-ICM), and maximization of posterior marginals (MPM). We conclude that because of the high speckle noise and adverse attenuation challenges of breast ultrasound, the MPM algorithm has the best performance. This is due to better localized segmentation than the other MAP techniques. We present results first with synthetic images then with breast ultrasound. Our new contributions for a 3-D breast ultrasound produce improved results using a model of the noise, in which the Gaussian mean is proportional to the image attenuation with depth, combined with a new prior probability model.

The problem of limited tagged training samples and unknown the number of classes is challenging for the classification of remote sensing scenes. This paper presents a new GMRF self-supervised algorithm for SAR image. We add a GOF process in the process of estimating GMM parameters by EM algorithm, which can not only dynamically select the best number of significant classes but also provides an initial feature parameter to calculate the MRF minimum energy. After the iterative region label and region growth cycle, iteration is combined with the Mll context model to obtain the best mark of each region. Since the initial feature parameter selection of the MRF is not random, the operation efficiency is also improved while reducing the number of iteration cycles of the algorithm. The experiment validates that our design not only solves the problem of manual input of the number of classes but also provide the better output result graph in terms of detail maintenance than the expert interpretation of the truth map in the unsupervised image classification process, and we hope that it could support operation and meet the real-time requirements.