Censored observations, repeated measures and mixed effects models: An approach using the EM algorithm and normal errors

The pharmaceutical industry is currently interested in the population approach and population models, also known as mixed effects models and random effects models depending on the precise form. Population models are useful in that they can account for both withinand between-individual sources of variability and serial correlation within individual observations when analyzing unbalanced repeated measures data. The modelling of population pharmacodynamic or pharmacokinetic profiles typically involves nonlinear random effects models. Each individual's observations are modelled by identical (up to unknown parameter values) nonlinear regression models, with the distribution of the observations, or a transformation of the observations, about expected responses taken to be normal, with the degree of variability described by a variance model. Between-individual variability is modelled by a population distribution for the individual regression parameter values (random effects). In a parametric analysis the population distribution is taken to be normal, the parameters of which, along with the parameters of the variance model, are known as the population parameters. Maximum likelihood estimation of the population parameters for nonlinear random effects models was pioneered by Beal and Sheiner (1979), and since then a number of algorithms have appeared for approximate maximum likelihood, including Steimer et al. (1984), Lindstrom and Bates (1990), Beal and Sheiner (1992), and Mentre and Gomeni (1995). All of these algorithms are approximate in some way. For a summary see Beal and Sheiner (1992), Wolfinger (1993), Pinheiro and Bates (1994), and Davidian and Giltinan (1995). In this paper an EM algorithm for exact maximum likelihood estimation is introduced. An EM algorithm obtaining maximum likelihood estimates for linear random effects models was introduced by Dempster, Laird, and Rubin (1977). Laird and Ware (1982), Lindstrom and Bates (1988), Jennrich and Schluchter (1986), and Liu and Rubin (1994) all describe hybrid EM algorithms for the linear random effects model. A true EM algorithm for the linear model is described by Jamshidian and Jennrich (1993). Mentre and Gomeni (1995) describe an approximate EM algorithm for nonlinear random effects models and, from the algorithm given in this paper, it can be seen clearly how their approximations arise. The present algorithm uses Monte Carlo methods to perform the E step, a strategy previously adopted in an altogether different model by Guo and Thompson (1994). Guo and Thompson require a Gibbs sampler, that is, a Markov chain Monte Carlo method for their E step, but the present algorithm uses independent samples. In Section 2 of this paper the nonlinear random effects model is described. Section 3 gives the EM algorithm without random effect covariates, while Section 4 gives the modified algorithm in the

This paper proposes a method for obtaining nonlinear models of distribution grid based on available measurements from the power grid. We formulate a maximum likelihood estimation (MLE) problem that estimates unknown line parameters—specifically, the impedance between nodes—using measured voltage magnitudes and net active and reactive power injections at each node. The nonlinear model for the distribution grid uses a nonlinear approximation of the DistFlow model, which includes line losses and is parameterized by the unknown line impedances. We solve the resulting MLE problem using an expectation maximization (EM) algorithm, tailored for the nonlinear model, and provide a numerically robust implementation. The proposed method is demonstrated on the IEEE 37-node test network, and we compare it with the state-of-the-art methods. The proposed method achieves a 70% reduction in voltage error and an error for state variables that is more than 10000 times smaller. A final comparison uses data from a real network, and the proposed method achieves parameter estimates with errors 100 times smaller than competing methods.

One effective iterative method for estimating parameters in latent variable probabilistic models is the Expectation-Maximization (EM) algorithm, which finds extensive use in classification issues. A versatile probabilistic framework for modeling data from multiple Gaussian distributions, Gaussian Mixture Models (GMMs) employ EM for parameter estimation, which is explored in this article. Problems with class membership estimation (including means, covariances, and mixing coefficients) are the primary focus of this study. Starting with initial parameter estimations, estimating posterior probabilities in the E-step, and repeatedly updating model parameters in the M-step until convergence, the technique entails applying the EM algorithm on both synthetic and real-world datasets. In order to measure performance, we compare it to K-means and other baseline classifiers using measures like classification accuracy, Adjusted Rand Index (ARI), and log-likelihood progression. When data is distributed according to a multimodal Gaussian distribution, the findings show that EM-GMM provides better parameter estimation and better classification accuracy. Initialization procedures have a substantial effect on performance, as convergence study further shows. In addition to reiterating the EM algorithm's usefulness for probabilistic classification, this paper also describes its advantages and disadvantages and offers suggestions for improving its performance in settings with high-dimensional data and noise.

This paper introduces structural identification using expectation maximization (STRIDE), a novel application of the expectation maximization (EM) algorithm and approach for output-only modal identification. The EM algorithm can be used to estimate the maximum likelihood parameters of a state-space model. In this context, the state-space model represents the equation of motion for a linear dynamic system. STRIDE is an iterative procedure that uses Kalman filtering and Rauch-Tung-Striebel (RTS) smoothing equations to produce estimates of the unobserved states; these calculations are based on the observed data and prior estimates of the state-space parameters. With this information, the conditional likelihood of the model is maximized and the state-space parameters are updated at each iteration. Once an iteration meets user-prescribed convergence criterion, the algorithm ends—yielding maximum likelihood estimates (MLE) for the state-space model parameters. The modal properties of the structure are th...

Examining the robustness properties of maximum likelihood (ML) estimators of parameters in exponential power and generalized t distributions has been considered together. The well-known asymptotic properties of ML estimators of location, scale and added skewness parameters in these distributions are studied. The ML estimators for location, scale and scale variant (skewness) parameters are represented as an iterative reweighting algorithm (IRA) to compute the estimates of these parameters simultaneously. The artificial data are generated to examine performance of IRA for ML estimators of parameters simultaneously. We make a comparison between these two distributions to test the fitting performance on real data sets. The goodness of fit test and information criteria approve that robustness and fitting performance should be considered together as a key for modeling issue to have the best information from real data sets.

This paper deals with maximum-likelihood (ML) estimation of synchronization parameters for coded transmission systems. In particular, we present a unified framework based on both the sum-product (SP) algorithm and the expectation-maximization (EM) algorithm for the design of iterative synchronizers. The proposed approach is shown to encompass some known iterative synchronizers. In particular, we revisit a previously proposed framework based on the EM algorithm only by means of our "SP-EM" approach. The performance of the proposed synchronization method is assessed in terms of mean-square error and bit-error rate by simulation results. In particular, we consider the joint synchronization of the timing epoch and the carrier phase offset in the case of convolutionally coded and turbo-coded transmissions

In reliability analysis, an investigator is often interested in the assessment of a specific risk in presence of other risk factors. It is well known as the competing risks problem in the statistical literature. In this paper, we consider the analysis of incomplete data in the presence of competing risks among several groups. We mainly consider the latent failure times model formulation, and it is assumed that the lifetime distributions of the different latent failure times of a particular group follow Weibull distributions with different scale parameters, but the same shape parameter. Maximum likelihood estimators of the different parameters are obtained using a simple iterative procedure, and also by EM algorithm. Asymptotic distributions of the maximum likelihood estimators of the different parameters are obtained, and based on the asymptotic distributions, asymptotic confidence intervals are also proposed. Testing equality of the parameters among several groups is performed. One data set has been analysed for illustrative purposes.

Maximum likelihood estimation of item parameters in the marginal distribution, integrating over the distribution of ability, becomes practical when computing procedures based on an EM algorithm are used. By characterizing the ability distribution empirically, arbitrary assumptions about its form are avoided. The Em procedure is shown to apply to general item-response models lacking simple sufficient statistics for ability. This includes models with more than one latent dimension.

Kim (J. Korean Stat. Soc. 37 (2008) 81–87) introduced an incorrect stochastic representation (SR) for the truncated Student-t (Tt) random variable. By pointing out that the gamma mixture based on a truncated normal distribution actually cannot result in a true Tt distribution, in this paper, we first propose three correct SRs and then recalculate the corresponding moments of the Tt distribution. Different from those derived by following the invalid SR of Kim (J. Korean Stat. Soc. 37 (2008) 81–87), the correct moments of the Tt distribution play a crucial role in parameter estimations. Based on the third SR proposed and the correct expressions of truncated moments, expectation–maximization (EM) algorithms are developed for calculating the maximum likelihood estimates of parameters in the Tt distribution. Extensions to a Tt regression model and a t interval-censored regression model are provided as well. Simulated experiments are conducted to evaluate the performance of the proposed methods. Finally, two real data analyses corroborate the theoretical results.

Overall anesthesia department quality of clinical supervision of trainees over a year evaluated using mixed effects models

Finding a new drug which is both safe and efficient is an expensive and time-consuming endeavour. In particular, establishing the ‘dose-effect relationship’ – how beneficial a drug is at different dosages – can be challenging. Predicting this curve requires gathering experimental data by exposing and recording how cells respond to various levels of the drug. However, extreme values are often observed at low and high dosages, potentially introducing errors that are hard to correct in the prediction process. Yet, these extreme observations are sometimes genuine so researchers cannot just ignore them. To improve dose-effect estimation, Zhou, Liu, Fang et al. developed a new general-purpose approach. It uses advanced statistical modelling to account for extremes in lab data. This strategy outperformed other methods when dealing with these observations while also providing higher efficiency in data analysis with more uniform data in experiments. To facilitate implementation, Zhou, Liu, Fang et al. set up a user-friendly tool baptized ‘REAP’; this free online resource allows scientists without advanced statistical experience to harness the new approach and to perform dose-effect analysis more easily and accurately. This could boost research across many different disciplines that examine the effects of chemicals on cells.

Purpose: In this paper, we propose a method to classify and detect normal, known anomalies, and unknown anomalies by combining the expectation–maximisation (EM-λ) algorithm and the Mahalanobis–Taguchi (MT) method. Methodology/Approach: The proposed method learns normal data that are expected to be homogeneous and known abnormal data and performs classification and detection by parameter estimation using the EM-λ algorithm. Conventional methods perform analysis based on parameter estimation using the EM algorithm. However, the EM algorithm can degrade classification accuracy if it does not assume that the data fits the model's generative process. Findings: We verify the performance of the proposed method using artificially generated data and real-world bean data for classification as data that do not satisfy this assumption. The validation results show up to 6% improvement over the conventional method in classification accuracy and unknown anomaly discrimination accuracy. Research Limitation/implication: We try various patterns for the parameter of the proposed method in the verification. However, this way is computationally expensive. Originality/Value of paper: Conventional methods perform analysis based on parameter estimation using the EM algorithm. Our proposal method seeks to improve accuracy by using the EM-λ algorithm for parameter estimation, which is expected to improve classification accuracy when the data do not conform to the generative assumptions of the EM algorithm's model.

As the amount of data increases, the data can obey either a single distribution, two distributions or even multiple distributions. If the distribution parameters and mixing ratio can be estimated for mixed data, it will be beneficial to further analysis and research in practical applications. In this paper, the EM algorithm and the Nelder-Mead(NM) algorithm are applied to the mixed parameter estimation of the two distributions, including mixing two identical distributions, mixing two different distributions, and comparing the EM algorithm and the Nelder-Mead algorithm to estimate the accuracy of the mixed distribution parameter estimation and its initial comparison. The stability of the value and other advantages and disadvantages. A large amount of data simulation results found that the EM algorithm has a good effect on the estimation of mixed distribution parameters, with high accuracy and fast convergence. The initial value selection has little effect on the results, and the preliminary derivation process is more complicated; the Nelder-Mead algorithm has a good effect on the estimation of mixed distribution parameters. High precision, fast convergence speed, the initial value selection has a greater impact on the result, the preliminary derivation process is relatively simple, and the application range is wide. When the initial value is close to the peak value of the real data, the parameter estimation effect is better; the two algorithms have poorer effect on the unimodal distribution parameter estimation for the data, and the bimodal distribution parameter estimation effect is better.

Unified computational methods for regression analysis of zero-inflated and bound-inflated data

Cognitive diagnostic models (CDMs) are increasingly being used in various assessment contexts to identify cognitive processes and provide tailored feedback. However, the most commonly used estimation method for CDMs, marginal maximum likelihood estimation with Expectation–Maximization (MMLE-EM), can present difficulties when sample sizes are small. This study compares the results of different estimation methods for CDMs under varying sample sizes using simulated and empirical data. The methods compared include MMLE-EM, Bayes modal, Markov chain Monte Carlo, a non-parametric method, and a parsimonious parametric model such as Restricted DINA. We varied the sample size, and assessed the bias in the estimation of item parameters, the precision in attribute classification, the bias in the reliability estimate, and computational cost. The findings suggest that alternative estimation methods are preferred over MMLE-EM under low sample-size conditions, whereas comparable results are obtained under large sample-size conditions. Practitioners should consider using alternative estimation methods when working with small samples to obtain more accurate estimates of CDM parameters. This study aims to maximize the potential of CDMs by providing guidance on the estimation of the parameters.