Riemannian optimization for variance estimation in linear mixed models

Bootstrap is a widely applicable computational statistical method. The focus of this paper lies in developing a bootstrap method for linear mixed models under homoscedasticity violation (heteroscedasticity) in variance of errors. We assume that the form of heteroscedasticity is unknown. To generate bootstrap response data as close as possible to the actual response data, we transform the marginal residuals to ensure that the variance of the modified marginal residuals in bootstrap samples is an unbiased estimator for the variance of the response variable in the linear mixed models. The consistency is proved, implying that the parameter estimators by the proposed method are asymptotically unbiased. The simulations are conducted with varying error terms and sample sizes to show improvement of the proposed method in parameter estimation. The simulation results are compared to demonstrate that the proposed method outperforms the other considered approaches including the wild method in the heteroscedastic linear mixed models under small samples and performs competitively with the parametric method and both methods outperform the other considered methods in homoscedastic linear mixed models. The proposed bootstrap method is applied to squid data for illustration of its effectiveness.

In this note, we recollect some formulas and facts about linear mixed models in relation to Cohen’s effect size measure $$f^2$$ f 2 . It is shown how an estimate of the variance-covariance matrix for the estimated fixed effects parameter vector may serve to compute $$f^2$$ f 2 in the presence of random effects. The advantage of the purported approach lies in the fact that any variance-covariance estimate already available may be applied. This also illuminates the circumstance that an actual computed effect size value necessarily depends on the employed linear mixed model estimation procedures. To demonstrate possible applications, it is shown how $$f^2$$ f 2 can be computed with the statistical software environment using . Based on an artificially generated data set, the possible impact of the inclusion of fixed and/or random effects is exemplarily discussed. In addition, the application to several random variables is demonstrated with a publicly available dat set.

This paper develops a model for repeated binary regression when a covariate is measured with error. The model allows for estimating the effect of the true value of the covariate on a repeated binary response. The choice of a probit link for the effect of the error-free covariate, coupled with normal measurement error for the error-free covariate, results in a probit model after integrating over the measurement error distribution. We propose a two-stage estimation procedure where, in the first stage, a linear mixed model is used to fit the repeated covariate. In the second stage, a model for the correlated binary responses conditional on the linear mixed model estimates is fit to the repeated binary data using generalized estimating equations. The approach is demonstrated using nutrient safety data from the Diet Intervention of School Age Children (DISC) study.

Tree growth is assumed to be mainly the result of three components: (i) an endogenous component assumed to be structured as a succession of roughly stationary phases separated by marked change points that are asynchronous among individuals, (ii) a time-varying environmental component assumed to take the form of synchronous fluctuations among individuals, and (iii) an individual component corresponding mainly to the local environment of each tree. To identify and characterize these three components, we propose to use semi-Markov switching linear mixed models, i.e., models that combine linear mixed models in a semi-Markovian manner. The underlying semi-Markov chain represents the succession of growth phases and their lengths (endogenous component) whereas the linear mixed models attached to each state of the underlying semi-Markov chain represent-in the corresponding growth phase-both the influence of time-varying climatic covariates (environmental component) as fixed effects, and interindividual heterogeneity (individual component) as random effects. In this article, we address the estimation of Markov and semi-Markov switching linear mixed models in a general framework. We propose a Monte Carlo expectation-maximization like algorithm whose iterations decompose into three steps: (i) sampling of state sequences given random effects, (ii) prediction of random effects given state sequences, and (iii) maximization. The proposed statistical modeling approach is illustrated by the analysis of successive annual shoots along Corsican pine trunks influenced by climatic covariates.

Advanced methods for dose-response assessments are used to estimate the minimum concentrations of a nutrient that maximizes a given outcome of interest, thereby determining nutritional requirements for optimal performance. Contrary to standard modeling assumptions, experimental data often present a design structure that includes correlations between observations (i.e., blocking, nesting, etc.) as well as heterogeneity of error variances; either can mislead inference if disregarded. Our objective is to demonstrate practical implementation of linear and nonlinear mixed models for dose-response relationships accounting for correlated data structure and heterogeneous error variances. To illustrate, we modeled data from a randomized complete block design study to evaluate the standardized ileal digestible (SID) Trp:Lys ratio dose-response on G:F of nursery pigs. A base linear mixed model was fitted to explore the functional form of G:F relative to Trp:Lys ratios and assess model assumptions. Next, we fitted 3 competing dose-response mixed models to G:F, namely a quadratic polynomial (QP) model, a broken-line linear (BLL) ascending model, and a broken-line quadratic (BLQ) ascending model, all of which included heteroskedastic specifications, as dictated by the base model. The GLIMMIX procedure of SAS (version 9.4) was used to fit the base and QP models and the NLMIXED procedure was used to fit the BLL and BLQ models. We further illustrated the use of a grid search of initial parameter values to facilitate convergence and parameter estimation in nonlinear mixed models. Fit between competing dose-response models was compared using a maximum likelihood-based Bayesian information criterion (BIC). The QP, BLL, and BLQ models fitted on G:F of nursery pigs yielded BIC values of 353.7, 343.4, and 345.2, respectively, thus indicating a better fit of the BLL model. The BLL breakpoint estimate of the SID Trp:Lys ratio was 16.5% (95% confidence interval [16.1, 17.0]). Problems with the estimation process rendered results from the BLQ model questionable. Importantly, accounting for heterogeneous variance enhanced inferential precision as the breadth of the confidence interval for the mean breakpoint decreased by approximately 44%. In summary, the article illustrates the use of linear and nonlinear mixed models for dose-response relationships accounting for heterogeneous residual variances, discusses important diagnostics and their implications for inference, and provides practical recommendations for computational troubleshooting.

We introduce a model to account for abrupt changes among repeated measures with non-monotone missingness. Development of likelihood inferences for such models is hard because it involves intractable integration to obtain the marginal likelihood. We use hierarchical likelihood to overcome such difficulty. Abrupt changes among repeated measures can be well described by introducing random effects in the dispersion. A simulation study shows that the resulting estimator is efficient, robust against misspecification of fatness of tails. For illustration we use a schizophrenic behaviour data presented by Rubin and Wu.

An algorithm for searching optimal variance component estimators in linear mixed models

Undirected graphical models such as Markov random fields or Boltzmann machines prove useful in many signal processing and machine learning tasks. However, parameter estimation in these models is difficult due to the intractable normalising constant in their probability density functions. One powerful technique for parameter estimation in such models is score matching. This technique makes use of an objective function which is independent of the normalising constant and constitutes locally consistent estimators for the parameters of such models. However, score matching is only applicable to fully-observed models. In this paper, we extend the applicability of score matching to models with latent variables. Our estimators are unbiased, based on Monte Carlo integration. Unbiased gradient estimators open the way to optimisation through stochastic approximation. We demonstrate the performance of our methodology on two synthetic problems.

ABSTRACTLinear mixed models have been widely used to analyze repeated measures data which arise in many studies. In most applications, it is assumed that both the random effects and the within-subjects errors are normally distributed. This can be extremely restrictive, obscuring important features of within-and among-subject variations. Here, quantile regression in the Bayesian framework for the linear mixed models is described to carry out the robust inferences. We also relax the normality assumption for the random effects by using a multivariate skew-normal distribution, which includes the normal ones as a special case and provides robust estimation in the linear mixed models. For posterior inference, we propose a Gibbs sampling algorithm based on a mixture representation of the asymmetric Laplace distribution and multivariate skew-normal distribution. The procedures are demonstrated by both simulated and real data examples.

When modeling multilevel data, it is important to accurately represent the interdependence of observations within clusters. Ignoring data clustering may result in parameter misestimation. However, it is not well established to what degree parameter estimates are affected by model misspecification when applying missing data techniques (MDTs) to incomplete multilevel data. We compare the performance of three MDTs with incomplete hierarchical data. We consider the impact of imputation model misspecification on the quality of parameter estimates by employing multiple imputation under assumptions of a normal model (MI/NM) with two-level cross-sectional data when values are missing at random on the dependent variable at rates of 10%, 30%, and 50%. Five criteria are used to compare estimates from MI/NM to estimates from MI assuming a linear mixed model (MI/LMM) and maximum likelihood estimation to the same incomplete data sets. With 10% missing data (MD), techniques performed similarly for fixed-effects estimates, but variance components were biased with MI/NM. Effects of model misspecification worsened at higher rates of MD, with the hierarchical structure of the data markedly underrepresented by biased variance component estimates. MI/LMM and maximum likelihood provided generally accurate and unbiased parameter estimates but performance was negatively affected by increased rates of MD.

Root-cause identification for quality-related problems is a key issue in quality and productivity improvement for a manufacturing process. Unfortunately, root-cause identification is also a very challenging engineering problem, particularly for a multistage manufacturing process. In this paper, root-cause identification is formulated as a problem of estimation and hypothesis testing of a general linear mixed model. First, a linear mixed fault-quality model is built to describe the cause-effect relationship between the process faults and product quality. Then, the estimation algorithms developed for a general linear mixed model are adapted to estimate the process mean and variance. Finally, a hypothesis testing method is developed to determine if process faults exist in terms of statistical significance. A detailed experimental study illustrated the effectiveness of the proposed methodology.<br><br>Note to Practitioners-Economic globalization brings intense competition among manufacturing enterprises. The key to succeed in this competitive climate is to rapidly respond to fast-changing market demands with high-quality and competitively priced products. To achieve this, we need to quickly identify root causes of quality-related problems in a complicated manufacturing system. However, the current widely adopted quality-control techniques focus more on monitoring than on root-cause identification. These techniques can efficiently detect the changes in the process but the root cause identification is often left to the plant engineers or operators. In this paper, a systematic estimation and testing method is proposed to identify the variational root causes in multistage manufacturing processes. First, a linear model is built based on the design information to describe the cause-effect relationship between the process faults and product quality. Then, an algorithm is developed to estimate the mean and variance of the process faults from the quality measurements of products. Finally, a statistical testing method is developed to determine if process faults (i.e. root causes) exist in terms of statistical significance. A detailed experimental study illustrates the effectiveness of this method. The method presented in this paper is a new quality-control technique and can be used for quality improvement of multistage manufacturing processes.

Network meta-analysis can be implemented by using arm-based or contrast-based models. Here we focus on arm-based models and fit them using generalized linear mixed model procedures. Full maximum likelihood (ML) estimation leads to biased trial-by-treatment interaction variance estimates for heterogeneity. Thus, our objective is to investigate alternative approaches to variance estimation that reduce bias compared with full ML. Specifically, we use penalized quasi-likelihood/pseudo-likelihood and hierarchical (h) likelihood approaches. In addition, we consider a novel model modification that yields estimators akin to the residual maximum likelihood estimator for linear mixed models. The proposed methods are compared by simulation, and 2 real datasets are used for illustration. Simulations show that penalized quasi-likelihood/pseudo-likelihood and h-likelihood reduce bias and yield satisfactory coverage rates. Sum-to-zero restriction and baseline contrasts for random trial-by-treatment interaction effects, as well as a residual ML-like adjustment, also reduce bias compared with an unconstrained model when ML is used, but coverage rates are not quite as good. Penalized quasi-likelihood/pseudo-likelihood and h-likelihood are therefore recommended.

"Prediction and Improved Estimation in Linear Models." Journal of the Operational Research Society, 30(1), pp. 88–89

The problem of optimal joint detection and estimation in linear models with Gaussian noise is studied. A simple closed-form expression for the joint posterior distribution of the (multiple) hypotheses and the states is derived. The expression crystalizes the dependence of the optimal detector on the state estimates. The joint posterior distribution characterizes the beliefs (“soft information”) about the hypotheses and the values of the states. Furthermore, it is a sufficient statistic for jointly detecting multiple hypotheses and estimating the states. The developed expressions give us a unified framework for joint detection and estimation under all performance criteria.

Principal components regression and r-k class predictions in linear mixed models