Bias approximation for maximum likelihood estimates

ABSTRACTSmall samples are common in growth models due to financial and logistical difficulties of following people longitudinally. For similar reasons, longitudinal studies often contain missing data. Though full information maximum likelihood (FIML) is popular to accommodate missing data, the limited number of studies in this area have found that FIML tends to perform poorly with small-sample growth models. This report demonstrates that the fault lies not with how FIML accommodates missingness but rather with maximum likelihood estimation itself. We discuss how the less popular restricted likelihood form of FIML, along with small-sample-appropriate methods, yields trustworthy estimates for growth models with small samples and missing data. That is, previously reported small sample issues with FIML are attributable to finite sample bias of maximum likelihood estimation not direct likelihood. Estimation issues pertinent to joint multiple imputation and predictive mean matching are also included and discussed.

The phenomenon of monotone likelihood is observed in the fitting process of a Cox model if the likelihood converges to a finite value while at least one parameter estimate diverges to +/- infinity. Monotone likelihood primarily occurs in small samples with substantial censoring of survival times and several highly predictive covariates. Previous options to deal with monotone likelihood have been unsatisfactory. The solution we suggest is an adaptation of a procedure by Firth (1993, Biometrika 80, 27-38) originally developed to reduce the bias of maximum likelihood estimates. This procedure produces finite parameter estimates by means of penalized maximum likelihood estimation. Corresponding Wald-type tests and confidence intervals are available, but it is shown that penalized likelihood ratio tests and profile penalized likelihood confidence intervals are often preferable. An empirical study of the suggested procedures confirms satisfactory performance of both estimation and inference. The advantage of the procedure over previous options of analysis is finally exemplified in the analysis of a breast cancer study.

Fitting N-mixture models to count data with unmodeled heterogeneity: Bias, diagnostics, and alternative approaches

The misspecification of gravity spatial interaction models has recently been described by the author. The bias in parameter estimates that results from such misspecification appears to produce the ‘map pattern effect’ or ‘spatial structure bias’ in estimated distance-decay parameters. A further aspect of the misspecification bias in gravity parameter estimates is explored here. The severity of the bias is shown to vary in a predictable manner with variations in spatial structure. In particular, the bias is shown to be dependent upon the pattern of accessibility that exists within a spatial system. The relevant aspects of this pattern are discussed for intraurban and interurban (or interregional) flow matrices. It is shown that from an examination of the spatial structure of centres in a spatial system it is possible, a priori, to identify whether significant bias will arise in the calibration of a gravity model. Certain configurations of centres are shown theoretically to produce maximal bias. The author thus answers the question, “why do gravity parameter estimates appear to be biased in some systems but not in others?”

The Maximum Likelihood (ML) and Best Linear Unbiased (BLU) estimators of the location and scale parameters of an extreme value distribution (Lawless [1982]) are compared under conditions of small sample sizes and Type I censorship. The comparisons were made in terms of the mean square error criterion. According to this criterion, the ML estimator of σ in the case of very small sample sizes (n < 10) and heavy censorship (low censoring time) proved to be more efficient than the corresponding BLU estimator. However, the BLU estimator for σ attains parity with the corresponding ML estimator when the censoring time increases even for sample sizes as low as 10. The BLU estimator of σ attains equivalence with the ML estimator when the sample size increases above 10, particularly when the censoring time is also increased. The situation is reversed when it came to estimating the location parameter μ, as the BLU estimator was found to be consistently more efficient than the ML estimator despite the improved performance of the ML estimator when the sample size increases. However, computational ease and convenience favor the ML estimators. Keywords: maximum likelihood estimatorsbest linear unbiased estimatorslife-testingreliability

BackgroundGroup sequential designs are increasingly employed to allow trials to stop early with statistical rigor. While existing work focuses on intention-to-treat effect on clinical endpoints, the properties of mediation analysis (commonly conducted in psychological trials to understand a causal mechanism) remain unknown under group sequential designs.MethodsConsidering a group sequential design with one interim analysis for early stopping for efficacy, we conduct a simulation study to evaluate existing analysis techniques when the treatment effect on a continuous outcome is partially or fully mediated by a continuous intermediate variable measuring a casual mechanism. We study the probability of rejecting the null hypotheses on the total effect (i.e., intention-to-treat effect), direct effect and indirect effect, respectively. We examine the bias of maximum likelihood estimator for these effects. We investigate if the penalized (and conditional) maximum likelihood estimator has smaller bias than the maximum likelihood estimator when a trial stopped (did not stop) early.ResultsThe presence of an intermediate variable reduces the power of a group sequential design when sample size calculation ignores the causal mechanism, though type I error control remains unaffected. The maximum likelihood estimator is unbiased only for the mediator-outcome path, impacting the properties of mediation analysis since existing methods typically rely on it to estimate the pathways. The penalized maximum likelihood estimator for other pathways has similar bias to the stage-one maximum likelihood estimator, while the conditional maximum likelihood estimator shows negligible or smaller bias than the usual maximum likelihood estimator for estimating the total and the direct effects only.ConclusionsMediation analysis needs additional consideration in group sequential designs. As with fixed trial designs, the sample size calculation of group sequential designs should account for the total variability underlying a causal mechanism when the treatment effect is hypothesized to be mediated by an intermediate variable, or risk the overall power to detect an intention-to-treat (total) effect being lower than the nominal value. We suggest reporting several estimators and acknowledging that they may be biased for some mediation pathways. More research is needed to develop methods for the analysis of indirect effect under group sequential designs.Supplementary InformationThe online version contains supplementary material available at 10.1186/s12874-025-02714-y.

In this article we introduce multiple stuttering-Poisson distributions and discuss their genesis, properties and applications. We consider a special case of triple stuttering-Poisson distribution with probability generating function G(t) = Exp [a(t − 1) + b(t 2 − 1) + c(t 3 − l)] and investigate (1) maximum likelihood estimation, (2) moment estimation, and (3) mixed moment estimation of the parameters a, b, c. First order terms in the expressions for the biases and covariances of moment estimators are presented. The joint asymptotic efficiencies of moment estimators are tabulated for a few selected parameter points. The first order terms in the biases of maximum likelihood estimators and moment estimators are also given for a few selected parameter points. As judged from the contribution of the first order term, the possible bias of maximum likelihood estimators can be substantial over a considerable region of the parameter space. The moment estimators have low joint asymptotic efficiency over a large par...

BackgroundIn investigating differentially expressed genes or other selected features, researchers conduct hypothesis tests to determine which biological categories, such as those of the Gene Ontology (GO), are enriched for the selected features. Multiple comparison procedures (MCPs) are commonly used to prevent excessive false positive rates. Traditional MCPs, e.g., the Bonferroni method, go to the opposite extreme: strictly controlling a family-wise error rate, resulting in excessive false negative rates. Researchers generally prefer the more balanced approach of instead controlling the false discovery rate (FDR). However, the q-values that methods of FDR control assign to biological categories tend to be too low to reliably estimate the probability that a biological category is not enriched for the preselected features. Thus, we study an application of the other estimators of that probability, which is called the local FDR (LFDR).ResultsWe considered five LFDR estimators for detecting enriched GO terms: a binomial-based estimator (BBE), a maximum likelihood estimator (MLE), a normalized MLE (NMLE), a histogram-based estimator assuming a theoretical null hypothesis (HBE), and a histogram-based estimator assuming an empirical null hypothesis (HBE-EN). Since NMLE depends not only on the data but also on the specified value of Π0, the proportion of non-enriched GO terms, it is only advantageous when either Π0 is already known with sufficient accuracy or there are data for only 1 GO term. By contrast, the other estimators work without specifying Π0 but require data for at least 2 GO terms. Our simulation studies yielded the following summaries of the relative performance of each of those four estimators. HBE and HBE-EN produced larger biases for 2, 4, 8, 32, and 100 GO terms than BBE and MLE. BBE has the lowest bias if Π0 is 1 and if the number of GO terms is between 2 and 32. The bias of MLE is no worse than that of BBE for 100 GO terms even when the ideal number of components in its underlying mixture model is unknown, but has high bias when the number of GO terms is small compared to the number of estimated parameters. For unknown values of Π0, BBE has the lowest bias for a small number of GO terms (2-32 GO terms), and MLE has the lowest bias for a medium number of GO terms (100 GO terms).ConclusionsFor enrichment detection, we recommend estimating the LFDR by MLE given at least a medium number of GO terms, by BBE given a small number of GO terms, and by NMLE given either only 1 GO term or precise knowledge of Π0.

The bias of maximum likelihood estimates calculated at the end of a sequential procedure is investigated. For the two sequential designs considered in detail, the sequential probability ratio test and the triangular test, this bias is appreciable. A method of calculating an adjusted estimate with reduced bias is described, and an approximation to the standard error of the new estimate is provided. Examples of the implementation of the method are given, and its advantages and disadvantages relative to alternative approaches are discussed.

Examining the robustness properties of maximum likelihood (ML) estimators of parameters in exponential power and generalized t distributions has been considered together. The well-known asymptotic properties of ML estimators of location, scale and added skewness parameters in these distributions are studied. The ML estimators for location, scale and scale variant (skewness) parameters are represented as an iterative reweighting algorithm (IRA) to compute the estimates of these parameters simultaneously. The artificial data are generated to examine performance of IRA for ML estimators of parameters simultaneously. We make a comparison between these two distributions to test the fitting performance on real data sets. The goodness of fit test and information criteria approve that robustness and fitting performance should be considered together as a key for modeling issue to have the best information from real data sets.

Firth's logistic regression has become a standard approach for the analysis of binary outcomes with small samples. Whereas it reduces the bias in maximum likelihood estimates of coefficients, bias towards one-half is introduced in the predicted probabilities. The stronger the imbalance of the outcome, the more severe is the bias in the predicted probabilities. We propose two simple modifications of Firth's logistic regression resulting in unbiased predicted probabilities. The first corrects the predicted probabilities by a post hoc adjustment of the intercept. The other is based on an alternative formulation of Firth's penalization as an iterative data augmentation procedure. Our suggested modification consists in introducing an indicator variable that distinguishes between original and pseudo-observations in the augmented data. In a comprehensive simulation study, these approaches are compared with other attempts to improve predictions based on Firth's penalization and to other published penalization strategies intended for routine use. For instance, we consider a recently suggested compromise between maximum likelihood and Firth's logistic regression. Simulation results are scrutinized with regard to prediction and effect estimation. We find that both our suggested methods do not only give unbiased predicted probabilities but also improve the accuracy conditional on explanatory variables compared with Firth's penalization. While one method results in effect estimates identical to those of Firth's penalization, the other introduces some bias, but this is compensated by a decrease in the mean squared error. Finally, all methods considered are illustrated and compared for a study on arterial closure devices in minimally invasive cardiac surgery. Copyright © 2017 John Wiley & Sons, Ltd.

Relative risks (RRs) are often considered the preferred measures of association in prospective studies, especially when the binary outcome of interest is common. In particular, many researchers regard RRs to be more intuitively interpretable than odds ratios. Although RR regression is a special case of generalized linear models, specifically with a log link function for the binomial (or Bernoulli) outcome, the resulting log-binomial regression does not respect the natural parameter constraints. Because log-binomial regression does not ensure that predicted probabilities are mapped to the [0,1] range, maximum likelihood (ML) estimation is often subject to numerical instability that leads to convergence problems. To circumvent these problems, a number of alternative approaches for estimating RR regression parameters have been proposed. One approach that has been widely studied is the use of Poisson regression estimating equations. The estimating equations for Poisson regression yield consistent, albeit inefficient, estimators of the RR regression parameters. We consider the relative efficiency of the Poisson regression estimator and develop an alternative, almost efficient estimator for the RR regression parameters. The proposed method uses near-optimal weights based on a Maclaurin series (Taylor series expanded around zero) approximation to the true Bernoulli or binomial weight function. This yields an almost efficient estimator while avoiding convergence problems. We examine the asymptotic relative efficiency of the proposed estimator for an increase in the number of terms in the series. Using simulations, we demonstrate the potential for convergence problems with standard ML estimation of the log-binomial regression model and illustrate how this is overcome using the proposed estimator. We apply the proposed estimator to a study of predictors of pre-operative use of beta blockers among patients undergoing colorectal surgery after diagnosis of colon cancer.

In this paper, we derive general formulae for second-order biases of maximum likelihood estimates of the regression, dispersion and precision parameters in nonlinear regression models with t distributed errors. Our formulae are easy to compute, giving the biases by means of ordinary linear regressions. They generalize some previous results due to Cook, Tsai and Wei (1986); Cordeiro and McCullagh (1991) and Cordeiro and Vasconcellos (1997). We derive simple closed-form expressions for these biases in special models. We present some simulations that indicate that the bias-corrected estimates produce a reduction in bias without a given corresponding increase in variability. The bias correction achieves a second-order reduction in the mean square errors of the corrected estimates.

This work proposes and evaluates two methods (CM1 and CM2) for detecting non-compliance using concentration-time data and for obtaining estimates of population pharmacokinetic model parameters in a population with prevalent non-compliance. CM1 estimates individual residual variability (RV) and identifies subjects with higher than average RV as non-compliant. Exclusion of subjects with high RV from the analysis dataset reduces the bias in the estimates of the model parameters. Various methods of identification and exclusion of non-compliant subjects were tested, compared, and shown to reduce or eliminate bias in parameter estimates associated with non-compliance. The tested methods were (i) a pre-defined cutoff value of the random effect on RV, (ii) sequential exclusion of subjects with the highest RV percentiles, and (iii) use of a mixture model for RV. CM2 is applicable for the data with a specific sampling pattern that includes a potentially non-compliant outpatient part with several trough samples followed by a dense profile after the inpatient (compliant) dose. It relies only on the doses known to be administered (e.g., inpatient doses). In this method, all concentration measurements during the outpatient part of the study (except the trough value immediately preceding the inpatient dose) are removed from the dataset and an additional parameter (individual relative bioavailability of the outpatient doses) is introduced. For a number of simulated datasets with various sampling schemes and non-compliance patterns the proposed methods allowed to identify subjects with compliance problems and to reduce or eliminate bias in the estimates of the model parameters.

The Multiple Comparison Procedures with Modeling Techniques (MCP-Mod) framework has been recently approved by the U.S. Food, Administration, and European Medicines Agency as fit-for-purpose for phase II studies. Nonetheless, this approach relies on the asymptotic properties of Maximum Likelihood (ML) estimators, which might not be reasonable for small sample sizes. In this paper, we derived improved ML estimators and correction for their covariance matrices in the censored Weibull regression model based on the corrective and preventive approaches. We performed two simulation studies to evaluate ML and improved ML estimators with their covariance matrices in (i) a regression framework (ii) the Multiple Comparison Procedures with Modeling Techniques framework. We have shown that improved ML estimators are less biased than ML estimators yielding Wald-type statistics that controls type I error without loss of power in both frameworks. Therefore, we recommend the use of improved ML estimators in the MCP-Mod approach to control type I error at nominal value for sample sizes ranging from 5 to 25 subjects per dose.