Multiple imputation strategies for a bounded outcome variable in a competing risks analysis.

Comparing the performance of different multiple imputation strategies for missing binary outcomes in cluster randomized trials: a simulation study Jinhui Ma,1–3 Parminder Raina,1,2 Joseph Beyene,1 Lehana Thabane1,3–51Department of Clinical Epidemiology and Biostatistics, McMaster University, Hamilton, ON, Canada; 2McMaster University Evidence-based Practice Center, Hamilton, ON, Canada; 3Biostatistics Unit, St Joseph's Healthcare Hamilton, Hamilton, ON, Canada; 4Centre for Evaluation of Medicines, St Joseph's Healthcare Hamilton, Hamilton, ON, Canada; 5Population Health Research Institute, Hamilton Health Sciences, Hamilton, ON, CanadaIntroduction: Although researchers have proposed various strategies to handle missing outcomes in cluster randomized trials (CRTs), limited attention has been paid to the performance of these strategies. Under the assumption of covariate-dependent missingness, the objective of this simulation study is to compare the performance of various strategies in handling missing binary outcomes in CRTs under different design settings.Methods: There are six missing data strategies investigated in this paper, which include complete case analysis, standard multiple imputation (MI) strategies using either logistic regression or Markov chain Monte Carlo (MCMC) method, within-cluster MI strategies using either logistic regression or MCMC method, and MI using logistic regression with cluster as a fixed effect. The performance of these strategies is evaluated through bias, empirical standard error, root mean squared error, and coverage probability.Results: Under the assumption of covariate-dependent missingness and applying the generalized estimating equations approach for fitting the logistic regression, it was shown that complete case analysis yields valid inferences when the percentage of missing outcomes is not large (<20%) for all the designs of CRTs considered in this paper. Standard MI strategies can be adopted when the design effect is small (variance inflation factor [VIF] ≤ 3); however, they tend to underestimate the standard error of treatment effect when the design effect is large. Within-cluster MI strategy using logistic regression is valid for imputation of missing data from CRTs, especially when the cluster size is large (>50) and the design effect is large (VIF > 3). In contrast, within-cluster MI strategy using MCMC method may yield biased estimates of treatment effect for CRTs with small cluster size (≤50). MI using logistic regression with cluster as a fixed effect may substantially overestimate the standard error of the estimated treatment effect when the intracluster correlation coefficient is small. It may also lead to biased estimated treatment effect.Conclusion: Findings from this simulation study provide researchers with quantitative evidence to guide selection of an appropriate strategy to deal with missing binary outcomes.Keywords: missing data, design effect, variance inflation factor

Accelerometer data are widely used in research to provide objective measurements of physical activity. Frequently, participants may remove accelerometers during their observation period resulting in missing data referred to as nonwear periods. Common approaches for handling nonwear periods include discarding data (days with insufficient hours or individuals with insufficient valid days) from analyses and single imputation (SI) methods.Purpose:This study evaluates the performance of various discard-, SI-, and multiple imputation (MI)-based approaches on the ability to accurately and precisely characterize the relationship between a summarized measure of accelerometer counts (mean counts per minute) and an outcome (body mass index).Methods:Realistic accelerometer data were simulated under various scenarios that induced nonwear. Data were analyzed using common and MI methods for handling nonwear. Bias, relative standard error, relative mean squared error, and coverage probabilities were compared across methods.Results:MI approaches were superior to commonly applied methods, with bias that ranged from −0.001 to −0.028 that was considerably lower than that of discard-based methods (ranging from −0.050 to −0.057) and SI methods (ranging from −0.061 to −0.081). We also reported substantial variation among MI strategies, with coverage probabilities ranging from .04 to .96.Conclusion:Our findings demonstrate the benefit of applying MI methods over more commonly applied discard- and SI-based approaches. Additionally, we show that how you apply MI matters, where including data from previously observed acceleration measurements in the imputation model when using MI improves model performance.

ObjectivesEpidemiological studies often have missing data, which are commonly handled by multiple imputation (MI). Standard (default) MI procedures use simple linear covariate functions in the imputation model. We examine the bias that may be caused by acceptance of this default option and evaluate methods to identify problematic imputation models, providing practical guidance for researchers. Study Design and SettingUsing simulation and real data analysis, we investigated how imputation model mis-specification affected MI performance, comparing results with complete records analysis (CRA). We considered scenarios in which imputation model mis-specification occurred because (i) the analysis model was mis-specified or (ii) the relationship between exposure and confounder was mis-specified. ResultsMis-specification of the relationship between outcome and exposure, or between exposure and confounder, can cause biased CRA and MI estimates (in addition to any bias in the full-data estimate due to analysis model mis-specification). MI by predictive mean matching can mitigate model mis-specification. Methods for examining model mis-specification were effective in identifying mis-specified relationships. ConclusionWhen using MI methods that assume data are MAR, compatibility between the analysis and imputation models is necessary, but not sufficient to avoid bias. We propose a step-by-step procedure for identifying and correcting mis-specification of imputation models.

BackgroundMultiple imputation is frequently used to deal with missing data in healthcare research. Although it is known that the outcome should be included in the imputation model when imputing missing covariate values, it is not known whether it should be imputed. Similarly no clear recommendations exist on: the utility of incorporating a secondary outcome, if available, in the imputation model; the level of protection offered when data are missing not-at-random; the implications of the dataset size and missingness levels.MethodsWe used realistic assumptions to generate thousands of datasets across a broad spectrum of contexts: three mechanisms of missingness (completely at random; at random; not at random); varying extents of missingness (20–80% missing data); and different sample sizes (1,000 or 10,000 cases). For each context we quantified the performance of a complete case analysis and seven multiple imputation methods which deleted cases with missing outcome before imputation, after imputation or not at all; included or did not include the outcome in the imputation models; and included or did not include a secondary outcome in the imputation models. Methods were compared on mean absolute error, bias, coverage and power over 1,000 datasets for each scenario.ResultsOverall, there was very little to separate multiple imputation methods which included the outcome in the imputation model. Even when missingness was quite extensive, all multiple imputation approaches performed well. Incorporating a secondary outcome, moderately correlated with the outcome of interest, made very little difference. The dataset size and the extent of missingness affected performance, as expected. Multiple imputation methods protected less well against missingness not at random, but did offer some protection.ConclusionsAs long as the outcome is included in the imputation model, there are very small performance differences between the possible multiple imputation approaches: no outcome imputation, imputation or imputation and deletion. All informative covariates, even with very high levels of missingness, should be included in the multiple imputation model. Multiple imputation offers some protection against a simple missing not at random mechanism.

Missing data are condition when there are some missing values or empty entries on several observations on data. It could inhibit statistical analysis process and might give a bias conclusion from the analysis if couldn't be handled properly. This problem can be found on some linear regression analysis. One way to handle this problem is using multiple imputation (MI) method named Predictive Mean Matching (PMM). PMM will matching the predictive mean distance of incomplete observations with the complete observations. To get the multiple imputation concept, the predictive mean of incomplete observations were estimated by Bayesian approach while the complete observations were estimated with ordinary least square. Thus, the complete observation that has the closest distance will be a donor value for the incomplete one. Simulation data with two variable (x and y), univariate missing data pattern (on y), and MAR mechanism is used to analyzed the effectiveness of PMM based on relative efficiency estimation result of missing covariate data. Regression analysis used x as independent variable and y as dependent variable. The result showed that PMM give a significant coefficient regression parameter at 5% level of significance and only loss 1 % of relative efficiency.

ABSTRACTIn certain settings, when conducting a randomized trial would be infeasible, electronic health records (EHR) can be used to emulate a target trial and estimate causal effects of an intervention. This process involves specifying the elements of a hypothetical trial protocol and applying these to the design of an observational study conducted with EHR data (or other observational data source). One element of target trial specification includes defining eligibility criteria. However, defining the eligible population with EHR can be complicated by missingness in eligibility‐defining variables. Multiple imputation (MI) is one common approach to missingness in EHR data, but it is unclear whether imputation of eligibility criteria should occur before or after excluding ineligible individuals. Motivated by a target trial emulation of two treatments for advanced breast cancer, we explore this question when estimating the average causal effect under a target trial framework with survival outcomes. We illustrate how alternative MI strategies perform using simulated data and in a real‐world analysis of oncology EHR data. We found that in most settings with high proportions of missingness in eligibility‐defining variables, imputing missing data using a flexible imputation model, such as a random forest, prior to excluding ineligible individuals resulted in lower bias than complete case analysis or imputation after excluding ineligible individuals. Choices about how to handle practical challenges such as this in the application of target trial emulation to messy, real‐world data sources can have substantial effects on causal parameter estimation and should be carefully considered to ensure that the results of observational studies are as rigorous as possible.

In population-based cancer survival studies, the cause-specific survival measures the net survival (excess mortality) due to cancer when the cause of death information is available and reliable. In contrast, when the cause of death is uncertain or unavailable, relative survival, the ratio of the survival rate due to all causes to the expected survival rate, is more appropriate. There is a large body of work on the modelling and hypothesis testing of cause-specific survival, but many of these methods are not directly applicable to relative survival. In this paper, we extend the multiple imputation (MI) methods (Stat. Methods Med. Res. 1999; 8:3-15) to the case of relative survival data. The MI methodology is combined with relative survival to estimate the net survival by changing relative survival data to cause-specific data. This facilitates the direct application to statistical methods developed for the cause-specific survival to the special situation of relative survival. The parameter estimates and the log-rank statistics are obtained by combining the results from multiple imputed cause-specific data. The likelihood-based methods for modelling relative survival data have been implemented by a Windows application called CANSURV (Comput. Meth. Prog. Biomed 2005). Although these methods produce accurate parameter estimates, the choice for models and diagnostic tools is limited. The MI method is presented as a simpler alternative. The relative survival data for the colorectal cancer patients from Surveillance, Epidemiology, and End Results (SEER) program (SEER Cancer Statistics Review, 1973-1999. National Cancer Institute: Bethesda, 2002) is used as an illustration. The results are compared with those obtained from the likelihood-based relative survival analysis methods. A sample SAS macro for the MI method is provided.

High‐dimensional datasets have become increasingly prominent in fields such as medicine, biology, and the social sciences. These datasets often contain missing values, which can significantly compromise the reliability and validity of analytical results. This review examines simulation studies in the literature that address the issue of missing data in high‐dimensional settings. It compares the effectiveness and applicability of multiple imputation (MI) methods, discussing their strengths and weaknesses. The review also explores the challenges encountered by MI techniques in high‐dimensional data and outlines directions for future research. Its primary aim is to guide researchers and practitioners in selecting appropriate MI methods to manage missing data issues in their own datasets. The reviewed simulation studies assess the performance of various MI approaches under different missing data mechanisms and data structures, highlighting their potential advantages and limitations in real‐world applications. In conclusion, MI methods for high‐dimensional data represent a dynamic and rapidly evolving area of research. While existing approaches offer substantial capabilities, there remains a pressing need for continual innovation and rigorous inquiry to develop MI solutions that are theoretically sound, practically feasible, and capable of producing more accurate, reliable, and efficient results. This article is categorized under: Statistical Models > Simulation Models Statistical and Graphical Methods of Data Analysis > Analysis of High Dimensional Data Statistical Models > Model Selection

BackgroundMissing data are a common problem in prospective studies with a long follow-up, and the volume, pattern and reasons for missing data may be relevant when estimating the cost of illness. We aimed to evaluate the effects of different methods for dealing with missing longitudinal cost data and for costing caregiver time on total societal costs in Alzheimer’s disease (AD).MethodsGERAS is an 18-month observational study of costs associated with AD. Total societal costs included patient health and social care costs, and caregiver health and informal care costs. Missing data were classified as missing completely at random (MCAR), missing at random (MAR) or missing not at random (MNAR). Simulation datasets were generated from baseline data with 10–40 % missing total cost data for each missing data mechanism. Datasets were also simulated to reflect the missing cost data pattern at 18 months using MAR and MNAR assumptions. Naïve and multiple imputation (MI) methods were applied to each dataset and results compared with complete GERAS 18-month cost data. Opportunity and replacement cost approaches were used for caregiver time, which was costed with and without supervision included and with time for working caregivers only being costed.ResultsTotal costs were available for 99.4 % of 1497 patients at baseline. For MCAR datasets, naïve methods performed as well as MI methods. For MAR, MI methods performed better than naïve methods. All imputation approaches were poor for MNAR data. For all approaches, percentage bias increased with missing data volume. For datasets reflecting 18-month patterns, a combination of imputation methods provided more accurate cost estimates (e.g. bias: −1 % vs −6 % for single MI method), although different approaches to costing caregiver time had a greater impact on estimated costs (29–43 % increase over base case estimate).ConclusionsMethods used to impute missing cost data in AD will impact on accuracy of cost estimates although varying approaches to costing informal caregiver time has the greatest impact on total costs. Tailoring imputation methods to the reason for missing data will further our understanding of the best analytical approach for studies involving cost outcomes.Electronic supplementary materialThe online version of this article (doi:10.1186/s12874-016-0188-1) contains supplementary material, which is available to authorized users.

ABSTRACTIntroductionIn analysis of time‐to‐event outcomes, a mixture cure (MC) model is preferred over a standard survival model when the sample includes individuals who will never experience the event of interest. Motivated by a cohort study of breast cancer patients with incomplete biomarkers, we develop multiple imputation (MI) methods assuming a Weibull proportional hazards (PH‐MC) analysis model with multiple prognostic factors. However, for MI with fully conditional specification, an incorrectly specified imputation model can impair accuracy of point and interval estimates.Objectives and MethodsOur goal is to propose imputation models that are compatible with the Weibull PH‐MC analysis models. We derive an exact conditional distribution (ECD) imputation model which involves the analysis model likelihood. Using simulation studies, we compare effect estimate bias and confidence interval (CI) coverage under alternative imputation models including the ECD model, an approximation that includes a cure indicator (cECD), and a comprehensive simple (CS) model. For robust parameter estimation in finite and/or sparse samples, we incorporate the Firth‐type penalized likelihood (FT‐PL) and combined likelihood profile (CLIP) methods into the MI.ResultsCompared to complete case analysis, MI with penalization reduces estimation bias and improves coverage. Although ECD and cECD perform similarly at higher event rates, ECD generates smaller bias and higher coverage at lower rates. CS has larger bias and lower coverage than ECD and cECD, but CIs are narrower than for cECD.ConclusionsIn analyses of biomarkers and composite subtypes for prognosis studies such as in breast cancer, the use of compatible imputation models and penalization methods is recommended for MC modeling in samples with low event numbers and/or with covariate imbalance.

Modern histocompatibility algorithms depend on the comparison and analysis of high-resolution HLA protein sequences and structures, especially when considering epitope-based algorithms, which aim to model the interactions involved in antibody or T cell binding. HLA genotype imputation can be performed in the cases where only low/intermediate-resolution HLA genotype is available or if specific loci are missing, and by providing an individuals' race/ethnicity/ancestry information, imputation results can be more accurate. This study assesses the effect of imputing high-resolution genotypes on molecular mismatch scores under a variety of ancestry assumptions. We compared molecular matching scores from "ground-truth" high-resolution genotypes against scores from genotypes which are imputed from low-resolution genotypes. Analysis was focused on a simulated patient-donor dataset and confirmed using two real-world datasets, and deviations were aggregated based on various ancestry assumptions. We observed that using multiple imputation generally results in lower error in molecular matching scores compared to single imputation, and that using the correct ancestry assumptions can reduce error introduced during imputation. We conclude that for epitope analysis, imputation is a valuable and low-risk strategy, as long as care is taken regarding epitope analysis context, ancestry assumptions, and (multiple) imputation strategy.

Missing data are nearly always a problem in research, and missing values represent a serious threat to the validity of inferences drawn from findings. Increasingly, social science researchers are turning to multiple imputation to handle missing data. Multiple imputation, in which missing values are replaced by values repeatedly drawn from conditional probability distributions, is an appropriate method for handling missing data when values are not missing completely at random. However, use of this method requires developing an imputation model from the observed data. This is typically a rigorous and time-consuming process. To encourage wider adoption of multiple imputation in social work research, a simple framework for designing imputation models is presented. The framework and its ability to generate unbiased estimates are demonstrated in a simulation study. KEY WORDS: missing data; multiple imputation; nonresponse ********** Missing data are ubiquitous in social research, and missingness or nonresponse can represent a threat to the validity of inferences because of undue effects on efficiency, power, and parameter bias (Shadish, Cook, & Campbell, 2002). Social work researchers are now addressing missing data in a more rigorous manner. Recently, Saunders et al. (2006) and Choi, Golder, Gillmore, and Morrison (2005) described important data imputation methods and dispelled misunderstandings regarding popular imputation methods, such as mean substitution. Recent advances in analytic methods, such as multiple imputation (MI), are taking hold in social work research. With MI, missing values are replaced with values repeatedly drawn from simulated conditional probability distributions (Schafer, 1997), thus creating multiple versions of the data set. Each version of the data set is analyzed according to the data analysis model, and the multiple results are combined into point estimates (Rubin, 1996). A critical task in MI is to devise an imputation model (Allison, 2002) or missing data model (Graham, Olchowski, & Gilreath, 2007), which involves specifying the measures that are putatively associated with the missing values. Although this process adds additional steps, the specification of an imputation model and the creation of multiple data sets can produce less-biased estimates in the presence of missing data across a wide variety of data analysis techniques (Schafer, 1997). Besides MI, there are many other methods for addressing missing data (Schafer, 1999; Schafer & Graham, 2002). An equally rigorous method known as direct or full information maximum likelihood (FIML) estimation can produce unbiased estimates and correct standard errors in the presence of missing data. When the number of imputations is sufficiently large, identical missing data models will produce the same estimates under MI and FIML (Graham et al., 2007). Unlike MI, FIML is limited to maximum likelihood analytic techniques and the missing data model must be included in the analysis model. Although we focus on MI, the steps we describe for developing an imputation model are equally appropriate for use in the missing data model for FIML. On the basis of the MI literature, this article describes a framework for developing an imputation model for use with any free or commercial software package that performs MI. BRIEF REVIEW OF MISSING DATA CONCEPTS Generally, both MI and a broad range of missing data issues have received ample attention in the applied literature (Graham et al., 2007). We briefly discuss the three types of distributions that describe the randomness of nonresponse given that this property has consequences for the development of an imputation model. For a discussion of general missing data concepts that are not critical to understanding our discussion of imputation model development, we refer readers to Schafer and Graham (2002). Distribution of Nonresponse The probability distribution of nonresponse--more frequently referred to as the missing data or nonresponse mechanism (Rubin, 1976)--is both an important factor in the decision to impute with MI and a context for the development of an imputation model. …

We consider the estimation of the regression of an outcome Y on a covariate X, where X is unobserved, but a variable W that measures X with error is observed. A calibration sample that measures pairs of values of X and W is also available; we consider calibration samples where Y is measured (internal calibration) and not measured (external calibration). One common approach for measurement error correction is Regression Calibration (RC), which substitutes the unknown values of X by predictions from the regression of X on W estimated from the calibration sample. An alternative approach is to multiply impute the missing values of X given Y and W based on an imputation model, and then use multiple imputation (MI) combining rules for inferences. Most of current work assumes that the measurement error of W has a constant variance, whereas in many situations, the variance varies as a function of X. We consider extensions of the RC and MI methods that allow for heteroscedastic measurement error, and compare them by simulation. The MI method is shown to provide better inferences in this setting. We also illustrate the proposed methods using a data set from the BioCycle study.

Objectives:Missing data is a recurrent issue in many fields of medical research, particularly in questionnaires. The aim of this article is to describe and compare six conceptually different multiple imputation methods, alongside the commonly used complete case analysis, and to explore whether the choice of methodology for handling missing data might impact clinical conclusions drawn from a regression model when data are categorical.Methods:In addition to the commonly used complete case analysis, we tested the following six imputation methods: multiple imputation using expectation–maximization with bootstrapping, multiple imputation using multiple correspondence analysis, multiple imputation using latent class analysis, multiple hot deck imputation and multivariate imputation by chained equations with two different model specifications: logistic regression and random forests. The methods are tested on real data from a questionnaire-based study in the Norwegian opioid maintenance treatment programme.Results:All methods performed relatively well when the sample size was large (n = 1000). For a smaller sample size (n = 200), the regression estimates depend heavily on the level of missing. When the amount of missing was ⩾20%, in particular, complete case analysis, hot deck and random forests had biased estimates with too low coverage. Multiple imputation using multiple correspondence analysis had the best performance all over.Conclusion:The choice of missing handling methodology has a significant impact on the clinical interpretation of the accompanying statistical analyses. With missing data, the choice of whether to impute or not, and choice of imputation method, can influence clinical conclusion drawn from a regression model and should therefore be given sufficient consideration.

Estimating treatment effects from longitudinal clinical trial data with missing values: comparative analyses using different methods