A multiple regression imputation method with application to sensitivity analysis under intermittent missingness

BackgroundMissing data are a common problem in prospective studies with a long follow-up, and the volume, pattern and reasons for missing data may be relevant when estimating the cost of illness. We aimed to evaluate the effects of different methods for dealing with missing longitudinal cost data and for costing caregiver time on total societal costs in Alzheimer’s disease (AD).MethodsGERAS is an 18-month observational study of costs associated with AD. Total societal costs included patient health and social care costs, and caregiver health and informal care costs. Missing data were classified as missing completely at random (MCAR), missing at random (MAR) or missing not at random (MNAR). Simulation datasets were generated from baseline data with 10–40 % missing total cost data for each missing data mechanism. Datasets were also simulated to reflect the missing cost data pattern at 18 months using MAR and MNAR assumptions. Naïve and multiple imputation (MI) methods were applied to each dataset and results compared with complete GERAS 18-month cost data. Opportunity and replacement cost approaches were used for caregiver time, which was costed with and without supervision included and with time for working caregivers only being costed.ResultsTotal costs were available for 99.4 % of 1497 patients at baseline. For MCAR datasets, naïve methods performed as well as MI methods. For MAR, MI methods performed better than naïve methods. All imputation approaches were poor for MNAR data. For all approaches, percentage bias increased with missing data volume. For datasets reflecting 18-month patterns, a combination of imputation methods provided more accurate cost estimates (e.g. bias: −1 % vs −6 % for single MI method), although different approaches to costing caregiver time had a greater impact on estimated costs (29–43 % increase over base case estimate).ConclusionsMethods used to impute missing cost data in AD will impact on accuracy of cost estimates although varying approaches to costing informal caregiver time has the greatest impact on total costs. Tailoring imputation methods to the reason for missing data will further our understanding of the best analytical approach for studies involving cost outcomes.Electronic supplementary materialThe online version of this article (doi:10.1186/s12874-016-0188-1) contains supplementary material, which is available to authorized users.

Missing data are present in most real world problems and need careful handling to preserve the prediction accuracy and statistical consistency in the downstream analysis. As the gold standard of handling missing data, multiple imputation (MI) methods are proposed to account for the imputation uncertainty and provide proper statistical inference. In this work, we propose Multiple Imputation via Generative Adversarial Network (MI-GAN), a deep learning-based (in specific, a GAN-based) multiple imputation method, that can work under missing at random (MAR) mechanism with theoretical support. MI-GAN leverages recent progress in conditional generative adversarial neural works and shows strong performance matching existing state-of-the-art imputation methods on high-dimensional datasets, in terms of imputation error. In particular, MI-GAN significantly outperforms other imputation methods in the sense of statistical inference and computational speed.

Objectives:Missing data is a recurrent issue in many fields of medical research, particularly in questionnaires. The aim of this article is to describe and compare six conceptually different multiple imputation methods, alongside the commonly used complete case analysis, and to explore whether the choice of methodology for handling missing data might impact clinical conclusions drawn from a regression model when data are categorical.Methods:In addition to the commonly used complete case analysis, we tested the following six imputation methods: multiple imputation using expectation–maximization with bootstrapping, multiple imputation using multiple correspondence analysis, multiple imputation using latent class analysis, multiple hot deck imputation and multivariate imputation by chained equations with two different model specifications: logistic regression and random forests. The methods are tested on real data from a questionnaire-based study in the Norwegian opioid maintenance treatment programme.Results:All methods performed relatively well when the sample size was large (n = 1000). For a smaller sample size (n = 200), the regression estimates depend heavily on the level of missing. When the amount of missing was ⩾20%, in particular, complete case analysis, hot deck and random forests had biased estimates with too low coverage. Multiple imputation using multiple correspondence analysis had the best performance all over.Conclusion:The choice of missing handling methodology has a significant impact on the clinical interpretation of the accompanying statistical analyses. With missing data, the choice of whether to impute or not, and choice of imputation method, can influence clinical conclusion drawn from a regression model and should therefore be given sufficient consideration.

Uncertain ascertainment of events in clinical trials has been noted for decades. To correct possible bias, Clinical Endpoint Committees (CECs) have been employed as a critical element of trials to ensure consistent and high-quality endpoint evaluation, especially for cardiovascular endpoints. However, the efficiency and usefulness of adjudication have been debated. The multiple imputation (MI) method was proposed to incorporate endpoint event uncertainty. In a simulation conducted to explain this methodology, the dichotomous outcome was imputed each time with subject-specific event probabilities. As the final step, the desired analysis was conducted based on all imputed data. This proposed method was further applied to real trial data from PARADIGM-HF. Compared with the conventional Cox model with adjudicated events only, the Cox MI method had higher power, even with a small number of uncertain events. It yielded more robust inferences regarding treatment effects and required a smaller sample size to achieve the same power. Instead of using dichotomous endpoint data, the MI method enables incorporation of event uncertainty and eliminates the need for categorizing endpoint events. In future trials, assigning a probability of event occurrence for each event may be preferable to a CEC assigning a dichotomous outcome. Considerable resources could be saved if endpoint events can be identified more simply and in a manner that maintains study power.

A Comparison of Multiple Imputation and Optimal Estimation for Missing and Uncertain Urban Air Toxics Data

High‐dimensional datasets have become increasingly prominent in fields such as medicine, biology, and the social sciences. These datasets often contain missing values, which can significantly compromise the reliability and validity of analytical results. This review examines simulation studies in the literature that address the issue of missing data in high‐dimensional settings. It compares the effectiveness and applicability of multiple imputation (MI) methods, discussing their strengths and weaknesses. The review also explores the challenges encountered by MI techniques in high‐dimensional data and outlines directions for future research. Its primary aim is to guide researchers and practitioners in selecting appropriate MI methods to manage missing data issues in their own datasets. The reviewed simulation studies assess the performance of various MI approaches under different missing data mechanisms and data structures, highlighting their potential advantages and limitations in real‐world applications. In conclusion, MI methods for high‐dimensional data represent a dynamic and rapidly evolving area of research. While existing approaches offer substantial capabilities, there remains a pressing need for continual innovation and rigorous inquiry to develop MI solutions that are theoretically sound, practically feasible, and capable of producing more accurate, reliable, and efficient results. This article is categorized under: Statistical Models > Simulation Models Statistical and Graphical Methods of Data Analysis > Analysis of High Dimensional Data Statistical Models > Model Selection

BackgroundMultiple imputation is frequently used to deal with missing data in healthcare research. Although it is known that the outcome should be included in the imputation model when imputing missing covariate values, it is not known whether it should be imputed. Similarly no clear recommendations exist on: the utility of incorporating a secondary outcome, if available, in the imputation model; the level of protection offered when data are missing not-at-random; the implications of the dataset size and missingness levels.MethodsWe used realistic assumptions to generate thousands of datasets across a broad spectrum of contexts: three mechanisms of missingness (completely at random; at random; not at random); varying extents of missingness (20–80% missing data); and different sample sizes (1,000 or 10,000 cases). For each context we quantified the performance of a complete case analysis and seven multiple imputation methods which deleted cases with missing outcome before imputation, after imputation or not at all; included or did not include the outcome in the imputation models; and included or did not include a secondary outcome in the imputation models. Methods were compared on mean absolute error, bias, coverage and power over 1,000 datasets for each scenario.ResultsOverall, there was very little to separate multiple imputation methods which included the outcome in the imputation model. Even when missingness was quite extensive, all multiple imputation approaches performed well. Incorporating a secondary outcome, moderately correlated with the outcome of interest, made very little difference. The dataset size and the extent of missingness affected performance, as expected. Multiple imputation methods protected less well against missingness not at random, but did offer some protection.ConclusionsAs long as the outcome is included in the imputation model, there are very small performance differences between the possible multiple imputation approaches: no outcome imputation, imputation or imputation and deletion. All informative covariates, even with very high levels of missingness, should be included in the multiple imputation model. Multiple imputation offers some protection against a simple missing not at random mechanism.

Estimating treatment effects from longitudinal clinical trial data with missing values: comparative analyses using different methods

Missing data are nearly always a problem in research, and missing values represent a serious threat to the validity of inferences drawn from findings. Increasingly, social science researchers are turning to multiple imputation to handle missing data. Multiple imputation, in which missing values are replaced by values repeatedly drawn from conditional probability distributions, is an appropriate method for handling missing data when values are not missing completely at random. However, use of this method requires developing an imputation model from the observed data. This is typically a rigorous and time-consuming process. To encourage wider adoption of multiple imputation in social work research, a simple framework for designing imputation models is presented. The framework and its ability to generate unbiased estimates are demonstrated in a simulation study. KEY WORDS: missing data; multiple imputation; nonresponse ********** Missing data are ubiquitous in social research, and missingness or nonresponse can represent a threat to the validity of inferences because of undue effects on efficiency, power, and parameter bias (Shadish, Cook, & Campbell, 2002). Social work researchers are now addressing missing data in a more rigorous manner. Recently, Saunders et al. (2006) and Choi, Golder, Gillmore, and Morrison (2005) described important data imputation methods and dispelled misunderstandings regarding popular imputation methods, such as mean substitution. Recent advances in analytic methods, such as multiple imputation (MI), are taking hold in social work research. With MI, missing values are replaced with values repeatedly drawn from simulated conditional probability distributions (Schafer, 1997), thus creating multiple versions of the data set. Each version of the data set is analyzed according to the data analysis model, and the multiple results are combined into point estimates (Rubin, 1996). A critical task in MI is to devise an imputation model (Allison, 2002) or missing data model (Graham, Olchowski, & Gilreath, 2007), which involves specifying the measures that are putatively associated with the missing values. Although this process adds additional steps, the specification of an imputation model and the creation of multiple data sets can produce less-biased estimates in the presence of missing data across a wide variety of data analysis techniques (Schafer, 1997). Besides MI, there are many other methods for addressing missing data (Schafer, 1999; Schafer & Graham, 2002). An equally rigorous method known as direct or full information maximum likelihood (FIML) estimation can produce unbiased estimates and correct standard errors in the presence of missing data. When the number of imputations is sufficiently large, identical missing data models will produce the same estimates under MI and FIML (Graham et al., 2007). Unlike MI, FIML is limited to maximum likelihood analytic techniques and the missing data model must be included in the analysis model. Although we focus on MI, the steps we describe for developing an imputation model are equally appropriate for use in the missing data model for FIML. On the basis of the MI literature, this article describes a framework for developing an imputation model for use with any free or commercial software package that performs MI. BRIEF REVIEW OF MISSING DATA CONCEPTS Generally, both MI and a broad range of missing data issues have received ample attention in the applied literature (Graham et al., 2007). We briefly discuss the three types of distributions that describe the randomness of nonresponse given that this property has consequences for the development of an imputation model. For a discussion of general missing data concepts that are not critical to understanding our discussion of imputation model development, we refer readers to Schafer and Graham (2002). Distribution of Nonresponse The probability distribution of nonresponse--more frequently referred to as the missing data or nonresponse mechanism (Rubin, 1976)--is both an important factor in the decision to impute with MI and a context for the development of an imputation model. …

Missing data frequently create problems in the analysis of population-based data sets, such as those collected by cancer registries. Restriction of analysis to records with complete data may yield inferences that are substantially different from those that would have been obtained had no data been missing. 'Naive' methods for handling missing data, such as restriction of the analysis to complete records or creation of a 'missing' category, have drawbacks that can invalidate the conclusions from the analysis. We offer a tutorial on modern methods for handling missing data in relative survival analysis. We estimated relative survival for 29 563 colorectal cancer patients who were diagnosed between 1997 and 2004 and registered in the North West Cancer Intelligence Service. The method of multiple imputation (MI) was applied to account for the common example of incomplete stage at diagnosis, under the missing at random (MAR) assumption. Multivariable regression with a generalized linear model and Poisson error structure was then used to estimate the excess hazard of death of the colorectal cancer patients, over and above the background mortality, adjusting for significant predictors of mortality. Incomplete information on stage, morphology and grade meant that only 55% of the data could be included in the 'complete-case' analysis. All cases could be included after indicator method (IM) or MI method. Handling missing data by MI produced a significantly lower estimate of the excess mortality for stage, morphology and grade, with the largest reductions occurring for late-stage and high-grade tumours, when compared with the results of complete-case analysis. In complete-case analysis, almost 50% of the information could not be included, and with the IM, all records with missing values for stage were combined into a single 'missing' category. We show that MI methods greatly improved the results by exploiting all the information in the incomplete records. This method also helped to ensure efficient inferences about survival were made from the multivariate regression analyses.

ObjectivesEpidemiological studies often have missing data, which are commonly handled by multiple imputation (MI). Standard (default) MI procedures use simple linear covariate functions in the imputation model. We examine the bias that may be caused by acceptance of this default option and evaluate methods to identify problematic imputation models, providing practical guidance for researchers. Study Design and SettingUsing simulation and real data analysis, we investigated how imputation model mis-specification affected MI performance, comparing results with complete records analysis (CRA). We considered scenarios in which imputation model mis-specification occurred because (i) the analysis model was mis-specified or (ii) the relationship between exposure and confounder was mis-specified. ResultsMis-specification of the relationship between outcome and exposure, or between exposure and confounder, can cause biased CRA and MI estimates (in addition to any bias in the full-data estimate due to analysis model mis-specification). MI by predictive mean matching can mitigate model mis-specification. Methods for examining model mis-specification were effective in identifying mis-specified relationships. ConclusionWhen using MI methods that assume data are MAR, compatibility between the analysis and imputation models is necessary, but not sufficient to avoid bias. We propose a step-by-step procedure for identifying and correcting mis-specification of imputation models.

ABSTRACTLongitudinal binary data are commonly encountered in clinical trials. Multiple imputation is an approach for getting a valid estimation of treatment effects under an assumption of missing at random mechanism. Although there are a variety of multiple imputation methods for the longitudinal binary data, a limited number of researches have reported on relative performances of the methods. Moreover, when focusing on the treatment effect throughout a period that has often been used in clinical evaluations of specific disease areas, no definite investigations comparing the methods have been available. We conducted an extensive simulation study to examine comparative performances of six multiple imputation methods available in the SAS MI procedure for longitudinal binary data, where two endpoints of responder rates at a specified time point and throughout a period were assessed. The simulation study suggested that results from naive approaches of a single imputation with non-responders and a complete case analysis could be very sensitive against missing data. The multiple imputation methods using a monotone method and a full conditional specification with a logistic regression imputation model were recommended for obtaining unbiased and robust estimations of the treatment effect. The methods were illustrated with data from a mental health research.

Accelerometer data are widely used in research to provide objective measurements of physical activity. Frequently, participants may remove accelerometers during their observation period resulting in missing data referred to as nonwear periods. Common approaches for handling nonwear periods include discarding data (days with insufficient hours or individuals with insufficient valid days) from analyses and single imputation (SI) methods.Purpose:This study evaluates the performance of various discard-, SI-, and multiple imputation (MI)-based approaches on the ability to accurately and precisely characterize the relationship between a summarized measure of accelerometer counts (mean counts per minute) and an outcome (body mass index).Methods:Realistic accelerometer data were simulated under various scenarios that induced nonwear. Data were analyzed using common and MI methods for handling nonwear. Bias, relative standard error, relative mean squared error, and coverage probabilities were compared across methods.Results:MI approaches were superior to commonly applied methods, with bias that ranged from −0.001 to −0.028 that was considerably lower than that of discard-based methods (ranging from −0.050 to −0.057) and SI methods (ranging from −0.061 to −0.081). We also reported substantial variation among MI strategies, with coverage probabilities ranging from .04 to .96.Conclusion:Our findings demonstrate the benefit of applying MI methods over more commonly applied discard- and SI-based approaches. Additionally, we show that how you apply MI matters, where including data from previously observed acceleration measurements in the imputation model when using MI improves model performance.

ABSTRACTIntroductionIn analysis of time‐to‐event outcomes, a mixture cure (MC) model is preferred over a standard survival model when the sample includes individuals who will never experience the event of interest. Motivated by a cohort study of breast cancer patients with incomplete biomarkers, we develop multiple imputation (MI) methods assuming a Weibull proportional hazards (PH‐MC) analysis model with multiple prognostic factors. However, for MI with fully conditional specification, an incorrectly specified imputation model can impair accuracy of point and interval estimates.Objectives and MethodsOur goal is to propose imputation models that are compatible with the Weibull PH‐MC analysis models. We derive an exact conditional distribution (ECD) imputation model which involves the analysis model likelihood. Using simulation studies, we compare effect estimate bias and confidence interval (CI) coverage under alternative imputation models including the ECD model, an approximation that includes a cure indicator (cECD), and a comprehensive simple (CS) model. For robust parameter estimation in finite and/or sparse samples, we incorporate the Firth‐type penalized likelihood (FT‐PL) and combined likelihood profile (CLIP) methods into the MI.ResultsCompared to complete case analysis, MI with penalization reduces estimation bias and improves coverage. Although ECD and cECD perform similarly at higher event rates, ECD generates smaller bias and higher coverage at lower rates. CS has larger bias and lower coverage than ECD and cECD, but CIs are narrower than for cECD.ConclusionsIn analyses of biomarkers and composite subtypes for prognosis studies such as in breast cancer, the use of compatible imputation models and penalization methods is recommended for MC modeling in samples with low event numbers and/or with covariate imbalance.

Clinical trials allow researchers to draw conclusions about the effectiveness of a treatment. However, the statistical analysis used to draw these conclusions will inevitably be complicated by the common problem of attrition. Resorting to ad hoc methods such as case deletion or mean imputation can lead to biased results, especially if the amount of missing data is high. Multiple imputation, on the other hand, provides the researcher with an approximate solution that can be generalized to a number of different data sets and statistical problems. Multiple imputation is known to be statistically valid when n is large. However, questions still remain about the validity of multiple imputation for small samples in clinical trials. In this paper we investigate the small-sample performance of several multiple imputation methods, as well as the last observation carried forward method.