Methods for repeated measures data analysis with missing values

Multiple imputation was first conceived as a tool that statistical agencies could use to handle nonresponse in large-sample public use surveys. In the last two decades, the multiple-imputation framework has been adapted for other statistical contexts. For example, individual researchers use multiple imputation to handle missing data in small samples, statistical agencies disseminate multiply-imputed data sets for purposes of protecting data confidentiality, and survey methodologists and epidemiologists use multiple imputation to correct for measurement errors. In some of these settings, Rubin's original rules for combining the point and variance estimates from the multiply-imputed data sets are not appropriate, because what is known—and thus the conditional expectations and variances used to derive inferential methods—differs from that in the missing-data context. These applications require new combining rules and methods of inference. In fact, more than 10 combining rules exist in the published literature. This article describes some of the main adaptations of the multiple-imputation framework, including missing data in large and small samples, data confidentiality, and measurement error. It reviews the combining rules for each setting and explains why they differ. Finally, it highlights research topics in extending the multiple-imputation framework.

The asymptotically distribution free (ADF) method is often used to estimate parameters or test models without a normal distribution assumption on variables, both in covariance structure analysis and in correlation structure analysis. However, little has been done to study the differences in behaviors of the ADF method in covariance versus correlation structure analysis. The behaviors of 3 test statistics frequently used to evaluate structural equation models with nonnormally distributed variables, χ2 test TAGLS and its small-sample variants TYB and TF(AGLS) were compared. Results showed that the ADF method in correlation structure analysis with test statistic TAGLS performs much better at small sample sizes than the corresponding test for covariance structures. In contrast, test statistics TYB and TF(AGLS) under the same conditions generally perform better with covariance structures than with correlation structures. It is proposed that excessively large and variable condition numbers of weight matrices are a cause of poor behavior of ADF test statistics in small samples, and results showed that these condition numbers are systematically increased with substantial increase in variance as sample size decreases. Implications for research and practice are discussed.

This research examine different covariance structures for experimental design with repeated measure data. Multiple responses taken sequentially from same experimental unit at different periods of time for quantitative data are referred as Repeated Measurement. Weight of 105 broilers in grams for six group obtained from jewel farm Gombe were used as research materials/data. Eleven different covariance structures including the modified one (UN, UNC, TOEP, TOEPH, ANTE(1), AR(1), ARH(1), CS, CSH, HF and ARFA(1)) were examined. AIC, AICC, BIC, HQIC, CAIC and the modified criteria ASIC were used to examine covariance structures and bring the best among them using the named information criteria. The result shows that sphericity assumptions was violated a such the best covariance structure was ARH(1) while the least structure was CSH. Also on the basis of goodness of fit criteria HQIC was found to be the best information criteria. When examined the best information criteria and covariance structure with the modified ones, the modified ASIC and ARFA(1) found to be the best. In conclusion examine different covariance structures with repeated measure data give a very good result defending on the kind of data.

BackgroundMissing data are a common problem in prospective studies with a long follow-up, and the volume, pattern and reasons for missing data may be relevant when estimating the cost of illness. We aimed to evaluate the effects of different methods for dealing with missing longitudinal cost data and for costing caregiver time on total societal costs in Alzheimer’s disease (AD).MethodsGERAS is an 18-month observational study of costs associated with AD. Total societal costs included patient health and social care costs, and caregiver health and informal care costs. Missing data were classified as missing completely at random (MCAR), missing at random (MAR) or missing not at random (MNAR). Simulation datasets were generated from baseline data with 10–40 % missing total cost data for each missing data mechanism. Datasets were also simulated to reflect the missing cost data pattern at 18 months using MAR and MNAR assumptions. Naïve and multiple imputation (MI) methods were applied to each dataset and results compared with complete GERAS 18-month cost data. Opportunity and replacement cost approaches were used for caregiver time, which was costed with and without supervision included and with time for working caregivers only being costed.ResultsTotal costs were available for 99.4 % of 1497 patients at baseline. For MCAR datasets, naïve methods performed as well as MI methods. For MAR, MI methods performed better than naïve methods. All imputation approaches were poor for MNAR data. For all approaches, percentage bias increased with missing data volume. For datasets reflecting 18-month patterns, a combination of imputation methods provided more accurate cost estimates (e.g. bias: −1 % vs −6 % for single MI method), although different approaches to costing caregiver time had a greater impact on estimated costs (29–43 % increase over base case estimate).ConclusionsMethods used to impute missing cost data in AD will impact on accuracy of cost estimates although varying approaches to costing informal caregiver time has the greatest impact on total costs. Tailoring imputation methods to the reason for missing data will further our understanding of the best analytical approach for studies involving cost outcomes.Electronic supplementary materialThe online version of this article (doi:10.1186/s12874-016-0188-1) contains supplementary material, which is available to authorized users.

Monitoring and management of large scale applications is already a complex task because of syntactic and unstructured nature of execution data. Traditional application monitoring and management solutions focused on employing analysis techniques on unstructured and syntactic log information become limited as unstructured information cannot be well utilized to find out related events information or correlate such information with other related information from applications. Our proposed solution of semantically formalized logging fills this gap by bringing formal semantics and combining it in a meaningful way to enable automated monitoring and management of applications. Such formalized and well-structured log information helps analytical solution to maximally automate the process of monitoring and management of applications. However, while formalizing and structuring the log information, we came across several missing and incomplete data which causes hindrance in this process. In this paper, we tackle this problem and propose a social network analysis based solution to handle incomplete and missing data from application execution, possibly compute it and use it by our proposed solution of semantically formalizing and structured logs with adapted data mining techniques to enable automated and effective application monitoring and management. We demonstrate from an industrial use-case application that how historical data from application execution is stored using semantic logging and utilized with standard social-network analysis techniques to find out missing values in incomplete data and perform application monitoring and management.

Offshore wind turbine (OWT) installations are continually expanding as they are considered an efficient mechanism for covering a part of the energy consumption requirements. The assessment of the energy potential of OWTs for specific offshore sites is the key factor that defines their successful implementation, commercialization and sustainability. The data used for this assessment mainly refer to wind speed measurements. However, the data may not present homogeneity due to incomplete or missing entries; this in turn, is attributed to failures of the measuring devices or other factors. This fact may lead to considerable limitations in the OWTs energy potential assessment. This paper presents two novel methodologies to handle the problem of incomplete and missing data. Computational intelligence algorithms are utilized for the filling of the incomplete and missing data in order to build complete wind speed series. Finally, the complete wind speed series are used for assessing the energy potential of an OWT in a specific offshore site. In many real-world metering systems, due to meter failures, incomplete and missing data are frequently observed, leading to the need for robust data handling. The novelty of the paper can be summarized in the following points: (i) a comparison of clustering algorithms for extracting typical wind speed curves is presented for the OWT related literature and (ii) two efficient novel methods for missing and incomplete data are proposed.

Cancer registries are important real-world data sources consisting of data abstraction from the medical record; however, patients with unknown or missing data are underrepresented in studies that use such data sources. To assess the prevalence of missing data and its association with overall survival among patients with cancer. In this retrospective cohort study, all variables within the National Cancer Database were reviewed for missing or unknown values for patients with the 3 most common cancers in the US who received diagnoses from January 1, 2006, to December 31, 2015. The prevalence of patient records with missing data and the association with overall survival were assessed. Data analysis was performed from February to August 2020. Any missing data field within a patient record among 63 variables of interest from more than 130 total variables in the National Cancer Database. Prevalence of missing data in the medical records of patients with cancer and associated 2-year overall survival. A total of 1 198 749 patients with non-small cell lung cancer (mean [SD] age, 68.5 [10.9] years; 628 811 men [52.5%]), 2 120 775 patients with breast cancer (mean [SD] age, 61.0 [13.3] years; 2 101 758 women [99.1%]), and 1 158 635 patients with prostate cancer (mean [SD] age, 65.2 [9.0] years; 100% men) were included in the analysis. Among those with non-small cell lung cancer, 851 295 patients (71.0%) were missing data for variables of interest; 2-year overall survival was 33.2% for patients with missing data and 51.6% for patients with complete data (P < .001). Among those with breast cancer, 1 161 096 patients (54.7%) were missing data for variables of interest; 2-year overall survival was 93.2% for patients with missing data and 93.9% for patients with complete data (P < .001). Among those with prostate cancer, 460 167 patients (39.7%) were missing data for variables of interest; 2-year overall survival was 91.0% for patients with missing data and 95.6% for patients with complete data (P < .001). This study found that within a large cancer registry-based real-world data source, there was a high prevalence of missing data that were unable to be ascertained from the medical record. The prevalence of missing data among patients with cancer was associated with heterogeneous differences in overall survival. Improvements in documentation and data quality are necessary to make optimal use of real-world data for clinical advancements.

Missing data result in less precise and possibly biased effect estimates in single studies. Bias arising from studies with incomplete outcome data is naturally propagated in a meta‐analysis. Conventional analysis using only individuals with available data is adequate when the meta‐analyst can be confident that the data are missing at random (MAR) in every study—that is, that the probability of missing data does not depend on unobserved variables, conditional on observed variables. Usually, such confidence is unjustified as participants may drop out due to lack of improvement or adverse effects. The MAR assumption cannot be tested, and a sensitivity analysis to assess how robust results are to reasonable deviations from the MAR assumption is important. Two methods may be used based on plausible alternative assumptions about the missing data. Firstly, the distribution of reasons for missing data may be used to impute the missing values. Secondly, the analyst may specify the magnitude and uncertainty of possible departures from the missing at random assumption, and these may be used to correct bias and reweight the studies. This is achieved by employing a pattern mixture model and describing how the outcome in the missing participants is related to the outcome in the completers. Ideally, this relationship is informed using expert opinion. The methods are illustrated in two examples with binary and continuous outcomes. We provide recommendations on what trial investigators and systematic reviewers should do to minimize the problem of missing outcome data in meta‐analysis.

(1) Background: Gene-expression data usually contain missing values (MVs). Numerous methods focused on how to estimate MVs have been proposed in the past few years. Recent studies show that those imputation algorithms made little difference in classification. Thus, some scholars believe that how to select the informative genes for downstream classification is more important than how to impute MVs. However, most feature-selection (FS) algorithms need beforehand imputation, and the impact of beforehand MV imputation on downstream FS performance is seldom considered. (2) Method: A modified chi-square test-based FS is introduced for gene-expression data. To deal with the challenge of a small sample size of gene-expression data, a heuristic method called recursive element aggregation is proposed in this study. Our approach can directly handle incomplete data without any imputation methods or missing-data assumptions. The most informative genes can be selected through a threshold. After that, the best-first search strategy is utilized to find optimal feature subsets for classification. (3) Results: We compare our method with several FS algorithms. Evaluation is performed on twelve original incomplete cancer gene-expression datasets. We demonstrate that MV imputation on an incomplete dataset impacts subsequent FS in terms of classification tasks. Through directly conducting FS on incomplete data, our method can avoid potential disturbances on subsequent FS procedures caused by MV imputation. An experiment on small, round blue cell tumor (SRBCT) dataset showed that our method found additional genes besides many common genes with the two compared existing methods.

Mixture regression for longitudinal data based on joint mean–covariance model

SummaryLinear regression (LR) is vastly used in data analysis for continuous outcomes in biomedicine and epidemiology. Despite its popularity, LR is incompatible with missing data, which frequently occur in health sciences. For parameter estimation, this short-coming is usually resolved by complete-case analysis or imputation. Both workarounds, however, are inadequate for prediction, since they either fail to predict on incomplete records or ignore missingness-induced reduction in prediction accuracy and rely on (unrealistic) assumptions about the missing mechanism. Here, we derive adaptive predictor-set linear model (aps-lm), capable of making predictions for incomplete data without the need for imputation. It is derived by using a predictor-selection operation, the Moore-Penrose pseudoinverse and the reduced QR-decomposition. aps-lm is an LR generalization that inherently handles missing values. It is applied on a reference dataset, where complete predictors and outcome are available, and yields a set of privacy-preserving parameters. In a second stage, these are shared for making predictions of the outcome on external datasets with missing entries for predictors without imputation. Moreover, aps-lm computes prediction errors that account for the pattern of missing values even under extreme missingness. We benchmark aps-lm in a simulation study. aps-lm showed greater prediction accuracy and reduced bias compared to popular imputation strategies under a wide range of scenarios including variation of sample size, goodness-of-fit, missing value type and covariance structure. Finally, as a proof-of-principle, we apply aps-lm in the context of epigenetic aging clocks, linear models that predict a person’s biological age from epigenetic data with promising clinical applications.

In situations with missing data, statistical analyses are usually limited to subjects with complete data. However, such estimates may be biased. The method of 'filling in' missing data is called imputation. This article aimed to present a multiple imputation method. From a data set of 470 surgical patients, logistic models were developed for death as the outcome. Two incomplete data sets were generated: one with 5% and another with 20% of missing data in a single variable. Logistic models were fitted for the complete and incomplete data sets and for the data set completed by multiple imputations. Estimates obtained for the data set with missing data were different from those observed in the complete data set, mainly in the situation with 20% of missing data. The multiple imputation used here appeared efficient, producing very similar results to those obtained with the complete data set. However, one coefficient became non-significant. The analysis using multiple imputations was considered superior to using the data sets that excluded incomplete cases from the analysis.

The data open sharing provides us opportunity to investigate mechanisms underlying autism spectrum disorder (ASD) by employing advanced techniques. In the current study, we employed structural covariance (SC) model to investigate the development of triple networks in ASD. Three hundred and seven ASD and 337 typical controls were collected and further classified into four distinct age cohorts. Night brain seeds belonging to default modal network, salience network, and central executive network were obtained. SC between those seeds, as well as its topological properties, was calculated for each group within each age cohort. Statistical analysis revealed that ASD had dynamic reconfigurations of SC of the triple networks, especially the right fronto-insular cortex. The results might indicate that ASD had specific mechanism within distinct age cohort. Additionally, the big data sharing, together with the SC modal, was able to facilitate understanding of the mechanism underlying ASD.

The missing values in industrial data restrict the applications. Although this incomplete data contains enough information for engineers to support subsequent development, there are still too many missing values for algorithms to establish precise models. This is because the engineering domain knowledge is not considered, and valuable information is not fully captured. Therefore, this article proposes an engineering domain knowledge-based framework for modelling incomplete industrial data. The raw datasets are partitioned and processed at different scales. Firstly, the hierarchical features are combined to decrease the missing ratio. In order to fill the missing values in special data, which is identified for classifying the samples, samples with only part of the features presented are fully utilized instead of being removed to establish local imputation model. Then samples are divided into different groups to transfer the information. A series of industrial data is analyzed for verifying the feasibility of the proposed method.

Fuzzy neuron modeling of incomplete data for missing value imputation