Exploratory Factor Analysis With Small Samples and Missing Data

Determining the number of factors in exploratory factor analysis is arguably the most crucial decision a researcher faces when conducting the analysis. While several simulation studies exist that compare various so-called factor retention criteria under different data conditions, little is known about the impact of missing data on this process. Hence, in this study, we evaluated the performance of different factor retention criteria—the Factor Forest, parallel analysis based on a principal component analysis as well as parallel analysis based on the common factor model and the comparison data approach—in combination with different missing data methods, namely an expectation-maximization algorithm called Amelia, predictive mean matching, and random forest imputation within the multiple imputations by chained equations (MICE) framework as well as pairwise deletion with regard to their accuracy in determining the number of factors when data are missing. Data were simulated for different sample sizes, numbers of factors, numbers of manifest variables (indicators), between-factor correlations, missing data mechanisms and proportions of missing values. In the majority of conditions and for all factor retention criteria except the comparison data approach, the missing data mechanism had little impact on the accuracy and pairwise deletion performed comparably well as the more sophisticated imputation methods. In some conditions, especially small-sample cases and when comparison data were used to determine the number of factors, random forest imputation was preferable to other missing data methods, though. Accordingly, depending on data characteristics and the selected factor retention criterion, choosing an appropriate missing data method is crucial to obtain a valid estimate of the number of factors to extract.

ABSTRACTSmall samples are common in growth models due to financial and logistical difficulties of following people longitudinally. For similar reasons, longitudinal studies often contain missing data. Though full information maximum likelihood (FIML) is popular to accommodate missing data, the limited number of studies in this area have found that FIML tends to perform poorly with small-sample growth models. This report demonstrates that the fault lies not with how FIML accommodates missingness but rather with maximum likelihood estimation itself. We discuss how the less popular restricted likelihood form of FIML, along with small-sample-appropriate methods, yields trustworthy estimates for growth models with small samples and missing data. That is, previously reported small sample issues with FIML are attributable to finite sample bias of maximum likelihood estimation not direct likelihood. Estimation issues pertinent to joint multiple imputation and predictive mean matching are also included and discussed.

Missing data (MD) is a common and inevitable problem facing data mining (DM)-based decision systems in e-health since many medical historical datasets contain a huge number of missing values. Therefore, a pre-processing stage is usually required to deal with missing values before building any DM-based decision system. The purpose of this paper is to evaluate the impact of MD techniques on classification systems in cardiovascular dysautonomias diagnosis. We analyzed and compared the accuracy rates of four classification techniques: random forest (RF), support vector machines (SVM), C4.5 decision tree, and Naive Bayes (NB), using two MD techniques: deletion or imputation with k-nearest neighbors (KNN). A total of 216 experiments were therefore carried out using three missingness mechanisms (MCAR: missing completely at random, MAR: missing at random and NMAR: not missing at random), two MD techniques (deletion and KNN imputation), nine MD percentages from 10 to 90% over a dataset collected from the autonomic nervous system (ANS) unit of the University Hospital Avicenne in Morocco. The results obtained suggest that using KNN imputation rather than deletion enhances the accuracy rates of the four classifiers. Moreover, the MD percentages have a negative impact on the performance of classification techniques regardless of the MD mechanisms and MD techniques used. In fact, the accuracy rates of the four classifiers decrease as the MD percentage increases. Graphical abstract.

When modeling multilevel data, it is important to accurately represent the interdependence of observations within clusters. Ignoring data clustering may result in parameter misestimation. However, it is not well established to what degree parameter estimates are affected by model misspecification when applying missing data techniques (MDTs) to incomplete multilevel data. We compare the performance of three MDTs with incomplete hierarchical data. We consider the impact of imputation model misspecification on the quality of parameter estimates by employing multiple imputation under assumptions of a normal model (MI/NM) with two-level cross-sectional data when values are missing at random on the dependent variable at rates of 10%, 30%, and 50%. Five criteria are used to compare estimates from MI/NM to estimates from MI assuming a linear mixed model (MI/LMM) and maximum likelihood estimation to the same incomplete data sets. With 10% missing data (MD), techniques performed similarly for fixed-effects estimates, but variance components were biased with MI/NM. Effects of model misspecification worsened at higher rates of MD, with the hierarchical structure of the data markedly underrepresented by biased variance component estimates. MI/LMM and maximum likelihood provided generally accurate and unbiased parameter estimates but performance was negatively affected by increased rates of MD.

Missing data are condition when there are some missing values or empty entries on several observations on data. It could inhibit statistical analysis process and might give a bias conclusion from the analysis if couldn't be handled properly. This problem can be found on some linear regression analysis. One way to handle this problem is using multiple imputation (MI) method named Predictive Mean Matching (PMM). PMM will matching the predictive mean distance of incomplete observations with the complete observations. To get the multiple imputation concept, the predictive mean of incomplete observations were estimated by Bayesian approach while the complete observations were estimated with ordinary least square. Thus, the complete observation that has the closest distance will be a donor value for the incomplete one. Simulation data with two variable (x and y), univariate missing data pattern (on y), and MAR mechanism is used to analyzed the effectiveness of PMM based on relative efficiency estimation result of missing covariate data. Regression analysis used x as independent variable and y as dependent variable. The result showed that PMM give a significant coefficient regression parameter at 5% level of significance and only loss 1 % of relative efficiency.

Background: PIn air quality studies, it is very often to have missing data due to reasons such as machine failure or human error. The approach used in dealing with such missing data can affect the results of the analysis. The main aim of this study was to review the types of missing mechanism, imputation methods, application of some of them in imputation of missing of PM10 and O3 in Tabriz, and compare their efficiency. Methods: Methods of mean, EM algorithm, regression, classification and regression tree, predictive mean matching (PMM), interpolation, moving average, and K-nearest neighbor (KNN) were used. PMM was investigated by considering the spatial and temporal dependencies in the model. Missing data were randomly simulated with 10, 20, and 30% missing values. The efficiency of methods was compared using coefficient of determination (R2 ), mean absolute error (MAE) and root mean square error (RMSE). Results: Based on the results for all indicators, interpolation, moving average, and KNN had the best performance, respectively. PMM did not perform well with and without spatio-temporal information. Conclusion: Given that the nature of pollution data always depends on next and previous information, methods that their computational nature is based on before and after information indicated better performance than others, so in the case of pollutant data, it is recommended to use these methods.

Several methods of factor extraction have recently gained popularity as a procedure for dealing with estimation problems associated with small sample sizes, which can be found in the various behavioral science disciplines, such as comparative psychology and behavior genetics. Two popular approaches for particularly small samples (below 50) include unweighted least squares factor analysis (ULS-FA) and regularized exploratory factor analysis (REFA). However, it is unclear how well each of the approaches performs with small samples in the context of exploratory bifactor modeling. In the current study, a comprehensive simulation study was conducted to evaluate the small sample behavior of the two approaches in terms of bifactor structure recovery under different sample size, factor loading, number of variables per factor, number of factors, and factor correlation experimental conditions. The results show that REFA is recommended for use over ULS-FA, particularly in the conditions involving low factor loadings, few group factors, or a small number of variables per factor.

This study aimed to describe the estimated means for weight, height, and body mass index (BMI) according to two imputation methods, using data from Vigitel (Risk and Protective Factors Surveillance System for Chronic Non-Communicable Diseases Through Telephone Interview). This was a cross-sectional study that used secondary data from the Vigitel survey from 2006 to 2017. The two imputation methods used in the study were hot deck and Predictive Mean Matching (PMM). The weight and height variables imputed by hot deck were provided by Vigitel. Two models were conducted with PMM: (i) explanatory variables - city, sex, age in years, race/color, and schooling; (ii) explanatory variables - city, sex, and age in years. Weight and height were the outcome variables in the two models. PMM combines linear regression and random selection of the value for imputation. Linear prediction is used as a measure of distance between the missing value and the possible donors, thereby creating the virtual space with the candidate cases for yielding the value for imputation. One of the candidates from the pool is randomly selected, and its value is assigned to the missing unit. BMI was calculated by dividing weight in kilograms by height squared. The result shows the means and standard deviations for weight, height, and BMI according to imputation method and year. The estimates used the survey module from Stata, which considers the sampling effects. The mean values for weight, height, and BMI estimated by hot deck and PMM were similar. The results with the Vigitel data suggest the applicability of PMM to the set of health surveys.

(1) Background: Gene-expression data usually contain missing values (MVs). Numerous methods focused on how to estimate MVs have been proposed in the past few years. Recent studies show that those imputation algorithms made little difference in classification. Thus, some scholars believe that how to select the informative genes for downstream classification is more important than how to impute MVs. However, most feature-selection (FS) algorithms need beforehand imputation, and the impact of beforehand MV imputation on downstream FS performance is seldom considered. (2) Method: A modified chi-square test-based FS is introduced for gene-expression data. To deal with the challenge of a small sample size of gene-expression data, a heuristic method called recursive element aggregation is proposed in this study. Our approach can directly handle incomplete data without any imputation methods or missing-data assumptions. The most informative genes can be selected through a threshold. After that, the best-first search strategy is utilized to find optimal feature subsets for classification. (3) Results: We compare our method with several FS algorithms. Evaluation is performed on twelve original incomplete cancer gene-expression datasets. We demonstrate that MV imputation on an incomplete dataset impacts subsequent FS in terms of classification tasks. Through directly conducting FS on incomplete data, our method can avoid potential disturbances on subsequent FS procedures caused by MV imputation. An experiment on small, round blue cell tumor (SRBCT) dataset showed that our method found additional genes besides many common genes with the two compared existing methods.

The goal of the present review was to examine whether or not the use of modern missing data techniques impacts the statistical conclusion validity of research on alcohol and drug use outcomes in survey-based research studies. We identified 28 papers and received complete case data from the authors of 12 studies. Seven studies (25%) reported the missing data pattern (missing not at random [MNAR], missing at random [MAR], missing completely at random [MCAR]), 15 studies (53.6%) indicated the amount of missing observations in the data set, and a significant proportion of studies ( n = 13, 46.4%) did not report any of the conditions or assumptions under which the missing data analysis was performed or implemented. Six of the 12 (50%) studies analyzed reported a different number of statistically significant associations between the complete case and full sample analyses. Efforts should be made to make missing data analysis more accessible, easy to implement and report to improve transparency and reproducibility of findings.

The objective of this study was to evaluate the impact of three different methods of handling missing data on the performance of Phase I Hotelling’s T 2 multivariate control chart. Using a Monte Carlo simulation, we studied the average, median, and standard deviation of the run length performance of multivariate data imputed using mean substitution, regression imputation, and predictive mean matching at three different levels of missingness ( 1 % , 10 % , and 25 % ) and three levels of variable correlation coefficients (0.2, 0.4, and 0.8). We found that predictive mean matching has average run length performance results comparable to that of the complete in-control data set at all levels of missingness and variable correlation, while the performance of mean substitution was adversely affected by high levels of missingness and by strong variable correlation. Based on the simulation (multivariate normal data), we concluded that predictive mean matching is superior to both regression imputation and mean substitution as a method for imputing missing values for the analysis of Phase I Hotelling’s T 2 control chart. Two applications were presented using the Altenrhein wastewater treatment plant and Olive oil datasets.

294 Background: Economic evaluations in oncology aim to assess the value of new therapies in the long term based on clinical trial data that often have restricted follow-up times (< 5 years) and small sample sizes (< 500 patients). This requires the use of extrapolation assumptions on long-term survival that go beyond the observed data. In this analysis, differences in survival extrapolation methods are tested in samples of sizes and follow-up reflecting typical clinical trials against a background of known survival in prostate cancer from a US based cancer registry. Methods: Data from the National Cancer Institute's Surveillance Epidemiology and End Results (SEER) registry on long-term survival in patients with stage IV prostate cancer were employed. The data set comprised those patients diagnosed between 1988 and 2003, with follow-up data available until 2012. Additional survival for those who received surgery (compared to those who did not), was estimated based on extrapolations using standard parametric statistical models (exponential, Weibull, log-logistic, log-normal, Gamma) fitted to the observed data. Survival analyses were run for 5 sample size scenarios (n = 27,670, 1000, 500, 200, 50) and 6 follow-up scenarios (follow-up years = 25, 20, 10, 5, 2, 1) yielding 30 combination scenarios. Performance of the methods was tested relative to the maximum follow-up, maximum sample size scenario (i.e. reference case) from the SEER registry. Results: Log-logistic and log-normal models were associated with flat tails which led to inflated survival estimations. For scenarios with smaller sizes, gamma models often did not converge. Exponential models were the most frequently reported as best model fit (in approximately 50% of scenarios). Also, gains in OS were consistent when exponential models were selected, and closely matched gain in OS from the reference case. Conclusions: Since clinical trials in oncology are often associated with small patient sample sizes and restricted follow-up, selecting an exponential model may lead to the most consistent and stable results based on the experiment constructed here. Further research should confirm these results for other types of cancer.

BackgroundMissing data are a common problem in prospective studies with a long follow-up, and the volume, pattern and reasons for missing data may be relevant when estimating the cost of illness. We aimed to evaluate the effects of different methods for dealing with missing longitudinal cost data and for costing caregiver time on total societal costs in Alzheimer’s disease (AD).MethodsGERAS is an 18-month observational study of costs associated with AD. Total societal costs included patient health and social care costs, and caregiver health and informal care costs. Missing data were classified as missing completely at random (MCAR), missing at random (MAR) or missing not at random (MNAR). Simulation datasets were generated from baseline data with 10–40 % missing total cost data for each missing data mechanism. Datasets were also simulated to reflect the missing cost data pattern at 18 months using MAR and MNAR assumptions. Naïve and multiple imputation (MI) methods were applied to each dataset and results compared with complete GERAS 18-month cost data. Opportunity and replacement cost approaches were used for caregiver time, which was costed with and without supervision included and with time for working caregivers only being costed.ResultsTotal costs were available for 99.4 % of 1497 patients at baseline. For MCAR datasets, naïve methods performed as well as MI methods. For MAR, MI methods performed better than naïve methods. All imputation approaches were poor for MNAR data. For all approaches, percentage bias increased with missing data volume. For datasets reflecting 18-month patterns, a combination of imputation methods provided more accurate cost estimates (e.g. bias: −1 % vs −6 % for single MI method), although different approaches to costing caregiver time had a greater impact on estimated costs (29–43 % increase over base case estimate).ConclusionsMethods used to impute missing cost data in AD will impact on accuracy of cost estimates although varying approaches to costing informal caregiver time has the greatest impact on total costs. Tailoring imputation methods to the reason for missing data will further our understanding of the best analytical approach for studies involving cost outcomes.Electronic supplementary materialThe online version of this article (doi:10.1186/s12874-016-0188-1) contains supplementary material, which is available to authorized users.

Cluster-randomized trials allow for the evaluation of a community-level or group-/cluster-level intervention. For studies that require a cluster-randomized trial design to evaluate cluster-level interventions aimed at controlling vector-borne diseases, it may be difficult to assess a large number of clusters while performing the additional work needed to monitor participants, vectors, and environmental factors associated with the disease. One such example of a cluster-randomized trial with few clusters was the "efficacy and risk of harms of repeated ivermectin mass drug administrations for control of malaria" trial. Although previous work has provided recommendations for analyzing trials like repeated ivermectin mass drug administrations for control of malaria, additional evaluation of the multiple approaches for analysis is needed for study designs with count outcomes. Using a simulation study, we applied three analysis frameworks to three cluster-randomized trial designs (single-year, 2-year parallel, and 2-year crossover) in the context of a 2-year parallel follow-up of repeated ivermectin mass drug administrations for control of malaria. Mixed-effects models, generalized estimating equations, and cluster-level analyses were evaluated. Additional 2-year parallel designs with different numbers of clusters and different cluster correlations were also explored. Mixed-effects models with a small sample correction and unweighted cluster-level summaries yielded both high power and control of the Type I error rate. Generalized estimating equation approaches that utilized small sample corrections controlled the Type I error rate but did not confer greater power when compared to a mixed model approach with small sample correction. The crossover design generally yielded higher power relative to the parallel equivalent. Differences in power between analysis methods became less pronounced as the number of clusters increased. The strength of within-cluster correlation impacted the relative differences in power. Regardless of study design, cluster-level analyses as well as individual-level analyses like mixed-effects models or generalized estimating equations with small sample size corrections can both provide reliable results in small cluster settings. For 2-year parallel follow-up of repeated ivermectin mass drug administrations for control of malaria, we recommend a mixed-effects model with a pseudo-likelihood approximation method and Kenward-Roger correction. Similarly designed studies with small sample sizes and count outcomes should consider adjustments for small sample sizes when using a mixed-effects model or generalized estimating equation for analysis. Although the 2-year parallel follow-up of repeated ivermectin mass drug administrations for control of malaria is already underway as a parallel trial, applying the simulation parameters to a crossover design yielded improved power, suggesting that crossover designs may be valuable in settings where the number of available clusters is limited. Finally, the sensitivity of the analysis approach to the strength of within-cluster correlation should be carefully considered when selecting the primary analysis for a cluster-randomized trial.

Rhinoplasty is a common operation in which shape and function are intimately related, whether the procedure is cosmetic, functional, or combined in nature. There is currently no properly developed and validated patient-reported outcome measure (PROM) to evaluate both functional and cosmetic components of rhinoplasty. To develop, validate, and field test the Standardized Cosmesis and Health Nasal Outcomes Survey (SCHNOS) to evaluate both functional and cosmetic outcomes of rhinoplasty. Survey development study between October 2016 and April 2017 in a tertiary referral facial plastic and reconstructive surgery clinic. Preoperative and postoperative adult patients undergoing rhinoplasty, whether cosmetic or reconstructive, were included. A fifth group of adult nonrhinoplasty patients (facial cosmetic or reconstructive) were also included for the field test. Generated and reduced items, psychometric validation measures of the SCHNOS, and differences on scales between groups. For survey development, a total of 18 patients and 5 experts were interviewed. Of these patients, 5 were male, and 13 were female. Their mean (SD) age was 38 (14.8) years (range, 20-64 years). Field testing included 191 patients (67% were women and the mean [SD] age was 41.5 [15.8] years). A total of 10 items were included after generation, cognitive interviews, and item reduction. The 10 items represent 2 domains: nasal obstruction (first 4 items) and nasal cosmesis (last 6 items). For both domains, Cronbach α was excellent: 0.94 (95% CI, 0.92-0.95) for obstruction and 0.94 (95% CI, 0.93-0.95) for cosmesis. Exploratory factor analysis using scree plots for each domain showed that the domains are unidimensional in nature with each domain evaluating what it is intended to assess (nasal obstruction and cosmesis). The factor loading estimates were high for all the items, varying from 0.74 to 0.92. Kruskal-Wallis testing showed a significance level of P < .001 when evaluating the difference between groups (preoperative cosmetic, postoperative cosmetic, preoperative functional, postoperative functional, and nonrhinoplasty) for all individual questions, composite scores, and Nasal Obstruction Symptom Evaluation (NOSE) score. Correlations between the obstruction composite score and the NOSE scores were r = 0.943 (P < .001), which is very strong. The obstruction and cosmesis composite scores were only weakly correlated (r = 0.388; P < .001). We have developed and validated a new PROM to evaluate both functional and cosmetic rhinoplasty patients. The domains of obstruction and cosmesis were found to be internally consistent and unidimensional. The SCHNOS provides a short, validated questionnaire that we recommend for use in all functional or cosmetic rhinoplasty patients. N/A.