Robust mean and covariance structure analysis.

Robust schemes in regression are adapted to mean and covariance structure analysis, providing an iteratively reweighted least squares approach to robust structural equation modeling. Each case is properly weighted according to its distance, based on first and second order moments, from the structural model. A simple weighting function is adopted because of its flexibility with changing dimensions. The weight matrix is obtained from an adaptive way of using residuals. Test statistic and standard error estimators are given, based on iteratively reweighted least squares. The method reduces to a standard distribution-free methodology if all cases are equally weighted. Examples demonstrate the value of the robust procedure.

The asymptotically distribution free (ADF) method is often used to estimate parameters or test models without a normal distribution assumption on variables, both in covariance structure analysis and in correlation structure analysis. However, little has been done to study the differences in behaviors of the ADF method in covariance versus correlation structure analysis. The behaviors of 3 test statistics frequently used to evaluate structural equation models with nonnormally distributed variables, χ2 test TAGLS and its small-sample variants TYB and TF(AGLS) were compared. Results showed that the ADF method in correlation structure analysis with test statistic TAGLS performs much better at small sample sizes than the corresponding test for covariance structures. In contrast, test statistics TYB and TF(AGLS) under the same conditions generally perform better with covariance structures than with correlation structures. It is proposed that excessively large and variable condition numbers of weight matrices are a cause of poor behavior of ADF test statistics in small samples, and results showed that these condition numbers are systematically increased with substantial increase in variance as sample size decreases. Implications for research and practice are discussed.

Research studying robustness of maximum likelihood (ML) statistics in covariance structure analysis has concluded that test statistics and standard errors are biased under severe non-normality. An estimation procedure known as asymptotic distribution free (ADF), making no distributional assumption, has been suggested to avoid these biases. Corrections to the normal theory statistics to yield more adequate performance have also been proposed. This study compares the performance of a scaled test statistic and robust standard errors for two models under several non-normal conditions and also compares these with the results from ML and ADF methods. Both ML and ADF test statistics performed rather well in one model and considerably worse in the other. In general, the scaled test statistic seemed to behave better than the ML test statistic and the ADF statistic performed the worst. The robust and ADF standard errors yielded more appropriate estimates of sampling variability than the ML standard errors, which were usually downward biased, in both models under most of the non-normal conditions. ML test statistics and standard errors were found to be quite robust to the violation of the normality assumption when data had either symmetric and platykurtic distributions, or non-symmetric and zero kurtotic distributions.

Asymptotically distribution free (ADF) type test statistics play an important role in covariance structure analysis with nonnormal data. However, empirical studies have indicated that ADF-type test statistics rejected the correct models too frequently at all but very large sample sizes. This phenomenon may similarly occur for tests involving nested models. Several statistics related to ADF-type nested tests are given. The new test statistics have the same asymptotic properties as the original ADF-type test statistics. When used to evaluate model restrictions with small sample sizes, our statistics are more conservative in rejecting correct models. Numerical comparison of critical values are given for selected sample sizes and model parameter differences.

Existing methods for structural equation modeling involve fitting the ordinary sample covariance matrix by a proposed structural model. Since a sample covariance is easily influenced by a few outlying cases, the standard practice of modeling sample covariances can lead to inefficient estimates as well as inflated fit indices. By giving a proper weight to each individual case, a robust covariance will have a bounded influence function as well as a nonzero breakdown point. These robust properties of the covariance estimators will be carried over to the parameter estimators in the structural model if a technically appropriate procedure is used. We study such a procedure in which robust covariances replace ordinary sample covariances in the context of the Wishart likelihood function. This procedure is easy to implement in practice. Statistical properties of this procedure are investigated. A fit index is given based on sampling from an elliptical distribution. An estimating equation approach is used to develop a variety of robust covariances, and consistent covariances of these robust estimators, needed for standard errors and test statistics, follow from this approach. Examples illustrate the inflated statistics and distorted parameter estimates obtained by using sample covariances when compared with those obtained by using robust covariances. The merits of each method and its relevance to specific types of data are discussed.

A major issue in the utilization of covariance structure analysis is model fit evaluation. Recent years have witnessed increasing interest in various test statistics and so-called fit indexes, most of which are actually based on or closely related to F 0, a measure of model fit in the population. This study aims to provide a systematic investigation about the performance of 4 available estimators of F 0. [Fcirc]01 is the conventional estimator and is based on noncentral chi-square approximation. [Fcirc]02 is newly proposed and does not assume noncentral chi-square approximation. [Fcirc]03 and [Fcirc]04 are variations of [Fcirc]02. A Monte Carlo simulation study is conducted to examine how these four estimators of F 0 perform across varying model misspecifications, data distributions, model sizes, and sample sizes. The results show that under normality all 4 quantities estimate F 0 equally well, and under nonnormality [Fcirc]02, [Fcirc]03, and [Fcirc]04 outperform [Fcirc]01. Issues related to these findings are discussed.

The asymptotically distribution-free (ADF) test statistic for covariance structure analysis (CSA) has been reported to perform very poorly in simulation studies, i.e. it leads to inaccurate decisions regarding the adequacy of models of psychological processes. It is shown in the present study that the poor performance of the ADF test statistic is due to inadequate estimation of the weight matrix (W = gamma -1), which is a critical quantity in the ADF theory. Bootstrap procedures based on Hall's bias reduction perspective are proposed to correct the ADF test statistic. It is shown that the bootstrap correction of additive bias on the ADF test statistic yields the desired tail behaviour as the sample size reaches 500 for a 15-variable-3-factor confirmatory factor-analytic model, even if the distribution of the observed variables is not multivariate normal and the latent factors are dependent. These results help to revive the ADF theory in CSA.

We provide a general framework for designing Generative Adversarial Networks (GANs) to solve high-dimensional robust statistics problems, which aim at estimating unknown parameter of the true distribution given adversarially corrupted samples. Prior work [1], [2] focus on the problem of robust mean and covariance estimation when the true distribution lies in the family of Gaussian distributions or elliptical distributions, and analyze depth or scoring rule based GAN losses for the problem. Our work extend these to robust mean estimation, second-moment estimation, and robust linear regression when the true distribution only has bounded Orlicz norms, which includes the broad family of sub-Gaussian, sub-exponential and bounded moment distributions. We also provide a different set of sufficient conditions for the GAN loss to work: we only require its induced distance function to be a cumulative density function of some light-tailed distribution, which is easily satisfied by neural networks with sigmoid activation. In terms of techniques, our proposed GAN losses can be viewed as a smoothed and generalized Kolmogorov-Smirnov distance, which overcomes the computational intractability of the original Kolmogorov-Smirnov distance used in the weaken the distance approach in [3].

Covariance structure analysis is used for inference and for dimension reduction with multivariate data. When data are not normally distributed, the asymptotic distribution free (ADF) method is often used to fit a proposed model. The ADF test statistic is asymptotically distributed as a chi-square variate. Experience with real data indicates that the ADF statistic tends to reject theoretically meaningful models. Empirical simulation shows that the ADF statistic rejects correct models too often for all but impractically large sample sizes. By comparing mean and covariance structure analysis with its analogue in the multivariate linear model, we propose some modified ADF test statistics whose distributions are approximated by F distributions. Empirical studies show that the distributions of the new statistics are more closely approximated by F distributions than are the original ADF statistics when referred to chi-square distributions. Detailed analysis indicates why the ADF statistic fails on large models and why F tests and corrections give better results. Implications for power analysis and model tests in other areas are discussed.

Evaluating the goodness of fit of logistic regression models is crucial to ensure the accuracy of the estimated probabilities. Unfortunately, such evaluation is problematic in large samples. Because the power of traditional goodness of fit tests increases with the sample size, practically irrelevant discrepancies between estimated and true probabilities are increasingly likely to cause the rejection of the hypothesis of perfect fit in larger and larger samples. This phenomenon has been widely documented for popular goodness of fit tests, such as the Hosmer-Lemeshow test. To address this limitation, we propose a modification of the Hosmer-Lemeshow approach. By standardizing the noncentrality parameter that characterizes the alternative distribution of the Hosmer-Lemeshow statistic, we introduce a parameter that measures the goodness of fit of a model but does not depend on the sample size. We provide the methodology to estimate this parameter and construct confidence intervals for it. Finally, we propose a formal statistical test to rigorously assess whether the fit of a model, albeit not perfect, is acceptable for practical purposes. The proposed method is compared in a simulation study with a competing modification of the Hosmer-Lemeshow test, based on repeated subsampling. We provide a step-by-step illustration of our method using a model for postneonatal mortality developed in a large cohort of more than 300000 observations.

When the assumption of multivariate normality is violated or when a discrepancy function other than (normal theory) maximum likelihood is used in structural equation models, the null distribution of the test statistic may not be χ2 distributed. Most existing methods to approximate this distribution only match up to 2 moments. In this article, we propose 2 additional approximation methods: a scaled F distribution that matches 3 moments simultaneously and a direct Monte Carlo–based weighted sum of i.i.d. χ2 variates. We also conduct comprehensive simulation studies to compare the new and existing methods for both maximum likelihood and nonmaximum likelihood discrepancy functions and to separately evaluate the effect of sampling uncertainty in the estimated weights of the weighted sum on the performance of the approximation methods.

This paper examines the problem of model modification in covariance structure analysis. Two methods of model modification are studied: the Modification Index (MI) which suggests modifications based on the largest drop in the overall value of the test statistic, and the Expected Parameter Change Statistic (EPC) which suggests modifications based on the removal of large and interesting specifications errors. Following a detailed discussion of the theory behind the MI and EPC, these methods are studied and applied to two specifications of the Wisconsin status attainment model. Additionally, a standardized version of the EPC statistic (SEPC) is proposed and applied to one of these models. Results indicate that the MI tends to suggest freeing substantively implausible parameters. The EPC and SEPC, by contrast, suggest freeing substantively interesting parameters. Results are discussed in terms of the practice of covariance structure modeling.

This study examines how organizational memory is formed and contributes to successful organizational learning from failure in Company A, a Japanese firm known for effectively learning from its failures. A hypothetical model extending the organizational memory mechanism model was tested using covariance structure analysis on data collected from 190 employees through a company-wide questionnaire. The results showed an acceptable model fit, suggesting that a well-functioning organizational memory supports effective organizational learning from failure. The study also revealed that, unlike in unspecified firms where documentation is directly prompted by matters of interest, Company A's distinctive characteristic is that documentation is not immediately triggered by matters of concern but is instead facilitated with computers. This finding suggests that Company A's approach of systematically recording failures using computers, rather than relying on immediate personal interest, contributes to the prevention of failure recurrence. The study highlights the importance of utilizing computers to enhance organizational memory and successfully implement organizational learning from failure. Further research is needed to validate these findings and determine whether they are key factors in preventing repeated failures in Company A.

This study proposes to modify improved Prony method by suggesting to use robust covariance matrix for the identification of low frequency power system modes utilising the real time data obtained from phasor measurement unit. The methods reported in the current literature are many based on l 2 ‐norm minimisation, the estimated modes from these methods are severely affected in the presence of an outlier or bad measurements. Hence, there is a need to develop the method which can provide a good estimate of the modes even in the presence of outliers in the measurements. In order to mitigate the effect of such bad data, this study proposes a robust Prony estimator for estimating the low frequency modes of power systems. The method is based on minimum covariance determinant technique to find the robust covariance followed by improved Prony to estimate the modes. In this study the proposed method is compared with other methods such as improved Prony, etc. on synthetic test signal at different signal‐to‐noise ratio in the presence of outliers. Furthermore, the effectiveness of the proposed method is also shown on two area system and real time probing data of the Western Electricity Coordinating Council network.

Principal component analysis (PCA) is one of the most popular dimension reduction methods. The usual PCA is known to be sensitive to the presence of outliers, and thus many robust PCA methods have been developed. Among them, the Tyler's M-estimator is shown to be the most robust scatter estimator under the elliptical distribution. However, when the underlying distribution is contaminated and deviates from ellipticity, Tyler's M-estimator might not work well. In this article, we apply the semiparametric theory to propose a robust semiparametric PCA. The merits of our proposal are twofold. First, it is robust to heavy-tailed elliptical distributions as well as robust to non-elliptical outliers. Second, it pairs well with a data-driven tuning procedure, which is based on active ratio and can adapt to different degrees of data outlyingness. Theoretical properties are derived, including the influence functions for various statistical functionals and asymptotic normality. Simulation studies and a data analysis demonstrate the superiority of our method.