Behavior of Asymptotically Distribution Free Test Statistics in Covariance Versus Correlation Structure Analysis

The asymptotically distribution-free (ADF) test statistic for covariance structure analysis (CSA) has been reported to perform very poorly in simulation studies, i.e. it leads to inaccurate decisions regarding the adequacy of models of psychological processes. It is shown in the present study that the poor performance of the ADF test statistic is due to inadequate estimation of the weight matrix (W = gamma -1), which is a critical quantity in the ADF theory. Bootstrap procedures based on Hall's bias reduction perspective are proposed to correct the ADF test statistic. It is shown that the bootstrap correction of additive bias on the ADF test statistic yields the desired tail behaviour as the sample size reaches 500 for a 15-variable-3-factor confirmatory factor-analytic model, even if the distribution of the observed variables is not multivariate normal and the latent factors are dependent. These results help to revive the ADF theory in CSA.

Research studying robustness of maximum likelihood (ML) statistics in covariance structure analysis has concluded that test statistics and standard errors are biased under severe non-normality. An estimation procedure known as asymptotic distribution free (ADF), making no distributional assumption, has been suggested to avoid these biases. Corrections to the normal theory statistics to yield more adequate performance have also been proposed. This study compares the performance of a scaled test statistic and robust standard errors for two models under several non-normal conditions and also compares these with the results from ML and ADF methods. Both ML and ADF test statistics performed rather well in one model and considerably worse in the other. In general, the scaled test statistic seemed to behave better than the ML test statistic and the ADF statistic performed the worst. The robust and ADF standard errors yielded more appropriate estimates of sampling variability than the ML standard errors, which were usually downward biased, in both models under most of the non-normal conditions. ML test statistics and standard errors were found to be quite robust to the violation of the normality assumption when data had either symmetric and platykurtic distributions, or non-symmetric and zero kurtotic distributions.

Asymptotically distribution free (ADF) type test statistics play an important role in covariance structure analysis with nonnormal data. However, empirical studies have indicated that ADF-type test statistics rejected the correct models too frequently at all but very large sample sizes. This phenomenon may similarly occur for tests involving nested models. Several statistics related to ADF-type nested tests are given. The new test statistics have the same asymptotic properties as the original ADF-type test statistics. When used to evaluate model restrictions with small sample sizes, our statistics are more conservative in rejecting correct models. Numerical comparison of critical values are given for selected sample sizes and model parameter differences.

Covariance structure analysis is a statistical technique in which a theoretical model, or a covariance structure, is constructed, and the covariances predicted by the theoretical model are compared with those of the observed data. The adequacy of the model in reproducing the sample covariances is reflected by estimates of the parameters of the model and measures indicating the goodness of fit. Covariance structure analysis is frequently used for analyzing data obtained in nonexperimental or quasiexperimental research, but is seldom employed in experimental research. In this paper, the applicability of this technique in experimental research is discussed and illustrated by covariance structure analysis studies in which two models for word translation—the symmetrical model and the asymmetrical model—are described, refined, and contrasted.

Even though data sets in psychology are seldom normal, the statistics used to evaluate covariance structure models are typically based on the assumption of multivariate normality. Consequently, many conclusions based on normal theory methods are suspect. In this paper, we develop test statistics that can be correctly applied to the normal theory maximum likelihood estimator. We propose three new asymptotically distribution-free (ADF) test statistics that technically must yield improved behaviour in samples of realistic size, and use Monte Carlo methods to study their actual finite sample behaviour. Results indicate that there exists an ADF test statistic that also performs quite well in finite sample situations. Our analysis shows that various forms of ADF test statistics are sensitive to model degrees of freedom rather than to model complexity. A new index is proposed for evaluating whether a rescaled statistic will be robust. Recommendations are given regarding the application of each test statistic.

Covariance structure analysis is used to evaluate hypothesized influences among unmeasured latent and observed variables. As implemented, it is not robust to outliers and bad data. Several robust methods in model fitting and testing are proposed. These include direct estimation of M-estimators of structured parameters and a two-stage procedure based on robust M- and S-estimators of population covariances. The large sample properties of these estimators are obtained. The equivalence between a direct M-estimator and a two-stage estimator based on an M-estimator of population covariance is established when sampling from an elliptical distribution. Two test statistics are presented in judging the adequacy of a hypothesized model; both are asymptotically distribution free if using distribution free weight matrices. So these test statistics possess both finite sample and large sample robustness. The two-stage procedures can be easily adapted into standard software packages by modifying existing asymptotically distribution free procedures. To demonstrate the two-stage procedure, S-estimator and M-estimators under different weight functions are calculated for some real data sets.

In covariance structure analysis, two-stage least-squares (2SLS) estimation has been recommended for use over maximum likelihood estimation when model misspecification is suspected. However, 2SLS often fails to provide stable and accurate solutions, particularly for structural equation models with small samples. To address this issue, a regularized extension of 2SLS is proposed that integrates a ridge type of regularization into 2SLS, thereby enabling the method to effectively handle the small-sample-size problem. Results are then reported of a Monte Carlo study conducted to evaluate the performance of the proposed method, as compared to its nonregularized counterpart. Finally, an application is presented that demonstrates the empirical usefulness of the proposed method.

This paper examines the problem of model modification in covariance structure analysis. Two methods of model modification are studied: the Modification Index (MI) which suggests modifications based on the largest drop in the overall value of the test statistic, and the Expected Parameter Change Statistic (EPC) which suggests modifications based on the removal of large and interesting specifications errors. Following a detailed discussion of the theory behind the MI and EPC, these methods are studied and applied to two specifications of the Wisconsin status attainment model. Additionally, a standardized version of the EPC statistic (SEPC) is proposed and applied to one of these models. Results indicate that the MI tends to suggest freeing substantively implausible parameters. The EPC and SEPC, by contrast, suggest freeing substantively interesting parameters. Results are discussed in terms of the practice of covariance structure modeling.

The two-sample problem for Cronbach's coefficient [Formula: see text], as an estimate of test or composite score reliability, has attracted little attention compared to the extensive treatment of the one-sample case. It is necessary to compare the reliability of a test for different subgroups, for different tests or the short and long forms of a test. In this paper, we study statistical procedures of comparing two coefficients [Formula: see text] and [Formula: see text]. The null hypothesis of interest is [Formula: see text], which we test against one-or two-sided alternatives. For this purpose, resampling-based permutation and bootstrap tests are proposed for two-group multivariate non-normal models under the general asymptotically distribution-free (ADF) setting. These statistical tests ensure a better control of the type-I error, in finite or very small sample sizes, when the state-of-affairs ADF large-sample test may fail to properly attain the nominal significance level. By proper choice of a studentized test statistic, the resampling tests are modified in order to be valid asymptotically even in non-exchangeable data frameworks. Moreover, extensions of this approach to other designs and reliability measures are discussed as well. Finally, the usefulness of the proposed resampling-based testing strategies is demonstrated in an extensive simulation study and illustrated by real data applications.

Similarity of genetic and phenotypic variation patterns among populations is important for making quantitative inferences about past evolutionary forces acting to differentiate populations and for evaluating the evolution of relationships among traits in response to new functional and developmental relationships. Here, phenotypic co variance and correlation structure is compared among Platyrrhine Neotropical primates. Comparisons range from among species within a genus to the superfamily level. Matrix correlation followed by Mantel's test and vector correlation among responses to random natural selection vectors (random skewers) were used to compare correlation and variance/covariance matrices of 39 skull traits. Sampling errors involved in matrix estimates were taken into account in comparisons using matrix repeatability to set upper limits for each pairwise comparison. Results indicate that covariance structure is not strictly constant but that the amount of variance pattern divergence observed among taxa is generally low and not associated with taxonomic distance. Specific instances of divergence are identified. There is no correlation between the amount of divergence in covariance patterns among the 16 genera and their phylogenetic distance derived from a conjoint analysis of four already published nuclear gene datasets. In contrast, there is a significant correlation between phylogenetic distance and morphological distance (Mahalanobis distance among genus centroids). This result indicates that while the phenotypic means were evolving during the last 30 millions years of New World monkey evolution, phenotypic covariance structures of Neotropical primate skulls have remained relatively consistent. Neotropical primates can be divided into four major groups based on their feeding habits (fruit-leaves, seed-fruits, insect-fruits, and gum-insect-fruits). Differences in phenotypic covariance structure are correlated with differences in feeding habits, indicating that to some extent changes in interrelationships among skull traits are associated with changes in feeding habits. Finally, common patterns and levels of morphological integration are found among Platyrrhine primates, suggesting that functional/developmental integration could be one major factor keeping covariance structure relatively stable during evolutionary diversification of South American monkeys.

Analysis of covariance structures is the common term for a number of techniques for analyzing multivariate data in order to detect and assess latent (unobserved) sources of variation and covariation in the observed measurements. The techniques of covariance structure analysis are general and flexible in that they can handle many types of covariance structures useful especially in the behavioral and social sciences. Although these techniques can be used for exploratory analysis, they have been most successfully applied to confirmatory analysis where the type of covariance structure is specified in advance. A covariance structure of a specified kind may arise because of a specified substantive theory or hypothesis, a given classificatory design for the measures, known experimental conditions, or because of results from previous studies based on extensive data. Sometimes the observed variables are ordered through time, as in longitudinal studies, or according to linear or circular patterns, as in Guttman’s (1954) simplex and circumplex models, or according to a given causal scheme, as in path analysis.

This paper states and proves the asymptotic properties of constrained generalized least squares estimators in the analysis of covariance structures in multiple populations with arbitrary distributions of variables. Asymptotic chi-square tests are also presented to permit evaluation of the goodness-of-fit of models. The currently known results for multiple population models based on variables that are multivariate normally distributed are obtained as a special case.

In this chapter, we bring together the notions of measurement error discussed in Chaps. 3 and 4 with the structural modeling of simultaneous relationships presented in Chap. 6. We demonstrate that a bias is introduced when estimating the relationship between two variables measured with error if that measurement error is ignored. We then present a methodology for estimating the parameters of structural relationships between variables that are not observed directly: analysis of covariance structures. We focus on the role of the measurement model as discussed in Chap. 4 with the confirmatory factor analytic model.

Keeping track of patients' medication histories is an important component of pharmacy practice. In this study, we conducted a questionnaire survey on medication histories among pharmacists and performed covariance structure analysis to investigate pharmacists' attitudes toward medication histories. The survey was conducted among pharmacists who work at pharmacies. With regard to the questions on medication histories, factor analysis and covariance structure analysis were performed to create a path diagram. The response rate to the questionnaire was 97.0%. The factor analysis revealed six factors, including: "patient assessment", "learning attitude", "practicing the recording of medication histories", "conscious effort to improve medication histories", "uniformity in medication histories" and "issues regarding medication histories". Meanwhile, covariance structure analysis revealed that the five-factor model excluding "issues regarding medication histories" was the best-fitted model. According to this model, it was clear that the pharmacists were extremely conscious about introducing improvements to the content of medication histories. In this study, covariance structure analysis enabled the effective analysis of the attitudes of pharmacists toward medication histories. We believe that creating and conducting a training program to clarify and resolve issues related to medication histories and then reviewing the outcome of such a training program will lead to quality improvement and standardization of the future management of medication histories.

This paper analyzed the relationship among the differential diagnosis, treatment, and results for allergic rhinitis using the covariance structural model. The data were collected from 274 patients with suspected allergic rhinitis who visited the Otorlaryngology Department of the Paik Hospital during 1991-1993. After each patient's characteristics was categorized and combined into several common factors, covariance structure analysis was performed to analyze the structural relationships among the differential diagnosis, treatment, and results of treatment using the significant factors obtained from discriminant analysis. The significant characteristics influenced the diagnosis were the results of skin test from mite/animal, and from mugworts, the results from laboratory tests, rhinorrhea and sneezing, and nasal polyps. The significant characteristics that influenced the method of treatment were: nasal polyps, headache/general symptom, family history/medication, and septal deviation. Headache/general symptom was the only significantly influencing factor for the treatment results.