Using generalizability theory with continuous latent response variables.

Classical Test Theory (CTT), also known as the true score theory, refers to the analysis of test results based on test scores. The statistics produced under CTT include measures of item difficulty, item discrimination, measurement error and test reliability. The term “Classical” is used in contrast to “Modern” test theory which usually refers to item response theory (IRT). The fact that CTT was developed before IRT does not mean that CTT is outdated or replaced by IRT. Both CTT and IRT provide useful statistics to help us analyse test data. Generally, CTT and IRT provide complementary results. For many item analyses, CTT may be sufficient to provide the information we need. There are, however, theoretical differences between CTT and IRT, and many researchers prefer IRT because of enhanced measurement properties under IRT. IRT also provides a framework that facilitates test equating, computer adaptive testing and test score interpretation. While this book devotes a large part to IRT, we stress that CTT is an important part of the methodologies for educational and psychological measurement. In particular, the exposition of the concept of reliability in CTT sets the basis for evaluating measuring instruments. A good understanding of CTT lays the foundations for measurement principles. There are other approaches to measurement such as generalizability theory and structural equation modelling, but these are not the focus of attention in this book.

Structural equation modeling (SEM) is a very general and flexible multivariate technique that allows relationships among variables to be examined. The roots of SEM are in the social sciences. In writing this textbook, the authors look to make SEM accessible to a wider audience of researchers across many disciplines, addressing issues unique to health and medicine. SEM is often used in practice to model and test hypothesized causal relationships among observed and latent (unobserved) variables, including in analysis across time and groups. It can be viewed as the merging of a conceptual model, path diagram, confirmatory factor analysis, and path analysis. In this textbook the authors also discuss techniques, such as mixture modeling, that expand the capacity of SEM using a combination of both continuous and categorical latent variables. Features: Basic, intermediate, and advanced SEM topics Detailed applications, particularly relevant for health and medical scientists Topics and examples that are pertinent to both new and experienced SEM researchers Substantive issues in health and medicine in the context of SEM Both methodological and applied examples Numerous figures and diagrams to illustrate the examples As SEM experts situated among clinicians and multidisciplinary researchers in medical settings, the authors provide a broad, current, on the ground understanding of the issues faced by clinical and health services researchers and decision scientists. This book gives health and medical researchers the tools to apply SEM approaches to study complex relationships between clinical measurements, individual and community-level characteristics, and patient-reported scales.

The monitoring and diagnosis of multivariate categorical processes (MCPs) have drawn increasing attention lately, as categorical variables have been frequently involved in modern quality control applications. In these applications, there may exist causal relationships among multiple categorical variables, where the attribute level of a cause variable influences that of its effect variable. In such a case, shifts occurring in a cause variable will propagate to its effect variable based on the causal structure. Furthermore, there usually exists natural order among the attribute levels of some categorical variables such as good, neutral, and bad for measuring the product quality. By assuming a latent continuous variable, the attribute levels of an ordinal categorical variable can be determined by classifying the value of the latent variable based on thresholds. In this paper, we leverage Bayesian networks (BNs) to characterize MCPs with a causal structure, where the categorical variables can be either nominal, ordinal or a combination of both. We develop one general control chart and one directional control chart, both of which fully exploit the causal relationships and the ordinal information for better process monitoring and diagnosis. Numerical simulations have demonstrated the superiority and robustness of our method in detecting and diagnosing the conditional probability shifts of nominal factors as well as the conditional latent location shifts of ordinal factors. Note to Practitioners-This paper aims at addressing the challenges in monitoring and diagnosing MCPs when there are causal relationships among the categorical variables. The developed method is greatly beneficial, especially when there are nominal and ordinal variables involved in the MCP. Specifically, a BN is employed to characterize the dependence structure of the variables involved in the process, and a latent continuous variable is utilized to model the orders of attribute levels of the ordinal variables. Then, a novel method is proposed to detect the probability shift in the nominal factors and the location shift on the latent variables of the ordinal variables based on the likelihood ratio test. A general monitoring control chart as well as a directional version which also facilitates diagnosis is proposed. In this paper, although our method is based on the assumption that the latent continuous variables of the ordinal variables follow logistic distributions, the proposed charts are demonstrated to perform efficiently and robustly in various cases as shown in the simulations and case studies.

A difficulty in joint modeling of continuous and discrete response variables is the lack of a natural multivariate distribution. For joint modeling of clustered observations on binary and continuous responses, we study a correlated probit model that has an underlying normal latent variable for the binary responses. Catalano and Ryan have factored the model into a marginal and a conditional component and used generalized estimating equations methodology to estimate the effects. We propose a Monte Carlo expectation–conditional maximization algorithm for finding maximum likelihood estimates of the mixed model itself, extending and accelerating an algorithm for models with binary responses. We demonstrate the methodology with a developmental toxicity study measuring fetal weight and a binary malformation status for several litters of mice. A simulation study suggests that efficiency gains of joint fittings over separate fittings of the response variables occur mainly for small datasets with strong correlations between the responses within cluster.

It is common in applied research to have large numbers of variables measured on a modest number of cases. Even with low rates of missingness of individual variables, such data sets can have a large number of incomplete cases with a mix of data types. Here, we propose a new joint modeling approach to address the high-dimensional incomplete data with a mix of continuous and binary data. Specifically, we propose a multivariate normal model encompassing both continuous variables and latent variables corresponding to binary variables. We apply a parameter-extended Metropolis–Hastings algorithm to generate the covariance matrix of a mixture of continuous and binary variables. We also introduce prior distribution families for unstructured covariance matrices to reduce the dimension of the parameter space. In several simulation settings, the method is compared with available-case analysis, a rounding method, and a sequential regression method.

Flexible methods that relax restrictive conditional independence assumptions of latent classanalysis (LCA) are described. Dichotomous and ordered category manifest variables are viewed asdiscretized latent continuous variables. The latent continuous variables are assumed to have a mixtureofmultivariate-normals distribution. Within a latent class, conditional dependence is modeled as the mutual association of all or some latent continuous variables with a continuous latent trait (or in special cases, multiple latent traits). The relaxation of conditional independence assumptions allows LCA to better model natural taxa. Comparisons of specific restricted and unrestricted models permit statistical tests of specific aspects of latent taxonic structure. Latent class, latent trait, and latent distribution analysis can be viewed as special cases of the mixed latent trait model. The relationship between the multivariate probit mixture model proposed here and Rost’s mixed Rasch (1990, 1991) model is discussed. Two studies illustrate different uses of the proposed model.

Linear models in social science, psychlology, economics, and epidemiology are often formullated in latent continuous variables, which can be measured only with error or through two or more observed indicators. Henice, the latent variables are connected with the observed variables by a measurement model. If the observed indicators are metric, the covariance matrix of the observed variables is analyzed by using covariance structure models. Covariance structure models have been extended by MuthMn (1984) to deal with indicators that are metric and ordered categorical rather than metric alone. By abandoning the analysis of covariance structures and returning to some of the original ideas of latent structure analysis, we can extend the class of variables that may serve as indicators for latent variables utsed in a simultaneous equation system to include variables on any level of measurement. This class comprises observed variables that may be metric, censored metric, counts, ordered, or unordered categorical. A limited marginal maximum likelihood approach is used to estimate the model. Numerical procedures necessary to implement such an estimation are discussed.

The Role of Providers in Mental Health Services Offered to American-Indian Youths

Short personality questionnaires are increasingly used in research and practice, with some scales including as few as two to five items per personality domain. Despite the frequency of their use, these short scales are often criticized on the basis of their reduced internal consistencies and their purported failure to assess the breadth of broad constructs, such as the Big 5 factors of personality. One reason for this might be the use of principles routed in Classical Test Theory during test construction. In this study, Generalizability Theory is used to compare psychometric properties of different scales based on the NEO-PI-R and BFI, two widely-used personality questionnaire families. Applying both Classical Test Theory (CTT) and Generalizability Theory (GT) allowed to identify the inner workings of test shortening. CTT-based analyses indicated that longer is generally better for reliability, while GT allowed differentiation between reliability for relative and absolute decisions, while revealing how different variance sources affect test score reliability estimates. These variance sources differed with scale length, and only GT allowed clear description of these internal consequences, allowing more effective identification of advantages and disadvantages of shorter and longer scales. Most importantly, the findings highlight the potential error proneness of focusing solely on reliability and scale length in test construction. Practical as well as theoretical consequences are discussed.

The reliability of event-related brain potential (ERP) scores depends on study context and how those scores will be used, and reliability must be routinely evaluated. Many factors can influence ERP score reliability; generalizability (G) theory provides a multifaceted approach to estimating the internal consistency and temporal stability of scores that is well suited for ERPs. G theory's approach possesses a number of advantages over classical test theory that make it ideal for pinpointing sources of error in observed scores. The current primer outlines the G-theory approach to estimating internal consistency (coefficients of equivalence) and test-retest reliability (coefficients of stability). This approach is used to evaluate the reliability of ERP measurements. The primer outlines how to estimate reliability coefficients that consider the impact of the number of trials, events, occasions, and groups. The uses of two different G-theory reliability coefficients (i.e., generalizability and dependability) in ERP research are elaborated, and a dataset from the companion manuscript, which examines N2 amplitudes to Go/NoGo stimuli, is used as an example of the application of these coefficients to ERPs. The developed algorithms are implemented in the ERP Reliability Analysis (ERA) Toolbox, which is open-source software designed for estimating score reliability using G theory. The toolbox facilitates the application of G theory in an effort to simplify the study-by-study evaluation of ERP score reliability. The formulas provided in this primer should enable researchers to pinpoint the sources of measurement error in ERP scores from multiple recording sessions and subsequently plan studies that optimize score reliability.

Background: Teacher performance evaluation plays a vital role in improving educational quality. However, achieving accuracy, fairness, and effectiveness in such evaluations remains a challenge. Psychometric models offer promising tools to enhance the validity and reliability of teacher assessments. Objective: This study aims to explore the application of psychometric models in evaluating teacher performance and to investigate how these models can improve the overall assessment process. Method: The research involves an in-depth analysis of key psychometric models, including Classical Test Theory (CTT), Item Response Theory (IRT), Generalizability Theory (GT), Structural Equation Modeling (SEM), and Multilevel Modeling (MLM). The integration of these models into teacher evaluation systems is critically examined. Result: Findings indicate that each model offers unique advantages for improving assessment accuracy and reliability. However, limitations such as potential biases, data interpretation challenges, and ethical concerns must be carefully addressed. Contextual factors such as teacher characteristics, institutional policies, and opportunities for professional development significantly affect the outcomes of psychometric-based evaluations. Conclusion: Psychometric models can strengthen teacher evaluation systems when applied thoughtfully and ethically. Their effective use requires alignment with institutional goals and attention to contextual influences. Contribution: This study contributes to the field of educational assessment by synthesizing current knowledge on psychometric models and offering insights into their practical, ethical, and policy-related implications in the context of teacher performance evaluation.

BackgroundHealth-related Quality of life (HRQOL) is now concerned worldwide in cancer clinical fields, with the specific scales for patients with hepatocellular carcinoma in China being seldom and no formal validation of Chinese Version of the FACT-Hep. This study was aimed to validate the Chinese FACT-Hep based on both Classical Test Theory and Generalizability Theory to rationalize its use in the Chinese population.MethodsOne hundred and fourteen in-patients with hepatocellular carcinoma were measured three times before and after treatments using the FACT-Hep. The FACT-Hep was evaluated by Cronbach α, Pearson correlation, intra-class correlation (ICC), and paired-t tests with standardized response mean. The Generalizability Theory was also applied to addresses the dependability of measurements deeply by estimation of multiple sources of variance.ResultsThe Cronbach’s α coefficients were greater than 0.70 for all domains, and test-retest reliability coefficients were all greater than 0.80 exception of EWB 0.74 with the range of 0.81–0.96. The domains of PWB, FWB and the overall scale had significant changes after treatments with SRM ranging from 0.40 to 0.69. G-coefficients and Ф-coefficients confirmed the reliability of the scale further with exact variance components.ConclusionThe Chinese version of FACT-Hep can be used to measure HRQOL for patients with hepatocellular carcinoma in China with being of good psychometric properties.

The concepts of relative risk and hazard ratio are generalized for ordinary ordinal and continuous response variables, respectively. Under the generalized concepts, the Cox proportional hazards model with the Breslow's and Efron's methods can be regarded as generalizations of the Mantel–Haenszel estimator for dealing with broader types of covariates and responses. When ordinal responses can be regarded as discretized observations of a hypothetical continuous variable, the estimated relative risks from the Cox model reflect the associations between the responses and covariates. Examples are given to illustrate the generalized concepts and wider applications of the Cox model and the Kaplan–Meier estimator.

Purpose: Since oxidative stress involves a variety of cellular changes, no single biomarker can serve as a complete measure of this complex biological process. The analytic technique of structural equation modeling (SEM) provides a possible solution to this problem by modelling a latent (unobserved) variable constructed from the covariance of multiple biomarkers.Methods: Using three pooled datasets, we modelled a latent oxidative stress variable from five biomarkers related to oxidative stress: F2-isoprostanes (FIP), fluorescent oxidation products, mitochondrial DNA copy number, γ-tocopherol (Gtoc) and C-reactive protein (CRP, an inflammation marker closely linked to oxidative stress). We validated the latent variable by assessing its relation to pro- and anti-oxidant exposures.Results: FIP, Gtoc and CRP characterized the latent oxidative stress variable. Obesity, smoking, aspirin use and β-carotene were statistically significantly associated with oxidative stress in the theorized directions; the same exposures were weakly and inconsistently associated with the individual biomarkers.Conclusions: Our results suggest that using SEM with latent variables decreases the biomarker-specific variability, and may produce a better measure of oxidative stress than do single variables. This methodology can be applied to similar areas of research in which a single biomarker is not sufficient to fully describe a complex biological phenomenon.

Structural equation model (SEM) is a multivariate analysis tool that has been widely applied to many fields such as biomedical and social sciences. In the traditional SEM, it is often assumed that random errors and explanatory latent variables follow the normal distribution, and the effect of explanatory latent variables on outcomes can be formulated by a mean regression-type structural equation. But this SEM may be inappropriate in some cases where random errors or latent variables are highly nonnormal. The authors develop a new SEM, called as quantile SEM (QSEM), by allowing for a quantile regression-type structural equation and without distribution assumption of random errors and latent variables. A Bayesian empirical likelihood (BEL) method is developed to simultaneously estimate parameters and latent variables based on the estimating equation method. A hybrid algorithm combining the Gibbs sampler and Metropolis-Hastings algorithm is presented to sample observations required for statistical inference. Latent variables are imputed by the estimated density function and the linear interpolation method. A simulation study and an example are presented to investigate the performance of the proposed methodologies.