Joint Maximum Likelihood Estimation for High-Dimensional Exploratory Item Factor Analysis.

The Rasch model is one of the most prominent item response models. In this article, different item parameter estimation methods for the Rasch model are systematically compared through a comprehensive simulation study: Different alternatives of joint maximum likelihood (JML) estimation, different alternatives of marginal maximum likelihood (MML) estimation, conditional maximum likelihood (CML) estimation, and several limited information methods (LIM). The type of ability distribution (i.e., nonnormality), the number of items, sample size, and the distribution of item difficulties were systematically varied. Across different simulation conditions, MML methods with flexible distributional specifications can be at least as efficient as CML. Moreover, in many situations (i.e., for long tests), penalized JML and JML with ε adjustment resulted in very efficient estimates and might be considered alternatives to JML implementations currently used in statistical software. Moreover, minimum chi-square (MINCHI) estimation was the best-performing LIM method. These findings demonstrate that JML estimation and LIM can still prove helpful in applied research.

Diffusion-based item response theory models for responses and response times in tests have attracted increased attention recently in psychometrics. Analyzing response time data, however, is delicate as response times are often contaminated by unusual observations. This can have serious effects on the validity of statistical inference. In this article, we compare three established and two new estimation approaches for diffusion-based item response theory models with respect to their robustness. The three established approaches are the marginal maximum likelihood (ML) estimator for continuous time, the marginal ML estimator for discrete time, and the weighted least squares (WLS) estimator. The new approaches are two modifications of the WLS estimator with better robustness properties. The performance of the estimators is compared in a simulation study. The simulation study illustrates that the new approaches are robust against some forms of random independent contamination. The marginal ML estimator for discrete time also performs well. The marginal ML estimator for continuous time is heavily affected by contamination.

Marginal maximum likelihood (MML) estimation is the preferred approach to fitting item response theory models in psychometrics due to the MML estimator's consistency, normality, and efficiency as the sample size tends to infinity. However, state-of-the-art MML estimation procedures such as the Metropolis-Hastings Robbins-Monro (MH-RM) algorithm as well as approximate MML estimation procedures such as variational inference (VI) are computationally time-consuming when the sample size and the number of latent factors are very large. In this work, we investigate a deep learning-based VI algorithm for exploratory item factor analysis (IFA) that is computationally fast even in large data sets with many latent factors. The proposed approach applies a deep artificial neural network model called an importance-weighted autoencoder (IWAE) for exploratory IFA. The IWAE approximates the MML estimator using an importance sampling technique wherein increasing the number of importance-weighted (IW) samples drawn during fitting improves the approximation, typically at the cost of decreased computational efficiency. We provide a real data application that recovers results aligning with psychological theory across random starts. Via simulation studies, we show that the IWAE yields more accurate estimates as either the sample size or the number of IW samples increases (although factor correlation and intercepts estimates exhibit some bias) and obtains similar results to MH-RM in less time. Our simulations also suggest that the proposed approach performs similarly to and is potentially faster than constrained joint maximum likelihood estimation, a fast procedure that is consistent when the sample size and the number of items simultaneously tend to infinity.

A ranking pattern approach is proposed to build item response theory (IRT) models for forced-choice (FC) items. This new approach is an addition to the two existing approaches, sequential selection and Thurstone's law of pairwise comparison. A new dominance IRT model, the multidimensional generalized partial preference model (MGPPM), is proposed for FC items with any number (greater than 1) of statements. The maximum marginal likelihood estimation using an expectation-maximization algorithm (MML-EM) and Markov chain Monte Carlo (MCMC) estimation are developed. A simulation study is conducted to show satisfactory parameter recovery on triplet and tetrad data. The relationships between the newly proposed approach/model and the existing approaches/models are described, and the MGPPM, Thurstonian IRT (TIRT) model, and Triplet-2PLM are compared when applied to simulated and real triplet data. The new approach offers more flexible IRT modeling than the other two approaches under different assumptions, and the MGPPM is more statistically elegant than the TIRT and Triple-2PLM.

在心理与教育测量中，项目反应理论（Item Response Theory，IRT）模型的参数估计方法是理论研究与实践应用的基本工具。最近，由于IRT模型的不断扩展与EM（expectation-maximization）算法自身的固有问题，参数估计方法的改进与发展显得尤为重要。这里介绍了IRT模型中边际极大似然估计的发展，提出了它的阶段性特征，即联合极大似然估计阶段、确定性潜在心理特质“填补”阶段、随机潜在心理特质“填补”阶段，重点阐述了它的潜在心理特质“填补”（dataaugmentation）思想。EM算法与Metropolis-Hastings Robbins—Monro（MH-RM）算法作为不同的潜在心理特质“填补”方法，都是边际极大似然估计的思想跨越。目前，潜在心理特质“填补”的参数估计方法仍在不断发展与完善。

The estimation of high-dimensional latent regression item response theory (IRT) models is difficult because of the need to approximate integrals in the likelihood function. Proposed solutions in the literature include using stochastic approximations, adaptive quadrature, and Laplace approximations. We propose using a second-order Laplace approximation of the likelihood to estimate IRT latent regression models with categorical observed variables and fixed covariates where all parameters are estimated simultaneously. The method applies when the IRT model has a simple structure, meaning that each observed variable loads on only one latent variable. Through simulations using a latent regression model with binary and ordinal observed variables, we show that the proposed method is a substantial improvement over the first-order Laplace approximation with respect to the bias. In addition, the approach is equally or more precise to alternative methods for estimation of multidimensional IRT models when the number of items per dimension is moderately high. Simultaneously, the method is highly computationally efficient in the high-dimensional settings investigated. The results imply that estimation of simple-structure IRT models with very high dimensions is feasible in practice and that the direct estimation of high-dimensional latent regression IRT models is tractable even with large sample sizes and large numbers of items.

The calibration of the one-parameter logistic ability-based guessing (1PL-AG) model in item response theory (IRT) with a modest sample size remains a challenge for its implausible estimates and difficulty in obtaining standard errors of estimates. This article proposes an alternative Bayesian modal estimation (BME) method, the Bayesian Expectation-Maximization-Maximization (BEMM) method, which is developed by combining an augmented variable formulation of the 1PL-AG model and a mixture model conceptualization of the three-parameter logistic model (3PLM). By comparing with marginal maximum likelihood estimation (MMLE) and Markov Chain Monte Carlo (MCMC) in JAGS, the simulation shows that BEMM can produce stable and accurate estimates in the modest sample size. A real data example and the MATLAB codes of BEMM are also provided.

This paper surveys the techniques used in item re sponse theory to estimate the parameters of the item characteristic curves fitted to item response data. The major focus is on the joint maximum likelihood esti mation (JMLE) procedure, but alternative approaches are also examined. The literature shows that both the theoretical asymptotic properties and the empirical properties of the JMLE results are well-established. Al though alternative approaches are available, such as Bayesian estimation and marginal maximum likelihood estimation, they do not appear to have an overwhelm ing advantage over the JMLE procedure. However, the properties of these alternative techniques have not been thoroughly studied as yet. It is also clear that the properties of the item parameter estimation techniques are inextricably intertwined with the computer pro grams used to implement them.

In this article we develop an estimation method based on the augmented data scheme and EM/SEM (Stochastic EM) algorithms for fitting one-parameter probit (Rasch) IRT (Item Response Theory) models. Instead of using the S steps of the SEM algorithm, that is, instead of simulating values for the unobserved variables (augmented data and the latent traits), we consider the conditional expectations of a set of unobserved variables on the other set of unobserved variables, the current estimates of the parameters and the observed data, based on the full conditional distributions from the Gibbs sampling algorithm. Our method, named the CADEM algorithm (conditional augmented data EM), presents straightforward E steps, which avoid the need to evaluate the usual integrals, also facilitating the M steps, without the need to use numerical methods of optimization. We use the CADEM algorithm to obtain both maximum likelihood estimates and maximum a posteriori estimates of the difficulty parameters for the one-parameter probit (Rasch) model. Also, we obtain estimates for the latent traits, based on conditional expectations. In addition, we show how to calculate the associated standard errors. Some directions are provided to extend our approach to other IRT models. In this respect, we perform a simulation study to compare the estimation methods. The results indicated that our approach is quite comparable to the usual marginal maximum likelihood (MML) and Gibbs sampling methods (GS) in terms of parameter recovery. However, CADEM is as fast as MML and as flexible as GS.

Standard 3.9 of the Standards for Educational and Psychological Testing () demands evidence of model fit when item response theory (IRT) models are employed to data from tests. Hambleton and Han () and Sinharay () recommended the assessment of practical significance of misfit of IRT models, but few examples of such assessment can be found in the literature concerning IRT model fit. In this article, practical significance of misfit of IRT models was assessed using data from several tests that employ IRT models to report scores. The IRT model did not fit any data set considered in this article. However, the extent of practical significance of misfit varied over the data sets.

Two common aspects of symptom measurement are 1) the occurrence or presence of symptoms, and 2) the intensity or severity of symptoms when they occur. We adopt a latent trait perspective based on item response theory (IRT), using both unidimensional and multidimensional IRT models. We demonstrate how to (a) prepare data for analysis, (b) specify, estimate, and compare models, (c) interpret model parameters, (d) compare scores from models, and (e) visualize analysis results. We develop the relevant sequential IRT model, noting its flexibility, congruence with the theorized data generating process for symptom measures, and its promise for facilitating additional research and practical applications. The sequential model is less frequently used than other IRT models for polytomous data such as the generalized partial credit or graded response models. However, estimation of the sequential model can be readily accomplished with standard latent variable modeling and IRT software for binary indicators that allows constraints on the discrimination parameters. We compare the sequential model to other modeling options. We provide discussion of future research directions.

Although item response theory (IRT) models such as the bifactor, two-tier, and between-item-dimensionality IRT models have been devised to confirm complex dimensional structures in educational and psychological data, they can be challenging to use in practice. The reason is that these models are multidimensional IRT (MIRT) models and thus are highly parameterized, making them only suitable for data provided by large samples. Unfortunately, many educational and psychological studies are conducted on a small scale, leaving the researchers without the necessary MIRT models to confirm the hypothesized structures in their data. To address the lack of modeling options for these researchers, we present a general Bayesian MIRT model based on adaptive informative priors. Simulations demonstrated that our MIRT model could be used to confirm a two-tier structure (with two general and six specific dimensions), a bifactor structure (with one general and six specific dimensions), and a between-item six-dimensional structure in rating scale data representing sample sizes as small as 100. Although our goal was to provide a general MIRT model suitable for smaller samples, the simulations further revealed that our model was applicable to larger samples. We also analyzed real data from 121 individuals to illustrate that the findings of our simulations are relevant to real situations.

Quality of life studies must be concerned with the accuracy, or reliability (as it is usually called in psychometrics) of measurement. A reliability coefficient gives an evaluation of a questionnaire’s ability to yield interpretable statements about the construct being measured. There are two classes of models for analyzing unidimensional scales: classical models and Item Response Theory (IRT) models. Classical models are based on a linear decomposition of the score, and reliability is estimated using Cronbach’s alpha coefficient. IRT models focus on the relationship between the probability of a correct answer and a latent variable. The Rasch model is a special IRT model which having good measurement properties. In the context of the Rasch model, or IRT models in general, no reliability coefficient similar to Cronbach Alpha is clearly defined. Some authors have studied the Fisher information of the latent parameter, which provides a measure of accuracy of the estimator. In this chapter, we present how reliability is estimated for classical models, and we propose a reliability coefficient for the Rasch model. Simulated data and data derived from the communication subscale of the Sickness Impact Profile are used to illustrate the methods.

Item response theory (IRT) models provide an appropriate alternative to the classical ordinal confirmatory factor analysis (CFA) during the development of patient-reported outcome measures (PROMs). Current literature has identified the assessment of IRT model fit as both challenging and underdeveloped (Sinharay & Johnson, 2003; Sinharay, Johnson, & Stern, 2006). This study evaluates the performance of Ordinal Bayesian Instrument Development (OBID), a Bayesian IRT model with a probit link function approach, through applications in two breast cancer-related instrument development studies. The primary focus is to investigate an appropriate method for comparing Bayesian IRT models in PROMs development. An exact Bayesian leave-one-out cross-validation (LOO-CV) approach (Vehtari & Lampinen, 2002) is implemented to assess prior selection for the item discrimination parameter in the IRT model and subject content experts’ bias (in a statistical sense and not to be confused with psychometric bias as in differential item functioning) toward the estimation of item-to-domain correlations. Results support the utilization of content subject experts’ information in establishing evidence for construct validity when sample size is small. However, the incorporation of subject experts’ content information in the OBID approach can be sensitive to the level of expertise of the recruited experts. More stringent efforts need to be invested in the appropriate selection of subject experts to efficiently use the OBID approach and reduce potential bias during PROMs development.

ABSTRACTMany standardized educational tests include groups of items based on a common stimulus, known as testlets. Standard unidimensional item response theory (IRT) models are commonly used to model examinees' responses to testlet items. However, it is known that local dependence among testlet items can lead to biased item parameter estimates when using standard IRT models, and to overestimated reliability. In this study, a general polytomous testlet model was proposed to account for local dependence in testlet‐based tests that contain both dichotomously and polytomously scored items. The proposed model and a standard IRT model were fit to simulated data and several real data sets from the reading sections of a large‐scale English‐language test, and model fit was evaluated. Item parameters and test information obtained from the two models were compared to check the impact of local item dependence. In addition, a multidimensional IRT model with simple structure was also fit to the real data sets. Results based on both simulated and real data suggested that local dependence had a small impact on item parameter estimates and a relatively larger impact on test information and reliability. It was also found that the multidimensional IRT model with simple structure fit the real data sets better than the general polytomous testlet model and the standard IRT model did.