Methodology Review: Item Parameter Estimation Under the One-, Two-, and Three-Parameter Logistic Models

Joint maximum likelihood (JML) estimation is one of the earliest approaches to fitting item response theory (IRT) models. This procedure treats both the item and person parameters as unknown but fixed model parameters and estimates them simultaneously by solving an optimization problem. However, the JML estimator is known to be asymptotically inconsistent for many IRT models, when the sample size goes to infinity and the number of items keeps fixed. Consequently, in the psychometrics literature, this estimator is less preferred to the marginal maximum likelihood (MML) estimator. In this paper, we re-investigate the JML estimator for high-dimensional exploratory item factor analysis, from both statistical and computational perspectives. In particular, we establish a notion of statistical consistency for a constrained JML estimator, under an asymptotic setting that both the numbers of items and people grow to infinity and that many responses may be missing. A parallel computing algorithm is proposed for this estimator that can scale to very large datasets. Via simulation studies, we show that when the dimensionality is high, the proposed estimator yields similar or even better results than those from the MML estimator, but can be obtained computationally much more efficiently. An illustrative real data example is provided based on the revised version of Eysenck's Personality Questionnaire (EPQ-R).

The Rasch model is one of the most prominent item response models. In this article, different item parameter estimation methods for the Rasch model are systematically compared through a comprehensive simulation study: Different alternatives of joint maximum likelihood (JML) estimation, different alternatives of marginal maximum likelihood (MML) estimation, conditional maximum likelihood (CML) estimation, and several limited information methods (LIM). The type of ability distribution (i.e., nonnormality), the number of items, sample size, and the distribution of item difficulties were systematically varied. Across different simulation conditions, MML methods with flexible distributional specifications can be at least as efficient as CML. Moreover, in many situations (i.e., for long tests), penalized JML and JML with ε adjustment resulted in very efficient estimates and might be considered alternatives to JML implementations currently used in statistical software. Moreover, minimum chi-square (MINCHI) estimation was the best-performing LIM method. These findings demonstrate that JML estimation and LIM can still prove helpful in applied research.

Diffusion-based item response theory models for responses and response times in tests have attracted increased attention recently in psychometrics. Analyzing response time data, however, is delicate as response times are often contaminated by unusual observations. This can have serious effects on the validity of statistical inference. In this article, we compare three established and two new estimation approaches for diffusion-based item response theory models with respect to their robustness. The three established approaches are the marginal maximum likelihood (ML) estimator for continuous time, the marginal ML estimator for discrete time, and the weighted least squares (WLS) estimator. The new approaches are two modifications of the WLS estimator with better robustness properties. The performance of the estimators is compared in a simulation study. The simulation study illustrates that the new approaches are robust against some forms of random independent contamination. The marginal ML estimator for discrete time also performs well. The marginal ML estimator for continuous time is heavily affected by contamination.

Marginal maximum likelihood (MML) estimation is the preferred approach to fitting item response theory models in psychometrics due to the MML estimator's consistency, normality, and efficiency as the sample size tends to infinity. However, state-of-the-art MML estimation procedures such as the Metropolis-Hastings Robbins-Monro (MH-RM) algorithm as well as approximate MML estimation procedures such as variational inference (VI) are computationally time-consuming when the sample size and the number of latent factors are very large. In this work, we investigate a deep learning-based VI algorithm for exploratory item factor analysis (IFA) that is computationally fast even in large data sets with many latent factors. The proposed approach applies a deep artificial neural network model called an importance-weighted autoencoder (IWAE) for exploratory IFA. The IWAE approximates the MML estimator using an importance sampling technique wherein increasing the number of importance-weighted (IW) samples drawn during fitting improves the approximation, typically at the cost of decreased computational efficiency. We provide a real data application that recovers results aligning with psychological theory across random starts. Via simulation studies, we show that the IWAE yields more accurate estimates as either the sample size or the number of IW samples increases (although factor correlation and intercepts estimates exhibit some bias) and obtains similar results to MH-RM in less time. Our simulations also suggest that the proposed approach performs similarly to and is potentially faster than constrained joint maximum likelihood estimation, a fast procedure that is consistent when the sample size and the number of items simultaneously tend to infinity.

在心理与教育测量中，项目反应理论（Item Response Theory，IRT）模型的参数估计方法是理论研究与实践应用的基本工具。最近，由于IRT模型的不断扩展与EM（expectation-maximization）算法自身的固有问题，参数估计方法的改进与发展显得尤为重要。这里介绍了IRT模型中边际极大似然估计的发展，提出了它的阶段性特征，即联合极大似然估计阶段、确定性潜在心理特质“填补”阶段、随机潜在心理特质“填补”阶段，重点阐述了它的潜在心理特质“填补”（dataaugmentation）思想。EM算法与Metropolis-Hastings Robbins—Monro（MH-RM）算法作为不同的潜在心理特质“填补”方法，都是边际极大似然估计的思想跨越。目前，潜在心理特质“填补”的参数估计方法仍在不断发展与完善。

The calibration of the one-parameter logistic ability-based guessing (1PL-AG) model in item response theory (IRT) with a modest sample size remains a challenge for its implausible estimates and difficulty in obtaining standard errors of estimates. This article proposes an alternative Bayesian modal estimation (BME) method, the Bayesian Expectation-Maximization-Maximization (BEMM) method, which is developed by combining an augmented variable formulation of the 1PL-AG model and a mixture model conceptualization of the three-parameter logistic model (3PLM). By comparing with marginal maximum likelihood estimation (MMLE) and Markov Chain Monte Carlo (MCMC) in JAGS, the simulation shows that BEMM can produce stable and accurate estimates in the modest sample size. A real data example and the MATLAB codes of BEMM are also provided.

A ranking pattern approach is proposed to build item response theory (IRT) models for forced-choice (FC) items. This new approach is an addition to the two existing approaches, sequential selection and Thurstone's law of pairwise comparison. A new dominance IRT model, the multidimensional generalized partial preference model (MGPPM), is proposed for FC items with any number (greater than 1) of statements. The maximum marginal likelihood estimation using an expectation-maximization algorithm (MML-EM) and Markov chain Monte Carlo (MCMC) estimation are developed. A simulation study is conducted to show satisfactory parameter recovery on triplet and tetrad data. The relationships between the newly proposed approach/model and the existing approaches/models are described, and the MGPPM, Thurstonian IRT (TIRT) model, and Triplet-2PLM are compared when applied to simulated and real triplet data. The new approach offers more flexible IRT modeling than the other two approaches under different assumptions, and the MGPPM is more statistically elegant than the TIRT and Triple-2PLM.

This article studies the estimation of a stepwise signal. To determine the number and locations of change-points of the stepwise signal, we formulate a maximum marginal likelihood estimator, which can be computed with a quadratic cost using dynamic programming. We carry out an extensive investigation on the choice of the prior distribution and study the asymptotic properties of the maximum marginal likelihood estimator. We propose to treat each possible set of change-points equally and adopt an empirical Bayes approach to specify the prior distribution of segment parameters. A detailed simulation study is performed to compare the effectiveness of this method with other existing methods. We demonstrate our method on single-molecule enzyme reaction data and on DNA array comparative genomic hybridization (CGH) data. Our study shows that this method is applicable to a wide range of models and offers appealing results in practice. Supplementary materials for this article are available online.

MIXNO provides maximum marginal likelihood estimates for mixed-effects nominal logistic regression analysis. These models can be used for analysis of correlated nominal response data, for example, data arising from a clustered or longitudinal design. For such data, the mixed-effects model assumes that data within clusters or sub jects are dependent. The degree of dependency is jointly estimated with the usual model parameters, thus adjusting for dependence resulting from nesting of the data. MIXNO uses marginal maximum likelihood estimation, utilizing a Fisher-scoring solution. For the scoring solution, the Cholesky factor of the random-effects variance-covariance matrix is estimated along with the (fixed) effects of explanatory variables. Examples illustrating usage and features of MIXNO are provided.

The present paper deals with the comparison of the performance of different estimation methods for regression models with correlated binary responses. Throughout, we consider probit models where an underlying latent continous random variable crosses a threshold. The error variables in the unobservable latent model are assumed to be normally distributed. The estimation procedures considered are (1) marginal maximum likelihood estimation using Gauss-Hermite quadrature, (2) generalized estimation equations (GEE) techniques with an extension to estimate tetrachoric correlations in a second step, and, (3) the MECOSA approach proposed by Schepers, Arminger and Kusters (1991) using hierarchical mean and covariance structure models. We present the results of a simulation study designed to evaluate the small sample properties of the different estimators and to make some comparisons with respect to technical aspects of the estimation procedures and to bias and mean squared error of the estimators. The results show that the calculation of the ML estimator requires the most computing time, followed by the MECOSA estimator. For small and moderate sample sizes the calculation of the MECOSA estimator is problematic because of problems of convergence as well as a tendency of underestimating the variances. In large samples with moderate or high correlations of the errors in the latent model, the MECOSA estimators are not as efficient as ML or GEE estimators. The higher the `true' value of an equicorrelation structure in the latent model and the larger the sample sizes are, the more is the efficiency gain of the ML estimator compared to the GEE and MECOSA estimators. Using the GEE approach, the ML estimates of tetrachoric correlations calculated in a second step are biased to a smaller extent than using the MECOSA approach.

Two psychometric models with very different parametric formulas and item response functions can make virtually the same predictions in all applications. By applying some basic results from the theory of hypothesis testing and from signal detection theory, the power of the most powerful test for distinguishing the models can be com puted. Measuring model misspecification by com puting the power of the most powerful test is proposed. If the power of the most powerful test is low, then the two models will make nearly the same prediction in every application. If the power is high, there will be applications in which the models will make different predictions. This measure, that is, the power of the most powerful test, places various types of model misspecifica tion— item parameter estimation error, multidi mensionality, local independence failure, learning and/or fatigue during testing—on a common scale. The theory supporting the method is presented and illustrated with a systematic study of misspecifica tion due to item response function estimation error. In these studies, two joint maximum likelihood estimation methods (LOGIST 2B and LOGIST 5) and two marginal maximum likelihood estimation methods (BILOG and ForScore) were contrasted by measuring the difference between a simulation model and a model obtained by applying an estimation method to simulation data. Marginal estimation was found generally to be superior to joint estimation. The parametric marginal method (BILOG) was superior to the nonparametric method only for three- parameter logistic models. The nonparametric mar ginal method (ForScore) excelled for more general models. Of the two joint maximum likelihood methods studied, LOGIST s appeared to be more accurate than LOGIST 2B.

A Bayesian wavelet approach is presented for estimating a partially linear model (PLM). A PLM consists of a linear part and a nonparametric component. The nonparametric component is represented with a wavelet series where the wavelet coefficients have assumed prior distributions. The prior for each coefficient consists of a mixture of a normal distribution and a point mass at 0. The linear parameters are assumed to have a normal prior. The hyperparameters are estimated by the marginal maximum likelihood estimator using the direct maximization. The model selection and model averaging methods give different estimates of the model parameters. MCMC computation is used for the estimation of the linear coefficients by model averaging method. Simulated examples illustrate the performance of the proposed estimators.

Type I error rates of the likelihood ratio test for the detection of differential item functioning (DIF) were investigated using monte carlo simulations. The graded response model with five ordered categories was used to generate datasets of a 30-item test for samples of 300 and 1,000 simulated examinees. All DIF comparisons were simulated by randomly pairing two groups of examinees. Three different sample size combinations of reference and focal groups were simulated under two ability matching conditions. For each of the six combinations of sample sizes by ability matching conditions, 100 replications of DIF detection comparisons were simulated. Item parameter estimates and likelihood values were obtained by marginal maximum likelihood estimation using the computer program MULTILOG. Irpe I eryor rates of the likelihood ratio test statistics for all six combinations of sample sizes and ability matching conditions were within theoretically expected values at each of the nominal alpha levels considered.

Cognitive diagnostic models (CDMs) are increasingly being used in various assessment contexts to identify cognitive processes and provide tailored feedback. However, the most commonly used estimation method for CDMs, marginal maximum likelihood estimation with Expectation–Maximization (MMLE-EM), can present difficulties when sample sizes are small. This study compares the results of different estimation methods for CDMs under varying sample sizes using simulated and empirical data. The methods compared include MMLE-EM, Bayes modal, Markov chain Monte Carlo, a non-parametric method, and a parsimonious parametric model such as Restricted DINA. We varied the sample size, and assessed the bias in the estimation of item parameters, the precision in attribute classification, the bias in the reliability estimate, and computational cost. The findings suggest that alternative estimation methods are preferred over MMLE-EM under low sample-size conditions, whereas comparable results are obtained under large sample-size conditions. Practitioners should consider using alternative estimation methods when working with small samples to obtain more accurate estimates of CDM parameters. This study aims to maximize the potential of CDMs by providing guidance on the estimation of the parameters.

Most item response theory models assume a uni-dimensional latent space. This study extended previous work on the effects of dimensionality on parameter estimation for dichotomous models to the polytomous graded response model. A multidimensional graded response model was developed to generate data in one, two, and three dimensions. The two- and three-dimension conditions contained datasets that varied in their interdimensional association. The effects of test length and the ratio of sample size to the number of item parameters estimated also were investigated. For the unidimensional data, a sample size ratio of 5:1 provided reasonably accurate estimation; increasing test length from 15 to 30 items did not have a significant impact on the accuracy of item parameter estimation. Regardless of the dimensionality of the data, the difficulty parameters were well estimated. For the multidimensional data, the correlations between the estimated item discrimination and the average (as well as the sum of the) dimensional discrimination were greater than the correlations between the estimated item discrimination and the individual dimensional discriminations. Fidelity coefficients between the mean trait level and the trait level estimate (θ) were greater than those between θ and the latent traits. The impact of equating on accuracy indexes in a multidimensional context is discussed. Index terms: dimensionality, graded response model, IRT, marginal maximum likelihood estimation, polytomous models.