The Data-augmentation Techniques in Item Response Modeling: Current Approaches and New Developments

Joint maximum likelihood (JML) estimation is one of the earliest approaches to fitting item response theory (IRT) models. This procedure treats both the item and person parameters as unknown but fixed model parameters and estimates them simultaneously by solving an optimization problem. However, the JML estimator is known to be asymptotically inconsistent for many IRT models, when the sample size goes to infinity and the number of items keeps fixed. Consequently, in the psychometrics literature, this estimator is less preferred to the marginal maximum likelihood (MML) estimator. In this paper, we re-investigate the JML estimator for high-dimensional exploratory item factor analysis, from both statistical and computational perspectives. In particular, we establish a notion of statistical consistency for a constrained JML estimator, under an asymptotic setting that both the numbers of items and people grow to infinity and that many responses may be missing. A parallel computing algorithm is proposed for this estimator that can scale to very large datasets. Via simulation studies, we show that when the dimensionality is high, the proposed estimator yields similar or even better results than those from the MML estimator, but can be obtained computationally much more efficiently. An illustrative real data example is provided based on the revised version of Eysenck's Personality Questionnaire (EPQ-R).

Marginal maximum likelihood (MML) estimation is the preferred approach to fitting item response theory models in psychometrics due to the MML estimator's consistency, normality, and efficiency as the sample size tends to infinity. However, state-of-the-art MML estimation procedures such as the Metropolis-Hastings Robbins-Monro (MH-RM) algorithm as well as approximate MML estimation procedures such as variational inference (VI) are computationally time-consuming when the sample size and the number of latent factors are very large. In this work, we investigate a deep learning-based VI algorithm for exploratory item factor analysis (IFA) that is computationally fast even in large data sets with many latent factors. The proposed approach applies a deep artificial neural network model called an importance-weighted autoencoder (IWAE) for exploratory IFA. The IWAE approximates the MML estimator using an importance sampling technique wherein increasing the number of importance-weighted (IW) samples drawn during fitting improves the approximation, typically at the cost of decreased computational efficiency. We provide a real data application that recovers results aligning with psychological theory across random starts. Via simulation studies, we show that the IWAE yields more accurate estimates as either the sample size or the number of IW samples increases (although factor correlation and intercepts estimates exhibit some bias) and obtains similar results to MH-RM in less time. Our simulations also suggest that the proposed approach performs similarly to and is potentially faster than constrained joint maximum likelihood estimation, a fast procedure that is consistent when the sample size and the number of items simultaneously tend to infinity.

A ranking pattern approach is proposed to build item response theory (IRT) models for forced-choice (FC) items. This new approach is an addition to the two existing approaches, sequential selection and Thurstone's law of pairwise comparison. A new dominance IRT model, the multidimensional generalized partial preference model (MGPPM), is proposed for FC items with any number (greater than 1) of statements. The maximum marginal likelihood estimation using an expectation-maximization algorithm (MML-EM) and Markov chain Monte Carlo (MCMC) estimation are developed. A simulation study is conducted to show satisfactory parameter recovery on triplet and tetrad data. The relationships between the newly proposed approach/model and the existing approaches/models are described, and the MGPPM, Thurstonian IRT (TIRT) model, and Triplet-2PLM are compared when applied to simulated and real triplet data. The new approach offers more flexible IRT modeling than the other two approaches under different assumptions, and the MGPPM is more statistically elegant than the TIRT and Triple-2PLM.

Diffusion-based item response theory models for responses and response times in tests have attracted increased attention recently in psychometrics. Analyzing response time data, however, is delicate as response times are often contaminated by unusual observations. This can have serious effects on the validity of statistical inference. In this article, we compare three established and two new estimation approaches for diffusion-based item response theory models with respect to their robustness. The three established approaches are the marginal maximum likelihood (ML) estimator for continuous time, the marginal ML estimator for discrete time, and the weighted least squares (WLS) estimator. The new approaches are two modifications of the WLS estimator with better robustness properties. The performance of the estimators is compared in a simulation study. The simulation study illustrates that the new approaches are robust against some forms of random independent contamination. The marginal ML estimator for discrete time also performs well. The marginal ML estimator for continuous time is heavily affected by contamination.

In this article we develop an estimation method based on the augmented data scheme and EM/SEM (Stochastic EM) algorithms for fitting one-parameter probit (Rasch) IRT (Item Response Theory) models. Instead of using the S steps of the SEM algorithm, that is, instead of simulating values for the unobserved variables (augmented data and the latent traits), we consider the conditional expectations of a set of unobserved variables on the other set of unobserved variables, the current estimates of the parameters and the observed data, based on the full conditional distributions from the Gibbs sampling algorithm. Our method, named the CADEM algorithm (conditional augmented data EM), presents straightforward E steps, which avoid the need to evaluate the usual integrals, also facilitating the M steps, without the need to use numerical methods of optimization. We use the CADEM algorithm to obtain both maximum likelihood estimates and maximum a posteriori estimates of the difficulty parameters for the one-parameter probit (Rasch) model. Also, we obtain estimates for the latent traits, based on conditional expectations. In addition, we show how to calculate the associated standard errors. Some directions are provided to extend our approach to other IRT models. In this respect, we perform a simulation study to compare the estimation methods. The results indicated that our approach is quite comparable to the usual marginal maximum likelihood (MML) and Gibbs sampling methods (GS) in terms of parameter recovery. However, CADEM is as fast as MML and as flexible as GS.

The calibration of the one-parameter logistic ability-based guessing (1PL-AG) model in item response theory (IRT) with a modest sample size remains a challenge for its implausible estimates and difficulty in obtaining standard errors of estimates. This article proposes an alternative Bayesian modal estimation (BME) method, the Bayesian Expectation-Maximization-Maximization (BEMM) method, which is developed by combining an augmented variable formulation of the 1PL-AG model and a mixture model conceptualization of the three-parameter logistic model (3PLM). By comparing with marginal maximum likelihood estimation (MMLE) and Markov Chain Monte Carlo (MCMC) in JAGS, the simulation shows that BEMM can produce stable and accurate estimates in the modest sample size. A real data example and the MATLAB codes of BEMM are also provided.

This study examined the application of the MML-EM algorithm to the parameter estimation problems of the normal ogive and logistic polytomous response models for Likert-type items. A rating-scale model was devel oped based on Samejima's (1969) graded response model. The graded response model includes a separate slope parameter for each item and an item response parameter. In the rating-scale model, the item re sponse parameter is resolved into two parameters: the item location parameter, and the category threshold parameter characterizing the boundary between re sponse categories. For a Likert-type questionnaire, where a single scale is employed to elicit different re sponses to the items, this item response model is ex pected to be more useful for analysis because the item parameters can be estimated separately from the threshold parameters associated with the points on a single Likert scale. The advantages of this type of model are shown by analyzing simulated data and data from the General Social Surveys. Index terms: EM algorithm, General Social Surveys, graded response model, item response model, Likert scale, marginal maximum likelihood, polytomous item response model, rating-scale model.

Item response theory (IRT) is the statistical paradigm underlying a dominant family of generative probabilistic models for test responses, used to quantify traits in individuals relative to target populations. The graded response model (GRM) is a particular IRT model that is used for ordered polytomous test responses. Both the development and the application of the GRM and other IRT models require statistical decisions. For formulating these models (calibration), one needs to decide on methodologies for item selection, inference, and regularization. For applying these models (test scoring), one needs to make similar decisions, often prioritizing computational tractability and/or interpretability. In many applications, such as in the Work Disability Functional Assessment Battery (WD-FAB), tractability implies approximating an individual’s score distribution using estimates of mean and variance, and obtaining that score conditional on only point estimates of the calibrated model. In this manuscript, we evaluate the calibration and scoring of models under this common use-case using Bayesian cross-validation. Applied to the WD-FAB responses collected for the National Institutes of Health, we assess the predictive power of implementations of the GRM based on their ability to yield, on validation sets of respondents, ability estimates that are most predictive of patterns of item responses. Our main finding indicates that regularized Bayesian calibration of the GRM outperforms the regularization-free empirical Bayesian procedure of marginal maximum likelihood. We also motivate the use of compactly supported priors in test scoring.

Most Facial Expression Recognition (FER) systems rely on machine learning approaches that require large databases for an effective training. As these are not easily available, a good solution is to augment the databases with appropriate data augmentation (DA) techniques, which are typically based on either geometric transformation or oversampling augmentations (e.g., generative adversarial networks (GANs)). However, it is not always easy to understand which DA technique may be more convenient for FER systems because most state-of-the-art experiments use different settings which makes the impact of DA techniques not comparable. To advance in this respect, in this paper, we evaluate and compare the impact of using well-established DA techniques on the emotion recognition accuracy of a FER system based on the well-known VGG16 convolutional neural network (CNN). In particular, we consider both geometric transformations and GAN to increase the amount of training images. We performed cross-database evaluations: training with the "augmented" KDEF database and testing with two different databases (CK+ and ExpW). The best results were obtained combining horizontal reflection, translation and GAN, bringing an accuracy increase of approximately 30%. This outperforms alternative approaches, except for the one technique that could however rely on a quite bigger database.

FRI0648 Directly comparing latent functional ability in adolescents with jia using the chaq and haq: an item response theory analysis

The estimation of high-dimensional latent regression item response theory (IRT) models is difficult because of the need to approximate integrals in the likelihood function. Proposed solutions in the literature include using stochastic approximations, adaptive quadrature, and Laplace approximations. We propose using a second-order Laplace approximation of the likelihood to estimate IRT latent regression models with categorical observed variables and fixed covariates where all parameters are estimated simultaneously. The method applies when the IRT model has a simple structure, meaning that each observed variable loads on only one latent variable. Through simulations using a latent regression model with binary and ordinal observed variables, we show that the proposed method is a substantial improvement over the first-order Laplace approximation with respect to the bias. In addition, the approach is equally or more precise to alternative methods for estimation of multidimensional IRT models when the number of items per dimension is moderately high. Simultaneously, the method is highly computationally efficient in the high-dimensional settings investigated. The results imply that estimation of simple-structure IRT models with very high dimensions is feasible in practice and that the direct estimation of high-dimensional latent regression IRT models is tractable even with large sample sizes and large numbers of items.

This paper surveys the techniques used in item re sponse theory to estimate the parameters of the item characteristic curves fitted to item response data. The major focus is on the joint maximum likelihood esti mation (JMLE) procedure, but alternative approaches are also examined. The literature shows that both the theoretical asymptotic properties and the empirical properties of the JMLE results are well-established. Al though alternative approaches are available, such as Bayesian estimation and marginal maximum likelihood estimation, they do not appear to have an overwhelm ing advantage over the JMLE procedure. However, the properties of these alternative techniques have not been thoroughly studied as yet. It is also clear that the properties of the item parameter estimation techniques are inextricably intertwined with the computer pro grams used to implement them.

Improving the prediction of extreme wind speed events with generative data augmentation techniques

BackgroundThere is a scarcity of studies that quantitatively assess the difficulty and importance of knowledge points (KPs) depending on students’ self-efficacy for learning (SEL). This study aims to validate the practical application of psychological measurement tools in physical therapy education by analyzing student SEL and course conceptual structure.MethodsFrom the “Therapeutic Exercise” course curriculum, we extracted 100 KPs and administered a difficulty rating questionnaire to 218 students post-final exam. The pipeline of the non-parametric Item Response Theory (IRT) and parametric IRT modeling was employed to estimate student SEL and describe the hierarchy of KPs in terms of item difficulty. Additionally, Gaussian Graphical Models with Non-Convex Penalties were deployed to create a Knowledge Graph (KG) and identify the main components. A visual analytics approach was then proposed to understand the correlation and difficulty level of KPs.ResultsWe identified 50 KPs to create the Mokken scale, which exhibited high reliability (Cronbach’s alpha = 0.9675) with no gender bias at the overall or at each item level (p > 0.05). The three-parameter logistic model (3PLM) demonstrated good fitness with questionnaire data, whose Root Mean Square Error Approximation was < 0.05. Also, item-model fitness unveiled good fitness, as indicated by each item with non-significant p-values for chi-square tests. The Wright map revealed item difficulty relative to SEL levels. SEL estimated by the 3PLM correlated significantly with the high-ability range of average Grade-Point Average (p < 0.05). The KG backbone structure consisted of 58 KPs, with 29 KPs overlapping with the Mokken scale. Visual analysis of the KG backbone structure revealed that the difficulty level of KPs in the IRT could not replace their position parameters in the KG.ConclusionThe IRT and KG methods utilized in this study offer distinct perspectives for visualizing hierarchical relationships and correlations among the KPs. Based on real-world teaching empirical data, this study helps to provide a research foundation for updating course contents and customizing learning objectives.Trial registrationNot applicable.

Two common aspects of symptom measurement are 1) the occurrence or presence of symptoms, and 2) the intensity or severity of symptoms when they occur. We adopt a latent trait perspective based on item response theory (IRT), using both unidimensional and multidimensional IRT models. We demonstrate how to (a) prepare data for analysis, (b) specify, estimate, and compare models, (c) interpret model parameters, (d) compare scores from models, and (e) visualize analysis results. We develop the relevant sequential IRT model, noting its flexibility, congruence with the theorized data generating process for symptom measures, and its promise for facilitating additional research and practical applications. The sequential model is less frequently used than other IRT models for polytomous data such as the generalized partial credit or graded response models. However, estimation of the sequential model can be readily accomplished with standard latent variable modeling and IRT software for binary indicators that allows constraints on the discrimination parameters. We compare the sequential model to other modeling options. We provide discussion of future research directions.