Parallel Computing with a Bayesian Item Response Model

Although item response theory (IRT) models such as the bifactor, two-tier, and between-item-dimensionality IRT models have been devised to confirm complex dimensional structures in educational and psychological data, they can be challenging to use in practice. The reason is that these models are multidimensional IRT (MIRT) models and thus are highly parameterized, making them only suitable for data provided by large samples. Unfortunately, many educational and psychological studies are conducted on a small scale, leaving the researchers without the necessary MIRT models to confirm the hypothesized structures in their data. To address the lack of modeling options for these researchers, we present a general Bayesian MIRT model based on adaptive informative priors. Simulations demonstrated that our MIRT model could be used to confirm a two-tier structure (with two general and six specific dimensions), a bifactor structure (with one general and six specific dimensions), and a between-item six-dimensional structure in rating scale data representing sample sizes as small as 100. Although our goal was to provide a general MIRT model suitable for smaller samples, the simulations further revealed that our model was applicable to larger samples. We also analyzed real data from 121 individuals to illustrate that the findings of our simulations are relevant to real situations.

An examination of the psychometric properties of the physical self-description questionnaire using a polytomous item response model

The estimation of high-dimensional latent regression item response theory (IRT) models is difficult because of the need to approximate integrals in the likelihood function. Proposed solutions in the literature include using stochastic approximations, adaptive quadrature, and Laplace approximations. We propose using a second-order Laplace approximation of the likelihood to estimate IRT latent regression models with categorical observed variables and fixed covariates where all parameters are estimated simultaneously. The method applies when the IRT model has a simple structure, meaning that each observed variable loads on only one latent variable. Through simulations using a latent regression model with binary and ordinal observed variables, we show that the proposed method is a substantial improvement over the first-order Laplace approximation with respect to the bias. In addition, the approach is equally or more precise to alternative methods for estimation of multidimensional IRT models when the number of items per dimension is moderately high. Simultaneously, the method is highly computationally efficient in the high-dimensional settings investigated. The results imply that estimation of simple-structure IRT models with very high dimensions is feasible in practice and that the direct estimation of high-dimensional latent regression IRT models is tractable even with large sample sizes and large numbers of items.

Item response theory (IRT) models provide an appropriate alternative to the classical ordinal confirmatory factor analysis (CFA) during the development of patient-reported outcome measures (PROMs). Current literature has identified the assessment of IRT model fit as both challenging and underdeveloped (Sinharay & Johnson, 2003; Sinharay, Johnson, & Stern, 2006). This study evaluates the performance of Ordinal Bayesian Instrument Development (OBID), a Bayesian IRT model with a probit link function approach, through applications in two breast cancer-related instrument development studies. The primary focus is to investigate an appropriate method for comparing Bayesian IRT models in PROMs development. An exact Bayesian leave-one-out cross-validation (LOO-CV) approach (Vehtari & Lampinen, 2002) is implemented to assess prior selection for the item discrimination parameter in the IRT model and subject content experts’ bias (in a statistical sense and not to be confused with psychometric bias as in differential item functioning) toward the estimation of item-to-domain correlations. Results support the utilization of content subject experts’ information in establishing evidence for construct validity when sample size is small. However, the incorporation of subject experts’ content information in the OBID approach can be sensitive to the level of expertise of the recruited experts. More stringent efforts need to be invested in the appropriate selection of subject experts to efficiently use the OBID approach and reduce potential bias during PROMs development.

Two common aspects of symptom measurement are 1) the occurrence or presence of symptoms, and 2) the intensity or severity of symptoms when they occur. We adopt a latent trait perspective based on item response theory (IRT), using both unidimensional and multidimensional IRT models. We demonstrate how to (a) prepare data for analysis, (b) specify, estimate, and compare models, (c) interpret model parameters, (d) compare scores from models, and (e) visualize analysis results. We develop the relevant sequential IRT model, noting its flexibility, congruence with the theorized data generating process for symptom measures, and its promise for facilitating additional research and practical applications. The sequential model is less frequently used than other IRT models for polytomous data such as the generalized partial credit or graded response models. However, estimation of the sequential model can be readily accomplished with standard latent variable modeling and IRT software for binary indicators that allows constraints on the discrimination parameters. We compare the sequential model to other modeling options. We provide discussion of future research directions.

Standard 3.9 of the Standards for Educational and Psychological Testing () demands evidence of model fit when item response theory (IRT) models are employed to data from tests. Hambleton and Han () and Sinharay () recommended the assessment of practical significance of misfit of IRT models, but few examples of such assessment can be found in the literature concerning IRT model fit. In this article, practical significance of misfit of IRT models was assessed using data from several tests that employ IRT models to report scores. The IRT model did not fit any data set considered in this article. However, the extent of practical significance of misfit varied over the data sets.

This study investigates the impact of item parameter drift (IPD) on parameter and ability estimation when the underlying measurement model fits a mixture distribution, thereby violating the item invariance property of unidimensional item response theory (IRT) models. An empirical study was conducted to demonstrate the occurrence of both IPD and an underlying mixture distribution using real-world data. Twenty-one trended anchor items from the 1999, 2003, and 2007 administrations of Trends in International Mathematics and Science Study (TIMSS) were analyzed using unidimensional and mixture IRT models. TIMSS treats trended anchor items as invariant over testing administrations and uses pre-calibrated item parameters based on unidimensional IRT. However, empirical results showed evidence of two latent subgroups with IPD. Results also showed changes in the distribution of examinee ability between latent classes over the three administrations. A simulation study was conducted to examine the impact of IPD on the estimation of ability and item parameters, when data have underlying mixture distributions. Simulations used data generated from a mixture IRT model and estimated using unidimensional IRT. Results showed that data reflecting IPD using mixture IRT model led to IPD in the unidimensional IRT model. Changes in the distribution of examinee ability also affected item parameters. Moreover, drift with respect to item discrimination and distribution of examinee ability affected estimates of examinee ability. These findings demonstrate the need to caution and evaluate IPD using a mixture IRT framework to understand its effects on item parameters and examinee ability.

The novelty of this study is to propose multidimensional item response theory (IRT) models for the investigation of heterogeneity in residents’ perceptions and attitudes toward the tourism industry. By following the social exchange theory, a sample of residents in the Romagna area (Italy) were invited to declare their agreement to some aspects related to benefits and costs of the tourism industry in 2010. The analysis suggests that the additive IRT model with two kind of latent variables (viz. the general latent trait measuring attitude toward tourism and two specific latent traits measuring perceptions on benefits and costs of tourism) fits the data adequately. Moreover, the estimated scores show that the tourism features are perceived differently by the residents, supporting the hypothesis that perceptions and attitudes change with regard to the individuals’ characteristics. Finally, the analysis proposes how the parameters of the IRT models can be interpreted in this new context.

BackgroundLongitudinal invariance is a perquisite for a valid comparison of oral health-related quality of life (OHRQoL) scores over time. Item response theory (IRT) models can assess measurement invariance and allow better estimation of the associations between predictors and latent construct. By extending IRT models, this study aimed to investigate the longitudinal invariance of the two 8-item short forms of the Child Perception Questionnaire (CPQ11–14) regression short form (RSF:8) and item-impact short form (ISF:8) and identify factors associated with adolescents’ OHRQoL and its change.MethodsAll students from S1 and S2 (equivalent to US grades 6 and 7) who were born in April 1997 and May 1997 (at age 12) from 45 randomly selected secondary schools were invited to participate in this study and followed up after 3 years. Data on the CPQ11–14 RSF:8 and CPQ11–14 ISF:8, demographics, oral health behavior and status were collected. Explanatory graded response models were fitted to both short forms of the CPQ11–14 data for assessing longitudinal invariance and factors associated with OHRQoL. The Bayesian estimation method – Monte Carlo Markov Chain (MCMC) with Gibbs sampling was adopted for parameter estimation and the credible intervals were used for inference.ResultsData from 649 children at age 12 at baseline and 415 children at age 15 at follow up were analyzed. For the 12 years old children, healthier oral health behavior, better gum status, families with both parents employed and parents’ education level were found to be associated with better OHRQoL. Four items among the 2 short forms lacked longitudinal invariance. With statistical adjustment of longitudinal invariance, OHRQoL were found improved in general over the 3 years but no predictor was associated with OHRQoL in follow-up. For those with decreased family income, their OHRQoL had worsened over 3 years.ConclusionsIRT explanatory analysis enables a more valid identification of the factors associated with OHRQoL and its changes over time. It provides important information to oral healthcare researchers and policymakers.

[Correction Notice: An Erratum for this article was reported in Vol 26(3) of Psychological Assessment (see record 2014-16017-001). The mean, standard deviation and alpha coefficient originally reported in Table 1 should be 74.317, 10.214 and .802, respectively. The validity coefficients in the last column of Table 4 are affected as well. Correcting this error did not change the substantive interpretations of the results, but did increase the mean, standard deviation, alpha coefficient, and validity coefficients reported for the Honesty subscale in the text and in Tables 1 and 4. The corrected versions of Tables 1 and Table 4 are shown in the erratum.] Item response theory (IRT) models were applied to dichotomous and polytomous scoring of the Self-Deceptive Enhancement and Impression Management subscales of the Balanced Inventory of Desirable Responding (Paulhus, 1991, 1999). Two dichotomous scoring methods reflecting exaggerated endorsement and exaggerated denial of socially desirable behaviors were examined. The 1- and 2-parameter logistic models (1PLM, 2PLM, respectively) were applied to dichotomous responses, and the partial credit model (PCM) and graded response model (GRM) were applied to polytomous responses. For both subscales, the 2PLM fit dichotomous responses better than did the 1PLM, and the GRM fit polytomous responses better than did the PCM. Polytomous GRM and raw scores for both subscales yielded higher test-retest and convergent validity coefficients than did PCM, 1PLM, 2PLM, and dichotomous raw scores. Information plots showed that the GRM provided consistently high measurement precision that was superior to that of all other IRT models over the full range of both construct continuums. Dichotomous scores reflecting exaggerated endorsement of socially desirable behaviors provided noticeably weak precision at low levels of the construct continuums, calling into question the use of such scores for detecting instances of "faking bad." Dichotomous models reflecting exaggerated denial of the same behaviors yielded much better precision at low levels of the constructs, but it was still less precision than that of the GRM. These results support polytomous over dichotomous scoring in general, alternative dichotomous scoring for detecting faking bad, and extension of GRM scoring to situations in which IRT offers additional practical advantages over classical test theory (adaptive testing, equating, linking, scaling, detecting differential item functioning, and so forth).

Quality of life studies must be concerned with the accuracy, or reliability (as it is usually called in psychometrics) of measurement. A reliability coefficient gives an evaluation of a questionnaire’s ability to yield interpretable statements about the construct being measured. There are two classes of models for analyzing unidimensional scales: classical models and Item Response Theory (IRT) models. Classical models are based on a linear decomposition of the score, and reliability is estimated using Cronbach’s alpha coefficient. IRT models focus on the relationship between the probability of a correct answer and a latent variable. The Rasch model is a special IRT model which having good measurement properties. In the context of the Rasch model, or IRT models in general, no reliability coefficient similar to Cronbach Alpha is clearly defined. Some authors have studied the Fisher information of the latent parameter, which provides a measure of accuracy of the estimator. In this chapter, we present how reliability is estimated for classical models, and we propose a reliability coefficient for the Rasch model. Simulated data and data derived from the communication subscale of the Sickness Impact Profile are used to illustrate the methods.

ABSTRACTMany standardized educational tests include groups of items based on a common stimulus, known as testlets. Standard unidimensional item response theory (IRT) models are commonly used to model examinees' responses to testlet items. However, it is known that local dependence among testlet items can lead to biased item parameter estimates when using standard IRT models, and to overestimated reliability. In this study, a general polytomous testlet model was proposed to account for local dependence in testlet‐based tests that contain both dichotomously and polytomously scored items. The proposed model and a standard IRT model were fit to simulated data and several real data sets from the reading sections of a large‐scale English‐language test, and model fit was evaluated. Item parameters and test information obtained from the two models were compared to check the impact of local item dependence. In addition, a multidimensional IRT model with simple structure was also fit to the real data sets. Results based on both simulated and real data suggested that local dependence had a small impact on item parameter estimates and a relatively larger impact on test information and reliability. It was also found that the multidimensional IRT model with simple structure fit the real data sets better than the general polytomous testlet model and the standard IRT model did.

The fuzzy TOPSIS (The Technique for Order of Preference by Similarity to Ideal Solution) is an attractive tool for measuring complex phenomena based on uncertain data. The original version of the method assumes that the object assessments in terms of the adopted criteria are expressed as triangular fuzzy numbers. One of the crucial stages of the fuzzy TOPSIS is selecting the fuzzy conversion scale, which is used to evaluate objects in terms of the adopted criteria. The choice of a fuzzy conversion scale may influence the results of the fuzzy TOPSIS. There is no uniform approach in constructing and selecting the fuzzy conversion scale for the fuzzy TOPSIS. The choice is subjective and made by researchers. Therefore, the aim of the article is to present a new, objective approach to the construction of fuzzy conversion scales based on Item Response Theory (IRT) models. The following models were used in the construction of fuzzy conversion scales: Polychoric Correlation Model (PM), Polytomous Rasch Model (PRM), Rating Scale Model (RSM), Partial Credit Model (PCM), Generalized Partial Credit Model (GPCM), Graded Response Model (GRM), Nominal Response Model (NRM). The usefulness of the proposed approach is presented on the example of the analysis of a survey’s results on measuring the quality of professional life of inhabitants of selected communes in Poland. The obtained results indicate that the choice of the fuzzy conversion scale has a large impact on the closeness coefficient values. A large difference was also observed in the spreads of triangular fuzzy numbers between scales based on IRT models and those used in the literature on the subject. The use of the fuzzy TOPSIS with fuzzy conversion scales built based on PRM, RSM, PCM, GPCM, and GRM models gives results with a greater range of variability than in the case of fuzzy conversion scales used in empirical research.

Model-data-fit of item response theory (IRT) models is generally assessed by comparing observed performance by examinees on individual items with performance that is predicted under the chosen IRT model. However, use of traditional chi-square methods to evaluate goodness-of-fit of IRT models is not appropriate when the underlying trait/ability is estimated imprecisely (e.g., shorter assessments). This article describes a goodness-of-fit statistic that considers directly the uncertainty with which ability is estimated as well as a resampling-based hypothesis testing procedure. A simulation study was conducted to evaluate the empirical power and Type I error rates for the proposed procedure. Results of the study indicated that the procedure should be useful for evaluating goodness-of-fit of IRT models for most testing applications where uncertainty in ability estimation is an issue.

In recent years, much effort has been invested in the development of new instruments for assessment of health-related quality of life (HRQOL). For many new instruments, modern psychometric methods, such as item response theory (IRT) models, have been used, either as supplemental to classical psychometric testing or as the primary methodological approach. We will use the term modern psychometric methods to refer to psychometric methods for multi-item scales that (1) examine the contribution of each item to the measurement properties of the overall scale and (2) recognize that items are categorical. The models include Rasch models (Rasch 1980; Fischer and Molenaar 1995), other IRT models (Samejima 1969; van der Linden and Hambleton 1997), and factor analytic models for categorical data (Muthen 1984). “Modern” psychometric methods have actually a rather long history within psychiatric research (both focusing on self-reported scales (Bech et al. 1978) and psychiatric outcome rating scales (Bech et al. 1984)). During the past 25 years, modern psychometric methods have increasingly been used in the analysis of patient-reported outcome measures (Teresi et al. 1989; Haley et al. 1994). For example, the NIH-sponsored Patient-Reported Outcomes Measurement Information System (PROMIS) project relies primarily on modern psychometric methods (Reeve et al. 2007). Similarly, modern psychometric analyses have started to be adopted for analysis of patient-reported HRQOL measures for patients with schizophrenia (D’haenen 1996; Pan et al. 2007; Boyer et al. 2010; Reise et al. 2011a; Laurens et al. 2012; Mojtabai et al. 2012; Chen et al. 2013; Michel et al. 2013; Park et al. 2015; Galindo-Garre et al. 2015; Norholm and Bech 2006). The present chapter provides an introduction to modern psychometric methods and discusses their potential use for analyses of HRQOL data from patients with schizophrenia. Rather than focusing on one particular approach, we will show what the methods have in common and how they can supplement each other.