Validating the Existential Quest Scale using item response theory

Classical Test Theory (CTT), also known as the true score theory, refers to the analysis of test results based on test scores. The statistics produced under CTT include measures of item difficulty, item discrimination, measurement error and test reliability. The term “Classical” is used in contrast to “Modern” test theory which usually refers to item response theory (IRT). The fact that CTT was developed before IRT does not mean that CTT is outdated or replaced by IRT. Both CTT and IRT provide useful statistics to help us analyse test data. Generally, CTT and IRT provide complementary results. For many item analyses, CTT may be sufficient to provide the information we need. There are, however, theoretical differences between CTT and IRT, and many researchers prefer IRT because of enhanced measurement properties under IRT. IRT also provides a framework that facilitates test equating, computer adaptive testing and test score interpretation. While this book devotes a large part to IRT, we stress that CTT is an important part of the methodologies for educational and psychological measurement. In particular, the exposition of the concept of reliability in CTT sets the basis for evaluating measuring instruments. A good understanding of CTT lays the foundations for measurement principles. There are other approaches to measurement such as generalizability theory and structural equation modelling, but these are not the focus of attention in this book.

Emerging adulthood is characterized by transitions toward independence, where separation anxiety may intensify. Despite its robust psychometric properties, previously investigated on adult samples, the Adult Separation Anxiety Questionnaire (ASA-27) lacks formal validation for emerging adults. This study aimed to investigate the psychometric properties of the ASA-27 on a sample of Italian emerging adults. 863 participants completed the ASA-27 and measures of anxiety and depression. With the aim of prospecting a shorter, optimized version of the tool, psychometric properties were assessed using the Classical Test Theory and Item Response Theory (IRT) approach. Convergent validity and measurement invariance across sex were also tested. The reduced 19-items version retained a unidimensional structure, strong internal consistency and convergent validity. Measurement invariance supported consistent use across sexes. The ASA-19 is a reliable and more user-friendly tool for assessing separation anxiety in emerging adults, contributing to improved psychological assessment and intervention strategies in this life stage.

PurposeTo validate the hepatitis B virus infection-related stigma scale (HBVISS) using Classical Test Theory and Item Response Theory in a sample of Chinese chronic HBV carriers.MethodsFeasibility, internal consistency reliability, split-half reliability and construct validity were evaluated using a cross-sectional validation study (n = 1,058) in Classical Test Theory. Content validity was assessed by COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) criteria. The Item Response Theory (IRT) model parameters were estimated using Samejima’s graded response model, after which item response category characteristic curves were drawn. Item information, test information, and IRT-based marginal reliability were calculated. Measurement invariance was assessed using differential item functioning (DIF). SPSS and R software were used for the analysis.ResultsThe response rate reached 96.4% and the scale was completed in an average time of 5 min. Content validity of HBVISS was sufficient (+) and the quality of the evidence was high according to COSMIN criteria. Confirmatory factor analysis showed acceptable goodness-of-fit (χ2/df = 5.40, standardized root mean square residual = 0.057, root mean square error of approximation = 0.064, goodness-of-fit index = 0.902, comparative fit index = 0.925, incremental fit index = 0.926, and Tucker-Lewis index = 0.912). Cronbach’s α fell in the range of 0.79–0.89 for each dimension and 0.93 for the total scale. Split-half reliability was 0.96. IRT discrimination parameters were estimated to range between 0.959 and 2.333, and the threshold parameters were in the range-3.767 to 3.894. The average score for test information was 12.75 (information >10) when the theta level reached between-4 and + 4. The IRT-based marginal reliability was 0.95 for the total scale and fell in the range of 0.83–0.91 for each dimension. No measurement invariance was detected (d-R2 < 0.02).ConclusionHBVISS exhibited good feasibility, reliability, validity, and item quality, making it suitable for assessing chronic Hepatitis B virus infection-related stigma.

In this study, the researchers assessed comparability of item statistics of 2017 basic education certificate mathematics examination of National Examinations Council (NECO) through Classical Test Theory (CTT) and Item Response Theory (IRT) measurement frameworks. The study adopted instrumentation design. A 60-item NECO basic education certificate education mathematics objective test paper I was administered to 978 basic nine examinees, randomly selected from Osogbo and Olorunda Local Government Area, Osun State, Nigeria. The responses of the examinees to the test data were analysed using Marginal Maximum Likelihood Estimation of JMETRIK software.&amp;nbsp; The result showed that the test data obey the assumption of unidimensionality of 3-parameter logistic model and Classical Test Theory measurement framework deleted more items 33 (55%) compare to IRT measurement framework 12 (20%). Also, it was observed that item statistics from the two contrasting frameworks (CTT and IRT) were not comparable. Moreover, further analysis showed that there was low correlation among the item statistics index. The implication of this is that NECO should jettison the use of Classical Test Theory and embrace utilization of Item Response Theory framework during their test development and item analysis.&amp;nbsp;

BackgroundIn medical education research, various questionnaires are often used to study possible relationships between strategies and approaches to teaching and learning and the outcome of these. However, judging the applicability of such questionnaires or the interpretation of the results is not trivial.MethodsAs a way to develop teacher thinking, teaching strategy profiles were calculated for teachers in a research intensive department at Karolinska Institutet. This study compares the sum score, that was inherent in the questionnaire used, with an Item Response Theory (IRT) approach. Three teaching dimensions were investigated and the intended sum scores were investigated by IRT analysis.ResultsAgreements as well as important differences were found. The use of the sum score seemed to agree reasonably with an IRT approach for two of the dimensions, while the third dimension could not be identified neither by a the sum score, nor by an IRT approach, as the items included showed conflicting messages.ConclusionsThis study emphasizes the possibilities to gain better insight and more relevant interpretation of a questionnaire by use of IRT. A sum score approach should not be taken for granted. Its use has to be thoroughly evaluated.

Interpreting change is a requisite component of clinical decision making for physical therapists. Physical therapists often interpret change using minimal detectable change (MDC) values. Current MDC formulas are informed by classical test theory and calculated with group-level error data. This approach assumes that measurement error is the same across a measure's scale and confines the MDC value to the sample characteristics of the study. Alternatively, an item response theory (IRT) approach calculates separate estimates of measurement error for different locations on a measure's scale. This generates a conditional measurement error for someone with a low, middle, or high score. Error estimates at the measure-level can then be used to determine a conditional MDC (cMDC) value for individual patients based on their unique pre- and post-score combination. cMDC values can supply clinicians with a means for using individual score data to interpret change scores while providing a personalized approach that should lower the threshold for change compared with the MDC and enhance the precision of care decisions by preventing misclassification of patients. The purpose of this Perspective is to present how IRT can address the limitations of MDCs for informing clinical practice. This Perspective demonstrates how cMDC values can be generated from item-level psychometrics derived from an IRT model using the patient-reported Activities-specific Balance Scale (ABC) commonly used in stroke rehabilitation and also illustrates how the cMDC compares to the MDC when accounting for changes in measurement error across a scale. Theoretical patient examples highlight how reliance on the MDC value can result in misclassification of patient change and how cMDC values can help prevent this from occurring. This personalized approach for interpreting change can be used by physical therapists to enhance the precision of care decisions.

Data-driven item selection for the Shirts and Shoes Test

Background Item response theory (IRT), particularly the graded response model, provides a latent trait (θ) that reflects balance ability and offers a complementary framework for psychometric evaluation. Although IRT is recommended for estimating clinical thresholds, its application to cutoff determination for stroke populations remains limited. We hypothesized that IRT-based approaches will yield cutoff values that differ from those derived using classical test theory and provide additional methodological insight for cutoff determination. Purpose To estimate and compare cutoff values for walking independence in inpatients with stroke using classical test theory and IRT. Methods Data from 165 inpatients with stroke were analyzed. Balance was assessed using the Brief Balance Evaluation Systems Test (Brief-BESTest). Cutoff scores were calculated using receiver operating characteristic (ROC), IRT combined with ROC (IRT+ROC), prevalence-based IRT method (old IRT), and embedded state-item IRT method (new IRT). Discriminative performance was evaluated using area under the ROC curve (AUC), sensitivity, specificity, likelihood ratios, and accuracy. Results The cutoff values for walking independence were 12.37 (ROC), 11.46 (IRT+ROC), 10.54 (old IRT), and 7.90 (new IRT). All methods demonstrated comparable performance, with AUC values ranging from 0.769 to 0.794, indicating good discriminative ability of the Brief-BESTest for identifying walking independence. The new IRT method achieved the highest AUC and sensitivity; ROC and IRT+ROC methods achieved similar AUCs. Although cutoff values and classification metrics differed slightly across methods, their confidence intervals overlapped. Conclusion To our knowledge, this is the first study to directly compare cutoff estimation methods incorporating IRT in stroke rehabilitation, suggesting that IRT-based approaches can yield cutoff values with stability comparable to that of classical test theory and providing methodological insight for future research and clinical assessment scale development.

BackgroundThe Fibromyalgia Impact Questionnaire - Revised (FIQR) is one of the most commonly used self-rating instruments developed to quantify the severity of disease burden and impact of disease on functional...

A classic topic in the fields of psychometrics and measurement has been the impact of the number of scale categories on test score reliability. This study builds on previous research by further articulating the relationship between item response theory (IRT) and classical test theory (CTT). Equations are presented for comparing the reliability and precision of scores within the CTT and IRT frameworks. This study presented new results pertaining to the relative precision (i.e., the test score conditional standard error of measurement for a given trait value) of CTT and IRT, and the new results shed light on the conditions where total scores and IRT estimates are more or less precisely measured. The relative reliability of CTT and IRT scores is examined as a function of item characteristics (e.g., locations, category thresholds, and discriminations) and subject characteristics (e.g., the skewness and kurtosis of the latent distribution). CTT total scores were more reliable when the latent distribution was mismatched with category thresholds, but the discrepancy between CTT and IRT declined as the number of scale categories increased. This article also considered the appropriateness of linear approximations of polytomous items and presented circumstances where linear approximations are viable. A linear approximation may be appropriate for items with two response options depending on the item discrimination and the match between the item location and latent distribution. However, linear approximations are biased whenever items are located in the tails of the latent distribution and the bias is larger for more discriminating items.

Item response theory (IRT) consists of a set of mathematical models for the probabilities of various responses to test items as a function of item and person characteristics. In longitudinal data, changes in measured variables can only be interpreted if important psychometric features of the measured variables are assumed invariant across time. Measurement invariance is invariance in the relation of a measure to the latent variable underlying it. Measurement invariance in longitudinal studies concerns invariance over time, and IRT provides a useful approach to investigating longitudinal measurement invariance. Commonly used IRT models are described, along with the representation of measurement invariance in IRT. The use of IRT for investigating invariance is then described, along with practical considerations in using IRT for this purpose. Conceptual issues, rather than technical details, are emphasized throughout.

Purpose: This study aims to show a creative approach to uniting the diversity of cultural and religious identities in musical harmony for efforts to build social harmony between multicultural and multireligious societies. Methodology: With an art-based qualitative analysis approach, this study analyzes elements of religious, cultural, and musical identities in the traditions and songs of the Maluku Protestant Church (GPM). Findings: This study found that GPM Songs have offered a new treasure in the tradition of religious songs by creatively and harmoniously combining elements of the cultural identity of the Maluku people who embrace all different religious identities through the shared language of pela-gandong, ain ni ain, kalwedo, in musical harmony for efforts to build and maintain social harmony in multicultural and multireligious societies. Implications: The findings of this study provide a new insight that can be applied to preserving peace in multicultural societies, namely that diverse cultural and religious identities can be harmonized through musical harmonization. Originality and Value: This study contributes to the discourse of music, rituals, and tradition studies, as well as multicultural and multireligious identities, and offers a new insight into creative diplomacy by using musical harmony in church song for social harmony.

ABSTRACTWe give an account of classical test theory (CTT) in terms of the more fundamental ideas of item response theory (IRT). This approach views CTT as a very general version of IRT, and the commonly used IRT models as detailed elaborations of CTT for special purposes. We then use this approach to CTT to derive some general results regarding the prediction of the true score of a test from an observed score on that test as well from an observed score on a different test. This leads us to a new view of linking tests that were not developed to be linked to each other. In addition we propose true‐score prediction analogues of the Dorans and Holland measures of the population sensitivity of test linking functions. We illustrate the accuracy of the first‐order theory using simulated data from the Rasch model and illustrate the effect of population differences using a set of real data.

BackgroundPrimary schoolteachers play a pivotal role in the education of children, highlighting the importance of addressing their psychological well-being and mental health. While the Depression, Anxiety, and Stress Scale-21 (DASS-21) is a globally recognized instrument for evaluating mental health, its systematic validation among primary schoolteachers, particularly in a cross-national context, remains understudied. The present cross-cultural study evaluated the psychometric properties of DASS-21 among primary schoolteachers in Spain and China to compare the DASS-21 between teachers across Western and Eastern cultures.MethodsThe study sample comprised 1,350 Spanish and 2,580 Chinese primary schoolteachers. The DASS-21, alongside the Emotional Exhaustion Scale for Teachers, was used to evaluate its psychometric properties. The construct and concurrent validity of the DASS-21 were examined using Rasch analysis, Confirmatory Factor Analysis (CFA), and Structural Equation Modeling (SEM). Additionally, measurement invariance was tested across two national teacher cohorts.ResultsRasch analysis confirmed the validity of most DASS-21 items among both the Spanish and Chinese teachers, with only the same item from the Anxiety subscale showing inadequate fit in both countries. CFA across both countries initially favored a bifactor model, which was subsequently excluded due to problematic factor loadings. Instead, a single-factor model provided the best fit for Chinese teachers, while a correlated three-factor model was optimal for Spanish teachers. SEM demonstrated the DASS-21’s concurrent validity with emotional exhaustion, with consistent findings across both samples. After excluding Differential Item Functioning (DIF) items, strict measurement invariance between Spanish and Chinese teachers was achieved, as verified by multi-group CFA.ConclusionThe DASS-21 exhibits commendable psychometric properties, rendering it a suitable instrument for evaluating the mental illness of primary schoolteachers in both Spain and China.

Tests used for such purposes as determining educational quality, defining educational needs, hiring an employee, student selection and placement and performing guidance and clinic services have an important place in education and psychology. Of course, they should have certain psychometric features related to test scores' validity and reliability. Various test theories have helped to create more valid and reliable measurements and, as a result, to make better decisions regarding individuals. In education and psychology, Classical Test Theory (CTT) and Item Response Theory (IRT) are both widely used. CTT assumes that an individual's observed score is the total of the true score and the error score, while IRT estimates an individual's ability or latent trait from responses to test items (Embretson & Reise, 2000).When IRT assumptions and model-data fit are ensured, item and ability parameters' invariance occurs; this is known as the most important advantage IRT has over CTT. Item and ability parameters' invariance means estimating ability parameters independently of item sample and estimating item parameters independently of ability sample. IRT's invariance feature makes it very practicable in many applications, for instance, test development, computerized adaptive testing, bias studies, test equating and item mapping (Hambleton & Swaminathan, 1985). IRT is classified under two main categories as parametric IRT (PIRT) and nonparametric IRT (NIRT) (Olivares, 2005; Sijtsma & Molenaar, 2002).To analyse ordered items, such as Likert-type attitude items, partial credit cognitive items or not ordered graded items such as multiple-choice test items, item response models are developed towards polytomous items in IRT (Ostini & Nering, 2006). In these models developed for polytomous items, a non-linear relationship between an individual's latent trait and the possibility of choosing a certain category of item answer is explained (Embretson & Reise, 2000). Graded Response Model (GRM), part of IRT models developed for polytomous items, is often preferred by researchers for applications since it is more useful in presentations, portfolios, essays and Likert-type items with ordered item categories (DeMars, 2010; Ostini & Nering, 2006). To scale tests that consist of polytomous items by making true estimates according to GRM, evaluating PIRT's assumptions and model-data fit is necessary. And to provide these assumptions and model-data fit, large samples are needed. At this point, NIRT models draw attention because they provide a practical advantage in determining psychometric properties of tests with fewer items and respondents (Stout, 2001).NIRT models are defined as statistical scaling methods that require fewer assumptions than PIRT models for measuring persons and items (Stochl, 2007). With their wide application area, NIRT models are used in ordinal scales, applied research areas, sociology, marketing research and health research on quality of life (Sijtsma, 2005). The literature reveals that two models, namely, the Mokken model and nonparametric regression estimation models, are employed. These two models are themselves divided into sub-models. The Mokken model consists of the sub-models Monotone Homogeneity Model (MHM) and the Double Monotonicity Model (DMM). Nonparametric regression estimation models consist of such sub-models as the Kernel Smoothing Approach Model (KSAM), the Isotonic Regression Estimation and the Smoothed Isotonic Regression Estimation models (Lee, 2007; Sijtsma & Molenaar, 2002). Along with theoretical studies being conducted, new sub-models are being added to nonparametric regression estimation models.As a NIRT model, MHM requires unidimensionality, local independence and monotonicity assumptions, and it defines the relationship that latent variables and items with homogeneous (unidimensional) and monotone item characteristic curve (ICC) have (Meijer & Baneke, 2004; Sijtsma & Molenaar, 2002). …