Hierarchical Diagnostic Classification Models Morphing into Unidimensional ‘Diagnostic’ Classification Models—A Commentary

In cognitive diagnostic assessment, multiple fine-grained attributes are measured simultaneously. Attribute hierarchies are considered important structural features of cognitive diagnostic models (CDMs) that provide useful information about the nature of attributes. Templin and Bradshaw first introduced a hierarchical diagnostic classification model (HDCM) that directly takes into account attribute hierarchies, and hence, HDCM is nested within more general CDMs. They also formulated an empirically driven hypothesis test to statistically test one hypothesized link (between two attributes) at a time. However, their likelihood ratio test statistic does not have a known reference distribution, so it is cumbersome to perform hypothesis testing at scale. In this article, we studied two exploratory approaches that could learn the attribute hierarchies directly from data, namely, the latent variable selection (LVS) approach and the regularized latent class modeling (RLCM) approach. An identification constraint was proposed for the LVS approach. Simulation results revealed that both approaches could successfully identify different types of attribute hierarchies, when the underlying CDM is either the deterministic input noisy and gate model or the saturated log-linear CDM. The LVS approach outperformed the RLCM approach, especially when the total number of attributes increases.

Matrix-assisted laser desorption/ionization time-of-flight mass spectrometry (MALDI-TOF-MS) was employed to analyze differential serum and urine peptides in patients with small cell lung cancer (SCLC) and healthy individuals, and SCLC diagnostic classification models were constructed. Serum and urine samples from 72 patients with SCLC, age- and gender-matched with 72 healthy individuals, were divided into training and testing sets in a 3:1 ratio. Serum and urine peptides were extracted using copper ion-chelating nanomagnetic beads, and mass spectra were obtained using MALDI-TOF-MS. Peptide spectra for the training set were analyzed, and the classification model was constructed using ClinProTools (CPT). The testing set was used for blinded model validation. For training-set sera, 122 differential peptide signal peaks with a mass of 0.8–10 kDa were observed, and 19 peptides showed significantly different expression [P<0.0005; area under curve (AUC) ≥0.80]. CPT screened 5 peptide peaks (0.8114, 0.83425, 1.86655, 4.11133 and 5.81192 kDa) to construct the classification model. The testing set was used for the blinded validation, which had 95.0% sensitivity and 90.0% specificity. For the training-set urine, 132 differential peptide signal peaks with m/z ratios of 0.8–10 kDa were observed, and 8 peptides had significantly different expression (P<0.0005; AUC ≥0.80). Then, 5 peaks (1.0724, 2.37692, 2.7554, 4.75475 and 4.7949 kDa) were used for classification model construction. The testing set was used for 36 blinded validation, which had 85.0% sensitivity and 80.0% specificity. Among the differential peptides, 3 had the same significant peaks at 2.3764, 0.8778 and 0.8616 kDa, identified as fibrinogen α, glucose-6-phosphate isomerase and cyclin-dependent kinase-1, respectively. The present study highlighted the differences that exist in serum and urine peptides between patients with SCLC and healthy individuals. Serum and urine peptide diagnostic classification models could be constructed using MALDI-TOF-MS, and showed high sensitivity and specificity.

Learners and educators in the health professions have called for more fine-grained information (subscores) from assessments, beyond a single overall test score. However, due to concerns over reliability, there have been limited uses of subscores in practice. Recent advances in latent class analysis have made contributions in subscore reporting by using diagnostic classification models (DCMs), which allow reliable classification of examinees into fine-grained proficiency levels (subscore profiles). This study examines the innovative and practical application of DCM framework to health professions educational assessments using retrospective large-scale assessment data from the basic and clinical sciences: National Board of Medical Examiners Subject Examinations in pathology (n = 2,006) and medicine (n = 2,351). DCMs were fit and analyzed to generate subscores and subscore profiles of examinees. Model fit indices, classification (reliability), and parameter estimates indicated that DCMs had good psychometric properties including consistent classification of examinees into subscore profiles. Results showed a range of useful information including varying levels of subscore distributions. The DCM framework can be a promising approach to report subscores in health professions education. Consistency of classification was high, demonstrating reliable results at fine-grained subscore levels, allowing for targeted and specific feedback to learners.

The purpose of our paper entitled Hierarchical Diagnostic Classification Models: A Family of Models for Estimating and Testing Attribute Hierarchies (Templin & Bradshaw, 2014) was two-fold: to create a psychometric model and framework that would enable attribute hierarchies to be parameterized as dependent binary latent traits, and to formulate an empirically driven hypothesis test for the purpose of falsifying proposed attribute hierarchies. The methodological contributions of this paper were motivated by a curious result in the analysis of a real data set using the log-linear cognitive diagnosis model, or LCDM (Henson, Templin, & Willse, 2009). In the analysis of the Examination for Certification of Proficiency in English (ECPE; Templin & Hoffman, 2013), results indicated that few, if any, examinees were classified into four of the possible eight attribute profiles that are hypothesized in the LCDM for a test of three binary latent attributes. Further, when considering the four profiles lacking examinees, it appeared that some attributes must be mastered before others, suggesting what is commonly called an attribute hierarchy (e.g., Leighton, Gierl, & Hunka, 2004). Although the data analysis alerted us to the notion that such a data structure might be present, we lacked the methodological tools to falsify the presence of such an attribute hierarchy. As such, we developed the Hierarchical Diagnostic Classification Model, or HDCM, in an attempt to fill the need for such tools. We note that the driving force behind the HDCM is one of seeking a simpler, or more parsimonious, solution when model data misfit is either evident from LCDM results or implied by the hypothesized theories underlying the assessed constructs. As a consequence of the ECPE data results, we worked to develop a more broadly defined set of models that would allow for empirical evaluation of hypothesized attribute hierarchies. We felt our work was timely, as a number of methods, both new and old, are now using implied attribute hierarchies to assess examinees in many large scale analyses—from so-called intelligent tutoring systems (e.g., Cen, Koedinger, & Junker, 2006) to large scale state assessment systems for alternative assessments using instructionally imbedded items (e.g. the Dynamic Learning Maps Alternate Assessment System Consortium Grant, 2010–2015). Moreover, such large scale analyses are based on tremendously large data sets, many of which simply cannot fit with the types of (mainly unidimensional) models often used in current large scale testing situations. Furthermore, newly developed standards in education have incorporated ideas of learning progressions which indirectly imply the existence of hierarchically structured attributes (e.g., Progressions for the Common Core State Standards in Mathematics, Common Core State Standards Writing Team, 2012). In short, the current and

Research on tourism resource demand offers critical decision support for the protection, development, and marketing of tourism resources. It also improves personalized experiences and satisfaction for tourists, thus fostering the advancement of the tourism industry into broader and higher realms. It also improves personalized experiences and satisfaction for tourists, thus fostering the advancement of the tourism industry into broader and higher realms. The objective of this study is to extract characteristics of 3A-level and above scenic spots nationwide, facilitating the integration and utilization of tourism resources in the country. The objective of this study is to extract characteristics of 3A-level and above scenic spots nationwide, facilitating the integration and utilization of tourism information. This facilitates the automation and efficiency of tourism resource classification and provides suggestions for improving services at these scenic areas. In response to the abundant resources of major scenic areas across the country, a new approach has been developed. In response to the abundant resources of major scenic areas across the country, this study introduces a hierarchical multi-label classification model for tourism resources. tourism resource classification theme system issued by the Ministry of Culture and Tourism, and leveraging FastText for pre-training on scenic area introductory texts, this research combines the advantages of the hierarchical multi-label classification model with the theme system of the Ministry of Culture and Tourism. introductory texts, this research combines the traditional LSTM model with an attention-based Transformer model. Additionally, a Graph-Convolutional Network (GCN) is used to classify tourism resources. Convolutional Network (GCN) is employed as a hierarchical structure-aware encoder to construct the MiLCT, a hierarchical multi-label classification model, enabling sophisticated multi-label classification to be applied to the text. model, enabling sophisticated multi-label classification of tourism scenic area resource data. Experimental comparisons demonstrate that, with increasing classification levels, the proposed model outperforms those lacking GCN and Transformer components in terms of micro-Precision, micro-Recision, micro-Recision, and micro-Recision. micro-Precision, micro-Recall, and micro-F1 scores, this indicates that the hierarchical structural information of the model can significantly enhance its performance. In comparison to the hierarchical multi-label correlation model, the proposed model demonstrates enhanced performance across the evaluated metrics, indicating its efficacy in integrating multimodal features and providing more comprehensive and accurate data characterization.

Diagnostic classification models (DCMs), aka cognitive diagnostic models (CDMs), have received considerable attention in L2 writing assessment thanks to their potential to offer fine-grained diagnostic information about learners’ writing strengths and weaknesses. However, the application of DCMs remains underexplored in classroom contexts, where nuanced diagnostic feedback is crucial for writing development. This study aimed to investigate the applicability of the Generalized Deterministic Inputs, Noisy “And” Gate (GDINA) model in a Thai EFL classroom writing assessment. Three experienced Thai EFL teachers used a 30-descriptor binary checklist to rate 210 students’ essays on five writing attributes: content fulfilment, organisational effectiveness, grammatical knowledge, vocabulary use, and mechanics. A Q-matrix specifying the relationship between descriptors and attributes was developed through expert judgement and empirical validation using fit indices and visual inspection (mesa plot and heat map). The findings indicated that despite a modest sample size and partial evidence of model-data misfit, the GDINA model could reasonably yield meaningful and nuanced diagnostic information about students’ writing strengths and weaknesses. At the group level, organisation was identified as the strongest attribute while content as the weakest. Specifically, while lower-proficiency students displayed relative strengths in vocabulary and mechanics, higher-proficiency students had relatively strong attributes in organisation and grammar. At the individual level, students with similar total scores exhibited distinct diagnostic profiles, highlighting the GDINA model’s ability to uncover fine-grained diagnostic information which is not uncovered by traditional raw scores. The findings have important implications for DCM-based diagnostic assessment in the classroom context.

The present study aimed at investigating invariance of a diagnostic classification model (DCM) for reading comprehension across gender. In contrast to models with continuous traits, diagnostic classification models inform mastery of a finite set of latent attributes, e.g., vocabulary or syntax in the reading context, and allow to provide fine grained feedback to learners and instructors. The generalized deterministic, noisy “and” gate (G-DINA) model was fit to item responses of 1000 male and female individuals to a high-stakes reading comprehension test. Use of the G-DINA model allowed for minimal assumption on the relationship of latent attribute profiles and item-specific response probabilities, i.e., the G-DINA model can represent compensatory or non-compensatory relationships of latent attributes and response probabilities. Item parameters were compared across the two samples, and only a small number of item parameters were statistically different between the two groups, corroborating the result of a formal measurement invariance test based on the multigroup G-DINA model. Neither correlations between latent attributes were significantly different across the two groups, nor mastery probabilities for any of the attributes. Model selection at item level showed that from among the 18 items that required multiple attributes, 16 items picked different rules (DCMs) across the groups. While this seems to suggest that the relationship among the attributes of reading comprehension differs across the two groups, a closer inspection of the rules picked by the items showed that almost in all cases the relationships were very similar. If a compensatory DCM was suggested by the G-DINA framework for an item in the female group, a model belonging to the same family resulted for the male group.

A practical illustration of multidimensional diagnostic skills profiling: Comparing results from confirmatory factor analysis and diagnostic classification models

Diagnostic classification models (DCMs) are statistical models with discrete latent variables (so-called skills) to analyze multiple binary variables (i.e., items). The one-parameter logistic diagnostic classification model (1PLDCM) is a DCM with one skill and shares desirable measurement properties with the Rasch model. This article shows that the 1PLDCM is indeed a latent class Rasch model. Furthermore, the relationship of the 1PLDCM to extensions of the DCM to mixed, partial, and probabilistic memberships is treated. It is argued that the partial and probabilistic membership models are also equivalent to the Rasch model. The fit of the different models was empirically investigated using six datasets. It turned out for these datasets that the 1PLDCM always had a worse fit than the Rasch model and mixed and partial membership extensions of the DCM.

The purpose of diagnostic classification models (DCMs) is to determine mastery or non-mastery of a set of attributes or skills. There are two statistics directly obtained from DCMs that can be used for mastery classification—the posterior marginal probabilities for attributes and the posterior probability for attribute profile.

Variable coded hierarchical fuzzy classification model using DNA coding and evolutionary programming

Psychiatric diseases are bringing heavy burdens for both individual health and social stability. The accurate and timely diagnosis of the diseases is essential for effective treatment and intervention. Thanks to the rapid development of brain imaging technology and machine learning algorithms, diagnostic classification of psychiatric diseases can be achieved based on brain images. However, due to divergences in scanning machines or parameters, the generalization capability of diagnostic classification models has always been an issue. We propose Meta-learning with Meta batch normalization and Distance Constraint (M2DC) for training diagnostic classification models. The framework can simulate the train-test domain shift situation and promote intra-class cohesion, as well as inter-class separation, which can lead to clearer classification margins and more generalizable models. To better encode dynamic brain graphs, we propose a concatenated spatiotemporal attention graph isomorphism network (CSTAGIN) as the backbone. The network is trained for the diagnostic classification of major depressive disorder (MDD) based on multi-site brain graphs. Extensive experiments on brain images from over 3261 subjects show that models trained by M2DC achieve the best performance on cross-site diagnostic classification tasks compared to various contemporary domain generalization methods and SOTA studies. The proposed M2DC is by far the first framework for multi-source closed-set domain generalizable training of diagnostic classification models for MDD and the trained models can be applied to reliable auxiliary diagnosis on novel data.

To obtain accurate, valid, and rich information from the questionnaires for internet addiction, a diagnostic classification test for internet addiction (the DCT-IA) was developed using diagnostic classification models (DCMs), a cutting-edge psychometric theory, based on DSM-5. A calibration sample and a validation sample were recruited in this study to calibrate the item parameters of the DCT-IA and to examine the sensitivity and specificity. The DCT-IA had high reliability and validity based on both CTT and DCMs, and had a sensitivity of 0.935 and a specificity of 0.817 with AUC = 0.919. More important, different from traditional questionnaires, the DCT-IA can simultaneously provide general-level diagnostic information and the detailed symptom criteria-level information about the posterior probability of satisfying each symptom criterion in DMS-5 for each patient, which gives insight into tailoring individual-specific treatments for internet addiction.

The Hierarchical Diagnostic Classification Model (HDCM) reflects on the sequences of the presentation of the essential materials and attributes to answer the items of a test correctly. In this study, a foreign language reading comprehension test was analyzed employing HDCM and the generalized deterministic-input, noisy and gate (G-DINA) model to determine and compare respondents’ mastery profiles in the test’s predefined skills and to illustrate the relationships among the attributes involved in the test to capture the influence of sequential teaching of materials on increasing the probability of getting an item a correct answer. Furthermore, Differential Item Functioning (DIF) analysis was applied to detect whether the test functions as a reason for the gender gap in participants’ achievement. Finally, classification consistency and accuracy indices are studied. The results showed that the G-DINA and one of the HDCMs fit the data well. However, although the results of HDCM showed the existence of attribute dependencies in the reading comprehension test, the relative fit indices highlight a significant difference between the G-DINA and HDCM, favoring G-DINA. Moreover, results indicate that there is a significant difference between males and females in six items in favor of females. Besides, classification consistency and accuracy indices specify that the Iranian University Entrance Examination holds a 71% chance of categorizing a randomly selected test taker consistently on two distinct test settings and a 78% likelihood of accurately classifying any randomly selected student into the true latent classes. As a result, it can be concluded that the Iranian University Entrance Examination can be considered as a valid and reliable test.

This study applied the newly developed Modified Ordinal Response Diagnostic Model (MORDM) to data reflecting organizational commitment in Korean bank employees. Using Bayesian estimation, the model converged and reasonably estimated the attribute profile distribution. However, the root mean square error of approximation (RMSEA) item fit indices and posterior prediction model checks showed unacceptable model-data fit. These findings call for further research into the development of more stable and comprehensive polytomous diagnostic classification models (DCMs) for ordinal data, and further evaluation of the adequacy of fit indices for polytomous DCMs.KeywordsPolytomous DCMsRMSEA item fitPosterior prediction model checking