ABSTRACT Large-scale assessments (LSAs) primarily support system-level monitoring, but their instructional diagnostic potential remains underused. Cognitive Diagnostic Models (CDMs) offer a promising avenue, though their application in LSAs poses theoretical and practical challenges. This study explores whether cognitive models used in item generation can directly inform Q-matrix construction for CDM analyses, enhancing diagnostic value in early numeracy assessment. Using data from Luxembourg’s school monitoring program (N = 2,704), we analyzed four cognitive attributes (counting, addition < 10, decomposition, addition > 10) using developmental frameworks. We compared a Single-Attribute Hierarchical Model, assuming linear progression, with a Multiple-Attribute Hierarchical Model, allowing skill interactions. Both hierarchical models reproduced expected developmental progressions, with decomposition emerging as a key threshold skill and socio-economic status showing the largest subgroup differences. Subgroup analyses revealed a smaller-than-expected impact of migration background, while math anxiety peaked at intermediate skill levels. Embedding cognitive models in LSAs can bridge system-level monitoring and instructional support.

ABSTRACT Response anchoring, the tendency for respondents to provide item responses equivalent (or in close proximity) to immediately preceding responses, can reflect disengaged responding and/or cognitive challenges in comprehending test items. Using Grade 4–12 student responses to a survey measure of four social-emotional learning constructs collected from the CORE Districts in the state of California, we examine student variables that predict anchoring, and consider potential score bias and survey design implications. Response anchoring is studied using both an index of sequential response separation as well as through an IRT-based anchoring index recently considered in the literature. Both approaches show anchoring behavior increases at higher grade levels; the IRT index, however, uniquely shows anchoring to be reduced for students higher on the SEL constructs, as might be theoretically anticipated, suggesting its greater validity. Both methodological and practical implications of the analyses are considered in discussion.

ABSTRACT This paper highlights efforts to investigate the possible impact(s) of using culturally responsive and antiracist stimuli (math) and passages (ELA) in large-scale assessment contexts. It describes the extent to which, as a result of engaging with these items, rights-holders (64 students, 5 parents, and 14 teachers) reported feeling: (a) like their lived experiences and/or sociocultural identities were reflected; (b) the content was interesting/engaging; (c) they learned something as a result of engaging with the content of the items; (d) the items were appropriate for a large-scale assessment; and (e) unharmed (emotionally) after engaging with the assessment items. Participant feedback revealed broad levels of support for including culturally responsive context in math and ELA items easing concerns about the appropriateness of the use of these items in large-scale assessment. The author also discusses limitations of the process (e.g., time) and implications for future research.

ABSTRACT For K-12 low-stakes assessments, testing later in the day was associated with greater disengagement and lower performance among 2nd to 8th grade students. We investigated whether similar patterns occur among college-aged students given a 24-hour testing window and how personal characteristics (e.g., personality) correlated with self-selection of testing start time. Students testing earlier in the day exhibited lower disengagement and higher performance compared to those testing later. Students testing earlier were more conscientious and had higher perceived test importance. Multiple regression analyses revealed that time of day predicted number of non-rapid responses after controlling for personal characteristics. Two-hour increases in test start time after 8am corresponded with one additional rapid response, on average (e.g., 1 additional rapid response if students began testing at 10am, 2 additional rapid responses if students began testing at noon, etc.). Results highlighted the need for more research on time of day in low-stakes testing settings.

ABSTRACT Assessment for Learning (AfL) practices provide opportunities for teachers and students to learn with and from each other. Less well established are the roles that the material and affective configurations of classroom spaces play in AfL interactions between teachers and students, students with peers and students with ideas. Feedback, dialogue, questioning and using exemplars all depend on interactions where students are expected to manage their bodies, emotions and physical materials. Students not only infer meaning through talk but infer meaning from where their teachers stand to survey their work and how they circulate. Data from twenty years of AfL observational studies in New Zealand and Australia highlight some of the often overlooked spatial and material dynamics that impact the enactment and effectiveness of AfL. A growing understanding of the importance of embodied perspectives of assessment is essential to promote sustainable, healthy, and manageable classroom assessment experiences.

ABSTRACT We explored three approaches to resolving or re-scoring constructed-response items in mixed-format assessments: rater agreement, person fit, and targeted double scoring (TDS). We used a simulation study to consider how the three approaches impact the psychometric properties of student achievement estimates, with an emphasis on person fit. We found that although TDS is not specifically designed to identify discrepant or misfitting score profiles, this approach can improve person fit for students who are also within a critical range of achievement. Our findings also suggest that TDS alone is not sufficient to provide a comprehensive picture of measurement quality because it does not directly consider person fit. We suggest that researchers and practitioners consider combining TDS with other approaches to score resolution. We consider the implications of our findings for research and practice.

ABSTRACT The transition from secondary to tertiary education varies from country to country. In many countries, secondary school is concluded with high-stakes, national exams, or high-stakes entry tests are used for admissions to tertiary education. In other countries, secondary-school grade point average (GPA) is the determining factor. In the Netherlands, both play a role. With administrative data of close to 180,000 students, we investigated whether national exam scores or secondary school GPA was a better predictor of tertiary first-year retention. For both university education and higher professional education, secondary school GPA was the better prediction of retention, to the extent that national exams did not explain any additional variance. Moreover, for students who failed their exam, being held back by the secondary school for an additional year and entering tertiary education one year later, GPA in the year of failure remained as predictive as for students who had passed their exams and started tertiary education immediately. National exam scores, on the other hand, had no predictive value at all for these students. It is concluded that secondary school GPA measures aspects of student performance that is not included in high-stakes national exams, but that are predictive of subsequent success in tertiary education.

ABSTRACT The present meta-analysis sought to quantify the average degree of aggregated test score distortion due to rapid guessing (RG). Included studies group-administered a low-stakes cognitive assessment, identified RG via response times, and reported the rate of examinees engaging in RG, the percentage of RG responses observed, and/or the degree of score distortion in aggregated test scores due to RG. The final sample consisted of 25 studies and 39 independent samples comprised of 443,264 unique examinees. Results demonstrated that an average of 28.3% of examinees engaged in RG (21% were deemed to engage in RG on a nonnegligible number of items) and 6.89% of item responses were classified as rapid guesses. Across 100 effect sizes, RG was found to negatively distort aggregated test scores by an average of 0.13 standard deviations; however, this relationship was moderated by both test content area and filtering procedure.

ABSTRACT Teacher assessment practice is affected by a complex set of cognitive and affective traits, as well as institutional contexts. There is a dearth of research on sociocultural influences on teachers’ assessment identity. This study presents a model illustrating how vocational education and training (VET) teachers enact their assessment identity across contexts. We draw on interview data from 18 VET teachers. Using metaphors found in the data set allowed us to capture key aspects of the data material and to situate this study in the existing literature on teachers’ assessment identity. Based on the findings, we conceptualize five “faces” of VET teachers’ assessment identity: 1) the quality controller, 2) the educator, 3) the fosterer, 4) the motivator, and 5) the negotiator. We then show how VET teachers enact these faces within and across contexts as a way of negotiating tensions and avoiding conflicts. Finally, we discuss the implications of this model.