Social changes have resulted in an increase of students living in households that do not include both a mother and a father, reducing the efficacy of common survey questionnaire approaches to measuring student socioeconomic status (SES). This paper presents two studies conducted to develop and test a new, more inclusive set of student SES items appropriate for students from a range of household types. In the first study, we held group interviews with 57 students in Grades 4, 8, and 12 who lived in four nontraditional household types. The study goal was, first, to understand how students thought about their household members and learn what they knew about the educational background and employment status of their caregivers and, second, to develop draft items based on these findings. In the second study, we held 51 individual cognitive interviews with a similar sample to evaluate draft item clarity and function. We found that although students may live with a broad range of family members and other adults, they understood the term caregiver to refer to a person who provides resources and support. Students found it easier to answer items when the items included the titles of their caregivers. Our results demonstrate that a customizable approach to measuring student SES allows more students to report information about their caregivers than the current standard of asking about mothers and fathers. We provide recommendations for student SES measurement and potential next steps for research on this topic.

Workforce development and career and technical education (CTE) have long provided reliable pathways to middle skill jobs and a gateway to the middle class. Given recent changes in middle skills jobs, the education landscape, and federal policy priorities, the role of CTE in the U.S. educational landscape is evolving more rapidly, encompassing a broader range of education, and practices are changing ahead of research. The first part of this report provides an overview of the current state of CTE in the United States, as well as the state of CTE research, and presents an argument for a broader definition of CTE that incorporates workforce development through postsecondary institutions. The second part provides operational definitions and typologies to facilitate future research. Our aim is to build a research framework for CTE that is grounded in a normative path through CTE: getting in (preparation and recruitment), getting through (retention and skill acquisition), getting out (completion and initial hire), and getting on (career progression). Key challenges and priorities for future research are discussed.

This study discusses the development of a basic electronics knowledge (BEK) assessment as a pretest activity for undergraduate students in engineering and related fields. The 28 BEK items represent 12 key concepts, including properties of serial circuits, knowledge of electrical laws (e.g., Kirchhoff's and Ohm's laws), and properties of digital multimeters. This paper first discusses a psychometric evaluation of the BEK assessment to understand its basic measurement properties and to examine various group‐level differences based on demographic, institutional, and instructor characteristics. Subsequently, the relationship between BEK scores on the 23 retained items and performance on an existing complex collaborative simulation‐based electronics task is discussed. Results demonstrated that basic content knowledge alone may not be sufficient for students to demonstrate knowledge of electronics skills on more complex tasks. The research also carries great importance given ongoing concerns about improving the overall state and diversity of the engineering workforce and its associated pipeline to meet the demands of the national economy.

A key instructional goal of English language arts instruction is teaching students to read and interpret complex literary texts. This report reviews the literature on the development and pedagogy of literary analysis skills. It analyzes literary analysis skills as akey practice, a bundle of disciplinary skills and strategies that form a key target for instruction. This practice focuses on applying a variety of strategies for building textual interpretations, including strategies for deepening comprehension, inferring the author's purpose, considering multiple perspectives, connecting text to context, generalizing and applying themes, and participating in cultural conversations about texts. It also identifies a sequence of activities that define literary interpretation as a social practice and identifies key goals and subgoals for those activities. If literary analysis is viewed as a key practice, it requires participants to habitually engage in the following practices: (a) read and reread texts to find cues that suggest new possibilities for interpretation, (b) participate in interpretive discussions of literary texts, (c) use writing both as a tool to support interpretive work and as a means to communicate interpretive arguments, and (d) maintain an openness to new interpretations and a willingness to revisit previous interpretive conclusions in the light of new evidence. The evidence indicates that this is a late‐developing practice, typically acquired during high school and never fully mastered by a large proportion of the population, though it can be encouraged and fostered in the upper elementary and middle school years with effective instruction.

In this study, we explored retest effects on test scores and response time for repeaters, examinees who retake an examination. We looked at two groups of repeaters: those who took the same form twice and those who took different forms on their two attempts for a certification and licensure test. Scores improved over the two test attempts, and repeaters taking the same form twice tended to have larger score gains than those taking different forms. This trend was more salient for female examinees than male and for White and Hispanic/Latino examinees than examinees from other ethnicity groups. In addition, over a third of the examinee responses were incorrect on the same form for both attempts, indicating that the knowledge required for these items was not addressed after these repeaters failed their initial attempt to answer those items correctly. Factors that may be related to these findings are discussed.

The redesigned four‐skills TOEIC Bridge® tests were designed to measure the listening, reading, speaking, and writing proficiency of beginning to low–intermediate English learners in the context of everyday life. In this paper, I describe two studies that were conducted to investigate claims about the meaningfulness of redesigned TOEIC Bridge test scores by comparing them to self‐assessments of listening, reading, speaking, and writing proficiency. In Study 1, test takers who participated in the redesigned TOEIC Bridge field test in Japan and Taiwan completed an online self‐assessment survey of their speaking and writing proficiency. In Study 2, test takers who participated in pretesting for the redesigned TOEIC Bridge Listening and Reading test in Japan and Taiwan completed a paper‐based self‐assessment survey of their listening and reading proficiency. Both studies were designed with the same research questions in mind in order to evaluate claims about the meaning of redesigned TOEIC Bridge test scores and to enhance the interpretation of test scores by relating them to test takers' self‐assessed ability to perform a variety of related tasks. The results indicated that self‐assessments had medium correlations with redesigned TOEIC Bridge Listening test scores (r = .55), Reading test scores (r = .54), Speaking test scores (r = .51), and Writing test scores (r = .46). These results compare favorably with the results of similar studies of the relationship between test scores and self‐assessments of language proficiency and provide evidence to support claims about the meaningfulness of TOEIC Bridge test scores. In addition, the pattern of results across TOEIC Bridge test proficiency levels and can‐do statements taken from language proficiency standards suggests that the tests meaningfully differentiate test‐takers' proficiency levels, an important consideration of test design.

Abstract K–12 students are expected to acquire competence in data display as part of developing statistical literacy. To support research, assessment design, and instruction, we developed a hypothesized learning progression (LP) using existing empirical literature in the fields of mathematics and statistics education. The data display LP posits a progression of student understanding and learning of data display in terms of two progress variables: constructing data displays and interpreting data displays. An initial data display LP was revised through expert review and then supported the design of a set of data display tasks to elicit evidence of student knowledge and skills in constructing and interpreting data display. The data display LP and tasks presented in this research report can inform assessment development as well as classroom instruction, and they can be used for further studies on potential interactions between the two progress variables in students' development of data display knowledge.

Collaborative problem solving (CPS) is an important 21st‐century skill that is crucial for both career and academic success. However, developing a large‐scale and standardized assessment of CPS that can be administered on a regular basis is very challenging. In this report, we introduce a set of psychometric considerations and a general scoring strategy around assessing CPS, summarized based on the results of the extensive empirical studies we conducted at Educational Testing Service (ETS) over the past 6 years. Using the ETS Collaborative Science Assessment Prototype as an example, we show how these psychometric considerations have been incorporated into the development of the assessment prototype and how the scoring strategy has been implemented.

Abstract Distractor analyses are routinely conducted in educational assessments with multiple‐choice items. In this research report, we focus on three item response models for distractors: (a) the traditional nominal response (NR) model, (b) a combination of a two‐parameter logistic model for item scores and a NR model for selections of incorrect distractors, and (c) a model in which the item score satisfies a two‐parameter logistic model and distractor selection and proficiency are conditionally independent, given that an incorrect response is selected. Model comparisons involve generalized residuals, information measures, scale scores, and reliability estimates. To illustrate the methodology, a study of an international assessment of proficiency of nonnative speakers of a single target language used to make high‐stakes decisions compares the models under study.

Abstract Cross‐validation is a common statistical procedure applied to problems that are otherwise computationally intractable. It is often employed to assess the effectiveness of prediction procedures. In this report, cross‐validation is discussed in terms of U‐statistics. This approach permits consideration of the statistical properties of cross‐validation as an approach to assessment of prediction accuracy.