Although research on young learners’ (YL) second language (L2) proficiency gains is on the rise, few studies have investigated YLs’ language progression in an English as a Foreign Language (EFL) context from a longitudinal perspective and the effects of background variables on their learning outcomes. This current study examined YL’s English proficiency gains over 6 months. It further investigated how the gains in proficiency were related to the learners’ educational context and background variables. In total, 110 YLs from three distinct educational settings in Mexico took the TOEFL® Primary tests at the beginning and the end of 6 months. They completed a set of background questionnaires at the start, midpoint, and end of the study. Results showed that the learners in the advantaged background with varying resources made significant gains in speaking proficiency throughout the 6 months, while those with limited resources did not. Also, educational context predicted learners’ proficiency gains over time, and low-proficiency learners exhibited more improvement than high-proficiency learners. The findings offer important implications to school administrators and testing agencies.

Assessment rubrics have increasingly been developed and deployed to evaluate the quality of language interpreting, yet understanding of rubric-based interpreting assessment remains limited. This systematic review aims to: (a) catalog rubrics, (b) examine rubric design features, (c) understand rubric use, and (d) evaluate rubric utility. A rigorous review process, involving database searching, citation tracking, and targeted review of core literature, identified 80 unique rubrics comprising a total of 265 (sub-)scales. A comprehensive analysis revealed that: (a) among 11 potential sources informing rubric development, test-external sources (e.g., literature review) were primarily used, whereas test-internal sources (e.g., performance samples) were much less consulted; (b) assessments primarily used analytic and task-type rubrics with an average of five performance levels and four scoring criteria. Rubric descriptors generally incorporated observable indicators of interpreting quality, employing both descriptive and evaluative rubric language; (c) rubrics were used by three main types of raters—interpreting practitioners, trainers, and students—to assess primarily spoken-language interpreting; and (d) rubric-based scores demonstrated moderate-to-high reliability and validity overall, though meta-analysis identified three significant moderators, including correlation coefficient type, assessment criterion, and rubric length. These findings are expected to provide guidance for assessment practice and research in interpreting.

Automated speaking assessment (ASA) of second language proficiency benefits both learners and educators. However, developing these systems for less commonly taught languages like Finnish and Finland Swedish is hindered by the need for large datasets with equal representation of all proficiency levels. Traditional machine learning algorithms used in ASA are data-driven and consequently struggle to generalize to underrepresented proficiency levels. This study leverages large language models (LLMs) to enhance scoring performance in underrepresented proficiency levels through two approaches: augmenting the learner’s corpus with LLM-generated transcripts (simulating data) and applying LLMs to score the transcripts of learners’ responses directly. Our findings show that both solutions are comparable to or better than a traditional machine learning model trained on the original data for proficiency levels with fewer examples. Additionally, we found that providing LLMs with examples of human grading at various proficiency levels significantly enhances their performance as graders, especially when compared to using a single demonstration or none at all. Finally, our study confirms that using automatic speech recognition transcripts instead of human transcripts does not compromise assessment quality, enabling the development of LLM-based systems that can generate proficiency ratings directly from audio input.

This paper reports the results of a study designed to explore the relationships between speaking test task variables and linguistic features of spoken responses on a speaking assessment for Grade 7 multilingual English learners (age 12–13) in U.S. public schools. Speaking task responses from 30 high-proficiency test takers were transcribed and coded for 14 complexity measures and 9 fluency measures. Subsequent analyses of simple main effects indicate that many of the complexity and fluency measures significantly differ by task variables. We observed more meaningful differences in the complexity measures, but fluency features also varied in the key area of pause time. The findings reveal that tasks designed to elicit extended spoken responses did indeed prompt spoken language with more complex features than tasks designed to elicit shorter, simpler responses across academic disciplines. Results of the effects of task variables on fluency measures indicate that Grade 7 students pause more in response to more complex tasks. The results provide some preliminary validity evidence for the assessment by indicating expected performance differences based on target task difficulty level and have implications for the design and scoring of speaking assessment tasks for young learners.

Corpus-based studies have offered the domain definition inference for test developers. Yet, corpus-based studies on temporal fluency measures (e.g., speech rate) have been limited, especially in the context of academic lecture settings. This made it difficult for test developers to sample representative fluency features to create authentic listening passages. To address this issue, the Fluency Corpus of Academic English Lectures (FCAEL) was created to offer insight into the thresholds for temporal fluency features in academic lecture settings. The current study compared the corpus data to the academic lecture passages in the Test of English as a Foreign Language Internet-based test (TOEFL iBT) and International English Language Testing System (IELTS) to examine the domain definition inference of these tests. In total, 14 temporal fluency measures were examined. A bootstrapped one-way multivariate analysis of variance (MANOVA), followed by a series of bootstrapped analyses of variances (ANOVAs), independent t-test, and Tukey tests showed some support for the tests, although many limitations were also found. The study suggests the 25th–75th percentile of FCAEL as tentative thresholds for each temporal fluency feature. The proposal may be useful for test developers to create and revise test materials. Coding schemes, analysis codes, and raw corpus data are available on the project’s Open Science Framework page, exemplifying how Open Science can provide benefits beyond the academic community.

This study is a systematic examination of the open access status of research in two flagship language testing and assessment journals: Language Testing and Language Assessment Quarterly. Coding and analysing 898 articles, we investigated (a) the prevalence of open access in four aspects—open manuscripts, open materials, open data, and open code, and (b) the relationship between open access and various characteristics of research, tests, and researchers. Our study revealed a positive trend in the adoption of open access over time, with open manuscripts and materials showing notable increases. Open code and data have remained scarce, though with a recent uptick from a low base. Notably, logistic regression results suggest inequitable participation in open access as authors from the Global South were less likely to have open manuscripts. Recognising the potential role of flagship journals as trend and standard setters, we call on the field to (a) shift towards more equitable open access models, (b) balance intellectual property concerns with validation needs, (c) recognise open code and open data with protected access via dedicated badges, and (d) adopt Research Transparency Statements, a new reporting structure inclusive of methodological and epistemological differences in open research practices.

Test preparation has garnered considerable attention in second language (L2) education due to the significant implications that successful performance on a language test may have for academic advancement, future career opportunities, and immigration prospects. Meanwhile, an overemphasis on test preparation has been criticized for encouraging the cultivation of construct-irrelevant test-taking strategies at the expense of developing general language proficiency. To systematically explore how test preparation has been investigated in the literature, we conducted a scoping review of 66 studies on L2 test preparation. Specifically, this study examined the key characteristics of publications on test preparation, the main themes explored, the study and participant characteristics, as well as the essential aspects of their research methodologies. The results of this review revealed various trends in the literature on L2 test preparation, such as the exclusive focus on English as the target language, the lack of diversity in stakeholders as participants, the dominance of international language tests, and the paucity of experimental studies that utilize advanced statistical techniques. In addition to interpreting the results of our analysis, we discuss the implications of this scoping review and outline several directions for future research on test preparation.

This article focuses on rater severity and consistency and their relation to major changes in the rating system in a high-stakes testing context. The study is based on longitudinal data collected from 2009 to 2019 from the second language (L2) Finnish speaking subtest in the National Certificates of Language Proficiency in Finland. We investigated whether rater severity and consistency changed over that period and whether the changes could be explained by major changes in the rating system, such as the change of lead examiner, the modus of rating and training (on-site or remote), and the composition of the rater group. The data consisted of 45 rating sessions with 104 raters and 59,899 examinees and were analysed using the many-facet Rasch model and generalized linear mixed models. The analyses indicated that raters as a group became somewhat more lenient over time. In addition, the results showed that the rater community and its practices, the lead examiners, and the modus of rating and training can influence rating behaviour. Finally, we elaborate on implications for both research and practice.

Given the lack of computational tools available for assessing second language (L2) production in Korean, this study introduces a novel automated tool called the Korean Syntactic Complexity Analyzer (KOSCA) for measuring syntactic complexity in L2 Korean production. As an open-source graphic user interface (GUI) developed in Python, KOSCA provides seven indices of syntactic complexity, including traditional and Korean-specific ones. Its validity was tested by investigating whether the syntactic complexity indices measured by it in L2 written and spoken production could explain the variability of L2 Korean learners’ proficiency. The results of mixed-effects regression analyses showed that all seven indices significantly accounted for learner proficiency in Korean. Subsequent stepwise multiple regression analyses revealed that the syntactic complexity indices explained 56.0% of the total variance in proficiency for the written data and 54.4% for the spoken data. These findings underscore the validity of the syntactic complexity indices measured by KOSCA as reliable indicators of L2 Korean proficiency, which can serve as a valuable resource for researchers and educators in the field of L2 Korean learning and assessment.