ABSTRACTThe Revised Preschool Language Assessment (RPLA) is a standardized measure for examining the language status of and determining potential language difficulties among preschoolers between 3 and 6 years old. To facilitate the applicability of the RPLA for use with Mandarin-speaking children, the present study adopted exploratory factor analysis (EFA) and Rasch analysis to assess the appropriateness of using a single total score to interpret children’s language ability. Mandarin-speaking children (N = 138) with no reported disabilities were recruited. Analyses suggested that the RPLA is a useful instrument for identifying differences in children based on language proficiency; however, analyses of total scores showed that the RPLA was not unidimensional; thus, reporting a single total score is not justified. Subsequently, EFA and Rasch analyses were performed in scales, yielding data reflecting separate unidimensional domains of language comprehension and oral expression. The person–item maps indicated that complexity in comprehension and expression increases and provides support for the validity of scores generated by using the measure. Thus, using separate ability indexes for language comprehension and oral expression rather than using one total score was suggested. Finally, applications involving screening and weighting items based on their difficulty levels in multidimensional language assessments were discussed.

ABSTRACTThis article considers the construct of fluency in second language assessment and how it might be informed by research in applied linguistics. It briefly describes the way fluency is conceptualized in four language tests, as embodied in their respective assessment criteria, to show where the field is at present. The article then takes into account recent insights from applied linguistics, and from the fields of psycholinguistics, discourse analysis, and sociolinguistics. The article questions the current conceptualization of fluency in language testing, in which it is defined as a concept that should be sought in the ear of the beholder, and where disfluency is only seen as a deficit. Future research should focus on finding ways to ensure that the measures used in testing reflect the ability to talk fluently and efficiently, rather than measures that only reflect listeners’ impressions about such ability, and to show that these can be marked appropriately and reliably.

Designing Listening Tests: A Practical Approach (DLT) by Green (2017) is the first invaluable new reference book on test development procedures focusing specifically on designing listening tasks. T...

ABSTRACTThis article reports on a study of foreign students’ success in higher education in Norway and focuses specifically on the relation between academic success and students’ proficiency level of Norwegian as measured by a CEFR-based university entrance test. This study is distinguished from prior studies because it sets out to investigate not only the relation between entrance test score and future academic success but also what level of target language proficiency is a necessary minimum requirement for foreign students to manage the linguistic demands they face in higher education. It includes students who passed the entrance test in question as well as students who failed, hence overcoming a common limitation of predictive validity studies. The results of the study show a significant positive correlation between scores on the university entrance test and self-reported academic success. Moreover, the study shows that students with proficiency levels below B2 seem to lack the necessary language skills to master their studies, regardless of discipline and faculty. The results are of relevance to other countries and educational contexts where the CEFR scale is used as a reference for university entrance tests. Together with similar studies of the predictive validity of CEFR-based entrance tests, the study contributes to an ongoing validation of the CEFR, which has been called for since its publication.

ABSTRACTMost English-medium programs at European universities require prospective students to take standardised tests for English as a foreign language (EFL) to be admitted. However, there are contexts in which individual teachers’ judgements serve the same function, thus having high-stakes consequences for the higher education entrance of their students. The present study investigates the accuracy of CEFR-based judgements of upper secondary school EFL teachers in Germany and compares it to the accuracy of conventional grading. Seventy-three teachers located each of the students in their EFL class onto a CEFR level. Teachers’ judgements were compared to the CEFR levels estimated from the TOEFL ITP using level, differentiation, and ranking component as indicators of judgement accuracy. Findings show that while CEFR-based judgements of relative EFL proficiency within classes (rank component) are rather appropriate, teachers substantially overestimate the total level of their students’ achievement (level component). Compared to conventional grading, however, CEFR-based judgements lead to more accurate estimations of variance between students in a class (differentiation component). The findings are discussed with particular reference to the consequences for higher education entrance policy concerning CEFR levels B2 and C1 as relevant benchmarks.

ABSTRACTThe purpose of the present study was twofold: (a) it examined the relationship between peer-rated likeability and peer-rated oral presentation skills of 96 student presenters enrolled in a science communication course, and (b) it investigated the relationship between student raters’ severity in rating presenters’ likeability and their severity in evaluating presenters’ skills. Students delivered an academic presentation and then changed roles to rate their peers’ performance and likeability, using an 18-item oral presentation scale and a 10-item likeability questionnaire, respectively. Many-facet Rasch measurement was used to validate the data, and structural equation modeling (SEM) was used to examine the research questions. At an aggregate level, likeability explained 19.5% of the variance of the oral presentation ratings and 8.4% of rater severity. At an item-level, multiple cause-effect relationships were detected, with the likeability items explaining 6–30% of the variance in the oral presentation items. Implications of the study are discussed.

ABSTRACTA growing number of stakeholders argue for the use of second language (L2) speaking assessments that measure the ability to orally communicate in real time. A Web-based virtual environment (VE) that allows live voice communication among individuals may have potential for aiding in delivering such assessments. While off-the-shelf voice communication VEs, such as Second Life, have been used for language learning and to a much lesser extent, language assessment, a very limited number of such environments have been designed with the explicit aim of aiding in assessing L2 oral communication. Moreover, test developers who desire to build such a VE have little theoretical basis to guide them. This conceptual article fills this void by discussing some of the theoretical issues that were considered when the researchers built a VE to help deliver L2 oral communication assessments.

ABSTRACTThe current study explores the nature and relationships among test takers’ performance appraisals, appraisal calibration, and reported cognitive and metacognitive strategy use in a language test situation. Performance appraisals are executive processes of strategic competence for judging test performance (e.g., evaluating the correctness or appropriateness of responses to given test tasks). Appraisal calibration denotes a perfect match between appraisal confidence and actual test performance. 294 English as a foreign language (EFL) students took an English test, which was designed to measure four language areas (listening, grammar, vocabulary, and reading). The students reported their level of appraisal confidence immediately after answering each test question. At the end of the test, they were asked to report their overall appraisal confidence and perceived cognitive and metacognitive strategy use in the test. First, it was found that test takers were not well calibrated in all test sections. Second, test takers’ appraisal confidence could predict just above one third of the test performance variance. Third, test takers tended to be underconfident in easy questions but overconfident in difficult questions. Fourth, appraisal calibration was not strongly related to reported metacognitive strategy use. The implications of the study in light of strategic competence research and recommendations for further research are discussed.

ABSTRACTThe Primary School Leaving Examination (PSLE) is taken by all Primary 6 students in Singapore. Test takers are assessed in four subject areas: English Language (EL), a Mother Tongue Language, Mathematics, and Science. Test scores of the high-stakes PSLE determine outcomes relating to secondary school placement of test takers. This article reviews the EL test of the PSLE. We provide background information, give a description of the PSLE EL test, and adopt Kunnan’s Test Fairness Framework to examine the fairness of the test in its validity, absence of bias, access, administration, and social consequences. Because of a lack of empirical studies on the PSLE EL test, we have occasionally based our evaluation on secondary sources (e.g., anecdotal evidence). Overall, our review shows that the validity of the test may have been enhanced because of improvements to the test design in the past two decades. We also find proactive access arrangements and stringent, efficacious administration of the test in place. Given that there is little empirical research on the PSLE EL test, more research is needed to further examine it.

ABSTRACTThe correct use of frequently occurring word combinations represents an important part of language proficiency in spoken and written discourse. This study investigates the use of English-language recurrent word combinations in low-level and high-level L2 English academic essays sourced from the Canadian Academic English Language (CAEL) assessment. Quantitative and qualitative measures of analysis were used to investigate how L2 English writers of differing proficiency levels made use of recurrent word combinations in their writing. Results indicate that low-level writers tend to use more stance and discourse-organizing expressions in their essays, and more of these identified expressions also appear in the articles included as part of the test materials (i.e., suggesting that low-level writers used verbatim copying as a test-taking strategy). High-level writers were found to be less dependent on the included reading articles and made greater use of referential bundles in their writing.