Technology in language testing

This article begins by exploring recent developments in the use of computers in language testing in four areas: (a) item banking, (b) computer-assisted language testing, (c) computerized-adaptive language testing, and (d) research on the effectiveness of computers in language testing.The article then examines the educational measurement literature in an attempt to forecast the directions future research on computers in language testing might take and suggests addressing the following issues: (a) piloting practices in computer adaptive language tests (CALTs), (b) standardizing or varying CALT lengths, (c) sampling CALT items, (d) changing the difficulty of CALT items, (e) dealing with CALT item sets, (f) scoring CALTS, (g) dealing with CALT item omissions, (h) making decisions about CALT cut-points, (i) avoiding CALT item exposure, (j) providing CALT item review opportunities, and (k) complying with legal disclosure laws when using CALTs.The literature on computer-assisted language learning indicates that language learners have generally positive attitudes toward using computers in the classroom (Reid, 1986;Neu & Scarcella, 1991;Phinney, 1991), and a fairly large literature has developed examining the effectiveness of computer-assisted language learning (for a review, see Dunkel, 1991).But less is known about the more specific area of computers in language testing.The purpose of this article is to examine recent developments in language testing that directly involve computer use including what we have learned in the process.The article will also examine the dominant issue of computer-adaptive testing in the educational measurement literature in an attempt to forecast some of the directions future research on computers in language testing might take. CURRENT STATE OF KNOWLEDGE ON COMPUTERS IN LANGUAGE TESTINGIn reviewing the literature on computers in language testing, I have found four recurring sets of issues: (a) item banking, (b) computer-assisted language testing, (c) computer-adaptive language testing, and (d) the effectiveness of computers in language testing.The discussion in this section will be organized under those four headings. Item BankingItem banking covers any procedures that are used to create, pilot, analyze, store, manage, and select test items so that multiple test forms can be created from subsets of the total "bank" of items.With a large item bank available, new forms of tests can be created whenever they are needed.Henning (1986) provides a description of how item banking was set up for the ESL Placement Examination at UCLA. (For further explanation and examples of item banking in educational testing, see Baker, 1989, pp.412-414, or Flaugher, 1990.)While the underlying aims of item banking can be accomplished by using traditional item analysis procedures (usually item facility and item discrimination indexes; for a detailed description of these

This chapter discusses the challenge of English as a Lingua Franca (ELF) for language assessment. It introduces the issue of ELF and language assessment. The chapter connects the specific challenges of ELF with broader debates around the nature of communicative competence. It also discusses how research in language testing and assessment has addressed the ELF challenge thus far, showing that tangible progress has been slow and more work needs to be done. The chapter describes what an ELF construct for assessment purposes might look like, and how this construct could be operationalised through tasks. It concludes with future challenges for integrating ELF perspectives into language assessment. ELF assessment opens up the question of who is to be tested; removing the privileged status of the non-native speaker, who may have no natural advantage over a non-native speaker in achieving successful ELF communication.

Contents: Introduction. Part I: Important Social Contexts in Research on Second Language Teaching and Learning. Introduction. M.E. Brisk, Bilingual Education. M.R. Hawkins, ESL in Elementary Education. P.A. Duff, ESL in Secondary Schools: Programs, Problematics, and Possibilities. D.E. Murray, ESL in Adult Education. S. Carkin, English for Academic Purposes. P. Master, English for Specific Purposes. C. Roberts, English in the Workplace. B. Tomlinson, English as a Foreign Language: Matching Procedures to the Context of Learning. Y. Kachru, Teaching and Learning of World Englishes. Part II: Methods in Second Language Research. Introduction. L. Harklau, Ethnography and Ethnographic Research on Second Language Teaching and Learning. L. van Lier, Case Study. A. Lazaraton, Quantitative Research Methods. D. Nunan, Classroom Research. A. Burns, Action Research. Part III: Applied Linguistics and Second Language Research. Introduction. T. Pica, Second Language Acquisition Research and Applied Linguistics. S.L. McKay, Sociolinguistics and Second Language Learning. J. Zuengler, K.M. Cole, Language Socialization and Second Language Learning. G. Kasper, C. Roever, Pragmatics in Second Language Learning. J.P. Lantolf, Sociocultural and Second Language Learning Research: An Exegesis. N. Markee, Conversation Analysis for Second Language Acquisition. R.B. Kaplan, Contrastive Rhetoric. S. Conrad, Corpus Linguistics and L2 Teaching. Part IV: Second Language Processes and Development. Introduction. G. Ioup, Age in Second Language Development. R. DeKeyser, A. Juffs, Cognitive Considerations in L2 Learning. Z. Han, L. Selinker, Fossilization in L2 Learners. M. Swain, The Output Hypothesis: Theory and Research. E. Tarone, Speaking in a Second Language. M. Rost, L2 Listening. T.G. Wiley, Second Language Literacy and Biliteracy. P. Byrd, Instructed Grammar. D.E. Eskey, Reading in a Second Language. I.S.P. Nation, Teaching and Learning Vocabulary. J.S. Hedgcock, Taking Stock of Research and Pedagogy in L2 Writing. E. Hinkel, Analyses of Second Language Text and What Can Be Learned From Them. Part V: Methods and Curricula in Second Language Teaching. Introduction. S.J. Savignon, Communicative Language Teaching: Strategies and Goals. S. Fotos, Traditional and Grammar Translation Methods for Second Language Teaching. J. Williams, Form-Focused Instruction. M.A. Snow, A Model of Academic Literacy for Integrated Language and Content Instruction. R. Ellis, Instructed Language Learning and Task-Based Teaching. M. Celce-Murcia, E. Olshtain, Discourse-Based Approaches: A New Framework for Second Language Teaching and Learning. C.A. Chapelle, Computer-Assisted Language Learning. N.J. Anderson, L2 Learning Strategies. Part VI: Second Language Testing and Assessment. T. McNamara, Introduction. A.J. Kunnan, Language Assessment From a Wider Context. A. Davies, C. Elder, Validity and Validation in Language Testing. M. Chalhoub-Deville, C. Deville, A Look Back at and Forward to What Language Testers Measure. T. Lumley, A. Brown, Research Methods in Language Testing. D. Douglas, Testing Languages for Specific Purposes. C. Leung, Classroom Teacher Assessment of Second Language Development: Construct as Practice. Part VII: Identity, Culture, and Critical Pedagogy in Second Language Teaching and Learning. Introduction. T. Ricento, Considerations of Identity in L2 Learning. M. Byram, A. Feng, Teaching and Researching Intercultural Competence. S. Canagarajah, Critical Pedagogy in L2 Learning and Teaching. Part VIII: Language Planning and Policy and Language Rights. R.B. Baldauf, Jr., Introduction. R.B. Baldauf, Jr., Language Planning and Policy Research: An Overview. T. van Els, Status Planning for Learning and Teaching. A.J. Liddicoat, Corpus Planning: Syllabus and Materials Development. R.B. Baldauf, Jr., R.B. Kaplan, Language-in-Education Planning. D.E. Ager, Prestige and Image Planning. S. May, Language Planning and Minority Language Rights.

Advances in natural language processing (NLP) and automatic speech recognition and processing technologies offer new opportunities for language testing. Despite their potential uses on a range of language test item types, relatively little work has been done in this area, and it is therefore not well understood by test developers, researchers or users in language assessment. This paper introduces NLP for language assessment as an area of inquiry and practice by describing the historical roots coming from computational linguistics, statistical NLP, speech recognition and processing technologies, language assessment, and computer-assisted language learning. It outlines uses of NLP and speech recognition and processing technologies in language assessment through illustrations of current testing projects, and identifies areas in need of further development.

This study is a follow-up to previous research conducted in 2012 on computer-assisted language testing (CALT) that applied a survey approach to investigate the use of technology in sign language testing worldwide. The goal of the current study was to replicate the 2012 study and to obtain updated information on the use of technology in sign language testing. Another goal was to broaden the scope of the previous study by also considering the use of automatic sign language recognition and production (by avatars) in applied testing scenarios through technologies that make use of artificial intelligence (AI). Thirty-two sign language testing professionals participated in the current study. The results of the survey confirm the findings from 2012, but also raise new issues for future research (trends) related to sign language testing, such as the use of automatic sign language recognition for automatic scoring and the problem of acquiring funding for developing and maintaining web-based sign language testing platforms.

Commissioned as the lead paper in the Volume, this chapter shows how the terms assessment, testing, examining and evaluation have become increasingly intertwined, and multiple new terms have emerged. The chapter takes a somewhat historical view as it lays out a picture of the ways that the field of language ‘assessment’ has expanded its knowledge and skills but also its socio-political effects. Taking the notion of ‘purposes’ as a broad organizing principle, the chapter is intended to introduce new readers in this field to the wide range of issues current in the field, and prepare them for the more specific chapters that follow.

The rapid change and use of computer technology has had a profound influence in the field of language teaching and assessment, and the changes that computer and technology has brought to language assessment in particular are not only in the delivery method, but also in the scoring systems (Chapelle & Douglas, 2006). For instance, a wide variety of tasks be given as a test online worldwide, and scores can be available immediately to the test takers. However, along with such advances come potential problems and concerns.

This paper looks into the shifts from traditional to innovative and inclusive language testing methods, focusing on developments such as alternative assessment, AI integration, and computer-adaptive testing. This study employs a desk-based approach to the literature which synthesizes relevant theories in order to highlight the increased focus on fairness, inclusivity and authentic assessment. The key findings highlight that dynamic and performance-based assessments address the different learning needs of the learners; promote instrumentality towards language use and incorporate socio-political and ethical dimensions in test construction. The paper also underlines the transformative power of technology to improve access and efficiency considering the potential challenges and inequalities associated with its use. By integrating traditional and modern practices, this study contributes to both the theoretical discourse and practical advancements in language assessment that uphold equitable, ethical and effective testing methods consistent with twenty-first century educational standards and learner diversity.

The concept of literacy has expanded significantly in the course of the last decade to include computer literacy or science literacy in academic and public discourse. The recent focus of attention on classroom-based language assessment has brought the aspect of language assessment literacy (LAL) to the forefront, with foreign language teachers as one important group of stakeholders whose professionalization of this concept is important. The current paper aims at relating the findings from part of a large-scale study that was undertaken in seven different European countries in order to shed light on perceived LAL levels of foreign language teachers and their training needs in language testing and assessment. The focus will be on the qualitative part of the mixed methods study that examines foreign language school teachers’ perceptions in three different educational contexts, namely Cyprus, Germany and Greece. The paper closes with an outline of the implications for teacher development and directions for future research.

The article substantiates the directions of informatization of the modern educational process in the institution of general secondary education. The analysis of normative maintenance is carried out and the needs of science, education and culture in informatization of educational process are defined. The purpose of the article: to substantiate the theoretical foundations of informatization of educational institutions. Research methods are used: analysis of scientific developments on the problem and normative sources; generalization and systematization − to identify promising areas of informatization; generalization of semantic characteristics of directions − for formulation of conclusions. The main directions of use of computer and information technologies in the institution of general secondary education are characterized, among which Computer and information technologies as an object of study, computer as a means of teaching various disciplines, as a tool to support subject lessons and other activities, as a means of development and upbringing of the child, the use of computers to perform educational and real tasks and to implement various activities, Information Technology as a means of diagnosing various functional systems of the child's body. The following arguments regarding the use of computer technology in the educational process of the school are emphasized: computer technology contributes to the improvement of teaching and increase the effectiveness of teaching; Computer literacy should be developed as early as possible so that students get used to and can quickly adapt to the constant updating of computer technology; thanks to computer technology, including the Internet, the mobility of the educational process increases, namely: students can communicate with teachers and students of other educational institutions, as well as professionals from both domestic and other countries, without leaving their class; computer technologies promote the development of self-regulation and self-control, teach students to manage their intellectual activity. It is concluded that the advantages of informatization include the following: increased activation of students' learning activities and motivation to learn; the emphasis is shifted to the development of practical skills, the development of creative thinking, memory, etc., the creation of conditions for active cooperation, the development of creativity, to develop communication skills.

The main objective of this research is to obtain an overview of pre-service English teachers' perceptions regarding the use of digital technology in making English language tests in the language assessment process. This research uses a mixed-method research model that combines qualitative research models and quantitative models to obtain more comprehensive and in-depth data regarding pre-service English teachers' perceptions on the use of digital technology in creating English tests. The research data source will be selected purposively, namely, pre-service teachers who have completed the Language Assessment course in the English Language Education study program, Faculty of Language and Literature, Universitas Negeri Makassar. In the data collection process, researchers used interviews, questionnaires, and document reviews. Qualitative data will be analyzed based on the stages of data reduction, data presentation, and drawing conclusions or verification. Meanwhile, to analyze quantitative data, researchers will use descriptive statistical analysis. The result of this study reveals that the pre-service English teachers portrayed a positive attitude toward the use of digital technology to create English tests in language assessment

This study is a systematic examination of the open access status of research in two flagship language testing and assessment journals: Language Testing and Language Assessment Quarterly. Coding and analysing 898 articles, we investigated (a) the prevalence of open access in four aspects—open manuscripts, open materials, open data, and open code, and (b) the relationship between open access and various characteristics of research, tests, and researchers. Our study revealed a positive trend in the adoption of open access over time, with open manuscripts and materials showing notable increases. Open code and data have remained scarce, though with a recent uptick from a low base. Notably, logistic regression results suggest inequitable participation in open access as authors from the Global South were less likely to have open manuscripts. Recognising the potential role of flagship journals as trend and standard setters, we call on the field to (a) shift towards more equitable open access models, (b) balance intellectual property concerns with validation needs, (c) recognise open code and open data with protected access via dedicated badges, and (d) adopt Research Transparency Statements, a new reporting structure inclusive of methodological and epistemological differences in open research practices.

In Malaysia, the need for speech therapy services has increased rapidly with the development of anawareness of their value in working with children who have speech and language disorders. Formallanguage assessment tools for use with this population in local clinics are limited. Such assessmentsare also required as tools in research into the development of language in children in Malaysia.Therefore, the present work will address the need for applicable language assessment tests,particularly for Malay-speaking children, by investigating two language tests - the MalaysianDevelopmental Language Assessment Kit (MDLAK) and the Pragmatics Profile of EverydayCommunication Skills in Children (PPECSC).The main objectives of this investigation were to 1) determine the suitability of the MDLAK for usewith Malay-speaking children by investigating the administration procedures of this test, and byinvestigating whether the age criteria given in the test were consistent with the scores obtained bythe participant group; and 2) determine the suitability of the PPECSC (translated) for use withMalay-speaking children. In doing so, three studies were undertaken: 1) the preliminary study todesign and trial modifications to the MDLAK, to translate the PPECSC into the Malay language,and to design a methodology for language sampling for case studies; 2) the primary study to test thesuitability of the modified MDLAK (MDLAK-M) and the translated PPECSC (PPECSC-T) for usewith the Malay-speaking children; and, 3) case studies to determine, on a very small scale, if thebehaviours demonstrated in the MDLAK-M and reported in the PPECSC-T were reflected innatural free play sessions.Six children participated in the preliminary study, and 13 in the primary study. They were alltypically-developing, aged 3;0 to 5;11 years, and all had the Malay language as their primarylanguage in their home setting. Three of the children from the primary study were selected atrandom to be included in the case studies.The pilot study revealed that further modifications were required for the MDLAK, but that thetranslated version of the PPECSC was suitable. The primary study indicated variability in thechildren’s performance in the Receptive (RLA) and the Expressive Language Assessment (ELA)when compared to the age-equivalent skills given in the MDLAK. Descriptive statistical analyseswere undertaken for the children’s results when calculated using an arbitrary scoring systemdeveloped for the MDLAK-M. Spearman’s analyses revealed positive correlations between thereceptive and expressive scores, and between those scores and the children’s ages. In addition, anegative correlation between the scores and time in Australia was found. Mann-Whitney U analysisindicated no significant effect of gender on the scores. For the PPECSC-T, the analysis for thescores for the younger group of children revealed a significant negative correlation between thenumber of skills reported and time in Australia; but a non-significant positive correlation was notedbetween their total skills and age. For the older children, similar patterns, although not significant,were found between the variables as for the younger children. No significant gender effect wasfound for either group. The case studies indicated that many of the receptive, expressive andpragmatic language abilities investigated in the MDLAK-M and PPECSC-T were reflected in anatural setting of play.The results of the present project suggest that, with modifications, both the MDLAK-M andPPECSC-T are suitable for research purposes with Malay-speaking children. Further investigationsare needed, however, to establish psychometric information for both of the language assessmentsprior to clinical use.

For young AAE speakers, how useful is the Developmental Sentence Scoring (DSS) compared with Index of Productive Syntax (IPSyn) in identifying developmental language disorder (DLD) in the presence of African American English (AAE)? Structured Review. PsycInfo®, Education Source, Education Resources Information Center (ERIC), Communication & Mass Media Complete (CMMC), PubMed, Scopus, ASHAWire. (1) "African American English" OR "African American Language" OR "African American Vernacular English" OR "Black English" OR "AAE" OR "AAVE" AND (2) "child" AND (3) "language assessment" OR "language testing" OR "speech evaluation". 3. DSS and IPSyn appear to be dialect-neutral measures of morphosyntax in young AAE speakers. DSS was better able to detect morphosyntactic differences between children with typical language development (TLD) and children with DLD. DSS and its variant, Black English Sentence Scoring (BESS), appear to be clinically useful language sampling analysis tools. Available evidence suggests that DSS is a more useful clinical tool over IPSyn for evaluating DLD within the context of AAE because it provides the opportunity to evaluate mastery and accuracy of grammatical features and not only the presence of structures.

In 1998, when University College Northampton (UCN) selected WebBoard as its campus‐wide virtual learning environment (VLE) and encouraged its adoption, e‐learning became a central feature of the French Department's learning and teaching strategy. By September 2003, all the campus‐focused modules in French had become VLE‐resourced. In April 2004, we conducted a paper‐based survey to measure our students' views on (1) ‘e‐novation’ in general; and (2) computer‐assisted language testing (CALT) in particular, with a view to (1) evaluating the French Department's e‐ffectiveness; and (2) making student‐informed decisions for 2004–2005. This paper concerns itself, initially, with the origins of e‐learning in our French Department, the latter's evolving range of web‐based learning offerings as well as the educational principles underpinning our e‐learning strategy. It then presents selected results of our investigation and offers some tentative conclusions with regard to ‘e‐ffectiveness’.