Commentaries on Validity Issues in Foreign and Second Language Assessment

This study examines the level of Language Assessment Literacy (LAL) of English language teachers at the university level in Pakistan by adopting a mixed-methods explanatory approach. The study offers a comprehensive analysis of language assessment awareness among English language teachers in the Pakistani context along with their language teaching-learning experiences. Likewise, English language teachers shared their perspectives regarding the role of LAL in enhancing language teaching and learning processes at the university level. The theoretical framework of the study, which is comprised of three seminal theories, viz. Classical Test Theory, Generalizability Theory and Item Response Theory, has informed the interview protocol used for qualitative data collection. In the same vein, the current study investigates three core areas including the current level of LAL among university English teachers, the challenges they face while assessing their students’ language proficiency, and the possible solutions for these challenges for the improvement of language assessment, teaching and research practices. For the said purpose, data is collected from 15 English language teachers who are currently serving at five different public sector universities including Kohat University of Science and Technology (KUST) Kohat, Abdul Wali Khan University Mardan (AWKUM), National University of Modern Languages (NUML) Islamabad, FATA University, Kohat, and Quaid-e-Azam University (QAU) Islamabad, through semi-structured questionnaire and semi-structured interviews. Correspondingly, quantitative findings of the study reveal that most of the participants (73.33%) are PhD degree holders but many of them lack professional language assessment trainings both pre-service and post-service. The study highlights that English language teachers have limited exposure to standardized testing frameworks like CEFR, IELTS, and ELLT. Among them, only 13.33% had studied language assessment as a course at the PhD level while 46.66% of language teachers had not attended any training in the last five years of their professional life. Given the gaps between language teaching and assessment practices as the qualitative data highlights, language assessment trainings need to be offered to teachers and students at the university level with a view to enhance their theoretical as well as practical knowledge regarding language assessment which will furnish their language assessment skills. The major findings of the study also have significant implications for English Language Teaching (ELT) and assessment practices and language policies with a view to bridge the gulf between language teaching and assessment practices in multilingual contexts like Pakistan.

Young Indigenous students with a language background of emergent, unrecognised, and/or non-prestigious contact languages are commonly by-passed as English language learners (ELLs). Consequently, they are taught and assessed via undifferentiated mainstream pedagogy and testing, generally characterised by reported underachievement. Long-term social and linguistic minoritisation has effectively created this inequitable situation, excluding this cohort from standard school ELL processes such as on-entry identification, and specialist ELL services of assessment and needs-based support. In these challenging social-educational contexts, where, furthermore, knowledge about the varieties resulting from language contact and shift is still developing, initiating language assessment for these young Language Minority ELLs pivots on the discernment of non-specialist classroom teachers. This is clearly a high stakes assessment exercise for this otherwise invisible ELL cohort, as this evidence makes the case for language-informed responses, such as differentiated delivery of classroom curriculum. Prioritising the social context in language assessment use and drawing on exploratory practice principles, the study investigates teachers’ views on their use of the state mandated classroom-based proficiency tool (adapted to be inclusive of this Indigenous ELL sub-group). This serves to illuminate the kind of support teachers need to conduct identification and assessment for these Indigenous ELLs and to highlight the broad gaps in basic research. An analysis of the interviews elicits the “puzzles” facing classroom teachers. Findings include that direct elucidation of tool puzzles for L2 pathways in contact language contexts would be most useful, not a proliferation of extra tools nor a complexification of the current tool.

The rapid change and use of computer technology has had a profound influence in the field of language teaching and assessment, and the changes that computer and technology has brought to language assessment in particular are not only in the delivery method, but also in the scoring systems (Chapelle & Douglas, 2006). For instance, a wide variety of tasks be given as a test online worldwide, and scores can be available immediately to the test takers. However, along with such advances come potential problems and concerns.

The paper presents the results of the first stage of the research into the specifics of ChatGPT application in assessment development for English as a Foreign Language (EFL) courses. The article aims to investigate the perceptions of English language teachers regarding the utilization of ChatGPT in their teaching and assessment practices obtained through a survey in order to establish the degree of their familiarity with the possibilities of ChatGPT for teaching a foreign language and the mechanics of AI-generated test tasks design. The participants are mostly experienced English language instructors, affiliated with higher education institutions in Ukraine, the EU and the USA. Findings reveal varying levels of respondents’ confidence in ChatGPT's functionalities. A considerable proportion of educators appeared to be either unfamiliar with ChatGPT or possess limited knowledge of its capabilities. Hesitancy in implementing ChatGPT in teaching practice primarily stems from a lack of guidance as to its effective utilization closely followed by academic integrity concerns. However, educators who have employed ChatGPT recognize its potential as an assistant in lesson planning, content creation, task design, and assessment, especially for vocabulary, grammar, and writing. Despite acknowledging the tool's benefits, respondents express reservations about ChatGPT-generated tasks’ accuracy for assessment purposes. Challenges include the need for meticulous proofreading, issues with biased information, and insufficient task complexity. Nonetheless, overall satisfaction levels among educators implementing ChatGPT range from moderate to high, substantiating its value in foreign language teaching. Thus the survey stage of the research revealed the growing interest in ChatGPT’s potential for foreign language teaching and assessment among Ukrainian and international EFL and ESP instructors and underscored the need for guidelines and recommendations to create effective ChatGPT-generated test tasks. Future research will develop criteria for evaluating ChatGPT-generated tasks, provide prompt design guidelines, and formulate ethical policies for AI in assessment, addressing educators' concerns and

Gamification in English language learning comprises of game mechanics which involve points, badges, leader boards and rewards. It is one of the approaches that has been used by the educators to change the pace of lesson in learning English. This approach has been used in the class because of the benefits that it can offer in English language learning. This paper focuses on the use of gamification in English language teaching (ELT) and learning by looking at the advantages it has brought to language skills. Gamification can be used in the teaching of the different skills in English and in terms of its benefits to English language learning, gamification makes learning more effective by making grammar learning more pleasurable, vocabulary memorisation more successful, communication and cooperation easier to be practiced and others. It is hoped that this paper will add to the literature on the applications of gamification in ELT and lead to the betterment of ELT in second language setting particularly in Malaysia.

Contents: Introduction. Part I: Important Social Contexts in Research on Second Language Teaching and Learning. Introduction. M.E. Brisk, Bilingual Education. M.R. Hawkins, ESL in Elementary Education. P.A. Duff, ESL in Secondary Schools: Programs, Problematics, and Possibilities. D.E. Murray, ESL in Adult Education. S. Carkin, English for Academic Purposes. P. Master, English for Specific Purposes. C. Roberts, English in the Workplace. B. Tomlinson, English as a Foreign Language: Matching Procedures to the Context of Learning. Y. Kachru, Teaching and Learning of World Englishes. Part II: Methods in Second Language Research. Introduction. L. Harklau, Ethnography and Ethnographic Research on Second Language Teaching and Learning. L. van Lier, Case Study. A. Lazaraton, Quantitative Research Methods. D. Nunan, Classroom Research. A. Burns, Action Research. Part III: Applied Linguistics and Second Language Research. Introduction. T. Pica, Second Language Acquisition Research and Applied Linguistics. S.L. McKay, Sociolinguistics and Second Language Learning. J. Zuengler, K.M. Cole, Language Socialization and Second Language Learning. G. Kasper, C. Roever, Pragmatics in Second Language Learning. J.P. Lantolf, Sociocultural and Second Language Learning Research: An Exegesis. N. Markee, Conversation Analysis for Second Language Acquisition. R.B. Kaplan, Contrastive Rhetoric. S. Conrad, Corpus Linguistics and L2 Teaching. Part IV: Second Language Processes and Development. Introduction. G. Ioup, Age in Second Language Development. R. DeKeyser, A. Juffs, Cognitive Considerations in L2 Learning. Z. Han, L. Selinker, Fossilization in L2 Learners. M. Swain, The Output Hypothesis: Theory and Research. E. Tarone, Speaking in a Second Language. M. Rost, L2 Listening. T.G. Wiley, Second Language Literacy and Biliteracy. P. Byrd, Instructed Grammar. D.E. Eskey, Reading in a Second Language. I.S.P. Nation, Teaching and Learning Vocabulary. J.S. Hedgcock, Taking Stock of Research and Pedagogy in L2 Writing. E. Hinkel, Analyses of Second Language Text and What Can Be Learned From Them. Part V: Methods and Curricula in Second Language Teaching. Introduction. S.J. Savignon, Communicative Language Teaching: Strategies and Goals. S. Fotos, Traditional and Grammar Translation Methods for Second Language Teaching. J. Williams, Form-Focused Instruction. M.A. Snow, A Model of Academic Literacy for Integrated Language and Content Instruction. R. Ellis, Instructed Language Learning and Task-Based Teaching. M. Celce-Murcia, E. Olshtain, Discourse-Based Approaches: A New Framework for Second Language Teaching and Learning. C.A. Chapelle, Computer-Assisted Language Learning. N.J. Anderson, L2 Learning Strategies. Part VI: Second Language Testing and Assessment. T. McNamara, Introduction. A.J. Kunnan, Language Assessment From a Wider Context. A. Davies, C. Elder, Validity and Validation in Language Testing. M. Chalhoub-Deville, C. Deville, A Look Back at and Forward to What Language Testers Measure. T. Lumley, A. Brown, Research Methods in Language Testing. D. Douglas, Testing Languages for Specific Purposes. C. Leung, Classroom Teacher Assessment of Second Language Development: Construct as Practice. Part VII: Identity, Culture, and Critical Pedagogy in Second Language Teaching and Learning. Introduction. T. Ricento, Considerations of Identity in L2 Learning. M. Byram, A. Feng, Teaching and Researching Intercultural Competence. S. Canagarajah, Critical Pedagogy in L2 Learning and Teaching. Part VIII: Language Planning and Policy and Language Rights. R.B. Baldauf, Jr., Introduction. R.B. Baldauf, Jr., Language Planning and Policy Research: An Overview. T. van Els, Status Planning for Learning and Teaching. A.J. Liddicoat, Corpus Planning: Syllabus and Materials Development. R.B. Baldauf, Jr., R.B. Kaplan, Language-in-Education Planning. D.E. Ager, Prestige and Image Planning. S. May, Language Planning and Minority Language Rights.

The English language functions as a global language that facilitates communication among people of different lingua-cultures. This background leads to the question of whether the traditional language assessment still fulfils the needs of the majority of language learners who will use English for various purposes with people from different linguistic and cultural backgrounds. This article illustrates the development and spread of the English language by focusing on two key paradigms: World Englishes (WE) and English as a lingua franca (ELF). This article argues that a native-oriented means of English pedagogy and assessment does not fit the current functions of the language and cannot meet the various and complex needs of the majority of English language learners. It therefore emphasizes the importance of an ELF-informed approach, such as the ‘post-method’ approach to English language teaching and more flexible language assessment focusing on students’ performance of tasks. The article concludes that English language teaching and assessment need to be more informed by ELF than by the entrenched, anachronistic native-oriented ideology.

Traditional language assessments have often relied on standardized tests, which emphasize discrete skills and overlook real-life language use. These assessments frequently result in test anxiety and reduced motivation among English language learners (ELLs). To address these challenges, policymakers in countries such as Finland, Australia, Japan, and Malaysia have implemented Classroom-Based Language Assessment (CBLA). This approach emphasizes teachers’ professional judgment, ongoing monitoring, and feedback. However, concerns persist regarding the subjectivity and reliability of CBLA in measuring language proficiency. It is therefore essential to examine whether CBLA is an effective alternative despite these criticisms. This scoping review investigates the impact of CBLA on ELLs’ language proficiency. Following Arksey and O’Malley’s framework for conducting scoping reviews, 15 studies published between 2011 and 2024 were retrieved from the Web of Science and Scopus databases, with additional relevant studies sourced from ERIC and JSTOR. The review focused the impact of CBLA on various areas of ELLs’ language proficiency. Key areas examined included language skills, grammar and vocabulary knowledge, and intercultural competence. The findings indicate that the effectiveness of CBLA depends on ELLs’ proficiency, age, targeted skills taught, and classroom settings. Future mixed-method studies should combine observation of classrooms, interviews of teachers and ELLs to develop CBLA models that promote effective language development for ELLs in diverse settings.

Two realities in American education in the last decade have combined to dramatically affect the professional lives of teachers-a school population that is increasingly more diverse in its makeup, and a national agenda for accountability that requires attention to the achievement of all students. The changing demographics of American schools, particularly an increase in the number of students for whom English is a new language, is part of the national conversation in education. Growth in the number of linguistically diverse learners has increased rapidly across the nation, particularly in the Southeast and Midwest. During the period between 1995 and 2005 the national growth rate of the K-12 English language learner (ELL) population was 61% while the overall growth rate for all students enrolled in public schools was 2.6% (National Clearinghouse for English Language Acquisition, Language Enhancement, and Academic Achievement for Limited English Proficient Students (NCELA), 2006). During the same period in Minnesota, the location of programs discussed in this chapter, ELL growth was over 161%, outpacing the national growth trend significantly, while the state total K-12 enrollment decreased by 6.5% (NCELA, 2006). The number of foreign-born immigrants doubled in the 1990s, from 2.6% of the state population to 5.8%. Minnesota has long been a state that welcomes refugees, and the refugee population has always been a significant portion of Minnesota immigrants. Between 2000 and 2005, the state resettled 21,743 refugees. Minnesota ranked second among states in refugee immigrant arrivals in 2004-2006. The Hmong population in Minnesota (estimated at 60,000) is second in size only to the California Hmong population, and the state is now home to 30,000 Somali residents-the largest community of Somalis in the United States. More than 60,000 ELL students are currently enrolled in Minnesota schools, representing more than 110 different language groups. Minnesota’s growing numbers of ELL and refugee students present educators with new challenges. The increase in refugee students and the fact that many of them have little formal education before arriving in Minnesota coincided with the full introduction of Minnesota’s accountability system, developed under Title I, and the new accountabilityprovisions in Title III, the federal education program for ELL students. Sadly, as suggested by Ruiz-de-Velasco and Fix (2000), the needs of immigrants with “significant education gaps” are simply overlooked-and correspondingly, drop-out rates are very high. The growth of second language learners statewide has led to a gap in achievement between these learners and other students that is of concern to both educators and policy-makers. Are teachers prepared for these challenges? The impact of increasing numbers of second language learners on teachers ill-prepared to meet their needs is substantial in many communities (Walker, Shafer, & Iiams, 2004). The pressure to meet achievement goals for all subgroups of learners requires that individual teachers as well as schools take responsibility for learning outcomes demonstrated by English language learners-a direct contrast to the historical practice of leaving this work exclusively to ESL and bilingual specialists. The task has fallen to in-service education to provide opportunities for practicing content and grade-level teachers to develop the expertise to successfully teach ELLs. At the same time, colleges and universities are faced with the daunting task of preparing pre-service teachers to attend to these language learners in their future classrooms. What is required for effective teacher development in both pre-service and in-service settings? In this chapter we examine the challenges inherent in preparing both groups to meet the complex linguistic and academic needs of second language learners by describing two very specific models of engagement with educators. We first describe the structure and content of a course module on English language learners at the University of Minnesota designed specifically for different K-12 pre-service content area programs. Data from a cohort in science education provide a sense of the challenges and outcomes of this approach. We then describe a professional development initiative designed for elementary school staff that was site-based, collaborative, and long-term (two years). Finally, as teacher educators, we offer thoughts on what other teacher development programs can do to maximize efforts to improve teacher learning regarding the education of English language learners.

Qatar is a Gulf state undergoing a sweeping educational reform where educational development is linked to ICT and the English language. ICT is promoted to upskill the future generation or the “digital natives”. English is seen as a key to a myriad of doors and a passport to a world class education. In all public schools, English is a compulsory subject taught from grade one where learners are exposed to English for more than 500 teaching hours every year. It is also the medium of instruction for maths and science. The purpose of my year-long study was to investigate the effectiveness of ICT in supporting the learning and teaching of English language in the Qatari context. It was guided by the critical paradigm and employed action research as the investigative methodology. The initial data collection process was carried out through a survey instrument. More than 200 students answered questions about their attitudes, perceived values, current uses and issues with ICT. The findings of the study have identified a number of challenges. One of which is the minimal support that ICT is providing to the teaching of the listening skill. An intervention programme was later designed in a form of a webquest lesson based on CLIL (content and language integrated learning) and informed by the survey findings, student’ voices and previous research in the field. After that, the intervention programme was trialed with two classes (n=50) and evaluated by qualitative (classroom observations) and quantitative instruments (questionnaires). Evidence from the evaluation data showed that ICT has the potential to support the teaching and learning of English language through its positive impact on autonomy and motivation. The results also revealed that teachers have limited knowledge of the potential of ICT and they could play an indirect role in impeding its use. This study has shown how action research, as a self-reflective inquiry, could broaden our understanding of the role of ICT in language classes. The study concluded that ICT-supported teaching of English in Qatar should be accompanied by a positive ICT culture, reflective research, proper staff training, teacher networking and supportive educational leadership.

ENGLISH LANGUAGE LEARNING AND TECHNOLOGY: LECTURES ON APPLIED LINGUISTICS IN THE AGE OF INFORMATION AND COMMUNICATION TECHNOLOGY. Carol Chapelle. Amsterdam: Benjamins, 2003. Pp. xv + 211. $39.95 paper. In this volume compiled from several years of recent lectures, Chapelle synthesizes research on technology and language learning to lay bare theoretical questions of SLA and second language (L2) assessment that can be reformulated, pushed forward, and reevaluated through technology. She argues that, in order to do so, it is necessary that the field of applied linguistics “engage more consciously and proactively with the complex language-technology reality in which the profession is working” (p. xii) and that technology must not be viewed simply as a tool of efficiency in L2 research and assessment.

Diagnostic language assessment has received increased research interest in recent years, with particular attention on methods through which diagnostic information can be gleaned from standardized proficiency tests. However, diagnostic procedures in the broader sense have been inadequately theorized to date, with the result that there is still little agreement on precisely what diagnosis in second and foreign language learning actually entails. In order to address this problem, this article investigated how diagnosis is theorized and carried out in a diverse range of professions with a view to finding commonalities that can be applied to the context of language assessment. Ten semi-structured interviews were conducted with professionals from the fields of car mechanics, IT systems support, medicine, psychology and education. Data were then coded, yielding five macro-categories that fit the entire data set: (i) definitions of diagnosis, (ii) means of diagnosis, (iii) key players, (iv) diagnostic procedures, (v) treatment/follow-up. Based on findings within these categories, a set of five tentative principles of diagnostic language assessment is drawn-up, as well as a list of implications for future research.

Language ideologies play a pivotal role in shaping language use and language learning, particularly in multilingual and multicultural regions such as Chongzuo City. This paper explores the parental language ideologies in English and the intricate relationship between parental language ideologies in English and their impact on familial factors within the context of Chongzuo City in the Guangxi border areas. Through a comprehensive analysis of quantitative data obtained from questionnaires, this study delves into the various language ideologies held by parents residing in this region, focusing specifically on the English language. The findings reveal that parents have a somewhat negative perception of English as a potentially disruptive language. However, they also acknowledge its importance for future opportunities and recognize the need to understand its cultural context. Conversely, parents hold a positive view of English as a language that embraces diversity. They believe that English has evolved through interactions with various languages and cultures. Parents perceive English as relevant but not highly practical in their daily lives within the local context. They feel that their children's bi-lingual or multilingual skills are not effectively developed through English language learning. Additionally, the study shows that parental education and interactions with schools play a significant role in shaping perceptions of how family language ideology affects the learning of English. Other family-related factors do not appear to have a significant impact. Based on the findings, schools, community organizations, and educators may implement recommendations for career counseling, cultural context education, promotion of linguistic diversity and cultural integration, contextualization of English learning, integration of English and local language learning, and family-school collaboration to enhance language ideology.

English language learner (EL) status has high stakes implications for determining when and how ELs should be evaluated for academic achievement. In the US, students designated as English learners are assessed annually for English language proficiency (ELP), a complex construct whose conceptualization has evolved in recent years to reflect more precisely the language demands of content area achievement as reflected in the standards of individual states and state language assessment consortia, such as WIDA and ELPA21. The goal of this paper was to examine the possible role for and utility of using content area assessments to validate language proficiency mastery criteria. Specifically, we applied mixture item response models to identify two classes of EL students: (1) ELs for whom English language arts and math achievement test items have similar difficulty and discrimination parameters as they do for non-ELs and (2) ELs for whom the test items function differently. We used latent class IRT methods to identify the two groups of ELs and to evaluate the effects of different subscales of ELP (reading, writing, listening, and speaking) on group membership. Only reading and writing were significant predictors of class membership. Cut-scores based on summary scores of ELP were imperfect predictors of class membership and indicated the need for finer differentiation within the top proficiency category. This study demonstrates the importance of linking definitions of ELP to the context for which ELP is used and suggests the possible value of psychometric analyses when language proficiency standards are linked to the language requirements for content area achievement.

This research article attempts to address the politics of English language assessment in Pakistani context. The focus of this qualitative research study is to unfold ideological factors, socio-cultural factors, political factors, economic agendas, societal elements, issues of power, teachers’ roles, educational considerations, and hierarchical structures in Pakistani context with special reference to English language assessment system. Thematic Document analysis is used for analyzing the data collected from one of the leading newspapers in Pakistan i.e. Dawn. It is found out that the politics of language assessment is pluralized in Pakistan owing to decide which strata of the society is failing (the lower class) and which one is succeeding (the upper class, the elite class) in political power. This study also reveals that politics of language assessment in Pakistan is firmly linked with the purposes of English language learning i.e. lucrative jobs, education and business. Furthermore, the findings of the study highlight that in the current scenario of Pakistan, assessment and testing in general education and language assessment in particular play a significant role in imposing political policies and power in the arena of academia and intelligentsia. Likewise, the major findings of the study exhibit that there is a hegemony of English that brought linguistic imperialism in Pakistan. In Pakistani context, proficiency in English language decides future of the candidates in term of jobs and businesses. English language assessment regime is affected by the ‘top-down’ policy driven initiatives rather than ‘bottom-up’ collaborative work. It is the domain of a particular group of experts who decide Others as opposed to us. It also contributes to the marketization of English language in educational sector of Pakistan. The elite class in Pakistan set targets for the educational system in order to implement their policies to achieve political power and interests.