Comprehensive Frameworks of School Learning in ILSAs

This chapter synthesizes the existing evidence on the role of socioeconomic status (SES) in determining educational performance, attainment, and a large variety of adult outcomes. Special attention is given to studies that have used International Large-Scale Assessments (ILSAs) to document the intensity of the associations between these outcomes and SES, to allow their comparison across countries, or to highlight conceptual and methodological issues associated with the SES construct. An analysis of the associations between various education and labor market outcomes by one proxy of SES – mothers’ educational attainment – using two ILSAs – PISA 2021 and PIAAC 2021 – is provided to highlight that significant inequities exist for different age groups and across selected countries, and, given these, to suggest that different countries may have provided more equal opportunities to the various generations at different points in time. The chapter stresses the value of adopting an equity-in-inputs angle, which discusses SES’s direct influence on outcomes and its association with other factors that predict outcomes, when examining gaps in outcomes in school and adulthood. In combination with the wealth of data in ILSAs, this approach can support policy research in the quest for strategies to improve equity in our education systems and in our societies.

International large-scale assessments (ILSAs) require national governments to invest significant resources in both time and money. With such investment national governments expect the results of ILSAs to provide policy and research communities with evaluative information on their educational system. Armed with this evaluative information, policymakers in many participating countries have used the results to stimulate reform. However, systematically tracking this influence and judging the validity of the claims has proven difficult for both the policy and research communities. There can be an erroneous expectation in the wider education community, and sometimes amongst policymakers themselves, that ILSA data automatically suggests policy solutions. Because of this error, a means for better systematizing the policymaking process responding to ILSA results is required. The model developed here can assist the policy and research community to better understand whether ILSAs are providing valid evidence to support their use in policy formation enactment and can be used to analyze ongoing consequences of that influence. Two worked examples demonstrate the utility of the model.

Education systems can be conceptualized as the scaffolding that supports the construction and development of student competences. Among other things, the size, location, and learning resources of schools can affect how efficient that system is at delivering the required support. Data from international large-scale assessments have indicated that the resources of rural schools may differ from those of urban schools; students in schools in urban and more economically developed environments often demonstrate higher achievement. Data from IEA’s Trends in International Mathematics and Science Study (TIMSS) 2019 from across the Dinaric region provides information on variations in the size of schools and allocation, student achievement, and the different kinds of scaffolding/support for learning in urban and rural schools. Secondary analyses of the TIMSS 2019 data for the Dinaric region, taking into account home and school factors, show that the types of support available for student learning differed between urban and rural schools. The findings suggest that policymakers should focus on improving the learning resources available to rural schools across the region, particularly in response to their lack of technological resources for developing students’ digital competencies. Concomitant investment is required for the development of teachers’ competencies for the effective use of such educational resources. Educators need to compensate for lack of family support for some students; in such situations, schools need to enhance the scaffolding for learning available to children.KeywordsAchievement gapGrade fourLocation of schoolScaffoldingTrends in International Mathematics and Science Study (TIMSS)

International large-scale assessments, such as PISA and TIMSS, have had a profound effect on educational policy and school accountability. In an open letter to the OECD in 2014, a large number of academics expressed their concerns over the continuous cycle of global testing and how it negatively affects student’s well-being and impoverishes classrooms, as it inevitably involves more and longer batteries of multiple-choice testing, more scripted “vendor”-made lessons, and less autonomy for teachers. Others have pointed to the ways in which international large-scale assessments have opened up education systems and built country’s assessment capacities. This chapter reviews the ways in which international standardized assessments permeate national policy and school-level decision-making, drawing on the sociology of numbers to understand the appeal of standardization and benchmarking. The OECD initiative “PISA for Schools” is described and how it potentially extends the influence of international assessments on the classroom. An alternative project within the OECD (the “Strategic Education Governance” project) is presented and how the approach of case studies and learning seminars allows countries to learn more holistically about how to hold schools accountable and improve education. Openness, situational awareness, and a consideration for the complexity of successful reform are presented as necessary “habits of mind” to successfully improve learning outcomes. The final conclusion reflects on the limitations and potential usage of international large-scale assessments for effective education reform.KeywordsAccountabilityCommensurationStandardizationEducational governance

Research related to the “teacher characteristics” dimension of teacher quality has proven inconclusive and weakly related to student success, and addressing the teaching contexts may be crucial for furthering this line of inquiry. International large-scale assessments are well positioned to undertake such questions due to their systematic sampling of students, schools, and education systems. However, researchers are frequently prohibited from answering such questions due to measurement invariance related issues. This study uses the traditional multiple group confirmatory factor analysis (MGCFA) and an alignment optimization method to examine measurement invariance in several constructs from the teacher questionnaires in the Trends in International Mathematics and Science Study (TIMSS) 2015 across 46 education systems. Constructs included mathematics teacher’s Job satisfaction, School emphasis on academic success, School condition and resources, Safe and orderly school, and teacher’s Self-efficacy. The MGCFA results show that just three constructs achieve invariance at the metric level. However, an alignment optimization method is applied, and results show that all five constructs fall within the threshold of acceptable measurement non-invariance. This study therefore presents an argument that they can be validly compared across education systems, and a subsequent comparison of latent factor means compares differences across the groups. Future research may utilize the estimated factor means from the aligned models in order to further investigate the role of teacher characteristics and contexts in student outcomes.

This paper seeks to further conceptualise participation in International Large-Scale Assessment (ILSA) as a global ritual of belonging and apply it to understand the appeal of PISA for Development (PISA-D) in Ecuador and Paraguay. The paper develops the theoretical underpinnings of the global ritual of belonging through policy borrowing and lending theory. The paper draws on ethnographic observations and interviews carried out with high level policy actors in Ecuador and Paraguay during their first implementation of an OECD education programme to understand how global policy tools resonate locally. Beyond demonstrating global accountability, mobilizing resources, acquiring technical capacity, attracting investors, and accessing the transnational accreditation PISA provides educational systems, it emerges that PISA-D is valued for the global stage it provides and the relationship it entails with the OECD. In the era of ILSAs, having a relationship with the OECD and appearing on the OECD maps provides governments with a form of legitimization and denotes states as modern and responsible.

The politics of ethnic identification involving the three major ethnic groups of Malay-Muslims, Chinese and Indians have resulted in power imbalances and hierarchies along the various social, educational and economic dimensions. These power dynamics have in turn produced a Malaysian education and schooling system that is highly politicised and ethnicised. This chapter examines the ways in which British colonial history, Malaysian state policies, contemporary ethnic politics and globalisation are played out within the Malaysian schooling and education system in relation to the inter and intra dynamics of the three major ethnic groups of Malay-Muslims, Chinese and Indians. Practices of social exclusion and inclusion within the education and schooling system will be discussed in relation to state policies such as the 1970 National Economic Policy (NEP), 1990 National Development Policy (NDP), 2001 New Vision Policy (NVP), and the National Education Policy. The ways in which the different types of Malaysian schools are ethnicised will be also examined. These schools vary in terms of Government funding, ethnic community support, resources and future educational opportunities. These schools are also used strategically by the Government and the ethnic collectives in the social and political positionings of these ethnic groups in contemporary Malaysia. The chapter begins with an overview of the politics of ethnicity in contemporary Malaysia as this provides the historical and socio-cultural context of this chapter.

Although the impact of education policy reform could inform future policymaking initiatives, studies on education policy reform of student assessment are considered scarce. This study attempts to scrutinise the drivers, impacts, and challenges of education policy reform of the national assessment system in Malaysia based on policy documents and previous empirical studies. The findings indicated that the education policy reform of the Malaysian national assessment system is driven by the decline of pupils’ performance in international large-scale assessments. There are changes in content, format, and structure of the public examinations and assessment delivery procedures. The emergence of school-based assessment shows a paradigm shift from cognitive learning to a more holistic manner. Challenges arise included teachers’ extra workload, teachers’ readiness, lack of reference materials, and time constraints in implementing school-based assessment. Findings inferred the needs in refining the monitoring and coaching system in order to ensure the student assessment implementation does not diverge from the missions of education policy reform. This study is hoped to provide useful insights for future policymaking initiatives, particularly in Malaysia. Keywords: Malaysia, policy reform, public examination, school-based assessment, student assessment. Cite as: Chin, H., Thien, L. M., & Chew, C. M. (2019). The reforms of national assessments in Malaysian education system. Journal of Nusantara Studies, 4(1), 93-111. http://dx.doi.org/10.24200/jonus.vol4iss1pp93-111

This paper explores the potential of abductive reasoning to enhance the analysis of international large-scale assessments which have traditionally relied on deductive and inductive reasoning. While these conventional methods have provided valuable insights into global student achievement, they often fail to capture the complexity of educational systems and the nuanced interplay of variables influencing student outcomes. By proposing abductive reasoning as a complementary analytical approach, this study argues for its ability to account for surprising data points and embrace the complexity inherent in educational systems. Abductive reasoning, characterized by the formulation of the most plausible explanations for observed phenomena, offers a promising avenue for generating new hypotheses and uncovering unexpected relationships. Through a detailed examination of the strengths and weaknesses of inductive and deductive reasoning in the context of ILSA data and the introduction of abductive reasoning’s unique characteristics, this paper highlights the benefits and practical applications of abduction in educational research. It also addresses potential limitations and challenges, advocating for a structured application to enhance rigor and mitigate biases. Ultimately, abductive reasoning is presented as a versatile and creative tool that, when used thoughtfully alongside traditional methods, can significantly advance our understanding of global education systems and inform more nuanced and impactful educational policies.

In international large-scale assessments, student performance comparisons across educational systems are frequently done to assess the state and development in different domains. These results often have a large impact on educational policy and on the perceptions of an educational system’s performance. Early assessments, such as the First and Second International Science Studies (FISS and SISS), have been used alongside recent studies to create unique scales for investigating changes in constructs. The implicit assumptions in system comparisons are that the measures are valid, reliable, and comparable. However, these assumptions have not always been investigated thoroughly. This study aims to investigate the validity and cross-system comparability of scores from the FISS and SISS, conducted by the International Association for the Evaluation of Educational Achievement in 1970–1971 and 1983–1984. Findings based on item response theory (IRT) modeling indicate that scores in most educational systems can be viewed as reliable measures of a single science construct, supporting the validity of test score interpretations in these educational systems individually. In a robust assessment of measurement invariance using standard IRT methods, an alignment-based method, and the root mean square difference (RMSD) fit statistic, we demonstrate that measurement invariance is violated across systems. The alignment-based method identified a well-fitting model with complex restrictions but no items exhibited invariance across all systems, a result supported by the RMSD statistics. These results question the appropriateness of score comparisons across systems in FISS and SISS. We discuss the implications of these results and outline consequences for score comparisons across time.

The purpose of this study was to determine the effect of industrial tax exemptions in the calendar year 1970 on local school revenue in Louisiana for the fiscal year 1970-1971* More specifically, the purpose was to determine and identify by school systems: (1) the extent of industrial tax exemptions, (2) the amount of revenue not available from the parishwide ad valorem Constitutional 5-Mill Tax because of industrial tax exemptions, (3) the amount of revenue not available from other parish-wide ad valorem school taxes because of industrial tax exemptions, (4) the amount of revenue not available from parish-wide ad valorem school taxes levied for bond redemption because of industrial tax exemptions. The population used in this investigation consisted of all local school systems of the State of Louisiana, except Orleans, Ouachita, and Washington Parishes and the city school systems of Bogalusa and Monroe. All data were secured from the official records and documents of the Office of the State Comptroller of Louisiana and the Louisiana Revised Statutes of 1950, as amended. Two assumptions were made: first, that one per cent of the ad valorem tax revenue would be uncollect ible and second, that exempted manufacturing plants would be vii (4) Twenty local school boards levied a parish-wide Maintenance Tax. These local school systems would have received $676,069 in additional revenue for maintenance of buildings and grounds. (5) Six local school boards levied a parish-wide Con struction or Improvement Tax. These local school systems would have received $1,013,440 in additional revenue which could have been used for construction. (6) Thirty local school boards levied a parish-wide school bond redemption tax. The increase in assessed valu ation for tax purposes would have permitted a significant reduction in millage levied for bond redemption. (7) Forty-nine of the sixty-one school systems included in this study would have derived increased revenue from parish-wide ad valorem school taxes, ranging from a low of one-tenth of one per cent in St. Helena to a high of 33$ per cent in Ascension.

This article challenges the dominant notion of the ‘high-performing education system’ and offers an alternative interpretation from a Daoist perspective. The paper highlights two salient characteristics of such a system: its ability to outperform other education systems in international large-scale assessments; and its status as a positive or negative ‘reference society’. It is contended that external standards are applied and imposed on educational systems across the globe, judging a system to be high- or low- performing, and consequently worthy of emulation or deserving of criticism. Three cardinal Daoist principles that are drawn from the Zhuangzi are expounded: a rejection of an external and oppressive dao (way); the emptying of one’s heart-mind; and an ethics of difference. A major implication is a celebration of a plurality of high performers and reference societies, each unique in its own dao but converging on mutual learning and appreciation.

School Reform: The District Policy Implications of the Effective Schools Literature

Much attention has been given to the Organization for Economic Co-operation and Development’s (OECD) Program for International Student Assessment (PISA) and its effects in different education systems, including the United States. This paper begins to address a major gap in this literature by focusing on the costs of participation in PISA. The paper draws on content analysis of official documents and budgets, as well as qualitative interviews with select government officials at federal and state levels. It details costs in three categories: direct overhead to the OECD, national implementation, and in-kind or human burden. As such, the paper details information on costs of PISA, important for eventual cost–benefit frameworks that may be useful for continued policy deliberations over US participation in PISA and other international large-scale assessments.

Large-scale international assessments, including PISA, might be useful for countries to receive feedback on their education systems. Measurement invariance studies are one of the active research areas for these assessments, especially cross-cultural and linguistic comparability have attracted attention. PISA questions are prepared in the English language, and students from many countries answer the translated form. In this respect, the purpose of our study is to investigate whether there is a measurement invariance problem across native English and non-native English speaker groups in the PISA-2015 reading skills subtest. The study sample included students from Canada, the USA, and the UK as the native speaker group and students from Japan, Thailand, and Turkey as the non-native speaker group. Measurement invariance studies taking into account the binary structure of the data set for these two groups revealed that eight of the twenty-eight items in the PISA-2015 reading skills test had possible limitations in equivalence