Using multimodal learning analytics as a formative assessment tool: Exploring collaborative dynamics in mathematics teacher education

The study focuses on two areas of chemistry education: misconceptions, which are undesirable elements of the learning process, and an ever-present problem in chemistry education; and the second area is classroom assessment, with an emphasis on formative assessment, which is still poorly applied in the practice of Slovak schools. In our paper, we aim to provide a brief overview of misconceptions in chemistry education and to characterize selected formative classroom assessment tools (FACT) that have also been instruments of empirical research exploring misconceptions. The paper also aims to present the results of empirical research aimed at exploring misconceptions about physical and chemical change using the selected formative assessment tool. The target population consisted of pupils aged 14–15 (N = 148, 63 males and 85 females). As a research tool, we used the FACT called “Always - Sometimes - Never” (A-S-N) (Keeley, 2015). Due to this formative assessment tool, we identified several alternative conceptions or misconceptions of the pupils described in other studies, and confirmed their existence in chemistry education.

A distinctive feature of game‐based learning environments is their capacity to create learning experiences that are both effective and engaging. Recent advances in sensor‐based technologies such as facial expression analysis and gaze tracking have introduced the opportunity to leverage multimodal data streams for learning analytics. Learning analytics informed by multimodal data captured during students’ interactions with game‐based learning environments hold significant promise for developing a deeper understanding of game‐based learning, designing game‐based learning environments to detect maladaptive behaviors and informing adaptive scaffolding to support individualized learning. This paper introduces a multimodal learning analytics approach that incorporates student gameplay, eye tracking and facial expression data to predict student posttest performance and interest after interacting with a game‐based learning environment, Crystal Island . We investigated the degree to which separate and combined modalities (ie, gameplay, facial expressions of emotions and eye gaze) captured from students ( n = 65) were predictive of student posttest performance and interest after interacting with Crystal Island . Results indicate that when predicting student posttest performance and interest, models utilizing multimodal data either perform equally well or outperform models utilizing unimodal data. We discuss the synergistic effects of combining modalities for predicting both student interest and posttest performance. The findings suggest that multimodal learning analytics can accurately predict students’ posttest performance and interest during game‐based learning and hold significant potential for guiding real‐time adaptive scaffolding.

The researcher conducted a qualitative study to develop a deeper understanding of teacher perceptions use, and understanding of homework as a formative assessment tool. The work of Black et al. (2004), Moss and Brookhart (2009), Stiggins et al. (2006) provided a lens to improve understanding of formative assessment. Cooper (2007), Kohn (2006), and Vatterott (2009) provided a lens to improve understanding of homework. Participants were teachers from three Midwestern high schools in one Midwestern state. Data were collected from focus group interviews, an online open-ended survey, and document analysis. All segments of the participants identified three categories as related to perception, use, and understanding homework as a formative assessment tool: teacher perception of the purpose of homework, the uses of homework as part of classroom assessment practice, and teacher understanding of homework as a formative assessment tool. The findings of this study have implications for teachers and administrators. One of the findings of this study was teachers formulate their perception of homework on assumptions embedded in educational culture. The other important finding was the lack of understanding of formative assessment and homework as formative assessment. Implications for future practice include recommendations for school leaders to take the lead in providing organizational learning, utilize the principles of transformational leadership, and be a provider of professional development in the areas of assessment. Implications for future research include recommendations for gaining an understanding of teacher attitudes before and after professional development in assessment, how effective change is achieved in the area of assessment, and student attitudes toward education in classroom which employ formative assessment versus those who do not.

The main purpose of formative assessment is to improve students' learning and it should be seen as a part of the learning process. Game-based learning has become more common in the education and one of the emerging game-based learning platform used in education institutions is Kahoot. This paper investigated the perception of students towards Kahoot as a formative assessment tool in undergraduate medical education and its association with gender. Methods: A cross-sectional study was carried out on first year medical students in a Malaysian public medical school. The study employed a survey that consists of 12 items through Kahoot survey platform. Result: A total of 113 subjects participated in this study; majority was female (68.1%) and Malay (58.4%). The students highly perceived Kahoot as fun, effective and better than e-learning platform for feedback as its median score was 4. The rest of items were satisfactorily perceived by the students as indicated by the score of 3 except for simplifying complex subjects that obtained unsatisfactory level. There were significant median score differences between male and female students for motivation and perceived knowledge retention, whereby males scored higher than females (p < 0.05). Conclusion: Kahoot is a promising formative assessment tool that is feasible, practical and makes learning fun and enjoyable. It can be used to motivate students to learn. However, Kahoot was not the best tool to simplify complex subjects as perceived by medical students.

Formative assessment is considered to be a promising teaching practice for promoting teaching and learning processes. The implementation of teaching practices into instruction involves intervening with a learning environment that is characterised by certain features of instructional quality. Our study aims to contribute to the understanding of formative assessment by analysing the interplay between a formative assessment intervention and aspects of general instructional quality. In a quasi-experimental study design, 15 teachers participated in a control group (n = 361 students) and 20 teachers in the intervention classes (n = 498 students) implemented a curriculum-embedded formative assessment tool in their ninth-grade mathematics classes. No effects were found for the intervention on the assessed aspects of general instructional quality (process-oriented instruction, teacher–student relationship, effective use of instructional time). However, multilevel regression analyses revealed an interaction effect between the intervention and process-orientation and the effective use of instruction time. Our findings suggest that implementing formative assessment tools do not seem to suffice regarding changes in general instructional quality, but that an intervention with detailed material and guidelines can counterbalance effects of instructional quality, fostering students’ achievement in classes with lower degrees of process orientation and a less effective use of instructional time.

Recent years have seen a growing interest in investigating visitor engagement in science museums with multimodal learning analytics. Visitor engagement is a multidimensional process that unfolds temporally over the course of a museum visit. In this paper, we introduce a multimodal trajectory analysis framework for modeling visitor engagement with an interactive science exhibit for environmental sustainability. We investigate trajectories of multimodal data captured during visitor interactions with the exhibit through slope-based time series analysis. Utilizing the slopes of the time series representations for each multimodal data channel, we conduct an ablation study to investigate how additional modalities lead to improved accuracy while modeling visitor engagement. We are able to enhance visitor engagement models by accounting for varying levels of visitors’ science fascination, a construct integrating science interest, curiosity, and mastery goals. The results suggest that trajectory-based representations of the multimodal visitor data can serve as the foundation for visitor engagement modeling to enhance museum learning experiences.KeywordsMuseum learningVisitor engagementMultimodal trajectory analytics

The new educational models such as smart learning environments use of digital and context‐aware devices to facilitate the learning process. In this new educational scenario, a huge quantity of multimodal students' data from a variety of different sources can be captured, fused, and analyze. It offers to researchers and educators a unique opportunity of being able to discover new knowledge to better understand the learning process and to intervene if necessary. However, it is necessary to apply correctly data fusion approaches and techniques in order to combine various sources of multimodal learning analytics (MLA). These sources or modalities in MLA include audio, video, electrodermal activity data, eye‐tracking, user logs, and click‐stream data, but also learning artifacts and more natural human signals such as gestures, gaze, speech, or writing. This survey introduces data fusion in learning analytics (LA) and educational data mining (EDM) and how these data fusion techniques have been applied in smart learning. It shows the current state of the art by reviewing the main publications, the main type of fused educational data, and the data fusion approaches and techniques used in EDM/LA, as well as the main open problems, trends, and challenges in this specific research area.This article is categorized under: Application Areas &gt; Education and Learning

Advancements in Multimodal Learning Analytics (MMLA) have the potential to enhance the development of effective teamwork skills and foster reflection on collaboration dynamics in physical learning environments. Yet, only a few MMLA studies have closed the learning analytics loop by making MMLA solutions immediately accessible to educators to support reflective practices, especially in authentic settings. Moreover, deploying MMLA solutions in authentic settings can bring new challenges beyond logistic and privacy issues. This paper reports the design and use of TeamSlides, a multimodal teamwork analytics dashboard to support teacher-guided reflection. We conducted an in-the-wild classroom study involving 11 teachers and 138 students. Multimodal data were collected from students working in team healthcare simulations. We examined how teachers used the dashboard in 22 debrief sessions to aid their reflective practices. We also interviewed teachers to discuss their perceptions of the dashboard’s value and the challenges faced during its use. Our results suggest that the dashboard effectively reinforced discussions and augmented teacher-guided reflection practices. However, teachers encountered interpretation conflicts, sometimes leading to mistrust or misrepresenting the information. We discuss the considerations needed to overcome these challenges in MMLA research.

ABSTRACTStudent self-assessment has been increasingly used as a formative assessment tool in interpreter education to help students foster learner autonomy, self-regulation and life-long learning. However, there seems to be little empirical research on how interpreting students perceive and view self-assessment. We therefore report a qualitative study, based on 38 students’ reflective observations, to shed light on potential benefits and drawbacks of self-assessment which was practiced throughout a ten-week consecutive interpreting course. Inductive content analysis reveals 21 emerging themes, categorized under four general dimensions: Comments (C), Benefits (B), Drawbacks (D) and Suggestions (S). Data analysis indicates that overall self-assessment was beneficial, making students self-aware and self-reflective, but it may also have produced differential effects on different student groups (e.g., motivating some students, while demoralizing others). The study also reveals emergent themes that have not been reported in previous interpreting literature, and identifies some implementation problems that require special attention. Practical implications of the research results are discussed regarding the use of self-assessment as a formative pedagogic exercise.

While Multimodal Learning Analytics (MMLA) is becoming a popular methodology in the LAK community, most educational researchers still rely on traditional instruments for capturing learning processes (e.g., click-stream, log data, self-reports, qualitative observations). MMLA has the potential to complement and enrich traditional measures of learning by providing high frequency data on learners’ behavior, cognition and affects. However, there is currently no easy-to-use toolkit for recording multimodal data streams. Existing methodologies rely on the use of physical sensors and custom-written code for accessing sensor data. In this paper, we present the EZ-MMLA toolkit. This toolkit was implemented as a website that provides easy access to the latest machine learning algorithms for collecting a variety of data streams from webcams: attention (eye-tracking), physiological states (heart rate), body posture (skeletal data), hand gestures, emotions (from facial expressions and speech), and lower-level computer vision algorithms (e.g., fiducial / color tracking). This toolkit can run from any browser and does not require special hardware or programming experience. We compare this toolkit with traditional methods and describe a case study where the EZ-MMLA toolkit was used in a classroom context. We conclude by discussing other applications of this toolkit, potential limitations, and future steps.

The article presents the idea of ​​using the portfolio as an opportunity to fully document the process of developing students' competencies. Its possibilities as a tool for formative assessment, are commented. On the basis of this type of formative assessment the competencies of the student are determined, which gives an idea of ​​the achievements of each of them at the end of a certain stage of the educational process. The motivating function of the portfolio is also considered. The results of a survey conducted among pedagogical specialists are also presented, the aim of which is to prove the possibilities of the portfolio as a tool for formative assessment in the educational process.

Multimodal learning analytics (MMLA) integrates novel sensing technologies and artificial intelligence algorithms, providing opportunities to enhance student reflection during complex, collaborative learning experiences. Although recent advancements in MMLA have shown its capability to generate insights into diverse learning behaviours across various learning settings, little research has been conducted to evaluate these systems in authentic learning contexts, particularly regarding students' perceived fairness, accountability, transparency, and ethics (FATE). Understanding these perceptions is essential to using MMLA effectively without introducing ethical complications or negatively affecting how students learn. This study aimed to address this gap by assessing the FATE of MMLA in an authentic, collaborative learning context. We conducted semi-structured interviews with 14 undergraduate students who used MMLA visualizations for post-activity reflection. The findings highlighted the significance of accurate and comprehensive data representation to ensure visualization fairness, the need for different levels of data access to foster accountability, the imperative of measuring and cultivating transparency with students, and the necessity of transforming informed consent from dichotomous to continuous and measurable scales. While students value the benefits of MMLA, they also emphasize the importance of ethical considerations, highlighting a pressing need for the LA and MMLA communities to investigate and address FATE issues actively.

Multimodal Learning Analytics (MMLA) innovations make use of rapidly evolving sensing and artificial intelligence algorithms to collect rich data about learning activities that unfold in physical spaces. The analysis of these data is opening exciting new avenues for both studying and supporting learning. Yet, practical and logistical challenges commonly appear while deploying MMLA innovations “in-the-wild”. These can span from technical issues related to enhancing the learning space with sensing capabilities, to the increased complexity of teachers’ tasks. These practicalities have been rarely investigated. This article addresses this gap by presenting a set of lessons learnt from a 2-year human-centred MMLA in-the-wild study conducted with 399 students and 17 educators in the context of nursing education. The lessons learnt were synthesised into topics related to (i) technological/physical aspects of the deployment; (ii) multimodal data and interfaces; (iii) the design process; (iv) participation, ethics and privacy; and (v) sustainability of the deployment.

ABSTRACTTechnology appears as one of the significant tools to create a meaningful process in all fields of life for living things, one of which is education. This situation drives related parties to create game-based learning platform for education. One of the emerging game-based learning platform used in education is Kahoot!. Educationally relevant game, such as Kahoot! makes students learn academic content with a lot of excitement. Recently, Kahoot! is used as a tool for formative assessment. Formative assessment is useful to improve students attainment. This study aims to investigate the effectiveness of Kahoot! used as formative assessment in education. This paper will elaborate the theoretical review as well as the application that underlay the hypothesis. The result of the study shows that Kahoot! is an alternative tool of interactive formative assessment. Kahoot! is feasible and practical to make learning fun and enjoyable because it has many benefits for education.

In learning situations that do not occur exclusively online, the analysis of multimodal evidence can help multiple stakeholders to better understand the learning process and the environment where it occurs. However, Multimodal Learning Analytics (MMLA) solutions are often not directly applicable outside the specific data gathering setup and conditions they were developed for. This paper focuses specifically on authentic situations where MMLA solutions are used by multiple stakeholders (e.g., teachers and researchers). In this paper, we propose an architecture to process multimodal evidence of learning taking into account the situation’s contextual information. Our adapter-based architecture supports the preparation, organisation, and fusion of multimodal evidence, and is designed to be reusable in different learning situations. Moreover, to structure and organise such contextual information, a data model is proposed. Finally, to evaluate the architecture and the data model, we apply them to four authentic learning situations where multimodal learning data was collected collaboratively by teachers and researchers.