Data-Driven Approaches to Enhancing Students’ Web Design Skills Using Multimodal Learning Analytics

The purpose of this research was to design a course based on the design-based learning (DBL) model to improve undergraduates’ web design skills, and to explore the differences in learning effectiveness of students with different levels of knowledge integration. The quasi-experimental design was adopted in this study. Ninety-two participants studying general education courses in a university in southern Taiwan were selected to participate in this study. The instruments included the Web Design Skills Test and the Knowledge Integration Scale. The study found that the use of the DBL strategy in teaching could improve the web design skills of the experimental group students, especially those students with insufficient web design skills in the initial stage. In addition, students with a high level of knowledge integration had better improvement in their web design skills. The implication of this research is that DBL is a suitable web design skills training strategy for students with insufficient web design skills. If students could be reminded to integrate knowledge while studying, their learning outcomes may be better.

Multimodal Learning is happening in different contexts, where different technologies are utilized. In such contexts, different modalities are used by learners in very unstructured ways. These modalities include video, audio, motion, eye tracking, to mention but a few. However, effective applications of Multimodal Learning Analytics remain challenging. Enabling educational technologies are underpinned by various pedagogical models that are designed to provide educational values of technology. Nevertheless, the link between Multimodal Learning Analytics and informal learning pedagogy is not very well established in the literature. Hence, this chapter aims at bridging the gap between multimodal learning analytics research concepts and approaches from one side and key informal learning pedagogical approaches such as self-directed learning and learning theories such as behaviorism and cognitivism from the other side. Establishing this link is expected to pave the ground for insightful and pedagogically informed learning analytics across multiple contexts. In addition, further explanations on what Multimodal learning analytics techniques and challenges are discussed to highlight the key concerns of MMLA applications.KeywordsMultimodal learning analyticsMultimodal dataPedagogyLearning process

A distinctive feature of game‐based learning environments is their capacity to create learning experiences that are both effective and engaging. Recent advances in sensor‐based technologies such as facial expression analysis and gaze tracking have introduced the opportunity to leverage multimodal data streams for learning analytics. Learning analytics informed by multimodal data captured during students’ interactions with game‐based learning environments hold significant promise for developing a deeper understanding of game‐based learning, designing game‐based learning environments to detect maladaptive behaviors and informing adaptive scaffolding to support individualized learning. This paper introduces a multimodal learning analytics approach that incorporates student gameplay, eye tracking and facial expression data to predict student posttest performance and interest after interacting with a game‐based learning environment, Crystal Island . We investigated the degree to which separate and combined modalities (ie, gameplay, facial expressions of emotions and eye gaze) captured from students ( n = 65) were predictive of student posttest performance and interest after interacting with Crystal Island . Results indicate that when predicting student posttest performance and interest, models utilizing multimodal data either perform equally well or outperform models utilizing unimodal data. We discuss the synergistic effects of combining modalities for predicting both student interest and posttest performance. The findings suggest that multimodal learning analytics can accurately predict students’ posttest performance and interest during game‐based learning and hold significant potential for guiding real‐time adaptive scaffolding.

Advancements in Multimodal Learning Analytics (MMLA) have the potential to enhance the development of effective teamwork skills and foster reflection on collaboration dynamics in physical learning environments. Yet, only a few MMLA studies have closed the learning analytics loop by making MMLA solutions immediately accessible to educators to support reflective practices, especially in authentic settings. Moreover, deploying MMLA solutions in authentic settings can bring new challenges beyond logistic and privacy issues. This paper reports the design and use of TeamSlides, a multimodal teamwork analytics dashboard to support teacher-guided reflection. We conducted an in-the-wild classroom study involving 11 teachers and 138 students. Multimodal data were collected from students working in team healthcare simulations. We examined how teachers used the dashboard in 22 debrief sessions to aid their reflective practices. We also interviewed teachers to discuss their perceptions of the dashboard’s value and the challenges faced during its use. Our results suggest that the dashboard effectively reinforced discussions and augmented teacher-guided reflection practices. However, teachers encountered interpretation conflicts, sometimes leading to mistrust or misrepresenting the information. We discuss the considerations needed to overcome these challenges in MMLA research.

This literature review delves into personalized learning recommendation systems (PLRSs) within educational contexts. It places a significant emphasis on the understanding of student behavior that is driven by Explainable AI (XAI). Additionally, it focuses on the mechanisms of teacher collaboration. The traditional educational models are not without their drawbacks. These limitations have instigated a transition towards personalized learning. This movement has, in turn, propelled the development of PLRSs. These systems are designed with the dual objectives of boosting learning efficiency and enhancing learning outcomes. To accomplish these goals, they offer customized learning resources and strategies. There are key trends. One trend is dynamic and adaptive recommendation strategies. Another trend is the use of explainable AI (XAI). XAI builds trust. XAI also builds transparency. In - depth student behavior understanding is a trend. Performance modeling is also a trend. Advanced content understanding is a trend. Semantic analysis is a trend as well. The application of collaborative filtering is a trend. The application of hybrid approaches is a trend. The emphasis on teacher collaboration is a trend. The emphasis on human - AI interaction is a trend too. Dynamic systems can adapt to students' changing needs. XAI makes AI - driven recommendations understandable and trustworthy. Precise student models improve the relevance of recommendations. Semantic analysis of educational content does the same. Hybrid approaches enhance the performance of collaborative filtering. Teacher - AI collaboration is important for the effective implementation of PLRSs. However, there are several challenges. Future research should focus on different things. It should develop more comprehensive student models. It should enhance XAI techniques for educational contexts. It should empirically study the impact of XAI on learning outcomes. It should design effective teacher collaboration mechanisms. It should create human - AI interaction strategies. It should address ethical issues and fairness in personalized learning. It should explore the integration of multimodal data and learning analytics. By dealing with these challenges, PLRSs can be optimized. This can create more intelligent, transparent, and human - centered personalized learning environments. In the end, this will enhance learning outcomes. It will also empower students and educators.

The availability of proper human capital represents a key-factor in order to support the current fast pace of diffusion of Internet-based applications. The careful assessment of the skills and competencies required to manage and develop Internet technologies in different application domains represents a necessary step to solve the emerging skill shortage and skill gap problems. While most existing analyses of skill needs in information and communication technologies deal with broad professional categories, the paper focuses the attention on a specific professional figure, the Web designer. By resorting to an on-line questionnaire administered to a sample of Italian Web designers, the paper shows that jobholders recognise greater usefulness to organisational rather than technical knowledge domains. However, perceived skill gaps are larger in the case of technical skills and competencies. Moreover, specialisation paths are emerging.

Multimodal learning analytics (MMLA) integrates novel sensing technologies and artificial intelligence algorithms, providing opportunities to enhance student reflection during complex, collaborative learning experiences. Although recent advancements in MMLA have shown its capability to generate insights into diverse learning behaviours across various learning settings, little research has been conducted to evaluate these systems in authentic learning contexts, particularly regarding students' perceived fairness, accountability, transparency, and ethics (FATE). Understanding these perceptions is essential to using MMLA effectively without introducing ethical complications or negatively affecting how students learn. This study aimed to address this gap by assessing the FATE of MMLA in an authentic, collaborative learning context. We conducted semi-structured interviews with 14 undergraduate students who used MMLA visualizations for post-activity reflection. The findings highlighted the significance of accurate and comprehensive data representation to ensure visualization fairness, the need for different levels of data access to foster accountability, the imperative of measuring and cultivating transparency with students, and the necessity of transforming informed consent from dichotomous to continuous and measurable scales. While students value the benefits of MMLA, they also emphasize the importance of ethical considerations, highlighting a pressing need for the LA and MMLA communities to investigate and address FATE issues actively.

Video analysis is a staple of the education research community. For many contemporary education researchers, participation in the video coding process serves as a rite of passage. However, recent developments in multimodal learning analytics may help to accelerate and enhance this process by providing researchers with a more nuanced glimpse into a set of learning experiences. As an example of how to use multimodal learning analytics towards these ends, this paper includes a preliminary analysis from 54 college students, who completed two engineering design tasks in pairs. Gesture, speech and electro-dermal activation data were collected as students completed these tasks. The gesture data was used to learn a set of canonical clusters (N=4). A decision tree was trained based on individual students' cluster frequencies, and pre-post learning gains. The nodes in the decision tree were then used to identify a subset of video segments that were human coded based on prior work in learning analytics and engineering design. The combination of machine learning and human inference helps elucidate the practices that seem to correlate with student learning. In particular, both engagement and disengagement seem to correlate with student learning, albeit in a somewhat nuanced fashion.

Traditional log-based Learning Analytics (LA) methods fail to capture the interactions that occurred during collaboration in collocated or face-to-face settings. Multimodal Learning Analytics (MMLA) has facilitated researchers a way for addressing this issue with the help of sensors for automated capturing of learning traces from real-world settings, and machine learning methods helped researchers to extract the pattern from the collected data to improve learning. My Ph.D. addresses the problem of automatically estimating the quality of collaboration in collocated settings using MMLA to support teachers when monitoring and assessing it in authentic classroom settings. This paper discusses the motivation, the research problem, and the research methodology of my Ph.D. It also reports the initial research findings in addition to the future work directions.

The article provides an answer to the frequently asked question about what to teach students so that they can easily enter the area of web development and become sought-after specialists in the modern IT industry upon graduation. We completed a review of publications and scientific discussions on this subject, performed an analysis of the relevant professional standards as well as the transformation dynamics of the requirements set out by employers for web developers. In addition, the paper examines the demand for designing educational programs. We interviewed representatives of professional communities and university graduates to understand the challenges which former students face while seeking employment in the field of web development and design. Six main skill groups of a modern successful specialist have been identified. They include basic web development skills, web programming skills, web design skills, development skills and development management skills, web resource promotion skills, universal skills. For each group, we identified the main skills and tools which are advisable to be studied as part of the educational program. It has been discovered that training should be based on using skill-oriented educational technologies since solving specific practical problems will enable each student to form their own future workspace. We also created a model of a graduate who aspires to work in the field of web development.

Simulation‐based learning provides students with unique opportunities to develop key procedural and teamwork skills in close‐to‐authentic physical learning and training environments. Yet, assessing students' performance in such situations can be challenging and mentally exhausting for teachers. Multimodal learning analytics can support the assessment of simulation‐based learning by making salient aspects of students' activities visible for evaluation. Although descriptive analytics have been used to study students' motor behaviours in simulation‐based learning, their validity and utility for assessing performance remain unclear. This study aims at addressing this knowledge gap by investigating how indoor positioning analytics can be used to generate meaningful insights about students' tasks and collaboration performance in simulation‐based learning. We collected and analysed the positioning data of 304 healthcare students, organised in 76 teams, through correlation, predictive and epistemic network analyses. The primary findings were (1) large correlations between students' spatial‐procedural behaviours and their group performances; (2) predictive learning analytics that achieved an acceptable level (0.74 AUC) in distinguishing between low‐performing and high‐performing teams regarding collaboration performance; and (3) epistemic networks that can be used for assessing the behavioural differences across multiple teams. We also present the teachers' qualitative evaluation of the utility of these analytics and implications for supporting formative assessment in simulation‐based learning. Practitioner notes What is currently known about this topic Assessing students' performance in simulation‐based learning is often challenging and mentally exhausting. The combination of learning analytics and sensing technologies has the potential to uncover meaningful behavioural insights in physical learning spaces. Observational studies have suggested the potential value of analytics extracted from positioning data as indicators of highly‐effective behaviour in simulation‐based learning. What this paper adds Indoor positioning analytics for supporting teachers' formative assessment and timely feedback on students' group/team‐level performance in simulation‐based learning. Empirical evidence supported the potential use of epistemic networks for assessing the behavioural differences between low‐performing and high‐performing teams. Teachers' positively validated the utility of indoor positioning analytics in supporting reflective practices and formative assessment in simulation‐based learning. Implications for practitioners Indoor positioning tracking and spatial analysis can be used to investigate students' teamwork and task performance in simulation‐based learning. Predictive learning analytics should be developed based on features that have direct relevance to teachers' learning design. Epistemic networks analysis and comparison plots can be useful in identifying and assessing behavioural differences across multiple teams.

Programming is a complex learning activity that involves coordination of cognitive processes and affective states. These aspects are often considered individually in computing education research, demonstrating limited understanding of how and when students learn best. This issue confines researchers to contextualize evidence-driven outcomes when learning behaviour deviates from pedagogical intentions. Multimodal learning analytics (MMLA) captures data essential for measuring constructs (e.g., cognitive load, confusion) that are posited in the learning sciences as important for learning, and cannot effectively be measured solely with the use of programming process data (IDE-log data). Thus, we augmented IDE-log data with physiological data (e.g., gaze data) and participants’ facial expressions, collected during a debugging learning activity. The findings emphasize the need for learning analytics that are consequential for learning, rather than easy and convenient to collect. In that regard, our paper aims to provoke productive reflections and conversations about the potential of MMLA to expand and advance the synergy of learning analytics and learning design among the community of educators from a post-evaluation design-aware process to a permanent monitoring process of adaptation.

Higher education institutions are developing the capacity for learning analytics. However, the technical development of learning analytics has far exceeded the consideration of ethical issues around learning analytics. We examined higher education academics’ knowledge, attitudes, and concerns about the use of learning analytics though four focus groups (N = 35). Thematic analysis of the focus group transcripts identified five key themes. The first theme, ‘Facilitating learning’, represents academics’ perceptions that, while currently unrealized, there could be several benefits to learning analytics that would help their students. Three themes; ‘Where are the ethics?’, ‘What about the students!’, and ‘What about me!’ represented academics’ perceptions of how learning analytics could pose some considerable difficulties within a higher education context. A final theme ‘Let’s move forward together’ reflected that despite some challenges and concerns about learning analytics, academics perceived scope for learning analytics to be beneficial if there is collaboration between academics, students, and the university. The findings highlight the need to include academics in the development of learning analytics policies and procedures to promote the suitability and widespread adoption of learning analytics in the higher education sector.

Persistent alignment, resource, and communication challenges constrain strategic implementation and wide-scale use of learning analytics (LA) in higher education context. Staff-facing LA solutions hold great potential benefits for higher education to offer targeted support for students but achieving them requires overcoming persistent staff involvement and LA adoption challenges. There are multiple frameworks that attempt to describe LA implementation, but the consideration of accountabilities of different stakeholders, especially non-managerial staff, is limited. Thus, in this study, we focus on the role and responsibilities of staff as active participants in LA implementation work. Through abductive analysis of individual and focus group interviews (N = 22), we examine non-managerial staff’s perspectives on LA implementation work in a case higher education institution undergoing a staff-facing LA dashboard implementation. We address the key dimensions of institutional LA implementation. This study provides insights into staff expectations and experiences of LA implementation, elaborating the present challenges and development needs. Our results highlight the importance of involving staff as active agents in LA implementation, considering what actionable insights the implemented LA solution provides to its users, and defining stakeholder accountability in LA. The results support the creation of aligned practices for user-centered implementation of technologies for education. This includes considering stakeholder communication, visions, and responsibilities, and ultimately leading to transparent institutional practices.

This paper provides documentation on dataset resources for establishing a new research area called multimodal learning analytics (MMLA). Research on this topic has the potential to transform the future of educational practice and technology, as well as computational techniques for advancing data analytics. The Math Data Corpus includes high-fidelity time-synchronized multimodal data recordings (speech, digital pen, images) on collaborating groups of students as they work together to solve mathematics problems that vary in difficulty level. The Math Data Corpus resources include initial coding of problem segmentation, problem-solving correctness, and representational content of students' writing. These resources are made available to participants in the data-driven grand challenge for the Second International Workshop on Multimodal Learning Analytics. The primary goal of this event is to analyze coherent signal, activity, and lexical patterns that can identify domain expertise and change in domain expertise early, reliably, and objectively, as well as learning-oriented precursors. An additional aim is to build an international research community in the emerging area of multimodal learning analytics by organizing a series of workshops that bring together multidisciplinary scientists to work on MMLA topics.