Learning analytics in seamless learning environments: a systematic review

The mobile-assisted seamless learning approach shows great promise in significantly enhancing vocabulary development among second language learners, particularly through artifact creation and engaging activities This study explores the experiences of tertiary-level learners studying Chinese as a Foreign Language (CFL) using a mobile-assisted seamless Chinese vocabulary learning (SCVL) approach. Specifically, the study examines the perspectives, reflections, and learning processes of CFL learners enrolled in a 16-week SCVL program. The research utilizes a qualitative design with inductive thematic analysis to analyze data collected from semi-structured interviews and reflective journals. The study involved a cohort of 32 intermediate-level students from three Chinese classes, with participants representing diverse Chinese proficiency levels. A purposive sampling method was used to select students who had actively engaged in the SCVL program. Thematic analysis of the data revealed two key themes: (1) affective engagement, which highlights students' emotional involvement and motivation in the seamless learning environment, and (2) issues and challenges, which focuses on the difficulties students encountered, such as technological barriers and frustration with peer reviews. Despite these challenges, the findings indicate that most students found the SCVL approach enjoyable and beneficial in enhancing their vocabulary learning, with many expressing positive views about the integration of mobile-assisted learning tools. This study contributes to the growing body of research on seamless learning by providing in-depth insights into the experiences of tertiary-level CFL learners. It offers novel contributions to the field by addressing the gaps in previous studies that have not fully captured students' perspectives and challenges in using mobile-assisted seamless learning for vocabulary acquisition. The findings have important pedagogical implications, particularly for enhancing the design and implementation of seamless learning environments in language education.

The use of seamless learning environments that have the potential to support lifelong learning anytime and anywhere has become a reality. In this sense, many educational institutions have started to consider introducing seamless learning environments into their programs. The aim of this study is to analyze how various educational university programs implement the design elements for seamless learning environments with mobile devices. For that purpose, three cases involved in a Finnish teaching development project are explored by conducting semi-structured interviews with key participants. The themes of the interviews were related to the theoretical background for (mobile) seamless learning environments from previous literature. This paper describes the findings of the three cases as common aspects for designing mobile, seamless learning environments; and it proposes a research agenda on challenges related to designing seamless learning environments with the integrated use of mobile devices in the curricula.

Tremendous proliferation in data generation in the past few years has paved the way for new research and the development of new and improved techniques and algorithms in different fields of science and education. Initially terms like educational data mining emerged as a branch of data mining borrowing techniques from its ancestor. The challenges brought about by this large and heterogeneous data are diverse and needs a greater serious technical treatment. New and emerging fields like learning analytics have been introduced to manage the complexities of this data deluge. Learning analytics deals with data in the context of learner and the learning environment to improve the overall learning experience. The ultimate aim of the field is to make use of the data about learners and their environments to gain insights into the learning process using some of the well-known techniques and algorithms from the fields of data mining and machine learning. The process involves collecting, analysis of data and reporting the results to understand and optimize the learning experience. The fields of data mining and academic analytics closely related to learning analytics. Systematic Literature Review (SLR) is a robust, organized and rigorous literature review and reporting process aimed at identifying, collecting and synthesizing the relevant literature on a research question according to specified criteria. The process is more unbiased and balanced by systematic sequence of steps. This paper presents a systematic literature review by first developing the systematic literature review protocol and then discussing the main findings of the literature review by especially focusing on the applications and uses of machine learning and data mining techniques in the domain of learning analytics.
 
 Index Terms—Systematic Literature Review (SLR), Learning Analytics (LA), Big Data, Educational Data Mining (EDM), Machine Learning (ML).

This paper describes a Human‐Centred Learning Analytics (HCLA) design approach for developing learning analytics (LA) dashboards for K‐12 classrooms that maintain both contextual relevance and scalability—two goals that are often in competition. Using mixed methods, we collected observational and interview data from teacher partners and assessment data from their students' engagement with the lesson materials. This DBR‐based, human‐centred design process resulted in a dashboard that supported teachers in addressing their students' learning needs. To develop the dashboard features that could support teachers, we found that a design refinement process that drew on the insights of teachers with varying teaching experience, philosophies and teaching contexts strengthened the resulting outcome. The versatile nature of the approach, in terms of student learning outcomes, makes it useful for HCLA design efforts across diverse K‐12 educational contexts. Practitioner notes What is already known about this topic Learning analytics that are aligned to both a learning theory and learning design support student learning. LA dashboards that support users to understand the associated learning analytics data provide actionable insight. Design‐based research is a promising methodology for Human‐Centred Learning Analytics design, particularly in the K‐12 educational context. What this paper adds Leveraging a longstanding, yet fluid, research‐practice partnership is an effective design‐based research adaptation for addressing the high variation in instructional practices that characterize K‐12 education. Using both quantitative and qualitative data that reflects students' developing knowledge effectively supports teachers' inquiry into student learning. Teachers' use of learning analytics dashboards is heavily influenced by their perspectives on teaching and learning. Implications for practice and/or policy Impact on student learning outcomes, alongside usability and feasibility, should be included as a necessary metric for the effectiveness of LA design. LA dashboard developers should both leverage learning data that reflect students' developing knowledge and position teachers to take responsive pedagogical action to support student learning. LA researchers and developers should utilize a long‐term, yet fluid, research‐practice partnership to form a multi‐stakeholder, multidisciplinary design team for Human‐Centred Learning Analytics design.

This paper reports an initial study on investigating inquiry-based learning in science in an upper primary class guided by an inquiry-based learning model in a seamless learning environment. Two questions are addressed: (1) how students advanced their domain knowledge?; and (2) how students developed their inquiry skills? One teacher and 27 Grade-four students from a local primary school were involved in the study. Six inquiry-based learning lessons focusing on a scientific “rustproofing” learning unit were conducted in a seamless learning environment, initiated in a digital classroom and extended to online discussions on a social network platform. Qualitative data were collected and analysed over two weeks. The results show ways that the students advanced their domain knowledge and developed their inquiry skills.

Learning analytics involve collecting, analyzing, and visualizing the digital 'footprints' that learners leave behind as they interact with digital learning environments. Learning analytics inform ongoing refinements to improve educational design and teaching practice. To date, research suggests that educators have not fully capitalized on learning analytics, despite the growing availability of data generated through digital learning platforms, programmatic assessment, and competency-based training. In this article, the authors adapted an established "learning analytics lifecycle" framework to evaluate a representative example of digital learning in HPE: eduCAST, an online, video-based website for orthopaedic surgery training. The framework provided a stepwise approach for exploring the potential of learning analytics in eduCAST, identifying implementation gaps, and developing a transferable list of recommendations for educators in HPE. Demographic and engagement data of 141 registered eduCAST users were analyzed after being collected via website analytics. The findings revealed limitations and opportunities related to: (1) planning for the learning environment and its users, (2) the scope and specificity of available learning analytics data, (3) the purposeful use of data analysis techniques, and (4) the relative absence of educators' data-informed actions. Generic website analytics showed significant limitations: high website traffic did not correspond to meaningful learner engagement, nor did it provide useful data to inform ongoing website refinement. Indeed, these analytics produced more data mysteries than meaningful data stories. Consistent with prior reviews in the HPE literature, this worked example demonstrated that educators appear to make diminishing investments of effort and resources across the learning analytics lifecycle. The authors argue that HPE educators can benefit from using learning analytics frameworks to guide the design, implementation, evaluation, and long-term sustainability of digital learning environments. When thoughtfully collected, analyzed, and interpreted, learning analytics can enhance learners' experiences and outcomes, support educators' professional development, and inform continuous program refinement.

Instructional designers use learning analytics information to evaluate designs of learning environments, learning materials, and tasks, and adjust difficulty levels, as well as measure the impact of interventions and feedback. Integrating real-time educational data and analysis into the design of learning environments, referred to as learning analytics design (LAD), seems to be a promising approach. Valid pedagogical recommendations may be suggested on the fly as learning analytics methodologies and visualizations evolve and as reliable tools become available and ready for classroom practice. This chapter aims to offer an overview on design and analytics of learning environments before reviewing opportunities of learning analytics design for optimizing learning environments in near real time. Learning analytics (LA) use static and dynamic information about learners and learning environments—assessing, eliciting, and analyzing them—for real-time modeling, prediction, and optimization of learning processes, learning environments, and educational decision-making.

A group of researchers had been working on a longitudinal mobile learning (m-learning) project in a primary school in Singapore. A curriculum design framework was proposed in the beginning of the project to guide the two-year design-enactment-reflection-refinement cycles of the mobilized curriculum. In this chapter, we narrate our implementation research approach by presenting a post hoc analysis of how the curriculum was progressively transformed for seamless learning (a learning notion that advocates perpetual learning across contexts) and how the design taps on the affordances for m-learning. The evaluation illuminates how various types of learning activities are systematically introduced in the two years of science curriculum to nurture inquiry learning across both formal and informal contexts, thus supporting notions of authentic learning. This chapter contributes to the literature on how to address challenges in translating learning theories and integrating mobile technology affordances into curriculum development and sustainable classroom practices.

This study introduces ‘seamless Chinese vocabulary learning’ (SCVL), a mobile-assisted initiative designed to bridge learners’ everyday experiences with formal language instruction. The SCVL framework integrates mobile technologies to foster vocabulary acquisition through artefact creation in a seamless learning environment. Employing a design-based research (DBR) methodology with mixed-methods design, 32 participants of varying proficiency levels took part in a 16-week intervention facilitated via the DingTalk platform. Quantitative analysis of 2,397 artefacts using the type–token ratio (TTR) indicated significant improvements in lexical variety across sentences, paragraphs, and essays across performance groups. Two-way ANOVA and post-hoc tests revealed significant main effects of time and group. A notable Group × Time interaction was observed for TTR Sentence and TTR Paragraph, but not for TTR Essay. These findings suggest that vocabulary development under SCVL is non-linear and shaped by learners’ proficiency levels. Qualitative data from interviews and learner feedback highlighted two major themes: (1) affective engagement, reflecting heightened motivation and emotional involvement, and (2) challenges, including technological barriers and frustration with peer review processes. Despite these difficulties, students reported that SCVL was both engaging and effective in enhancing their vocabulary learning. This study contributes to seamless learning research by addressing limitations in earlier work that often overlooked learners’ perspectives and the obstacles they encounter. Pedagogically, the findings highlight the potential of the ‘sentence–paragraph–essay’ artefact creation process to guide language instructors in developing more effective, sustainable strategies for vocabulary instruction.

Business Intelligence (BI) represents a pivotal advancement in leveraging information technology to enhance organizational performance. BI tools serve as crucial aids in decision-making processes by furnishing requisite insights. In higher education institutions, BI can contribute to leaders and managers in providing perspectives related to academics, learning, and management. Central to BI development is the meticulous gathering of requirements, a process pivotal in identifying organizational informational and knowledge needs. This involves employing various methods such as interviews, observation, and analysis, including leveraging learning analytics to discern data utility for enhanced learning processes. Various studies show that learning analytics contributes to improving the learning and education process. On the other hand, learning analytics requires activity data that is integrated, subject oriented, and time series which are aligned with the characteristics of the data warehouse (DWH) as the main component of BI. This research endeavors to develop BI utilizing academic and e-learning data, exemplified through a case study of Telkom University's Academic Systems and Learning Management Systems (LMS). This study aims to provide actionable insights into the intersection of BI and learning analytics, ultimately enhancing educational processes and organizational decision-making capabilities. By integrating learning analytics into BI development, the resultant BI systems can cater not only to current managerial demands but also anticipate future analytical needs. The implementation of the multidimensional schema was successfully executed. This process involved mapping data from the academic information system and the LMS as data sources to the data warehouse, the Extract, Transform, and Load (ETL) process, and development of the prototype. The testing on the prototype indicated that the prototype meets the intended requirements and provides valuable insights through its comprehensive reporting capabilities. This demonstrates the effectiveness of the implemented multidimensional schema, ETL process, and the overall design of the reporting dashboard.

Educational institutions are increasingly turning to learning analytics to identify and intervene with students at risk of underperformance or discontinuation. However, the extent to which the current evidence base supports this investment is currently unclear, and particularly so in relation to the effectiveness of interventions based on predictive models. The aim of the present paper was to conduct a systematic review and quality assessment of studies on the use of learning analytics in higher education, focusing specifically on intervention studies. Search terms identified 689 papers, but only 11 studies evaluated the effectiveness of interventions based on learning analytics. These studies highlighted the potential of such interventions, but the general quality of the research was moderate, and left several important questions unanswered. The key recommendation based on this review is that more research into the implementation and evaluation of scientifically driven learning analytics is needed to build a solid evidence base for the feasibility, effectiveness and generalizability of such interventions. This is particularly relevant when considering the increasing tendency of educational institutions around the world to implement learning analytics interventions with only little evidence of their effectiveness. Practitioner Notes What is already known about this topic? Drop‐out rates and underachivement is a significant issue at most Western universities. Learning analytics have been shown to predict student performance and risk of dropping out. Interventions based on learning analytics have emerged in recent years, some reportedly successful. What this paper adds The paper also reviews and synthesizes the evidence on the effectiveness of learning analytics interventions targeting student underperformance, experience and discontinuation. The paper compares and contrasts past and current learning analytics methods and foci, and makes recommendations for the future research and practice. It critically synthesizes the current evidence base on learning analytics interventions, which is a field that is in constant flux and development. Implications for practice and/or policy The paper focuses on an increasing part of higher education with the goal of validating learning analytics methods and usefulness. The paper makes evidence‐based recommendations for institutions wishing to implement learning analytics programs and/or interventions. The paper makes evidence‐based recommendations for instructors as well as researchers in the field.

In this paper, we present the development of the Altered States Database (ASDB), an open-science project based on a systematic literature review. The ASDB contains psychometric questionnaire data on subjective experiences of altered states of consciousness (ASC) induced by pharmacological and non-pharmacological methods. The systematic review follows the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. Scientific journal articles were identified through PubMed and Web of Science. We included studies that examined ASC using the following validated questionnaires: Altered States of Consciousness Rating Scale (APZ, 5D-ASC, 11-ASC), Phenomenology of Consciousness Inventory (PCI), Hallucinogen Rating Scale (HRS), or Mystical Experience Questionnaire (MEQ30). The systematic review resulted in the inclusion of a total of 165 journal articles, whereof questionnaire data was extracted and is now available on the Open Science Framework (OSF) website (https://osf.io/8mbru) and on the ASDB website (http://alteredstatesdb.org), where questionnaire data can be easily retrieved and visualized. This data allows the calculation of comparable psychometric values of ASC experiences and of dose-response relationships of substances inducing ASC.

Technologies and teaching practices can provide a rich log data, which enables learning analytics (LA) to bring new insights into the learning process for ultimately enhancing student success. This type of data has been used to discover student online learning patterns, relationships between online learning behaviors and assessment performance. Previous studies have provided empirical evidence that not all log variables were significantly associated with student academic achievement and the relationships varied across courses. Therefore, this study employs a systematic review with meta‐analysis method to provide a comprehensive review of the log variables that have an impact on student academic achievement. We searched six databases and reviewed 88 relevant empirical studies published from 2010 to 2021 for an in‐depth analysis. The results show different types of log variables and the learning contexts investigated in the reviewed studies. We also included four moderating factors to do moderator analyses. A further significance test was performed to test the difference of effect size among different types of log variables. Limitations and future research expectations are provided subsequently. Practitioner notes What is already known about this topic Significant relationship between active engagement in online courses and academic achievement was identified in a number of previous studies. Researchers have reviewed the literature to examine different aspects of applying LA to gain insights for monitoring student learning in digital environments (eg, data sources, data analysis techniques). What this paper adds Presents a new perspective of the log variables, which provides a reliable quantitative conclusion of log variables in predicting student academic achievement. Conducted subgroup analysis, examined four potential moderating variables and identified their moderating effect on several log variables such as regularity of study interval, number of online sessions, time‐on‐task, starting late and late submission. Compared the effect of generic and course‐specific, basic and elaborated log variables, and found significant difference between the basic and elaborated. Implications for practice and/or policy A depth of understanding of these log variables may enable researchers to build robust prediction models. It can guide the instructors to timely adjust teaching strategies according to their online learning behaviors.

This paper presents a comprehensive review of the integration of Learning Analytics (LA), Artificial Intelligence (AI), and STEM education within classroom settings, aimed at enhancing educational outcomes. By examining the synergistic effects and interactions among LA, AI, and STEM disciplines, this review highlights how these technologies can collectively transform educational practices. It discusses the potential of LA and AI to personalize learning experiences, thereby improving engagement and academic success in STEM subjects. The paper also explores various case studies and success stories, illustrating practical implementations and the significant impact these technologies have made in schools. Additionally, it addresses the challenges and considerations related to the ethical use of AI and data privacy, providing insights into how educators and policymakers can navigate these issues. Overall, this review underscores the critical role of technology in shaping the future of education by fostering more adaptive and inclusive learning environments.

This paper attempts to understand the broader implications for integrating learning analytics in e-learning systems. This is also the commencement of a collaborative effort for developing research bases around learning analytics between the University of Mauritius and the University of KwaZuluNatal. The focus on e-assessments is deliberate: the tendancy in both our universities to appraise student learning as summative and quantitative does not reflect contemporary workplace and scholarly needs where learning processes are more valuable than outcomes. The necessity for epistemological rethinking and developing systems based on contextual requirements is important, whether it is related to curriculum development or workshop organisation or even conducting a class. After a survey of the literature on the potentials and hypes around learning analytics, we look towards the literature linking learning analytics, data mining and e-assessment for educators to interface more effectively and efficiently with the LMS platform. We conclude this concept-paper with a workshop outline which can help educational technologists organise practical, sustainable and meaningful hands-on sessions with educators.