“Belonging analytics”: A proposal

The prevalence of mental distress among young adults, including those at university, has increased. In this context, learning analytics, students’ digital trace data, are increasingly being used to understand student mental health. In line with calls for more research on learning analytics from student perspectives, as part of a broader focus group study, 44 undergraduate students from three United Kingdom universities were invited to consider how they felt about having a digital footprint on their virtual learning environment (VLE). Two main themes were constructed using reflexive thematic analysis. First, students’ responses depended on the perceived threat to their privacy and identity. Some students were indifferent if no threat was perceived, but expressed unease if there was. Second, some students expressed personal preference for autonomy over use of their VLE data. Two uses identified were for non-judgmental personalized support, and using aggregated data to improve student learning. These themes suggest how the use of educational digital data can, under some circumstances, impact wellbeing negatively. The students’ perspectives garnered from the focus groups could have implications for policy and practice concerning privacy and surveillance, the possibility for misuse or misinterpretation of data, and informed consent. This small study supports the importance of partnering with students to develop and implement guidance for how VLE learning analytics data are used and interpreted by students and staff, including lecturers, to protect and enhance student mental wellbeing.

Many researchers rely on digital trace data, which are organically produced by sociotechnical systems and can provide insights into new digitally related phenomena. Yet theorizing from contexts observed through digital trace data is challenging. Digital trace data are part of these contexts. For some, these contexts may be long-established but changing, while for others, they may be new but perhaps not fully unprecedented. Theorizing from contexts observed through digital trace data introduces a dilemma between staying true to these contexts and developing new theory, which can overlook key aspects of the contexts or overstate how distinctive the observations really are. Theorizing from contexts with digital trace data thus involves developing new theory that both makes sense of and transcends these contexts. This study elaborates on the challenges and opportunities associated with theorizing from contexts with digital trace data and provides guidance on how to do so. It explains how researchers can engage in probing and elucidating contexts when analyzing digital trace data. Probing contexts involves surfacing the omnibus context and identifying, scanning, and connecting the discrete contexts from which digital trace data originate. Elucidating contexts involves situating, depicting, and explaining contexts to answer contextual where, when, what, who, how, and why questions. The progression of these elucidating questions can help researchers build theory from contexts observed through digital trace data. This study illustrates this framework using three papers that rely on digital trace data and examine distinct contexts. This framework can help researchers deepen their engagement with contexts as they analyze digital trace data and provide inspiration to build new theory.

Understanding the measurement inaccuracy and bias introduced by self-reports of smartphone use is essential for making meaningful inferences about smartphone use and its effects. Evidence for the self-reports of smartphone use in intensive longitudinal studies is largely missing. Based on self-reported and digital trace data from 137 Czech adolescents (41% girls, Mage = 14.95 years), this study examined the accuracy, directional bias, and convergent validity of daily end-of-day and single-time reports of screen time and phone-checking behavior. Overall, the study found considerable discrepancies between self-reported smartphone use and digital trace and low between-person convergent validity for all self-reports considered for the study. Respondents usually reported shorter screen time and lower frequency of phone-checking behavior as compared to digital trace, both in daily and single-time self-reports. The within-person convergent validity between daily reports and digital tracking was low, indicating poor self-reports ability to capture the actual day-to-day fluctuations in smartphone use. This study adds to the existing evidence showing that self-reports based insights into how people use smartphones differ considerably from digital trace data and shows that both person and situational levels contribute to explaining the discrepancy between digital trace and self-report data among adolescents.

This study investigates secondary education students' self-regulated learning (SRL) processes with digital trace data, particularly whether SRL processes found in secondary education are comparable to those observed in higher education. We therefore adapted a digital learning environment and rule-based AI algorithm originally designed to measure SRL in higher education and collected multi-trace data from 13-year-old students ( N = 179) across three European countries during an essay-writing task. Hidden Markov modeling was employed to capture latent SRL processes. Four latent SRL processes emerged: orientation, first-reading, writing, and re-reading combined with monitoring. By clustering sequences of these latent SRL processes, we identified four sequential patterns of SRL processes at the task level: writing with metacognitive monitoring, writing intensively, reading first, writing next, and reading and writing simultaneously. Our findings highlight how AI and multi-trace data can be used to measure SRL during learning, providing a basis for enhancing personalized support. Self-regulated learning (SRL) is vital in the digital world. In this study, we investigated secondary education students' SRL processes with digital trace data. We also demonstrated that the instrumentation of a digital learning environment and rule-based AI algorithm originally designed to measure SRL processes in higher education can be leveraged to measure SRL processes in secondary education students. The real-time measurement of covert SRL processes is important as this information can 1) raise students' awareness on their learning, 2) help teachers to support students' learning, and 3) form the basis for providing personalized support for these processes with the help of AI-enhanced learning technologies. • Collected multi-trace data from 13-year-old students during an essay-writing task. • Found four SRL processes: orientation, reading, writing, re-reading with monitoring. • Identified four sequential patterns of SRL processes at the task level. • Students' task knowledge was associated with SRL at the task level.

As the use of online services grows and capabilities in data storage as well as analytics keep on rising, researchers become increasingly interested in what digital trace data might tell them about patterns of human behavior. These data sources potentially hold new information on the mechanisms of human interaction and social phenomena. But before we can unlock the potential of these data, we have to establish how they are connected with social phenomena of interest. While there is an increasing body of research linking offline phenomena to patterns in digital trace data, there is surprisingly little research focusing on the mechanisms leading users to interact with online services. This is a serious deficit in the literature, as digital data traces do not offer us a direct view on social phenomena or events; instead, they offer us a view of reality mediated through the interests and behavior of users moving in the constricted, semi-public communication spaces provided by various digital services. In this chapter, I develop a framework for the use of digital trace data in social science. Key to this framework is a realization that the reflection of reality emerging from digital trace data might be biased through various mediating factors leading users to interact with digital services. This chapter will discuss the nature of digital trace data, various approaches for their use in the analysis of social phenomena, and close by introducing a framework for their application in social science.KeywordsSocial PhenomenonOnline ServicePolitical EventTwitter UserDigital MethodThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Educational institutions are increasingly turning to learning analytics to identify and intervene with students at risk of underperformance or discontinuation. However, the extent to which the current evidence base supports this investment is currently unclear, and particularly so in relation to the effectiveness of interventions based on predictive models. The aim of the present paper was to conduct a systematic review and quality assessment of studies on the use of learning analytics in higher education, focusing specifically on intervention studies. Search terms identified 689 papers, but only 11 studies evaluated the effectiveness of interventions based on learning analytics. These studies highlighted the potential of such interventions, but the general quality of the research was moderate, and left several important questions unanswered. The key recommendation based on this review is that more research into the implementation and evaluation of scientifically driven learning analytics is needed to build a solid evidence base for the feasibility, effectiveness and generalizability of such interventions. This is particularly relevant when considering the increasing tendency of educational institutions around the world to implement learning analytics interventions with only little evidence of their effectiveness. Practitioner Notes What is already known about this topic? Drop‐out rates and underachivement is a significant issue at most Western universities. Learning analytics have been shown to predict student performance and risk of dropping out. Interventions based on learning analytics have emerged in recent years, some reportedly successful. What this paper adds The paper also reviews and synthesizes the evidence on the effectiveness of learning analytics interventions targeting student underperformance, experience and discontinuation. The paper compares and contrasts past and current learning analytics methods and foci, and makes recommendations for the future research and practice. It critically synthesizes the current evidence base on learning analytics interventions, which is a field that is in constant flux and development. Implications for practice and/or policy The paper focuses on an increasing part of higher education with the goal of validating learning analytics methods and usefulness. The paper makes evidence‐based recommendations for institutions wishing to implement learning analytics programs and/or interventions. The paper makes evidence‐based recommendations for instructors as well as researchers in the field.

Higher education institutions are developing the capacity for learning analytics. However, the technical development of learning analytics has far exceeded the consideration of ethical issues around learning analytics. We examined higher education academics’ knowledge, attitudes, and concerns about the use of learning analytics though four focus groups (N = 35). Thematic analysis of the focus group transcripts identified five key themes. The first theme, ‘Facilitating learning’, represents academics’ perceptions that, while currently unrealized, there could be several benefits to learning analytics that would help their students. Three themes; ‘Where are the ethics?’, ‘What about the students!’, and ‘What about me!’ represented academics’ perceptions of how learning analytics could pose some considerable difficulties within a higher education context. A final theme ‘Let’s move forward together’ reflected that despite some challenges and concerns about learning analytics, academics perceived scope for learning analytics to be beneficial if there is collaboration between academics, students, and the university. The findings highlight the need to include academics in the development of learning analytics policies and procedures to promote the suitability and widespread adoption of learning analytics in the higher education sector.

Purpose The purpose of this paper is to present a systematic review of the mounting research work on learning analytics. Design/methodology/approach This study collects and summarizes information on the use of learning analytics. It identifies how learning analytics has been used in the higher education sector, and the expected benefits for higher education institutions. Empirical research and case studies on learning analytics were collected, and the details of the studies were categorized, including their objectives, approaches, and major outcomes. Findings The results show the benefits of learning analytics, which help institutions to utilize available data effectively in decision making. Learning analytics can facilitate evaluation of the effectiveness of pedagogies and instructional designs for improvement, and help to monitor closely students’ learning and persistence, predict students’ performance, detect undesirable learning behaviours and emotional states, and identify students at risk, for taking prompt follow-up action and providing proper assistance to students. It can also provide students with insightful data about their learning characteristics and patterns, which can make their learning experiences more personal and engaging, and promote their reflection and improvement. Originality/value Despite being increasingly adopted in higher education, the existing literature on learning analytics has focussed mainly on conventional face-to-face institutions, and has yet to adequately address the context of open and distance education. The findings of this study enable educational organizations and academics, especially those in open and distance institutions, to keep abreast of this emerging field and have a foundation for further exploration of this area.

Research background: Higher education institutions are generating multiple formats of data from diverse sources across the globe. The data ingestion layer is responsible for collecting data and transform for analysis. Learning analytics plays a vital role in providing decision-making support and selection of suitable timely intervention. The lack of tailored big-data ingestion architectures for academics led to several implementation challenges. Purpose of the article: The purpose of this article is to propose data ingestion architecture enabled for big data learning analytics. Methods: The study reviews existing literature to examine big-data ingestion tools and frameworks; and identify big-data ingestion challenges. An optimized framework for the real world learning analytics application was not yet in place at global higher educations. Consequently, the big-data ingestion pipeline is experiencing challenges of inefficient and complex data access, slow processing time, and security issues associated with transferring data to the system. The proposed data ingestion architecture is based on review of recent literature and adapts best international practices, guidelines, and techniques to meet the demand of current big-data ingestion issues. Findings & value added: This study identifies the current global challenges in implementing learning analytics projects. Review of recent big data ingestion techniques has been done based on defined metrics tuned for learning analytics purposes. The proposed data ingestion framework would increase the effectiveness of collecting, importing, processing and storing of learning data. Besides, the proposed architecture contributes to the construction of full-fledged big-data learning analytics ecosystem of higher educations.

Digital trace data have become one of the central pillars of media research methods. Despite the opportunities for better understanding individual users’ true behaviors in the personalized media environment, many scholars have pointed out the potential for bias in trace data collections, questioning the generalizability of findings based on them. In this study, we propose two statistical bias correction methods–Inverse Probability Weighting (IPW) and Type II Tobit, which are designed to remedy selection bias of inference from digital trace data donated by research participants. Applying these methods to Facebook take-out data, we demonstrate how the correction methods can change estimated effect sizes, which is important for the translation of academic findings into real-world impacts. We conduct two simulation studies, one under fully synthetic and another under partially simulated conditions, and find that Type II Tobit generally provides a more robust and cost-efficient correction method for digital trace data.

Potential benefits of learning analytics (LA) for improving students’ performance, predicting students’ success, and enhancing teaching and learning practice have increasingly been recognized in higher education. However, the adoption of LA in higher education institutions (HEIs) to date remains sporadic and predominantly small in scale due to several socio-technical challenges. To better understand why HEIs struggle to scale LA adoption, it is needed to untangle adoption challenges and their related factors. This paper presents the findings of a study that sought to investigate the associations of adoption factors with challenges HEIs face in the adoption of LA and how these associations are compared among HEIs at different scopes of adoption. The study was based on a series of semi-structured interviews with senior managers in HEIs. The interview data were thematically analysed to identify the main challenges in LA adoption. The connections between challenges and other factors related to LA adoption were analysed using epistemic network analysis (ENA). From senior managers’ viewpoints, ethical issues of informed consent and resistance culture had the strongest links with challenges of learning analytic adoption in HEI; this was especially true for those institutions that had not adopted LA or who were in the initial phase of adoption (i.e., preparing for or partially implementing LA). By contrast, among HEIs that had fully adopted LA, the main challenges were found to be associated with centralized leadership, gaps in the analytic capabilities, external stakeholders, and evaluations of technology. Based on the results, we discuss implications for LA strategy that can be useful for institutions at various stages of LA adoption, from early stages of interest to the full adoption phase.

Learning analytics (LA) employs educational data to improve the timeliness of support for learners. Apart from technical aspects, there is a need to understand social complexities brought about by different stakeholders, so as to systematise the adoption of LA in Higher Education (HE). We present an analysis of the situation, needs and challenges of LA in the context of Spanish HE, considering managers’, teachers’ and students’ perspectives. Qualitative research is employed using recursive abstraction. Results reveal that the level of institutional adoption is low and none of the analysed institutions had an LA policy. Furthermore, only two of these institutions had an initial LA strategy. While the institutions shared some commonalities in their objectives for LA, chosen tools and adoption challenges, the distinct differences in the political contexts and institutional practices among the institutions reaffirmed that LA solutions and services cannot be implemented in the same manner. Moreover, different needs for LA and concerns are identified about its adoption among managers, students and teachers. These observations lead to our conclusion that the main challenges to implement LA in Spain are not related to technological issues but to the social and cultural issues rooted in institutions and those associated with different stakeholders.

Persistent alignment, resource, and communication challenges constrain strategic implementation and wide-scale use of learning analytics (LA) in higher education context. Staff-facing LA solutions hold great potential benefits for higher education to offer targeted support for students but achieving them requires overcoming persistent staff involvement and LA adoption challenges. There are multiple frameworks that attempt to describe LA implementation, but the consideration of accountabilities of different stakeholders, especially non-managerial staff, is limited. Thus, in this study, we focus on the role and responsibilities of staff as active participants in LA implementation work. Through abductive analysis of individual and focus group interviews (N = 22), we examine non-managerial staff’s perspectives on LA implementation work in a case higher education institution undergoing a staff-facing LA dashboard implementation. We address the key dimensions of institutional LA implementation. This study provides insights into staff expectations and experiences of LA implementation, elaborating the present challenges and development needs. Our results highlight the importance of involving staff as active agents in LA implementation, considering what actionable insights the implemented LA solution provides to its users, and defining stakeholder accountability in LA. The results support the creation of aligned practices for user-centered implementation of technologies for education. This includes considering stakeholder communication, visions, and responsibilities, and ultimately leading to transparent institutional practices.

The development and implementation of learning analytics as a mechanism to support student success has been an emerging trend within Higher Education. Previous research identifies that learning analytics is an innovative educational development but recognises that little attention has been paid to evaluating its effectiveness or pedagogic usefulness. Researchers recognise that learning analytics is a new field in need of further research to aid its credibility within the educational arena. This research study provides a better understanding of learning analytics to support student success through the examination of opportunities and challenges of learning analytics from a multi-stakeholder perspective. This study also demonstrates how learning analytics can be successfully implemented within Higher Education. Through an interpretivist paradigm, this cross-sectional research study captures the unique experiences of students, academic staff and learning analytics experts. Data collected through twelve semi-structured interviews and three student focus groups enabled the researcher to gather a broad understanding of learning analytics from those involved and provides an holistic portrayal from this cultural group. The main findings of this study suggest that learning analytics need to have a clear context and purpose within Higher Education to ensure successful development, effectiveness and pedagogic usefulness. Effective organisational change, culture, academic and student engagement, ownership and motivation are paramount. Findings also indicate disparities in the implementation of learning analytics within Higher Education, which require resolution to ensure success, and there is some discussion about how challenges can be overcome to ensure effective institutional adoption and student success.

Learning analytics leverages the use of technology to gather and analyse data about student performance, engagement, and learning behaviours, which can help educators make informed decisions about how to improve learning outcomes. In higher education, learning analytics can provide insights into student engagement, performance, and learning pathways. Collaborative learning in higher education involves a group of students working together towards a common goal or task. Collaborative learning encourages students to work together to solve problems, analyse information, and make decisions, especially in self-directed learning environment. Through discussion and debate, students develop critical thinking skills and learn to approach problems from different perspectives to enhance problem-solving skills. Collaborative learning provides opportunities for students to work with others from diverse backgrounds, helping them develop interpersonal skills such as communication, teamwork, and leadership. While collaborative learning helps students to foster critical thinking and develop interpersonal skills, students’ activities and engagement in collaborative learning are not properly assessed and measured. Student performance indicators are highly dependent on the learning activities and resources used in the learning management system based on individual basis. The ability and potential of learning analytics to track students’ behaviour and performance in team, and to monitor the effectiveness of their sharing and communication is not fully utilised in higher education. This paper addresses this issue and aims to provide an overview of collaborative learning analytics. The overview elaborates essential elements in collaborative learning and defines features in analytics to support collaborative learning. The overview is expected to guide to educators and developers in promoting and assessing students’ performance based on collaborative works.