Assessing participant learning in online environments

Although recommender systems have the potential to advance self-regulated learning (SRL) in online environments, implementation challenges impede their development. To systematically investigate the barriers to the effective use of recommender systems to enhance SRL, this study developed a three-tiered analytical framework that considers challenges in three areas: data and experiments, user experience/interface design, and technical factors. A systematic search of five bibliographic databases identified 33 empirical studies. The findings revealed the following key challenges in the three analytical levels: (a) data and experiments: scarcity, validity, and arrangement; (b) user experience and engagement: poor matching and inadequate scaffolding for SRL (e.g., weak feedback mechanisms) and suboptimal interface and function designs; and (c) technical factors: lack of stability and security of recommender systems and limitations in modeling complex learning behaviors and balancing personalization with pedagogical goals. These challenges interact dynamically to form a self-reinforcing recursive constraint, where poor user experiences lead to disengagement, resulting in sparse, low-quality data that further constrains algorithmic capabilities. Current algorithmic approaches typically address isolated problems, failing to break this cycle by overlooking these systemic interdependencies. This underscores the critical need to move beyond isolated fixes and develop comprehensive, pedagogically-aware algorithmic architectures. To address this, this study proposes a four-phase development roadmap for creating an SRL-Scaffolding Architecture. This roadmap integrates multi-modal data validation, hybrid algorithmic intelligence combining emotional and cognitive modeling, adaptive scaffolding interfaces, and a continuous human-in-the-loop feedback cycle. By mapping these interconnected challenges and proposing a structured architectural framework, this study proposes roadmap to address these barriers and develop recommender systems that effectively empower SRL in online environments.

The Framework for DIRECTED Online Learning Environments

Online and blended learning requires the reconstruction of instructor and learner roles, relations, and practices in many aspects. Assessment becomes an important issue in non-traditional learning environments. Assessment literacy, i.e., understanding assessment and assessment strategies, is critical for both instructors and students in creating online and blended environments that are effective for teaching and learning. Instructors need to identify and implement assessment strategies and methods appropriate to online or blended learning. This includes an understanding of the potential of a variety of technology tools for monitoring student learning and improving their teaching effectiveness. From the students’ perspective, good assessment practices can show them what is important to learn and how they should approach learning; hence, engaging them in goal-oriented and self-regulatory cognitions and behaviors.The book targets instructors, instructional designers, and educational leaders who are interested in understanding and implementing either summative or formative assessment in online and blended learning environments. This book will assist the relevant audience in the theory and practice of assessment in online and blended learning environments. Providing both a research and practice perspective, this book can help instructors make the connection between pedagogy and technology tools to maximize their teaching and student learning. Among the questions addressed in this book are:• What assessment strategies can be used in online or blended learning?• How can instructors design effective assessment strategies?• What methods or technology tools can be used for assessment in online or blended learning?• How does peer-assessment work in online or blended learning environments?

Many music educators are concerned about building healthy classroom communities as they prepare to transition to online or blended learning environments. Research involving distance education has indicated that several techniques in the design and delivery of online instruction can increase students’ sense of community, commitment to the course, and overall learning experience. The purpose of this research-to-resource article is to provide music educators with research-based strategies for establishing and connecting students to a sense of community in courses delivered partially or entirely online.

Research on how Chinese students perceive their social presence, especially in the UK's higher education and online environment, has received less attention. This qualitative study will fill the research gap by providing data to understand Chinese students' attitudes towards Zoom-based student-led online learning projects, aiming to explore how Chinese students perceive social presence and what factors influence their social presence. We collected data by conducting semi-structured interviews with seven postgraduate students who attended an online Zoom-based student-led project, "Holistic Academic and Research Reflective Practice". It is found that students' perceived social presence can be grouped into three themes: conceptualization of social presence; the level of perceived social presence in the online environments; and the perceived importance of social presence. The factors that affect the social presence of Chinese students can be put into three categories: social context, online communication, and interactivity. We suggest that students should always be aware of the impact of emotional values and human interaction in online learning environments in order to have a positive learning experience. Chinese students who want to take part in more interactive online learning environments should think about how to build a healthy sense of social presence.

Technology makes possible abundant new opportunities to capture and display data in online learning environments. We describe here an example of using these opportunities to improve students' use of the rich supports available in online learning environments. We describe an example of a blended learning experience that uses an online inquiry‐based middle‐school science curriculum in which we provided sixth graders (n = 126) data aligned with the Universal Design for Learning instructional framework. Students were provided and asked to reflect on their own data not only about performance but also about use of optional embedded supports vis‐à‐vis their perceptions of difficulty of key concepts. We determined that students were generally able to understand and interpret these rich data and that providing these data influenced subsequent help seeking in the online environment. We discuss implications for supporting help seeking and designing assessment and feedback within online learning environments.

Collecting and analyzing data for continuous improvement of student learning has been a staple of the United States education system for the past two decades, but K-12 education is experiencing a paradigm shift to online learning. Teachers in face-to-face environments have experience monitoring student learning and collecting data through formative assessment. Assessment strategies implemented in face-to-face classrooms can be modified for an online learning environment. Additionally, higher education has utilized online platforms far longer than K-12 education and have extensive knowledge on data collection in an online learning environment. K-12 pre-service and in-service teachers can learn from the trials and tribulations of higher education in online learning. Lastly, teacher educators need to respond to this change in the instructional environment by altering their current curriculum to prepare preservice teachers to teach in an online environment. This chapter addresses these issues and provides practical examples and strategies for data collection in K-12 online learning environments.

Open, online learning environments, such as massive open online courses (MOOCs) and open learning communities have been promoted as a way to expand equitable access to quality education. Such learning experiences are potentially enriched via extensive networks of peer learners. Even though challenges exist to realize these aspirations, open, online learning environments can serve as a mechanism for how we provide transformative learning experiences. This workshop aims to bring researchers and practitioners from diverse disciplinary backgrounds to explore how the body of CSCW knowledge can better support the vision of sustaining peer-to-peer learning in online environments. Integrating contributions from designers, researchers, and practitioners at the intersection of CSCW & education, participants will co-create future visions and proposed implementations for open, online learning environments.

Online learning environments (OLE) play a vital role in delivering quality learning outcomes. However, despite calls for improved learning environments, the practice of translating traditional face-to-face delivery into quality online offerings remains patchy. OLEs have implications for student experience, and thus student retention. In this paper we examine three domains that shape student experience, ‘relevance of online content’, ‘alignment of online content with student aspirations’, and ‘navigation with the online environment’ and propose that students evaluate online materials based on what they find interesting, and what they deem a value-add investment in exchange for their time. Drawing from the literature, as well as our experience in the field, we present a conceptual framework, the Tri-layered Student Online Experience Framework (TSOEF), which aims to act as a practical resource for academic and education technologists for informing the design of online units. To illustrate how our Framework can be operationalised, we provide an implementation case study centred on a third-year undergraduate unit at the University of Tasmania in Australia. This paper offers a practical guide for providing students with value-driven offerings.

Low Higher Order Thinking Skills (HOTS) among students is a persistent and pervasive challenge in modern classroom. The problem remains for years despite the intensive manoeuvres to solve it. Yet, the need to enhance HOTS is imperative in ensuring quality education. The purpose of this research is to enhance HOTS among students in Mathematics using an online learning environment. The online learning environment was developed based on Polya’s problem solving model. However, the system is not discussed in this paper. There are three research objectives which are to investigate the current HOTS achievement among students, to study the effect of the developed online learning environment in nurturing HOTS among samples and finally to study how learning process taken place to achieve HOTS among sample while engaging the system. This research implemented experimental research design with a set of pre and posttest and the developed online learning environment as intervention. A total of 76 samples were sampled for the preliminary study and another 32 samples for the four weeks treatment using cluster random sampling technique. Samples are from three schools from a district in Malaysia. Finding shows that the current HOTS achievement among students are still low at mean = 32.05, σ = 15.87. During the intervention, samples show improvement in their HOTS from mean = 39.52, σ = 11.65 in the pretest to higher level in the posttest at mean = 77.84, σ = 12.84. The Paired-Samples T Test conducted to identify the differences between the pre and posttest shows a statistically significant difference at t (31) = -11.58, p < 0.05. Indicating that an online learning environment using Polya’s problem solving for four weeks enhance HOTS among samples. The process of how samples achieve HOTS is also reported. This research shows the importance of integrating technology and inquiry in modern classroom to increase students’ thinking skill ability. Finding suggests that HOTS can be nurtured via technology enhanced learning environment.

This study utilized Expectancy Confirmation Theory (ECT) and Technology Acceptance Model (TAM) to examine satisfaction of students and instructors toward online learning tools and resources in online learning environments. The study findings indicate that student expectation was the most important factor that helped the instructors to design and provide appropriate and efficient technology tools and resources to enhance student learning. Further, the instructors were satisfied with the technology tools and resources that were provided, as these helped them to achieve their instructional expectations. These findings could help to stimulate reflections on ways to improve and design useful e-learning tools and resources that could enhance effective teaching and learning in online environments.

The first aim of this chapter is to present a literature review regarding two very important concepts for the online learning environment: social presence and personality. The second aim is to present the findings of an exploratory study that measured students’ perceptions regarding different aspects of their experience with the online learning environment including social presence and personality of lecturers in the online environment in particular. An online survey (developed using Qualtrix) was emailed to 474 off-campus postgraduate and 699 undergraduate students who are pursuing a coursework degree in management. To date, 62 responses from postgraduate and 41 from undergraduate students have been received, which indicates a response rate of 13 percent and 5.9 percent, respectively. Results show that social presence is not being successfully developed by lecturers, with discussion boards and chat rooms showing relatively low evaluations. In terms of lecturers’ personalities in the online environment, results show that some lecturers do not have a clear structure for their discussion boards and chat rooms and that the language used by lecturers in the online environment (especially discussion boards and chat rooms) differs from their language in face-to-face contexts. Other online learning tools, such as special podcasts and special vodcasts, show less difference in the personality of lecturers in the online and face-to-face contexts, especially in terms of language and tone of voice used by lecturers.

Understanding the effectiveness of peer mentor training in online learning environments prepares students for their roles as online peer mentors. In the Learning Assistant peer mentor program, Learning Assistants (LAs) play a crucial role in supporting student learning and engagement in STEM courses. Due to the shift to online learning during the COVID-19 pandemic, many LAs began working in online synchronous and asynchronous courses. We gathered information on challenging interactions encountered by LAs, and the frequencies of these interactions were analyzed and compared between the two learning environments. The results revealed that while no new challenging interactions emerged in the online environment, there were nuances in how existing interactions manifested online. In this article, we present new challenging interaction scenarios designed for online learning environments. These scenarios aim to enhance the training in the LA pedagogy course. Furthermore, the study highlights the significance of behavioral engagement barriers, such as unprepared and disinterested students, which had higher frequencies in both in-person and online environments. These findings help create a foundation for online LA training by incorporating new scenarios and focusing on behavioral engagement barriers to help the LA pedagogy course better equip LAs to navigate challenging interactions and support student learning.

Online learning environments have become a crucial means to provide flexible and personalised pedagogical material, and a major driving cause is due to the COVID-19 pandemic. This has rapidly forced the migration and implementation of online education strategies across the world. Online learning environments have a requirement for high trust and confidence in establishing a student’s identity and the authenticity of their work, and this need to lessen academic malpractices due to increased online delivery and assure the quality in education has accelerated. In addition to this, due to the ubiquity of mobile devices such as smartphones, tablets and laptops, students use a variety of devices to access online learning environments. Therefore, authentication systems for online learning environments should operate effectively on those devices to authenticate and invigilate online students. Confidence in authentication systems is also crucial to detect cheating and plagiarism for online education as strong authorisation and protection mechanisms for sensitive information and services are bypassed if authentication confidence is low. In this paper, we examine issues of existing authentication solutions for online learning environments and propose a design for an adaptive biometric authentication system for online learning environments that will automatically detect and adapt to changes in the operating environment. Multi-modal biometrics are applied in the proposed system which will dynamically select combinations of biometrics depending on a user’s authenticating device. The adaptation strategy updates two thresholds (decision and adaptation) as well as the user’s biometric template they are using the authentication system.

This chapter employed qualitative and case study approaches to elicit valuable insights from De La Salle University tertiary teachers' use of digital educational technology and AnimoSpace in addressing the challenges posed by online learning environments and the issues of equity and inclusion. Analysis of interviews and AnimoSpace statistical data was conducted through the lens of the task-technology fit perspective outlining the tasks and roles teachers perform in the online environment and the characteristics of AnimoSpace. Findings revealed that similar tasks and roles were performed when teaching and learning were shifted online. AnimoSpace and other digital technology fostered equity and inclusion in the online learning environment. However, additional roles/role changes were uncovered, specifically Technologist, Basic Troubleshooter, Instructional Designer, and Alternative Assessor. In addition, the lack of cost-effective or free proctoring tools compelled teachers to seek alternative forms of assessment to measure learning outcomes.