Enhancing Peer Assessment Validity with Engagement Behaviors: A Structural Equation Modeling Approach

This chapter analyzes the instructional quality and learning design of different categories of online courses and their history, with a special focus on massive open online courses (MOOCs). Online courses have a long tradition that has gained public attention, broad interest, and huge numbers of participants thanks to the introduction of free MOOCs accessible online for all interested learners worldwide. In this chapter, we first define MOOCs, their characteristics, and history. Afterward, theoretical frameworks and practical instruments and tools based on scientific research are presented. From the beginning, the quality of MOOCs (and of online courses in general) has been debated. That led to discussions about the learning design and outcomes of MOOCs, which we introduce in the next section. Key research findings and practical validated instruments for designing and evaluating MOOCs (and online courses in general) are presented. Then following, the key benefits of MOOCs and the main arguments and scenarios for their usage are summarized. Based on our analysis of the research results, practices, and standards, a framework for categories and types of (massive open) online courses is proposed, called the typologies of online courses (TOC) framework. As part of the global community for open educational resources (OER) and in combination with the UNESCO recommendation on OER, MOOCs can play a significant role in achieving the SDG4 of the United Nations: inclusive and equitable quality education for all. This is true in particular during times of public lockdowns, such as during the ongoing COVID-19 pandemic.

Peer grading affords a scalable and sustainable way of providing assessment and feedback to a massive student population, and has been used in massive open online courses (MOOCs) on the Coursera platform. However, currently there is little empirical evidence to support the credentials of peer grading as a learning assessment method in the MOOC context. To address this research need, this study examined 1825 peer grading assignments collected from a Coursera MOOC with the purpose of investigating the reliability and validity of peer grading as well as its perceived effects on students’ MOOC learning experience. The empirical findings proved that the aggregate ratings of student graders can provide peer grading scores that were fairly consistent and highly similar to the instructor grading scores. Student responses to a survey also show that the peer grading activity was well received as the majority of MOOC students believed it was fair, useful, beneficial, and would recommend it to be included in future MOOC offerings. Based on the empirical results, this study concludes with a set of principles for designing and implementing peer grading activities in the MOOC context.

Open online distance learning in higher education has quickly gained popularity, expanded, and evolved, with Massive Open Online Courses (MOOCs) as the most recent development. New web technologies allow for scalable ways to deliver video lecture content, implement social forums and track student progress in MOOCs. However, we remain limited in our ability to assess complex and open-ended student assignments. In this paper, we present a study on the quality of self- and peer assessments in three MOOCs. In general, the quality of self-assessments and peer assessments was low to moderate, suggesting that both self-assessment and peer assessment should be used as assessment for learning instead of assessment of learning. Based on low correlations with final exam grades and other assessment forms, we conclude that self-assessments might not be a valid way to assess students’ performance in MOOCs. Yet the weekly quizzes and peer assessment significantly explained differences in students’ final exam scores, with one of the weekly quizzes as the strongest predictor. Future research on MOOCs implies a reconceptualization of education variables, including the role of assessment of students’ achievements.

A massive open online course on the laws of thermodynamics has been designed using the 5Es (Engage, Explore, Explain, Elaborate, Evaluate) lesson planning methodology and delivered using everyday engineering examples to provide familiar contexts for both a global cohort of more than 3000 learners ranging in age from teenagers to septuagenarians in 137 different countries plus a large class of 355 university undergraduates. In addition to the structured pedagogy, the massive open online course included innovations in the use of practical exercises, that could be performed in the learner’s kitchen, and worked examples to enhance the learning experience. The massive open online course was provided in parallel with an introductory undergraduate course and as an additional learning resource for the undergraduate students. A little less than one-fifth of the undergraduates participated in the massive open online course and just over half of these responded that it would be an acceptable substitute for traditional lectures. The percentage completing the massive open online course was relatively high at 28% compared to 21% for other massive open online courses on the same platform, which is attributed to the pedagogy and strong support for social learning based on a survey of fully participating learners in the last week of the five-week course.

The running security and stability of massive open online course's teaching network directly affect the implementation process of massive open online course's teaching tasks. In order to provide valuable reference data for the management and maintenance of the network, this paper puts forward an analysis method of abnormal traffic of higher vocational massive open online course's teaching network based on deep convolution neural network. According to the structure of the teaching network of massive open online course in higher vocational colleges, a network model is built. Under this model, the flow data is collected, and the preprocessing of the initial flow data is completed through data cleaning, standardized conversion and clustering. The deep convolution neural network is established, and the characteristics of network traffic data are extracted through back propagation iteration. After discrete detection and feature matching, the abnormal traffic in the teaching network of higher vocational massive open online course is detected, and the visual analysis results are obtained. Compared with the traditional network abnormal traffic analysis method, it is found that the detection error and missed detection rate of the optimized design method are reduced by 3.75MB and 1.02% respectively, the accuracy of traffic abnormal type analysis is increased by 1%, and the analysis speed is obviously improved.KeywordsDeep Convolution Neural NetworkTeaching in Massive Open Online CourseTeaching NetworkAbnormal Flow Analysis

MOOC(Massive Open Online Course)란 대단위로 누구나 무료로 수강할 수 있는 온라인 강좌이다. 이러한 MOOC가 기존 대학체계를 무너뜨리고, 교육의 새 지평을 열 것이라는 전망이 있었다. 그러나 MOOC는 기존의 고등교육기관을 위협할 것이라는 최초의 전망과는 다른 양상으로 진행되고 있다. 그래서 본 연구는 MOOC의 허와 실을 드러냄으로써 MOOC를 우리 고등교육의 실정에 맞게 제작하고 활용할 수 있는 논의의 시발점을 마련하고자 한다. MOOC는 기본적으로 온라인 교육이므로 기존 온라인 교육의 본질적인 문제점인 '상호작용, 감독 및 평가'의 문제를 그대로 계승할 수밖에 없다. 그리고 새로이 추가된 'Massive' 개념은 교육 현장에서 가장 민감한 부분인 강좌당 학생 수 문제를 불러일으키며, 'Open' 개념은 MOOC가 불특정 다수를 염두에 둔 강좌이므로 맞춤형 교육에 역행된다. 결론적으로 MOOC는 자기 주도적 학습 능력이 매우 탁월한 학습자들과 전통적인 온라인 고등교육기관에도 접근할 수 없는 사람들을 위한 교육 서비스로 자리매김 되고, 대학에서는 Flipped Learning의 수단으로 사용될 가능성이 매우 높다. 그러므로 향후 MOOC에서는 이 점이 고려되어 강좌의 대상에 맞게 차별화되어 개발될 필요가 있다. MOOC (Massive Open Online Course) is an online course aimed at limited participation and open access via the web. There is even the prospect that MOOC may break down the existing university system and open a new horizon of education. However, MOOC is going differently from the first prospect that it would threat the existing higher education institutions. So this study wishes to provide a starting point of discussion that we can produce and utilize MOOC to fit our situation of higher education by revealing the inherent problems of MOOC. MOOC is a basically online education, so that it cannot help inheriting the essential weakness of the existing one: the interaction, supervision and evaluation. The newly added concept 'massive' raises the problem of class size which is the most sensitive part in the field of education. The concept 'open' reverses a customized education because MOOC is a kind of video clips of a lecture for unspecific massive learners. As a conclusion, we predicts that MOOC will be positioned as a higher education service for very prominent learners in self-led learning ability and people who cannot access even traditional online higher education institutions. Also MOOC is highly likely to be used as a means of Flipped Learning in universities. Therefore, considering these points in the future, we need to develop MOOC to suit these targets.

BackgroundMassive open online courses (MOOCs) have become immensely popular in a short span of time. However, there is very little research exploring MOOCs in the discipline of health and medicine.ObjectiveWe aim to provide a review of MOOCs related to health and medicine offered by various MOOC platforms in 2013, by analyzing and comparing the various offerings, their target audience, typical length of course, and credentials offered. We also discuss opportunities and challenges presented by MOOCs in health and medicine.MethodsHealth and medicine–related MOOCs were gathered using several methods to ensure the richness and completeness of data. Identified MOOC platform websites were used to gather the lists of offerings. In parallel, these MOOC platforms were contacted to access official data on their offerings. Two MOOC aggregator sites (Class Central and MOOC List) were also consulted to gather data on MOOC offerings. Eligibility criteria were defined to concentrate on the courses that were offered in 2013 and primarily on the subject of health and medicine. All language translations in this paper were done using Google Translate.ResultsThe search identified 225 courses, of which 98 were eligible for the review. Over half (58%, 57/98) of the MOOCs considered were offered on the Coursera platform, and 94% (92/98) of all the MOOCs were offered in English. Universities offered 90 MOOCs, and the John Hopkins University offered the largest number of MOOCs (12/90). Only three MOOCs were offered by developing countries (China, West Indies, and Saudi Arabia). The duration of MOOCs varied from 3-20 weeks with an average length of 6.7 weeks. On average, MOOCs expected a participant to work on the material for 4.2 hours a week. Verified certificates were offered by 14 MOOCs, while three others offered other professional recognition.ConclusionsThe review presents evidence to suggest that MOOCs can be used as a way to provide continuous medical education. It also shows the potential of MOOCs as a means of increasing health literacy among the public.

One of the major challenges facing Massive Open Online Courses (MOOCs) is assessing the learner performances beyond traditional automated assessment methods. This leads to a bottleneck problem due to the massiveness of course participants, especially in the context of problem solving. To tackle this issue, peer assessment has been proposed as an effective method. However, the validity of this process is still under discussion, suffers from a lack of credibility and has many weaknesses, particularly with regards to group formation. This paper develops a new method of peer assessment for MOOCs to improve the accuracy and exactitude of the learner grade. Our proposition is based on three main steps: the formation of learner groups, the assessment and synthesis of the results. First, the group definition process can use different elements of the learner model and enables to build heterogeneous groups. After, each learner is required to grade a small number of peer productions. Finally, a synthesis of the various grades is proposed using both data about the ability to assess of each learner and complexity of problems. To evaluate the proposed peer assessment process, we conducted an experimentation devoted to teaching Software Quality Assurance to beginners with computer science during the first university cycle.

Problem and purpose. The article analyzes the development of massive open online courses (MOOC), their structure and application in the practice of higher education, the introduction of new experience and strategy of distance learning at the Department of Physical Culture of the Siberian Federal University. The purpose of the article is to analyze the stages of the emergence and development of one of the areas of distance learning – massive open online courses (MOOC) and to analyze their application in the educational process of the University. The methodology of the research is based on the analysis of theoretical materials and research works of foreign and domestic scientists on the development of massive open online courses and the experience in the implementation of massive open online courses. Results. The article is devoted to the theory and experience of using MOОС in the world and in the educational process of the University. The authors describe the main intermediate results of the massive open online courses introduction in the educational process at the Department of Physical Culture of the Siberian Federal University. The problems, errors in the work with the MOОС and the aspects correcting the errors in future are analyzed. Conclusion. The results obtained so far allow us to conclude that the possible option of using distance learning technologies in the form of MOОС in teaching the discipline of „Physical Culture” is effective but requires additional research and further considerations.

Purpose: Building a diverse healthcare workforce warrants efforts to scale health careers education for underrepresented and nontraditional students, as health career opportunity programs are not accessible to many students. A massive open online course (MOOC) is a new model for education that offers knowledge and resources for free. Accessible globally to anyone with an internet connection, MOOCs have the potential to impart interdisciplinary knowledge and skills toward students' pursuits of health related careers. Here, we describe the design, implementation, and evaluation of the MOOC, “Career 911: Your Future Job in Medicine and Healthcare.” Career 911 seeks to enable students to explore health career options and build skills for their career journey, including: articulating a personal story, resume and cover letter writing, job search, interviewing, professional networking, and professional communications. We examine the reach and characteristics of enrolled learners from the first run of Career 911 (April 13 – May 24, 2015) and investigate the challenges and concerns students had about pursuing their healthcare career goals. Methods: Data collected included students' self-reported demographic information from a pre-course survey as well as students' content usage and engagement from participant tracking logs and activity metadata. Self-reported career challenges were parsed from homework submissions and coded using standard qualitative methods. Results: A total of 8,338 students from 159 different countries (44% from the United States) enrolled in the first run of Career 911. Whereas MOOC student populations tend to be male, older, highly educated, and employed, Career 911 students were female (56%), younger (mean = 34.2 and 18% were under the age of 22), currently still in school (43%), and not currently employed full time (63%). The most frequently cited challenges and concerns students had about pursuing their career goals included finances, time management, competition of getting jobs/getting into programs, and family commitments. In the six week run of the course, 450 students explored career websites; 220 students mapped their 1, 2, and 5-year career steps to achieve their target profession/job; 163 students put together a professional networking plan with informational interview questions; 132 students created a cover letter; and 125 students created a resume. A sizeable percentage of learners (11%) indicated that they were educators and were interested in using the course content for their own students. Conclusion: MOOCs have the potential to scale health careers exploration and career readiness resources to global populations, supporting thousands of students through content that can scale. We discuss several key challenges and opportunities for future MOOCs to enhance entry into health care careers and impact health equity globally. Citation Format: Melissa A. Simon, Laura S. Tom, Kasey Brown, Shaneah Taylor, Nadia Hajjar, Emmanuel Cordova Telles, Erika de la Riva. Career 911: A massive open online course (MOOC) to promote healthcare workforce diversity. [abstract]. In: Proceedings of the Eighth AACR Conference on The Science of Health Disparities in Racial/Ethnic Minorities and the Medically Underserved; Nov 13-16, 2015; Atlanta, GA. Philadelphia (PA): AACR; Cancer Epidemiol Biomarkers Prev 2016;25(3 Suppl):Abstract nr B01.

The dominant direction in the development of online education, actively discussed by the pedagogical community, is training using massive open online courses (MOOCs). The key factor in the effectiveness of the implementation of MOOCs in the practice of educational organizations is primarily the availability of elite education. In modern conditions, which determine the priority of lifelong education (lifelong learning), it is particularly relevant to form schoolchildrens readiness for self-education, which can be facilitated by training on MOOC platforms. The broad discussion of MOOC implementation in educational practice is more concerned with university education, additional education and the system of advanced training. The use of MOOCs in school education in the publications of teachers-researchers is not considered fully enough and is mostly related to natural sciences. The paper analyzes the MOOC on the example of humanities and the features of the implementation of mixed learning models (inverted learning, autonomous group, individual trajectory) based on the use of online courses. This analysis suggests that there should be a need for more in-depth research in this area and the need to introduce elements of learning into school practice through massive open online courses. This will resolve the contradiction between the need for differentiation and individualization of training and the potential of existing unused massive open online courses.

Massive Open Online Courses (MOOCs), as one type of disruptive technology, has been described as the “campus tsunami” that is purportedly poised to change the face of higher education. Peer assessment has been adopted as a strategy to stimulate students’ active engagement in learning in MOOCs. While it is considered an effective pedagogical strategy to empower students and support learning by some, it is regarded as one of the main criticisms by others. It has been the focus of the dispute among developers of MOOCs in the field in the past few years. Based on a case of the MOOC entitled, The Red Chamber Dream, on Coursera, this study aims to explore the relationship between the perception of usefulness of peer assessment and the quality of learner assignments, and whether community knowledge is advanced in MOOCs facilitated by peer assessment. All notes related to learning in the discussion board were retrieved for further analysis. Notes indicating students’ perceptions about peer assessment were downloaded for analysis. Finally, students’ essays available on the course site were also saved in individual files for further analysis. Research found that there is no correlation between the quality of students’ essay writing and note writing and the reported usefulness of providing feedback/grading to their peers. Results also suggest that no correlation exists between the quality of students’ essay writing and note writing and the reported usefulness of receiving peer feedback. Meanwhile, results suggest that students were engaged in discussions and advanced their understanding. They produced a large number of explanations and knowledge building discourse, all of which involved knowledge advancement in each week. This improvement in knowledge may result in part from their expertise and in part from peer review of peers’ work. However, we cannot identity improvement of students’ work between different weeks.

Massive open online courses (MOOCs) have received both praise and condemnation in higher education. Massive open online courses (MOOCs) debuted in 2008, and the phenomenon quickly acquired global traction. The present study aims to increase understanding of the effects of MOOCs on higher education and teachers' attitudes towards this emerging type of education by analysing existing research and data. The current study is quantitative, and the survey method of data collection was chosen due to the nature of the study and its demand. The population of the study included all higher education teachers of West Bengal. The analysis shows that there is no significant difference in male and female teachers' attitudes towards Massive Open Online Courses (MOOCs), nor is there a significant difference in teachers' attitudes towards Massive Open Online Courses (MOOCs) based on teaching experience. The major findings show that MOOCs have a positive influence on the attitude of higher education teachers.

What knowledge is available on massive open online courses in nursing and academic healthcare sciences education? A rapid review

Purpose The purpose of this study is to propose a research model based on the stimulus-organism-response (S-O-R) model to explore whether interactivity (INT) and virtual rewards giving (VRG) as environmental stimuli to learners’ learning engagement (LE) elicited by massive open online courses (MOOCs) affect their learning persistence (LP) and learning outcome (LO) in MOOCs. This study further examines whether usage experience (UE) moderates the path relationships within the research model. Design/methodology/approach Sample data for this study were collected from learners who had experience in taking VR-based MOOCs provided by the MOOCs platform launched by a well-known university in Taiwan, and 313 useable questionnaires were analyzed using structural equation modeling. Findings This study proved that learners’ perceived INT and VRG in MOOCs positively influenced their psychological LE and social LE elicited by MOOCs, which concurrently expounded their LP in MOOCs, and in turn uplifted their LO in MOOCs. Besides, this study showed that learners’ UE partially moderated the path relationships in the research model. The research model respectively explains 71.8 and 56.6% of the variance (R2) in learners’ LP and LO in MOOCs. This study first reveals that learners’ perceived INT in MOOCs is the largest antecedent that can maximize their LP and LO in MOOCs via their PLE elicited by MOOCs. Next, learners’ perceived VRG in MOOCs is the strongest antecedent that can maximize their LP and LO in MOOCs via their SLE elicited by MOOCs. Originality/value This study utilizes the S-O-R model as a theoretical groundwork to construct learners’ LO in MOOCs as a series of the psychological process, which is affected by INT and VRG. Noteworthily, while the S-O-R model has been extensively employed in previous literature, little research uses the S-O-R model to explain the impacts of INT and VRG on learners’ LP and LO in MOOCs and examine whether learners’ UE can moderate the model relationships. Hence, this study enriches the research for understanding how learners value their learning gains via INT and VRG to support them in MOOCs.