Evaluating the User Experience of a Gamification-Based Moodle LMS

Introduction Remote learning has been increasingly required after the COVID-19 pandemic. In this regard, an integrative Learning Management System (LMS) that connects external assets for technical and nontechnical skills training in minimally invasive surgery has been implemented. Before its wider use as a training tool, a comparison of usability perception between experts and students has been performed. Methods The system usability scale (SUS) was used. Users rated the 10-item questionnaire on a 5-point Likert-scale (5-Strongly agree, 1-Strongly disagree), with odd questions referring to positive aspects and even questions to negative ones. Two profiles were considered: experts in surgical training and students and residents. They were requested to use the LMS and complete any of the implemented courses. Comparison between groups was done with a Mann-Whitney U-test. Results Seven experts and 110 students participated in the study. Overall, SUS score by experts reached a mean value of 67.14 points [Q1 = 68.05, Q3= 76.81; interquartile range (IQR) = 9.67], while the mean SUS score given by students was 73.09 [Q1 = 65.00, Q3= 84.38; IQR = 19.38]. When comparing both groups, significant differences were not found (p-value=0.639). Conclusions Since SUS scores are usually considered to be valid when the average is greater than 68, experts were slightly below, while students surpassed this threshold. Nevertheless, since there were no significant differences, it might be assumed that usability of the LMS is acceptable and it will not hinder the training process, although further studies to prove its pedagogical value are required.

Voice-based User Interfaces (VUIs) challenge our existing conceptions of usability since the standardized evaluation tools we use were typically developed for interfaces with visual feedback, whereas VUI's have predominantly eyes-free interactions. We experimented with the use of a well-validated tool, the System Usability Scale (SUS), to evaluate two existing, commercially available VUIs. We administered the SUS to 12 participants after they completed a set of scenario tasks on Amazon's Alexa and Apple's Siri. The results were consistent with previous studies comparing subjective rating scores and SUS, suggesting that the SUS is a valid evaluation tool for VUIs. Additionally, despite large, significant differences in adjective scale ratings and SUS scores, both systems performed similarly on the learnability items. We conclude with recommendations on the use of the SUS for evaluating VUIs.

BackgroundUnderstanding the impact of socio-economic inequality on health outcomes is arguably more relevant than ever before given the global repercussions of Covid-19. With limited resources, innovative methods to track disease, population needs, and current health and social service provision are essential. To best make use of currently available data, there is an increasing reliance on technology. One approach of interest is the implementation and integration of mapping software. This research aimed to determine the usability and acceptability of a methodology for mapping public health data using GIS technology.MethodsPrototype multi-layered interactive maps were created demonstrating relationships between socio-economic and health data (vaccination and admission rates). A semi-structured interview schedule was developed, including a validated tool known as the System Usability Scale (SUS), which assessed the usability of the mapping model with five stakeholder (SH) groups. Fifteen interviews were conducted across the 5 SH and analysed using content analysis. A Kruskal-Wallis H test was performed to determine any statistically significant difference for the SUS scores across SH. The acceptability of the model was not affected by the individual use of smart technology among SHs.ResultsThe mean score from the SUS for the prototype mapping models was 83.17 out of 100, indicating good usability. There was no statistically significant difference in the usability of the maps among SH (p = 0.094). Three major themes emerged with respective sub-themes from the interviews including: (1) Barriers to current use of data (2) Design strengths and improvements (3) Multiple benefits and usability of the mapping model.ConclusionIrrespective of variations in demographics or use of smart technology amongst interviewees, there was no significant difference in the usability of the model across the stakeholder groups. The average SUS score for a new system is 68. A score of 83.17 was calculated, indicative of a “good” system, as falling within the top 10% of scores. This study has provided a potential digital model for mapping public health data. Furthermore, it demonstrated the need for such a digital solution, as well as its usability and future utilisation avenues among SH.

BACKGROUND Preventative self-care can reduce dental disease that disproportionately burdens vulnerable populations. Personalized digital oral self-care behavioral interventions offer a promising solution. However, the success of these digital interventions depends on toothbrushing data collection e-platforms attuned to the needs and preferences of vulnerable communities. OBJECTIVE The aim of this study is to assess the usability and feasibility of the Remote Oral Behaviors Assessment System (ROBAS), which has been adapted to address the unique requirements of socioeconomically disadvantaged minority individuals. METHODS A cohort of 53 community-clinic participants, including 31 (58%) Latino and 22 (42%) Black individuals with no prior experience using electric toothbrushes, were recruited to use ROBAS, with planned assessments at baseline, 2 months, and 4 months. Beyond evaluating ROBAS’s technical performance, extensive feedback was gathered to gauge users’ experiences, viewpoints, and overall contentment. The System Usability Scale (SUS) served as a primary metric for assessing user satisfaction and acceptability. RESULTS ROBAS exhibited largely reliable and consistent data-gathering capabilities. SUS scores (mean 75.6, SD 14.5) reflected participant contentment within a range of values for other commonly used digital devices and technologies. Among participants who answered questions about willingness to pay for ROBAS, 97% (30/31) indicated that they were willing to pay for ROBAS either as a one-time payment or as a subscription-based service. Additionally, 87.5% of participants expressed that they would endorse it to acquaintances. Most participants expressed no reservations about privacy; among those who expressed privacy concerns (n=20, 50%), the concerns included exposure of information (n=18, 45%), monitoring of brushing habits (n=12, 30%), and collection of information (n=14, 35%), although these concerns did not significantly correlate with specific participant traits. In qualitative terms, users valued ROBAS's ability to monitor brushing habits but called for refinements, especially in Wi-Fi and application connectivity. Recommendations for system improvements encompassed enhanced app functionality, individualized coaching, more comprehensive brushing data, and the addition of flossing activity tracking. CONCLUSIONS The research highlights ROBAS's promise as a digital platform for unobtrusively tracking daily oral self-care activities in marginalized communities. The system proved to be both feasible, as evidenced by its stable and accurate data capture of brushing behaviors, and user-friendly, as reflected by strong SUS scores and positive user feedback. Influential factors for its uptake included ease of learning and operation, and the feedback provided.

BackgroundRepeated applications of short-term dietary assessment instruments are recommended for estimating usual dietary intake. For this purpose, NutriDiary, a smartphone app for collecting weighed dietary records (WDRs) in the German population, was developed.ObjectiveWe aim to describe NutriDiary and evaluate its usability and acceptability.MethodsNutriDiary was developed as a WDR, allowing users to enter food items via text search, barcode scanning, or free text entry. The sample for the evaluation study included 74 participants (n=51, 69% female, aged 18‐64 years), including 27 (37.5%) experts and 47 (63.5%) laypersons (including n=22, 30%, nutrition students). Participants completed a 1-day WDR and entered a predefined sample meal (n=17 foods) the following day by using NutriDiary. An evaluation questionnaire was answered from which the system usability scale (SUS) score (0‐100) was calculated. A backward selection procedure (PROC REG in SAS; SAS Institute) was used to identify potential predictors for the SUS score (age, sex, status [expert or laypersons], and operating system [iOS or Android]).ResultsThe median SUS score of 75 (IQR 63‐88) indicated good usability. Age was the only characteristic identified as a potential predictor for a lower SUS score (P<.001). The median completion time for an individual WDR was 35 (IQR 19‐52) minutes. Older participants took longer to enter the data than younger ones (18‐30 y: median 1.5, IQR 1.1‐2.0 min/item vs 45‐64 y: median 1.8, IQR 1.3‐2.3 min/item). Most participants expressed a preference for NutriDiary over the traditional paper-based method.ConclusionsGood usability and acceptability make NutriDiary promising for use in epidemiological studies.

BackgroundWearable devices have the potential to promote a healthy lifestyle because of their real-time data monitoring capabilities. However, device usability is a critical factor that determines whether they will be adopted on a large scale. Usability studies on wearable devices are still scarce.ObjectiveThis study aims to compare the functions and attributes of seven mainstream wearable devices and to evaluate their usability.MethodsThe wearable devices selected were the Apple Watch, Samsung Gear S, Fitbit Surge, Jawbone Up3, Mi Band, Huawei Honor B2, and Misfit Shine. A mixed method of feature comparison and a System Usability Scale (SUS) evaluation based on 388 participants was applied; the higher the SUS score, the better the usability of the product.ResultsFor features, all devices had step counting, an activity timer, and distance recording functions. The Samsung Gear S had a unique sports track recording feature and the Huawei Honor B2 had a unique wireless earphone. The Apple Watch, Samsung Gear S, Jawbone Up3, and Fitbit Surge could measure heart rate. All the devices were able to monitor sleep, except the Apple Watch. For product characteristics, including attributes such as weight, battery life, price, and 22 functions such as step counting, activity time, activity type identification, sleep monitoring, and expandable new features, we found a very weak negative correlation between the SUS scores and price (r=−.10, P=.03) and devices that support expandable new features (r=−.11, P=.02), and a very weak positive correlation between the SUS scores and devices that support the activity type identification function (r=.11, P=.02). The Huawei Honor B2 received the highest score of mean 67.6 (SD 16.1); the lowest Apple Watch score was only 61.4 (SD 14.7). No significant difference was observed among brands. The SUS score had a moderate positive correlation with the user’s experience (length of time the device was used) (r=.32, P<.001); participants in the medical and health care industries gave a significantly higher score (mean 61.1, SD 17.9 vs mean 68.7, SD 14.5, P=.03).ConclusionsThe functions of wearable devices tend to be homogeneous and usability is similar across various brands. Overall, Mi Band had the lowest price and the lightest weight. Misfit Shine had the longest battery life and most functions, and participants in the medical and health care industries had the best evaluation of wearable devices. The perceived usability of mainstream wearable devices is unsatisfactory and customer loyalty is not high. A consumer’s SUS rating for a wearable device is related to their personal situation instead of the device brand. Device manufacturers should put more effort into developing innovative functions and improving the usability of their products by integrating more cognitive behavior change techniques.

The purpose of this study was to determine if children's prior performance experience was a mediating factor in their performance of a dominant or novel task in an audience or no audience situation. Eighty 9-year-old boys were divided into experienced (n = 40) and nonexperienced (n = 40) groups based on prior youth sport experience and the absence of any performance experience before a formal audience. Half of each group learned a rotary pursuit task until they could perform the task with at least 60% accuracy. The other half did not practice the task. Groups were again divided for task performance in an audience or no audience situation so that the following treatments were observed for both experienced and nonexperienced groups: dominant task, no audience; dominant task, evaluative audience; novel task, no audience; novel task, evaluative audience. Task performance for each subject was five 20-sec trials on the photoelectric rotary pursuit task. The mean score of each set of five was used for data analysis. An audience of four passive adults was present in each audience condition and made evaluative notations following each performance. Results of a 2 × 2 × 2 (experience × task dominance × audience) ANOVA failed to support Zajonc's (1965) social facilitation theory and Cottrell's (1968) modification of this theory. The well-learned task was inhibited by the presence of an evaluative audience while performance of a novel task was enhanced. No significant experience effects were evident.

Usability engineering analyzes the interaction between the intended users and a device. Its implementation is mandatory for manufacturers to obtain regulatory approval for the European market. The aim of this evaluation was assessing the role of usability testing in the development process. For this purpose, a continuous glucose monitoring (CGM) device under development was investigated to determine whether it could be used safely and effectively by the intended users. Conduct of the usability testing was based on the international standard IEC 62366-1. Medical device use of CGM-experienced and non-experienced users (n = 15 each) was observed without initial training in use scenarios containing 18 tasks. The success rate of task completion was determined and the System Usability Scale (SUS) score was calculated from a questionnaire. A prototype of the FiberSense CGM System (EyeSense GmbH, Großostheim, Germany), comprising of a single-use sensor and a reusable detector, was investigated. Most use errors made by both user groups were related to ease of handling of the reusable detectors. The SUS scores achieved in this study were below the pre-defined SUS score acceptance criterion of ≥68. The most frequently mentioned reason for use errors was an incomprehensible and non-chronological instructions for use (IFU). The evaluation provides valuable insights on how to improve usability of the prototype device and demonstrates the value of conducting structured usability testing prior to product finalization. The results reflected areas for improvement of the user interface, mainly by restructuring the IFU, provision of an additional leaflet, and device training prior to use.

This study evaluates the KBGAN iHealth© and KBGAN iFeed© mobile apps designed for buffalo health and feeding management, particularly for agricultural extension professionals (AEPs) in selected Philippine municipalities. These apps aim to address challenges in buffalo management, such as limited access to veterinary expertise, personalized recommendations, organized data, communication channels, and difficulties in calculating ideal feed compositions and meeting the distinct needs of smallholder farmers and AEPs. Despite System Usability Scale (SUS) scores indicating marginal acceptability for both apps, weighted mean scores by AEPs for statements assessed on a 5-point Likert scale (1 as strongly disagree and 5 as strongly agree), demonstrate that AEPs reported high confidence in the accuracy of buffalo health diagnostics (Mean of 4.20) and health management recommendations (Mean of 4.17) provided by KBGAN iHealth©. Similarly, KBGAN iFeed© received favorable ratings, with AEPs expressing agreement on the accuracy of feeding recommendations (Mean of 3.89) and the facilitation of feeding ration computations (Mean of 4.00). These positive perceived performance outcomes, coupled with increased confidence (Mean of 4.03) and motivation (Mean of 4.00) among AEPs, suggest the potential for frequent and consistent app usage despite usability concerns. Chi-square tests examining the relationship between AEP characteristics and SUS scores for the apps reveal significant associations between AEPs' education and experience levels and SUS scores for KBGAN iFeed©. The choice of IT device also influences KBGAN iHealth© SUS scores. Proposed enhancements by AEPs include refining algorithms, improving the user interface for navigation, speed, and efficiency, and incorporating features such as photo uploads and geotagging.

Recent studies in Uganda show that Learning Management Systems (LMS) were not usable, and some were complex to use, making it difficult for lecturers to use the platforms. As a result, there has been low uptake and usage of LMS in many universities. To investigate this claim, the study set out to evaluate the usability of LMS in four public universities in Uganda, namely Makerere University, Makerere University Business School, Busitema University and Gulu University. Google Docs was used to design a System Usability Scale (SUS) which was used to evaluate the usability of the LMS in the four public universities. An open-ended questionnaire was also included in the SUS to capture qualitative data. This questionnaire was sent to different staff mailing lists in the four universities. A total of 80 questionnaires were filled and analyzed. The findings showed that the average System Usability Scale score for the LMS used in the four public universities was 60, representing about 30% usability, which is poor. The study identified several usability challenges and ways to address the usability challenges. This paper, therefore, presents an IDSS usability model for improving LMS usability in learning institutions.

Background and importanceAn online personalised health record (PHR) is a valid tool to reduce medication discrepancies (MDs), defined as unexplained differences among medication regimens. The success of a PHR depends...

Higher education in today's digital era faces the challenge of integrating technology effectively to improve the quality of learning. One of the technologies used is the Learning Management System (LMS), which facilitates interaction between lecturers and students in teaching and learning activities. Universitas Hindu Indonesia (UNHI) has adopted LMS as part of its education digitization strategy. However, the effectiveness of using this LMS has not been thoroughly evaluated, especially in terms of ease of use and user satisfaction. This study aims to assess the usability of the LMS at UNHI using the System Usability Scale (SUS) method. By analyzing data from 97 respondents, the results showed a score of 61, which falls within the 'OK' category and grade 'D'. This indicates that although the LMS provides benefits, there are some areas that require improvement. Key findings show that while ease of use is relatively good, there are significant issues with the mobile interface, notification system, and technical capacity. Recommendations for improvement include enhancing the interface for mobile devices, improving the notification system, and increasing technical capacity to improve the user experience. These findings provide important directions for the development of a more efficient and satisfactory LMS at UNHI.

As higher education moves to blended learning environments, a divide is emerging in the Australian higher education sector. This divide is predicated on differing skills and usage patterns, not access to devices. Access is not perceived to be the issue as numerous Australian secondary schools offer a school-issued laptop scheme. Yet many students transitioning to university are grappling with the necessary skills required to participate in a setting. Referred to as digital natives, these young people were expected to be digitally proficient. This thesis challenges the existence of Mark Prensky's (2001) Digital Native and provides an analysis of how differing fluency stages influence perceived preparedness for university study. Conceptualising the growing inequalities arising from a widening divide, the thesis investigates impacts on the student experience, fluency and secondary schooling opportunities. The thesis reports on three studies drawn from three research questions. Using a mixed-mode approach centred on Critical Theory and Ajzen's Theory of Planned Behaviour (TPB), the thesis provides an analysis of the divide in Australian higher education. Study 1 reports on RQ1: What is the relationship between socioeconomic, sociocultural/ geographic indicators and the divide? Four hundred and nine first-year business students were surveyed at regional and urban Australian universities. This study provides empirical data on the divide and determines a link between fluency, socioeconomic status, sociocultural capital, identity and student self-reported preparedness and skills. Study 2 reports on RQ2: Is fluency a precursor to preparedness for university study? Fifteen of the surveyed respondents completed a test with usability testing software prior to an in-depth interview. Study 2 provides a link between access and application of environments in schooling and the development of fluency. This study presents data showing disadvantage indicators can be alleviated through access to learning environments during schooling. Study 3 reports on RQ3: What enhances and develops fluencies? and examines the divide from a student's perceptive. Case studies were developed from in-depth interviews and presented as techno-biographies to determine respondents' fluency stage. These techno-biographies outline differing experiences and opportunities for skills development between secondary schools. Study 3 explores prior experience to identify influences, skills, knowledge, attitude and mindset. The study suggests that influences and prior experiences contribute to fluency and perceived preparedness for university study. The three studies are intertwined in their investigation of an association between disadvantage indicators, prior experience and stages of fluency. Particular attention is placed on examining the distribution and allocation of digitally resourcing in secondary schools. The three studies culminate in a concept model to illustrate the link between the distribution of resources, fluency and preparedness for university study. The thesis demonstrates a link between access to a learning management system (LMS) or curriculum during secondary school and disadvantage indicators. Access to a school LMS consistently produced higher self-reported skills than those without, even when disadvantage indicators were present. The issue of perceived preparedness for university study and/or a learning environment was also linked to participants who had access to a school LMS. Rural, regional, low socioeconomic, low sociocultural capital and state-school participants were less likely to have had access to a curriculum during secondary schooling and therefore less likely to report preparedness for university study. Conversely, these disadvantage indicators were overcome if participants had access to an LMS or curriculum. The thesis identifies a divide in higher education emanating from the distribution, use and allocation of secondary schooling resources and prior experience. The resourcing of secondary schools with school-issued laptops did not increase fluency or perceived preparedness for university study. However, the implementation of a curriculum or LMS produced significant outcomes in the development of fluency. These findings illustrate the influence of immersion in the formation of fluency. Resourcing schools without a clear curriculum does not increase fluency. If the divide is to be conquered, the appropriate application of resources in secondary schools must be implemented to enable the development of fluency.

Rational: Lymphangioleiomyomatosis (LAM) is a rare, progressive interstitial lung disease (ILD) predominantly affecting women of reproductive age. Improving access to structured exercise through home-based programs is attractive in LAM and other ILDs, but monitoring exercise pulse oximetry and outcomes can be challenging for patients far from specialized pulmonology centres. A preliminary study of a remotely-monitored exercise program demonstrated improvement in 6-minute walk test (6MWT) distance without safety incident, indicating mobile technology with remote monitoring may provide a feasible solution for broader access. For use during LAMFit, a fully-remote, structured exercise program for LAM, we aimed to develop and test a mobile health application to enable remote assessment of 6MWT and home monitoring of pulse oximetry during exercise. Methods: Application development involved multi-stakeholder engagement and collaboration between LAMFit investigators, patient representatives, LAM care providers, and software developers. Feature design was informed by 2002 American Thoracic Society (ATS) guidelines for the 6MWT. The application was deployed in a 28-week LAMFit pilot in 25 women with LAM (mean±SD, age 52.6±14 years). Weekly exercise sessions were remotely monitored via the application, and remote 6MWT and validated patient questionnaires were completed at weeks 2, 14, and 27. System Usability Scale (SUS) scores, a standardized metric to assess user perceptions and ease of system use, were collected at weeks 2 and 28.Results: Collaborative co-design facilitated the creation of a mobile application for the LAMFit program. Iterative design, testing and refinement, guided by steering team feedback, incorporated adjustments in language, font size, icons, and audio prompts, and enabled 6MWT to continue during brief connection-loss with the Bluetooth-enabled pulse oximeter. All 25 patients successfully used the application to complete surveys and multiple 6MWT. System usability (SUS) at week 2 was 50.625 (SD 3.227), and 52.62 (SD 4.292) at week 28. On exit survey, 88% of participants found it feasible to conduct a remote 6MWT at home using the mobile health app. Conclusion: A collaborative co-design approach enabled alignment of the application with users’ needs, enabling successful implementation of the technology. SUS scores increased during the study, suggesting improvements in perceptions of usability with increasing familiarity. However, both scores are below the industry average SUS of 68, indicating potential for further improvement in application usability. Examination of application usability during LAMFit's pilot study provides insights to inform the refinement of the application prior to scaling up the program, optimizing a digital-health solution that supports patients with LAM exercise routines

Perceived usability is an important requirement of interactive systems, affecting greatly the overall user experience. The Post-Study System Usability Questionnaire (PSSUQ) is a popular instrument, developed in English, for assessing perceived usability of interactive systems. In this paper, a first step towards standardization of a Greek version of PSSUQ, the PSSUQ-GR, is presented. To this end, the PSSUQ was translated and adapted to Greek by two HCI experts. Next, 359 participants interacted with two websites and then completed the System Usability Scale (SUS) and the PSSUQ-GR immediately after interacting with each website. Factor analysis successfully revealed a three-factor structure of the PSSUQ-GR with 12 items and item-subscales alignment close to the original PSSUQ. Reliability analysis using Cronbach's alpha indicated that the PSSUQ-GR scale and the three subscales have a high reliability. In addition, the participants' SUS scores and PSSUQ-GR scores were significantly and strongly correlated, which supports the validity of the PSSUQ-GR. Thus, this study found that the PSSUQ-GR (N=12 items) is a reliable and valid tool for measuring perceived usability, with a factor structure consistent with the English version of the PSSUQ.