Evaluating System Usability, Workload Suitability, and User Experience of Game-Based Virtual Reality in Spaceflight Education and Training

Virtual environments have been widely adopted for design and training tasks in the industrial domain. Low-cost automation (LCA) is a technology that automatizes some activities using mostly standard automation mechanisms available off the shelf. However, LCA systems should adapt to existing standard production lines and workstations. Thus, workers must customize standard LCA templates and perform adaptation and customization steps. This activity can be very time consuming with physical LCA systems, and in case of errors, it may be necessary to rebuild many parts from scratch. Thus, LCA systems would greatly benefit from a design and prototyping step experienced in a virtual simulation environment. An immersive virtual reality (IVR) application for rapid and easy prototyping of LCA solutions has been investigated in previous work; the assessment of the system usability proved that the users highly appreciated the proposed solutions. This research explores further improvements to exploit the existing IVR application as a training tool for LCA prototyping trainees. The proposed application now provides users with two different interaction paradigms based on the VIVE controllers and the Manus Prime II data gloves. The application’s interface has been revised to allow a proper comparison of the two interaction models. The two interfaces have been compared, involving 12 participants in an LCA building task. The System Usability Scale (SUS) and the NASA Task Load Index (TLX) questionnaires have been used to assess the usability and workload of the two solutions.

Assessment of Radiation Therapy Technologists’ Workload and Situation Awareness: Monitoring 2 Versus 3 Collocated Display Monitors

• Helicopter crews solved rescue missions with rope and hoist, in randomized sequence. • NASA Task Load Index workload, heart rate, and heart rate variability were analyzed. • Workload was lower in hoist operations for all crewmembers, most markedly for pilots. • Findings were consistent across mission terrain types and operational flight phases. • Physiological measures aligned with workload among pilots and rescuers, but not among physicians. High workloads can deplete cognitive capacity, impairing individuals’ ability to meet task demands and increasing the likelihood of human error. Helicopter rescue operations are inherently complex and require effective multitasking by all crewmembers. This study aimed to quantify subjective workload and physiological activation associated with two different rescue methods, hoist operations and rope operations, in a three-crew Helicopter Emergency Medical Service. The National Aeronautics and Space Administration (NASA) Task Load Index was used to assess workload. Heart rate and heart rate variability were used as markers for physiological activation. Crews performed fifty-five standardized rescue missions covering four distinct terrain categories: alpine, gorge, forest, and vertical environments. Each mission was solved with both rope and hoist rescue methods, in randomized sequence. Linear mixed-model regression analyses were used to account for repeated measures within individuals. Across all mission phases, pilots (n = 4), physicians (n = 3), and rescuers (n = 4) reported lower workloads during hoist operations than during rope operations. For pilots and rescuers, hoist operations were also associated with lower physiological activation. The opposite was found in physicians, who acted as hoist operators. Rescue method affected pilots most markedly, their mean workload scores during active mission being 35 [19,52] for hoisting versus 53 [37,70] for rope operations (R 2 = 0.76, p <0.0001). Analogously, pilot mean heart rate was 80 [57,103] during hoisting compared to 88 [65,111] during rope operations (R 2 = 0.91, p <0.0001). A reduction in workload could contribute to improved flight safety, favouring hoist as rescue method.

Electronic health records (EHRs) have been associated with information overload, causing providers to miss critical information, make errors, and delay care. Information overload can be especially prevalent in medical intensive care units (ICUs) where patients are often critically ill and their charts contain large amounts of data points such as vitals, test and laboratory results, medications, and notes. We propose to study the relationship between information overload and EHR use among medical ICU providers in 4 major United States medical centers. In this study, we examined 2 prominent EHR systems in the United States to generate reproducible and generalizable findings. Our study collected physiological and objective data through the use of a screen-mounted eye-tracker. We aim to characterize information overload in the EHR by examining ICU providers' decision-making and EHR usability. We also surveyed providers on their institution's EHR to better understand how they rate the system's task load and usability using the NASA (National Aeronautics and Space Administration) Task Load Index and Computer System Usability Questionnaire. Primary outcomes include the number of eye fixations during each case, the number of correct decisions, the time to complete each case, and number of screens visited. Secondary outcomes include case complexity performance, frequency of mouse clicks, and EHR task load and usability using provided surveys. This EHR usability study was funded in 2021. The study was initiated in 2022 with a completion date of 2025. Data collection for this study was completed in December 2023 and data analysis is ongoing with a total of 81 provider sessions recorded. Our study aims to characterize information overload in the EHR among medical ICU providers. By conducting a multisite, cross-sectional usability assessment of information overload in 2 leading EHRs, we hope to reveal mechanisms that explain information overload. The insights gained from this study may lead to potential improvements in EHR usability and interface design, which could improve health care delivery and patient safety. DERR1-10.2196/66127.

During the process of digital-system development from prototype to live implementation, differences in user interactions, perceived usability, and overall satisfaction can emerge. These differences often arise due to various factors, which may include the fidelity of the software prototype, the limitations of the prototyping tool, and the complexity of the live digital system. Recognizing these potential usability discrepancies between prototypes and live digital systems, assessment of how well user experience (UX) test approaches, such as usability testing and the System Usability Scale (SUS), reflect the UX in using the digital-system prototype and its counterpart deployed live system emerged as an important research gap. To address this gap, this study compares usability testing and SUS results among a Figma web prototype and its counterpart live web digital system, for the telecom service extension process as a representative digital-system case study. The research study involved a testing process with a total of 10 participants across the Figma prototype and live-web-system test environments, in which different sensing devices that included versatile types of mobile phones were utilized. The research study presents usability testing results related to the overlap in perceived usability issues for the same digital-product developments in both testing environments, which are experienced on different types of mobile sensing devices. The usability testing results are presented as reports on the frequency of occurrence of web system usability issues and corresponding severity levels. The obtained results demonstrated that prototype testing is highly effective for detecting a wide range of usability issues early in the digital-product development phase. The paper also evaluates the predictive capabilities of SUS assessment for the case of the Figma web prototype and its counterpart live web system in the phase of digital-product development. The results show that the SUS evaluation, when applied to digital-system prototype testing, can provide early in the development process a reliable indication of the perceived usability of its counterpart digital system, once it is developed and deployed. The findings presented in the paper offer valuable guidance for software designers and developers seeking to make prototypes and their counterpart real digital-product deployments with improved digital-product overall user experience.

BackgroundTo address the epidemic burden of diet-related diseases, adequate dietary intake assessments are needed to determine the actual nutrition intake of a population. In this context, the eNutri web app has been developed, providing online automated personalized dietary advice, based on nutritional information recorded via an integrated and validated food frequency questionnaire (FFQ). Originally developed for a British population and their dietary habits, the eNutri tool has specifically been adapted to the German population, taking into account national eating habits and dietary recommendations.ObjectiveThe primary aim of this study is to evaluate the system usability and users’ experience and feedback on the eNutri app in a small-scale preliminary study. The secondary aim is to investigate the efficacy of personalized nutrition (PN) recommendations versus general dietary advice in altering eating habits.MethodsThe app was piloted for 4 weeks by 106 participants from across Germany divided into a PN group and a control group. The groups differed according to the degree of personalization of dietary recommendations obtained.ResultsAn overall System Usability Scale (SUS) score of 78.4 (SD 12.2) was yielded, indicating an above average user experience. Mean completion time of the FFQ was 26.7 minutes (SD 10.6 minutes). Across subgroups (age, sex, device screen sizes) no differences in SUS or completion time were found, indicating an equal performance for all users independent of the assigned experimental group. Participants’ feedback highlighted the need for more personalized dietary advice for controls, while personalized nutritional recommendations improved the awareness of healthy eating behavior. Further improvements to the eNutri app were suggested by the app users.ConclusionsIn total, the eNutri app has proven to be a suitable instrument to capture the dietary habits of a German population sample. Regarding functionality, system usability, and handling, direct user feedback was quite positive. Nutritional advice given was rated ambivalent, pointing to several weaknesses in the eNutri app, minimizing the system’s full potential. A higher level of personalization within nutritional advice subjectively improved the app’s usability. The insights gained will be used as a basis to further develop and improve this digital diet assessment tool.

To analyse the relationship between health app quality with user ratings and the number of downloads of corresponding health apps. Utilising a dataset of 881 Android-based health apps, assessed via the 300-point objective Organisation for the Review of Care and Health Applications (ORCHA) assessment tool, we explored whether subjective user-level indicators of quality (user ratings and downloads) correlate with objective quality scores in the domains of user experience, data privacy and professional/clinical assurance. For this purpose, we applied spearman correlation and multiple linear regression models. For user experience, professional/clinical assurance and data privacy scores, all models had very low adjusted R squared values (< .02). Suggesting that there is no meaningful link between subjective user ratings or the number of health app downloads and objective quality measures. Spearman correlations suggested that prior downloads only had a very weak positive correlation with user experience scores (Spearman = .084, p = .012) and data privacy scores (Spearman = .088, p = .009). There was a very weak negative correlation between downloads and professional/clinical assurance score (Spearman = -.081, p = .016). Additionally, user ratings demonstrated a very weak correlation with no statistically significant correlations observed between user ratings and the scores (all p > 0.05). For ORCHA scores multiple linear regression had adjusted R-squared = -.002. This study highlights that widely available proxies which users may perceive to signify the quality of health apps, namely user ratings and downloads, are inaccurate predictors for estimating quality. This indicates the need for wider use of quality assurance methodologies which can accurately determine the quality, safety, and compliance of health apps. Findings suggest more should be done to enable users to recognise high-quality health apps, including digital health literacy training and the provision of nationally endorsed "libraries".

Functional Electrical Stimulation Assisted Cycling (FES-cycling) is a rehabilitation intervention for individuals with Spinal Cord Injury (SCI), offering benefits like increased muscle trophism, improved cardiopulmonary function, and reduced bone demineralization. Despite numerous studies on its physical advantages, few have focused on user experience. This study evaluates the user experience, usability, acceptability, and human-device interaction of two FES-cycling systems: a recumbent FES-bike prototype from Politecnico di Milano and the commercial BerkelBike Pro. We recruited 15 participants with SCI, aged 18-65, with varying injury levels. The user experience (primary outcome), acceptability, usability, human-device interaction, and ergonomics of both systems were investigated by means of 4 questionnaires: the User Experience Questionnaire (UEQ), the Technological Acceptance Measure 3 (TAM-3), the System Usability Scale (SUS), and a customized questionnaire to evaluate human-device interaction both in terms of ergonomic aspects and psychological factors. The user experience received positive evaluations across all dimensions (scores ≥ 1.5 on a scale from -3 to 3), with slightly higher, though not statistically significant, scores for the FB. Secondary outcomes indicated strong acceptability (global TAM-3 scores > 5.5/7 for both devices), high usability ratings (SUS scores ≥ 75/100 for both devices), and favorable interaction and ergonomics, emphasizing ease of use, comfort, and alignment with user expectations. The study underscores the positive user experience as the primary outcome, along with strong acceptance and usability of both devices, emphasizing their potential to enhance Sport-Therapy and the value of integrating user perspectives in future development.

BackgroundMobile health technologies have the potential to enhance the management, communication, and overall quality of life of patients with cancer. The MyPal project, using a participatory design approach, aims to provide palliative care support for patients with cancer through an electronic patient-reported outcomes eHealth platform.ObjectiveThis study aims to evaluate the usability and user experience of “MyPal for adults,” a co-designed palliative care mobile app designed to support adults with cancer.MethodsRepresentative users participated in a 4-step usability study using a “think-aloud” protocol, complemented with feature satisfaction, difficulty perceived, and design impression surveys along with a short, structured interview. Participants were also asked to provide quantitative feedback via the postuse System Usability Scale, User Experience Questionnaire, and Post-Study System Usability Questionnaire. The data were analyzed along the lines of the International Organization for Standardization (ISO) 9241‐210 framework.ResultsAll participants found the intervention content useful, and they reported satisfactory usability, with a mean Post-Study System Usability Questionnaire score of 2.458 (SD 1.08) and a System Usability Scale score of 68.9. All aspects of the User Experience Questionnaire (attractiveness, perspicuity, efficiency, dependability, stimulation, and novelty) surpassed usability quality benchmarks. The qualitative analysis identified 43 usability issues, primarily related to effectiveness and efficiency as defined in ISO 9241‐210.ConclusionsIn this study, we conducted a usability evaluation of the “MyPal for adults” app, a digital tool designed to enhance the palliative care experience. This approach identifies real-world usability issues, enabling iterative improvements in the eHealth platform’s design. Our findings reveal a user-friendly interface and positive patient experiences. This study emphasizes the need to enhance mobile health platform usability, offering insights to improve digital palliative care.

BackgroundUltrasound education is transitioning from in-person training to remote methods using mixed reality (MR) and 5G networks. Previous studies are mainly experimental, lacking randomized controlled trials in direct training scenarios.ObjectiveThis study aimed to compare an MR-based telesupervised ultrasound education platform on private 5G networks with traditional in-person training for novice doctors.MethodsConducted at a tertiary academic hospital from November to December 2023, the prospective unblinded randomized controlled pilot study assigned doctors without prior abdominal ultrasound education experience to either the telesupervision group (TG; n=20) or direct supervision group (DG; n=20). Participants received a 15-minute video lecture, conducted ultrasound on a phantom, and had 18 images scored by 2 blinded experts. Additionally, the TG received 5 minutes of training on the basic operation of a head-mounted display. Communication between doctors in the TG and supervisors was facilitated through a head-mounted display, whereas those in the DG interacted directly with supervisors. Primary outcomes were image quality scores, while secondary outcomes included procedure time, number of supervisor interventions, user experience using National Aeronautics and Space Administration-Task Load Index (NASA-TLX), System Usability Scale (SUS), and self-confidence through pre- and postsurveys.ResultsImage quality scores and procedure times showed no significant differences between the groups (TG: 66.8 [SD 10.3] vs DG: 66.8 [SD 10.4], P=.84; TG: 23.8 [SD 8.0] min vs DG: 24.0 [SD 8.1] min, P=.95, respectively). However, the TG engaged in more educational interventions (TG: 4.0 [SD 2.5] vs DG: 0.8 [SD 1.1], P<.001), reflecting a more interactive training environment. TG participants reported lower NASA-TLX scores for mental demand (43.8 [SD 24.8] vs 60.6 [SD 22.4], P=.03), effort (43.1 [SD 22.9] vs 67.9 [SD 17], P<.001), and frustration (26.9 [SD 20.3] vs 45.2 [SD 27.8], P=.02), indicating a reduced cognitive load compared to the DG. The mean SUS score was also higher in the TG (66.6 [SD 9.1] vs 60.2 [SD 10.4], P=.046), suggesting better usability. Both groups showed significant improvements in confidence, with the TG showing notably greater improvement in abdominal ultrasound proficiency (pre-education TG: 1.6 [SD 0.9] vs DG: 1.7 [SD 0.9], P=.73; post-education TG: 3.8 [SD 0.9] vs DG: 2.8 [SD 1.0], P=.006).ConclusionsAlthough no significant differences in image quality scores were observed between groups, considerable differences in positive educational interactions, workload, and usability were evident. These findings emphasize the platform’s potential to enhance the ultrasound training experience, suggesting more interactive and efficient learning.

Evaluation of Premium Ultrasound Systems With Sonographers: How Comparative Usability Testing Can Reveal Human Factors Issues Prior to Errors in the Exam Room1

BACKGROUND By 2050, nearly 22% of the world's population will be 60 years of age or older, with Europe experiencing remarkably rapid demographic ageing. This demographic shift coincides with the increasing prevalence of chronic diseases and functional decline among older adults, challenging traditional healthcare delivery models. The World Health Organization's concept of healthy aging emphasizes the preservation of functional capabilities through the optimization of Intrinsic Capacity (IC), the totality of all physical and mental abilities across five domains: locomotion, vitality, sensory, cognition, and psychology. Digital Health Solutions (DHS) offers promising approaches to support aging populations through remote monitoring and integrated care; however, significant barriers remain, including low digital literacy and usability challenges among older adults. OBJECTIVE To evaluate the impact of a multidimensional, AI-driven digital health platform (CAREUP) on the dynamic monitoring and personalized enhancement of Intrinsic Capacity in community-dwelling older adults, integrating real-time physiological, cognitive, and behavioral data streams. METHODS A mixed-methods pilot study was conducted across three European countries (Italy, Romania, and Austria) from September to December 2024, enrolling 66 primary users (older adults aged 65 years or older) and 17 secondary users (caregivers). The CAREUP platform integrates Android applications (Careplan and Positive Health apps), smart devices for health monitoring, and an online dashboard. Primary outcomes included system usability, assessed by the System Usability Scale (SUS); user experience, measured by the User Experience Questionnaire short version (UEQ-S); and acceptance, evaluated through qualitative questionnaires. Secondary outcomes encompassed physical and mental health measures, quality of life indicators, and caregiver workload assessments at three-time points (T0, T1, T2). RESULTS Of 66 enrolled participants, 61 primary users (37 women, 24 men; mean age 73.9 years) completed the study. The overall SUS score was 68, indicating acceptable but improvable usability, with significant variations across countries. The UEQ-S score of 1.45 indicated a good overall user experience, with high Hedonic Quality (1.66) suggesting emotional appeal and Pragmatic Quality (1.23) reflecting above-average usability. Most participants (49/61, 80.3%) evaluated the platform as interesting, helpful, and efficient, with 48/61 (78.7%) willing to recommend it to friends. Secondary outcomes demonstrated stable physical and mental health, as well as maintained independence in daily activities. Technical connectivity issues and device complexity emerged as primary barriers to adoption. CONCLUSIONS The CAREUP platform demonstrated the feasibility of monitoring Intrinsic Capacity in older adults, achieving an acceptable level of usability and a positive user experience despite technical challenges. Country-specific variations in acceptance highlight the importance of cultural adaptation and digital literacy considerations in platform deployment. Future iterations should prioritise technical reliability, interface simplification, and enhanced user support to optimise adoption and sustained engagement among ageing populations. CLINICALTRIAL The study was registered on ClinicalTrials.gov with the ID number NCT05963399.

Objective:Cancer is a non-communicable disease that accounts for 71% of deaths globally. The prevalence of people living with cancer in Indonesia increased from 1.4/1000 population (2013) to 1.8/1,000 population (2018). This study aims to evaluate usability, user experience, and user feedback on the use of the Oncodoc m-health application. Methods: The research method uses mixed methods with a cross-sectional approach and online survey sampling techniques (677 respondents), in December 2021-January 2022 in Indonesia. The instruments used are the System Usability Scale and User Experience Questionnaire. The variables measured include the characteristics of respondents, usability, user experience, and responses to using applications qualitatively. Quantitative data were analysed descriptively and the User Experience Questionnaire’s analysis tool. Qualitative data were analysed thematically. Results:Evaluation of the usability of Oncodoc’s m-health application is in the “acceptable” category (70.88), and the adjective rating is in the “Good” category. Evaluations of the relative quality of user experience are in the “good” and “very good” types. The average user experience scale is (mean; SD): attractiveness (1.80; 0.99), perspicuity (1.82; 1.05), efficiency (1.78; 1.09), dependability (1.56; 1.01), stimulation (1.81; 1.06), and novelty (1.32; 1.10). The findings of this study are 1) the ability to accommodate user needs, 2) usage barriers, and 3) user expectations for the Oncodoc application. Conclusion:Overall, the Oncodoc m-health application is acceptable to users. We recommend developing a cancer early detection application with an approach that refers to user needs for further research.

Virtual Loupes: A Pilot Study on the Use of Video Passthrough Augmented Reality in Plastic Surgery.

The rapid development of technology has advanced the financial sector, leading to the rise of digital wallets and mobile banking services. However, calculating or splitting the bill can be difficult in social activities like group dinners. Even though it seems simple, determining the exact amount each person must pay is often complex due to different orders, taxes, and service charges. This research compares user experience based on effectiveness and satisfaction when using the Split Bill feature in GoPay and LINE. The System Usability Score (SUS) method assesses system quality for experienced users, while the Cognitive Walkthrough (CW) method evaluates success rates for new users. The results show that based on SUS, the Split Bill feature in both apps fall into the "Marginal" category, with scores below the ideal threshold: 67.4 for LINE and 66.2 for GoPay. Meanwhile, the CW method indicates that GoPay has a task completion success rate of 89.44%, 8.83% higher than LINE's 80.61%. Improvement recommendations are based on tasks in the CW method with the lowest success rates in each app to enhance usability and user experience. By addressing these weaknesses, both applications can improve their Split Bill features to offer smoother and more efficient user experience.