MitsuDomoe

Extended-Reality Technologies: An Overview of Emerging Applications in Medical Education and Clinical Care.

This chapter will be focused on contributing to the increase of universal design competencies of future engineers, educators, and designers through the use of mixed reality technologies, closing the gap between theory and field application of principles, towards a more inclusive world and promoting health and wellbeing for all. The experience of a situation where limitations arise in relation to what is taken for granted is an important experience that leads to a personal knowledge of the difficulties. By the use of simulators, especially virtual (VR) and mixed reality (MR) technologies, it is possible to create such experiences. Training based on MR can prepare future and current professionals for up-to-date requirements of the labor market. In addition, it can ensure that the standards such as barrier-free concepts, broader accessibility, adaptive and assistive technology will be familiar to trainees.

Though virtual reality (VR) has been advanced to certain levels of maturity in recent years, the general public, especially the population of the blind and visually impaired (BVI), still cannot enjoy the benefit provided by VR. Current VR accessibility applications have been developed either on expensive head-mounted displays or with extra accessories and mechanisms, which are either not accessible or inconvenient for BVI individuals. In this paper, we present a mobile VR app that enables BVI users to access a virtual environment on an iPhone in order to build their skills of perception and recognition of the virtual environment and the virtual objects in the environment. The app uses the iPhone on a selfie stick to simulate a long cane in VR, and applies Augmented Reality (AR) techniques to track the iPhone’s real-time poses in an empty space of the real world, which is then synchronized to the long cane in the VR environment. Due to the use of mixed reality (the integration of VR & AR), we call it the Mixed Reality cane (MR Cane), which provides BVI users auditory and vibrotactile feedback whenever the virtual cane comes in contact with objects in VR. Thus, the MR Cane allows BVI individuals to interact with the virtual objects and identify approximate sizes and locations of the objects in the virtual environment. We performed preliminary user studies with blind-folded participants to investigate the effectiveness of the proposed mobile approach and the results indicate that the proposed MR Cane could be effective to help BVI individuals in understanding the interaction with virtual objects and exploring 3D virtual environments. The MR Cane concept can be extended to new applications of navigation, training and entertainment for BVI individuals without more significant efforts.

Recent technology advances in both Virtual Reality and Augmented Reality are creating an opportunity for a paradigm shift in the design of human-computer interaction systems. Delving into the Reality-Virtuality Continuum, we find Mixed Reality - systems designed to augment the physical world with virtual entities that embody characteristics of real world objects. In the medical field, Mixed Reality systems can overlay real-time and spatially accurate results onto a patient's body without the need for external screens. The complexity of these systems previously required specialized prototypes, but newly available commercial products like the Microsoft HoloLens make the technology more available. Through a combination of literature review, expert analysis, and prototyping we explore the use of Mixed Reality in healthcare. From the experience of prototyping Patiently and HoloSim, two applications for augmenting medical training, we outline considerations for the future design and development of virtual interfaces grounded in reality.

The article considers the possibility of using virtual and mixed reality in studying the geometry of the crystal lattice in the framework of the course of solid state physics. The application of mixed and virtual reality technologies in the study of various branches of physics is receiving more and more attention. Virtual reality headsets were used in practical and laboratory classes, where students had to create crystal structures and carry out modeling of various processes. When studying complex spatial objects of crystallography, virtual reality technology allowed students to familiarize themselves with the object of study in more detail, understand the structure of crystals and, accordingly, complete the task in a timely manner. At the same time, the prolonged use of virtual reality headsets led to rapid fatigue of students.

Two instructors collaborated across programs in a School of Education to bring mixed reality simulation to preservice early childhood educators in an effort to utilize content knowledge to practice challenging conversations. The scenario chosen was a parent-teacher conference focused on supporting a child with disabilities in an inclusive early childhood preschool classroom. The authors discuss the process of implementing mixed reality simulation in Early Childhood Studies and potential implications for lifelong impact of family engagement throughout a child’s schooling experiences and the use of mixed reality in teacher preparation. Interdisciplinary collaboration across programs within a school of education are considered.

Finding enjoyable and effective long-term approaches to rehabilitation for improving the upper limb (UL) function of people with multiple sclerosis (MS) is challenging. Virtual reality (VR) with its engagement, interaction, and real time feedback could be a solution, with immersive VR systems having preliminary evidence in stroke to be more effective in improving UL outcomes in comparison to non-immersive VR. However, immersive VR research is limited within the realm of MS rehabilitation. Therefore, the main aim of the PhD was to explore immersive VR as an approach for UL rehabilitation for people with MS (pwMS). This was achieved in four stages: 1) a systematic review of the current evidence for VR in UL rehabilitation in MS; 2) predevelopment focus groups to determine requirements for the development of bespoke immersive VR games; 3) software development of bespoke VR games to potentially improve UL function for pwMS; and 4) a feasibility study of the games and immersive VR for pwMS’ UL rehabilitation. The systematic review initially yielded ten articles with varying study design, VR intervention and UL outcome measures used. There were some statistically significant improvements after VR interventions in outcomes such as in nine-hole peg test (NHPT), hand grip strength and active range of motion. There were however conflicting results regarding if VR was more effective than conventional approaches. Whilst the review demonstrated there were positive results for VR improving UL function, most of the studies had small sample sizes and were inadequately powered, and the level of immersion on effectiveness in MS remains unknown. The predevelopment focus groups recruited ten pwMS and eight specialist clinicians to explore their views on immersive VR as an approach to UL rehabilitation and determine requirements to develop VR games for this purpose. PwMS commonly reported that their UL problems interfered with activities of daily living and resulted in the loss of meaningful hobbies such as writing. Many pwMS neglected UL exercise and found strategies for adapting to their UL impairments. Similarly, clinicians stated UL rehabilitation was neglected within their service and that it was challenging to find interesting treatment strategies. VR was suggested by both participant groups as a potential solution, as it was convenient for pwMS to access and could provide a more engaging and disguised approach to exercise. Although, there were shared concerns about cybersickness and disengagement while using VR approaches. Finally, both groups agreed games should be meaningful and adaptable for users but suggested different VR activities, with clinicians suggesting games directly reflecting activities of daily living and pwMS suggesting more abstract activities. Recommendations were made for the development of VR rehabilitation games which could be personalised and customisable for the varying abilities of pwMS. Using theme-based content analysis and requirement analysis, requirements for the software development of immersive VR games were created and developed using Unity game engine. Three games, “Piano”; “Catch-a-falling-star” and “VR Whack-a-mole” which targeted dexterity and strength for pwMS were developed, with different difficulty levels implemented to adapt to varying ability of pwMS. The final stage of this PhD was a randomised, controlled feasibility study to investigate an intervention of immersive VR involving the three co-produced games for pwMS’ UL rehabilitation. Participants were randomised to either an intervention group consisting of 30 minutes of VR co-produced games, twice/week, for eight weeks, or to a control group of usual care. Participants were recruited from the NHS and third sector organisations. Nineteen pwMS were recruited (intervention n=11, control n=8), with a recruitment rate of 3.2 participants/month. Only minor adverse events were reported (n=9), such as fatigue and UL pain. Adherence was good with 87% (±12%) session completed, and two dropouts were reported in the control group. There were positive trends for improving NHPT and hand grip strength, but there was an observed ceiling effect in the Action Research Arm Test. Participants reported high levels of satisfaction and good usability of the co-produced VR games, as measured by the USE questionnaire. Thematic analysis revealed participants enjoyed the distraction and atmospheres that immersive VR provided, and believed the games were fit for purpose with the majority of participants reporting improvements in their UL function. Participants suggested more variety and progression of exercises for future development. Overall, these studies show that immersive VR has the potential to be beneficial for UL rehabilitation and is feasible and safe for pwMS. Therefore, future work should refine VR games for long-term application and explore the clinical and cost effectiveness of this approach.

In this paper, we investigate the validity of Mixed Reality (MR) Simulation by conducting an experiment studying the effects of the visual realism of the simulated environment on various search tasks in Augmented Reality (AR). MR Simulation is a practical approach to conducting controlled and repeatable user experiments in MR, including AR. This approach uses a high-fidelity Virtual Reality (VR) display system to simulate a wide range of equal or lower fidelity displays from the MR continuum, for the express purpose of conducting user experiments. For the experiment, we created three virtual models of a real-world location, each with a different perceived level of visual realism. We designed and executed an AR experiment using the real-world location and repeated the experiment within VR using the three virtual models we created. The experiment looked into how fast users could search for both physical and virtual information that was present in the scene. Our experiment demonstrates the usefulness of MR Simulation and provides early evidence for the validity of MR Simulation with respect to AR search tasks performed in immersive VR.

The perceptual behavior of consumers on a product displayed in the market has a vital role in analyzing the importance given to that product. Therefore, various strategies have been developed to understand this consumer behavior in the selection of products. Immersive technologies like virtual, augmented, and mixed reality are among them. With the foremost feature of immersion in the virtual world and interaction of users with virtual objects, virtual reality, and augmented reality have unlocked their potential in research and a user-friendly tool for analyzing consumer behavior. In addition to these technologies, mixed reality also has a significant role in investigating consumer behavior. Studies on immersive technologies in food applications are vast, hence this review focuses on the applications of virtual, augmented, and mixed reality in the food selection behavior of consumers. The behavioral studies are elicited to develop new products based on consumer needs, to understand the shopping behavior in supermarkets for real-time usage, and to know the influence of emotions in a selection of products. The findings suggest that virtual, augmented, and mixed reality induce immersion of the users in food selection behavioral studies. Information on the technological advancements in the tools used for bringing immersion and interaction are discussed for its futuristic applications in food. Though immersive technology gives users a realistic virtual environment experience, its application in food systems is in the budding stage. More research on human response studies would contribute to its innovative and inevitable application in the future.

The use of Mixed Reality (MR) in Information Systems learning and teaching pedagogies is becoming more widely accepted as providing a reflection of possible realities. The aim of using MR in learning design is to extend the student experience by helping them not only to ‘see the unseen’ through the capacity of MR but also to visualise and interact with complex and abstract concepts. This study expands on the current literature by applying design science to use, survey and evaluate Microsoft HoloLens, a MR device, in an Information Systems classroom. The significance of this study is the use of MR in an Information Systems class to understand and learn the technology and provide an authentic learning experience to prepare the students for technological disruption and the future of work. Student survey responses were positive, with high student satisfaction in the classroom, demonstrating critical and creative engagement with the technological limitations and challenges with the user experience. This paper concludes with suggestions of specific pedagogical models that could be used in Information Systems education.

BackgroundVirtual reality (VR) dental simulators are currently used in preclinical skills training. However, with the development of extended reality technologies, the use of mixed reality (MR) has shown significant advantages over VR.ObjectiveThis study aimed to describe the research and development of a newly developed MR and haptic–based dental simulator for tooth preparation and to conduct a preliminary evaluation of its face validity.MethodsA prototype of the MR dental simulator for tooth preparation was developed by integrating a head-mounted display (HMD), special force feedback handles, a foot pedal, computer hardware, and software program. We recruited 34 participants and divided them into the Novice group (n=17) and Skilled group (n=17) based on their clinical experience. All participants prepared a maxillary right central incisor for an all-ceramic crown in the dental simulator, completed a questionnaire afterward about their simulation experience, and evaluated hardware and software aspects of the dental simulator.ResultsOf the participants, 74% (25/34) were satisfied with the overall experience of using the Unidental MR Simulator. Approximately 90% (31/34, 91%) agreed that it could stimulate their interest in learning, and 82% (28/34) were willing to use it for skills training in the future. Differences between the 2 study groups in their experience with the HMD (resolution: P=.95; wearing comfort: P=.10), dental instruments (P=.95), force feedback of the tooth (P=.08), simulation of the tooth preparation process (P=.79), overall experience with the simulation (P=.47), and attitude toward the simulator (improves skills: P=.47; suitable for learning: P=.36; willing to use: P=.89; inspiring for learning: P=.63) were not significant. The Novice group was more satisfied with the simulator’s ease of use (P=.04). There were significant positive correlations between the overall experience with the simulation and the HMD’s resolution (P=.03) and simulation of the preparation process (P=.001).ConclusionsThe newly developed Unidental MR Simulator for tooth preparation has good face validity. It can achieve a higher degree of resemblance to the real clinical treatment environment by improving the positional adjustment of the simulated patients, for a better training experience in dental skills.

(1) Background: The development of new technologies means that the use of virtual reality is increasingly being implemented in rehabilitative approaches for adult stroke patients. Objective: To analyze the existing scientific evidence regarding the application of immersive and non-immersive virtual reality in patients following cerebrovascular incidents and their efficacy in achieving dynamic and static balance. (2) Data sources: An electronic search of the databases Medline, Cochrane Library, PEDro, Scopus, and Scielo from January 2010 to December 2020 was carried out using the terms physiotherapy, physical therapy, virtual reality, immersive virtual reality, non-immersive virtual reality, stroke, balance, static balance, and dynamic balance. Selection of studies: Randomized controlled trials in patients older than 18 developed with an adult population (>18 years old) with balance disorders as a consequence of suffering a stroke in the previous six months before therapeutic intervention, including exercises harnessing virtual reality in their interventions and evaluations of balance and published in English or Spanish, were included. A total of two hundred twenty-seven articles were found, ten of which were included for review and of these, nine were included in the subsequent meta-analysis. (3) Data extraction: Two authors selected the studies and extracted their characteristics (participants, interventions, and validation instruments) and results. The methodological quality of the studies was evaluated using the PEDro scale, and the risk of bias was determined using the Cochrane risk-of-bias tool. Data synthesis: Of the selected studies, three did not show significant improvements and seven showed significant improvements in the intervention groups in relation to the variables. (4) Conclusions: Non-immersive virtual reality combined with conventional rehabilitation could be considered as a therapeutic option.

Lower body tracking and user experience analysis are ongoing challenges in immersive technologies and biomechanics research, respectively. Particularly, in immersive technologies like mixed reality, there are limited research applications in user comfort and experience, and locomotion. In this paper, we aimed to conduct a cross-disciplinary research (by combining the elements of Biomechanics and human–computer interaction) in understanding the impact of user experience and immersive technologies induced symptoms in virtual and mixed reality environments for obstacle clearance tasks. For this, we created obstacle clearance applications for Oculus Quest and Microsoft HoloLens. Also, we conducted a user study with 12 participants under two conditions (virtual reality and mixed reality) in obstacle clearance behavior of varying heights. The results from our study indicated that the levels of immersion and enjoyment were significantly higher in virtual reality compared to mixed reality. In terms of virtual reality-induced symptoms, the participants found the mixed reality environment to be slightly more comfortable than virtual reality.

Museums and exhibitions often present physical artefacts which may contain rich histories or deep meaning associated with them. These additional contents are often installed physically as informational panels shown on the wall. However, these may sometimes be challenging to deploy due to space constraints. In order to address this challenge, we introduce the use of mixed reality. Mixed reality offers an immersive and interactive experience through the use of head mounted displays and in-air gestures. Visitors can discover additional content virtually, without changing the physical space. For a small-scale exhibition at a cafe, we developed a Microsoft HoloLens application to create an interactive experience on top of a collection of historic physical items. Through public experiences at the cafe, we received positive feedback of our system. In this paper, we discuss the design and implications of our system, survey results, as well as challenges that were encountered in deploying our mixed reality experience in a public setting.

Texting while walking (TeWW) has become common among people of all ages, and mobile phone use during gait is increasingly associated with pedestrian injury. Although dual-task walking performance is known to decline with age, data regarding the effect of age on dual-task performance in ecological settings are limited. The objective of this study was to evaluate the effect of age, environment (indoors/outdoors), and mixed reality (merging of real and virtual environments) on TeWW performance. A cross-sectional design was used. Young (n=30; 27.8±4.4 years) and older (n=20; 68.9±3.9 years) adults performed single- and dual-task texting and walking indoors and outdoors, with and without a mixed reality display. Participants also completed evaluations of visual scanning and cognitive flexibility (Trail Making Test) and functional mobility (Timed "Up & Go" Test). Indoors, similar interference to walking and texting occurred for both groups, but only older adults' gait variability increased under dual task conditions. Outdoors, TeWW was associated with larger age-related differences in gait variability, texting accuracy, and gait dual-task costs. Young adults with better visual scanning and cognitive flexibility performed TeWW with lower gait costs (r=0.52-0.65). The mixed reality display was unhelpful and did not modify walking or texting. Older adults tested in this study were relatively high functioning. Gaze of participants was not directly monitored. Although young and older adults possess the resources necessary for TeWW, older adults pay an additional "price" when dual-tasking, especially outdoors. TeWW may have potential as an ecologically valid assessment and/or an intervention paradigm for dual-task performance among older adults as well as for clinical populations.