Spelland: Situated Language Learning with a Mixed-Reality Spelling Game through Everyday Objects

This chapter will be focused on contributing to the increase of universal design competencies of future engineers, educators, and designers through the use of mixed reality technologies, closing the gap between theory and field application of principles, towards a more inclusive world and promoting health and wellbeing for all. The experience of a situation where limitations arise in relation to what is taken for granted is an important experience that leads to a personal knowledge of the difficulties. By the use of simulators, especially virtual (VR) and mixed reality (MR) technologies, it is possible to create such experiences. Training based on MR can prepare future and current professionals for up-to-date requirements of the labor market. In addition, it can ensure that the standards such as barrier-free concepts, broader accessibility, adaptive and assistive technology will be familiar to trainees.

BackgroundVirtual reality (VR) dental simulators are currently used in preclinical skills training. However, with the development of extended reality technologies, the use of mixed reality (MR) has shown significant advantages over VR.ObjectiveThis study aimed to describe the research and development of a newly developed MR and haptic–based dental simulator for tooth preparation and to conduct a preliminary evaluation of its face validity.MethodsA prototype of the MR dental simulator for tooth preparation was developed by integrating a head-mounted display (HMD), special force feedback handles, a foot pedal, computer hardware, and software program. We recruited 34 participants and divided them into the Novice group (n=17) and Skilled group (n=17) based on their clinical experience. All participants prepared a maxillary right central incisor for an all-ceramic crown in the dental simulator, completed a questionnaire afterward about their simulation experience, and evaluated hardware and software aspects of the dental simulator.ResultsOf the participants, 74% (25/34) were satisfied with the overall experience of using the Unidental MR Simulator. Approximately 90% (31/34, 91%) agreed that it could stimulate their interest in learning, and 82% (28/34) were willing to use it for skills training in the future. Differences between the 2 study groups in their experience with the HMD (resolution: P=.95; wearing comfort: P=.10), dental instruments (P=.95), force feedback of the tooth (P=.08), simulation of the tooth preparation process (P=.79), overall experience with the simulation (P=.47), and attitude toward the simulator (improves skills: P=.47; suitable for learning: P=.36; willing to use: P=.89; inspiring for learning: P=.63) were not significant. The Novice group was more satisfied with the simulator’s ease of use (P=.04). There were significant positive correlations between the overall experience with the simulation and the HMD’s resolution (P=.03) and simulation of the preparation process (P=.001).ConclusionsThe newly developed Unidental MR Simulator for tooth preparation has good face validity. It can achieve a higher degree of resemblance to the real clinical treatment environment by improving the positional adjustment of the simulated patients, for a better training experience in dental skills.

The industrial landscape is about to undergo a revolution thanks to the convergence of emerging technologies. Specifically, the integration of blockchain with the digital twin-enabled industrial internet of things (IIoT) within mixed reality environments has the potential to do just that. This chapter presents a thorough analysis of the applications, advantages, and difficulties of various technologies while examining their potential for synergy. The digital twin provides real-time data monitoring, analysis, and predictive maintenance capabilities. The industrial internet of things establishes connections between tangible objects and sensors, enabling smooth communication and interchange of data. This chapter investigates the use of mixed reality (MR) technology to integrate blockchain technology with the IIoT that is enabled by digital twins. The potential for improving data security, trust, and transparency in industrial applications through the integration of blockchain with IIoT and MR could aid in the development of the Industry 4.0 paradigm.

Digital hydraulics is a discrete technology that integrates advanced dynamic system controls, digital electronics, and machine learning to enhance fluid power systems’ performance, overall efficiency, and controllability. A mechanically actuated inline three-piston variable displacement digital pump was previously proposed and designed. The inline three-piston pump incorporates complex mechanical and hydraulic subsystems and highly coupled mechanisms. The complexity of the utilized subsystems poses challenges when assessing the viability of the conceptual design. Therefore, this work focuses on designing, developing, and implementing a collaborative virtual platform involving a digitized module showcasing the internal mechanical structure of the digital pump utilizing mixed reality (MR) technology. MR technology is acknowledged as the forthcoming evolution of the human–machine interface in the real–virtual environment utilizing computers and wearables. This technology permits running simulations that examine the complexity of highly coupled systems, like the digital pump, where understanding the physical phenomenon is far too intricate. The developed MR platform permits multiple users to collaborate in a synchronized immersive MR environment to study and analyze the applicability of the pump’s design and the adequacy of the operated mechanisms. The collaborative MR platform was designed and developed on the Unity game engine, employing Microsoft Azure and Photon Unity Networking to set up the synchronized MR environment. The platform involves a fully interactive virtual module on the digital pump design, developed in multiple stages using Microsoft’s Mixed Reality Tool Kit (MRTK) for Unity and deployed in the synchronized MR environment through a HoloLens 2 MR headset. A research study involving 71 participants was carried out at Purdue University. The study’s objective was to explore the impact of the collaborative MR environment on understanding the complexity and operation of the digital pump. It also sought to assess the effectiveness of MR in facilitating collaboration among fluid power stakeholders in a synchronized digital reality setting to study, diagnose, and control their complex systems. Surveys were designed and completed by all 71 participants after experiencing the MR platform. The results indicate that approximately 75% of the participants expressed positive attitudes toward their overall MR platform experience, with particular appreciation for its immersive nature and the synchronized collaborative environment it provided. More than 70% of the participants agreed that the pump’s collaborative MR platform was essential for studying and understanding the complexity and intricacy of the digital pump’s mechanical structure. Overall, the results demonstrate that the MR platform effectively facilitates the visualization of the complex pump’s internal structure, inspection of the assembly of each of the involved subsystems, and testing the applicability of the complicated mechanisms.

Two instructors collaborated across programs in a School of Education to bring mixed reality simulation to preservice early childhood educators in an effort to utilize content knowledge to practice challenging conversations. The scenario chosen was a parent-teacher conference focused on supporting a child with disabilities in an inclusive early childhood preschool classroom. The authors discuss the process of implementing mixed reality simulation in Early Childhood Studies and potential implications for lifelong impact of family engagement throughout a child’s schooling experiences and the use of mixed reality in teacher preparation. Interdisciplinary collaboration across programs within a school of education are considered.

Recent technology advances in both Virtual Reality and Augmented Reality are creating an opportunity for a paradigm shift in the design of human-computer interaction systems. Delving into the Reality-Virtuality Continuum, we find Mixed Reality - systems designed to augment the physical world with virtual entities that embody characteristics of real world objects. In the medical field, Mixed Reality systems can overlay real-time and spatially accurate results onto a patient's body without the need for external screens. The complexity of these systems previously required specialized prototypes, but newly available commercial products like the Microsoft HoloLens make the technology more available. Through a combination of literature review, expert analysis, and prototyping we explore the use of Mixed Reality in healthcare. From the experience of prototyping Patiently and HoloSim, two applications for augmenting medical training, we outline considerations for the future design and development of virtual interfaces grounded in reality.

Extended-Reality Technologies: An Overview of Emerging Applications in Medical Education and Clinical Care.

The use of Mixed Reality (MR) in Information Systems learning and teaching pedagogies is becoming more widely accepted as providing a reflection of possible realities. The aim of using MR in learning design is to extend the student experience by helping them not only to ‘see the unseen’ through the capacity of MR but also to visualise and interact with complex and abstract concepts. This study expands on the current literature by applying design science to use, survey and evaluate Microsoft HoloLens, a MR device, in an Information Systems classroom. The significance of this study is the use of MR in an Information Systems class to understand and learn the technology and provide an authentic learning experience to prepare the students for technological disruption and the future of work. Student survey responses were positive, with high student satisfaction in the classroom, demonstrating critical and creative engagement with the technological limitations and challenges with the user experience. This paper concludes with suggestions of specific pedagogical models that could be used in Information Systems education.

PD30-08 EVALUATION OF MIXED REALITY (MR) TECHNOLOGIES FOR REMOTE GUIDANCE DURING ULTRASOUND (US)-GUIDED PERCUTANEOUS RENAL ACCESS SIMULATION: A PROSPECTIVE, RANDOMIZED COMPARATIVE TRIAL

The steady increase in the aging population worldwide is expected to cause a shortage of doctors and therapists for older people. This demographic shift requires more efficient and automated systems for rehabilitation and physical ability evaluations. Rehabilitation using mixed reality (MR) technology has attracted much attention in recent years. MR displays virtual objects on a head-mounted see-through display that overlies the user's field of vision and allows users to manipulate them as if they exist in reality. However, tasks in previous studies applying MR to rehabilitation have been limited to tasks in which the virtual objects are static and do not interact dynamically with the surrounding environment. Therefore, in this study, we developed an application to evaluate cognitive and motor functions with the aim of realizing a rehabilitation system that is dynamic and has interaction with the surrounding environment using MR technology. The developed application enabled effective evaluation of the user's spatial cognitive ability, task skillfulness, motor function, and decision-making ability. The results indicate the usefulness and feasibility of MR technology to quantify motor function and spatial cognition both for static and dynamic tasks in rehabilitation.

ObjectiveTo provide an overview of the current state of mixed reality (MR) technology’s application in medical education and to review a research scope of the area.MethodsAccording to the Joanna Briggs Institute’s scope-based review procedures, searches were conducted in the following databases: PubMed, Embase, Web of Science Core Collection, Cochrane Library, CNKI, WanFang Database, and VIP Database. The included literatures were compiled and examined, and the search period was limited to the period from the database’s creation to April 20, 2025.ResultsIn all, 4118 records were found. After eligibility assessment, 28 studies in English or Chinese that examine the use of MR in education for medical students were included. Data extraction focused on theoretical/skills test scores, students’ satisfaction, testing and evaluation of the system operation as well as students’ learning interest of MR. The majority of studies indicated MR technology effectively enhances students’ theoretical knowledge and skill proficiency. Eighteen studies consistently reported a high students’ satisfaction level with MR-assisted instruction; MR systems demonstrated favorable usability due to its high consistency. Ten studies showed students in the MR-assisted learning groups exhibit heightened learning interest.ConclusionMR Technology has demonstrated excellent performance in enhancing students’ mastery of theories/skills, improving students’ satisfaction, and increasing students’ interest. However, it has not yet been widely adopted due to the high costs associated with teacher training and the purchase and maintenance of equipment. The future should prioritize continuously developing the immersive experience and engagement of MR technology, integrating ecological instantaneous evaluation methodologies and artificial intelligence technology, and combining economic benefit analysis and feasibility testing to perform higher-quality original research.

Museums and exhibitions often present physical artefacts which may contain rich histories or deep meaning associated with them. These additional contents are often installed physically as informational panels shown on the wall. However, these may sometimes be challenging to deploy due to space constraints. In order to address this challenge, we introduce the use of mixed reality. Mixed reality offers an immersive and interactive experience through the use of head mounted displays and in-air gestures. Visitors can discover additional content virtually, without changing the physical space. For a small-scale exhibition at a cafe, we developed a Microsoft HoloLens application to create an interactive experience on top of a collection of historic physical items. Through public experiences at the cafe, we received positive feedback of our system. In this paper, we discuss the design and implications of our system, survey results, as well as challenges that were encountered in deploying our mixed reality experience in a public setting.

Texting while walking (TeWW) has become common among people of all ages, and mobile phone use during gait is increasingly associated with pedestrian injury. Although dual-task walking performance is known to decline with age, data regarding the effect of age on dual-task performance in ecological settings are limited. The objective of this study was to evaluate the effect of age, environment (indoors/outdoors), and mixed reality (merging of real and virtual environments) on TeWW performance. A cross-sectional design was used. Young (n=30; 27.8±4.4 years) and older (n=20; 68.9±3.9 years) adults performed single- and dual-task texting and walking indoors and outdoors, with and without a mixed reality display. Participants also completed evaluations of visual scanning and cognitive flexibility (Trail Making Test) and functional mobility (Timed "Up & Go" Test). Indoors, similar interference to walking and texting occurred for both groups, but only older adults' gait variability increased under dual task conditions. Outdoors, TeWW was associated with larger age-related differences in gait variability, texting accuracy, and gait dual-task costs. Young adults with better visual scanning and cognitive flexibility performed TeWW with lower gait costs (r=0.52-0.65). The mixed reality display was unhelpful and did not modify walking or texting. Older adults tested in this study were relatively high functioning. Gaze of participants was not directly monitored. Although young and older adults possess the resources necessary for TeWW, older adults pay an additional "price" when dual-tasking, especially outdoors. TeWW may have potential as an ecologically valid assessment and/or an intervention paradigm for dual-task performance among older adults as well as for clinical populations.

In this study, we propose the use of mixed reality (MR) for the purposes of biological education. Our objective is to create an interactive edutainment MR framework for users to learn about nature and human beings. MtsuDomoe, an interactive ecosystem simulator of virtual creatures in a petri dish, comprises three species of primitive artificial creatures. MitsuDomoe simulates the predation chain of the virtual creatures in the petri dish, and users can interact with this ecosystem via the petri dish interface. Users can also experience immersive observation by wearing HMD. By combining the MR petri dish and immersive virtual reality (VR) interfaces, we synergistically improve user understanding of the experience.

BackgroundThis was a prospective comparative study of mixed reality (MR) technology assisted lumbar pedicle screws placement and traditional lumbar pedicle screws placement.Material/MethodsFifty cases of lumbar pedicle screws placement were randomly divided into 2 groups: 25 cases with MR technology in group A, and 25 cases without MR technology in group B. All patients had their scores on the Oswestry disability index (ODI) of low back pain and the visual analog scale (VAS) of the affected lower limb recorded at pre-operation. Blood loss, operative duration, success rate of first penetration by tap, and number of times C-arm fluoroscopy was performed were recorded at intraoperation. The postoperative drainage was recorded. The ODI of low back pain and VAS of the affected lower limb were recorded at 1, 3, and 6 months after operation.ResultsGroup A had less bleeding, shorter operation time, higher success rate of first penetration by tap, and fewer times using C-arm fluoroscopy at intraoperation (P<0.05). There was significant difference in ODI scores and VAS scores at 1 mouth after operation (P<0.05). The postoperative drainage of group A was less than group B (P<0.05). The implantation accuracy of group A was higher than group B (P<0.05). The postoperative recovery rate of low back pain of group A was faster than group B (P<0.05).ConclusionsThe safety of spinal surgery and implantation accuracy of pedicle screw fixation system could be increased by MR technology.