Mixed Reality and Haptic–Based Dental Simulator for Tooth Preparation: Research, Development, and Preliminary Evaluation

Extended-Reality Technologies: An Overview of Emerging Applications in Medical Education and Clinical Care.

This chapter will be focused on contributing to the increase of universal design competencies of future engineers, educators, and designers through the use of mixed reality technologies, closing the gap between theory and field application of principles, towards a more inclusive world and promoting health and wellbeing for all. The experience of a situation where limitations arise in relation to what is taken for granted is an important experience that leads to a personal knowledge of the difficulties. By the use of simulators, especially virtual (VR) and mixed reality (MR) technologies, it is possible to create such experiences. Training based on MR can prepare future and current professionals for up-to-date requirements of the labor market. In addition, it can ensure that the standards such as barrier-free concepts, broader accessibility, adaptive and assistive technology will be familiar to trainees.

Recently MR (Mixed Reality) techniques are used in many fields, and one of them is work support. Work support using MR techniques can display work instructions directly in the work space and help user to work effectively especially in assembling tasks. MR work support for assembling tasks often uses HMD (Head Mounted Display) to construct MR environment. However, there are some problems for the use of HMD, such as a burden to the head, a narrow view and a motion picture sickness. One of the techniques to solve such problems is pepper's ghost which is an optical illusion using a glass. In this paper, we propose the naked eye MR work supporting system for assembling tasks using pepper's ghost. This system enables a beginner to assemble some blocks into one object by the naked eye with a few burdens.

This work explores the use of mixed-reality (MR) technology to enable situated language learning using everyday objects in the environment around the learners. The learning method is based on Presentation, Practice, and Production (PPP), which cultivates the habit of independent learning through repetition, practice, and demonstration. In our game design, the learners first interact with real-world objects via MR, and the objects’ spelling and their pronunciation will appear (Presentation), the learners repeat the pronunciation (Practice) to collect the letters of this objects, and finally the learner use the collected letters to spell out the target words (Production), which then transform into interactive 3D objects. We designed the learning experience and content tools using gestural UI, voice input, and object-to-word engine. Children in the preliminary user study found the game to be immersive, helpful in learning the spelling of the everyday objects and the target words, and additionally showed increased interests in learning about other nearby objects after playing the game.

Two instructors collaborated across programs in a School of Education to bring mixed reality simulation to preservice early childhood educators in an effort to utilize content knowledge to practice challenging conversations. The scenario chosen was a parent-teacher conference focused on supporting a child with disabilities in an inclusive early childhood preschool classroom. The authors discuss the process of implementing mixed reality simulation in Early Childhood Studies and potential implications for lifelong impact of family engagement throughout a child’s schooling experiences and the use of mixed reality in teacher preparation. Interdisciplinary collaboration across programs within a school of education are considered.

Recent technology advances in both Virtual Reality and Augmented Reality are creating an opportunity for a paradigm shift in the design of human-computer interaction systems. Delving into the Reality-Virtuality Continuum, we find Mixed Reality - systems designed to augment the physical world with virtual entities that embody characteristics of real world objects. In the medical field, Mixed Reality systems can overlay real-time and spatially accurate results onto a patient's body without the need for external screens. The complexity of these systems previously required specialized prototypes, but newly available commercial products like the Microsoft HoloLens make the technology more available. Through a combination of literature review, expert analysis, and prototyping we explore the use of Mixed Reality in healthcare. From the experience of prototyping Patiently and HoloSim, two applications for augmenting medical training, we outline considerations for the future design and development of virtual interfaces grounded in reality.

The use of Mixed Reality (MR) in Information Systems learning and teaching pedagogies is becoming more widely accepted as providing a reflection of possible realities. The aim of using MR in learning design is to extend the student experience by helping them not only to ‘see the unseen’ through the capacity of MR but also to visualise and interact with complex and abstract concepts. This study expands on the current literature by applying design science to use, survey and evaluate Microsoft HoloLens, a MR device, in an Information Systems classroom. The significance of this study is the use of MR in an Information Systems class to understand and learn the technology and provide an authentic learning experience to prepare the students for technological disruption and the future of work. Student survey responses were positive, with high student satisfaction in the classroom, demonstrating critical and creative engagement with the technological limitations and challenges with the user experience. This paper concludes with suggestions of specific pedagogical models that could be used in Information Systems education.

Currently, the research on urban air mobility and particularly on air taxi concepts experiences a major boom, not only in the aviation industry. One goal of DLL’s project HorizonUAM is to assess the passenger acceptance during this new way to travel. Therefore, we developed a mixed reality air taxi simulator that exploits the capabilities of a video-see-through head-mounted display. In this paper, we present and evaluate four different mixed reality setups that differ in the amount of real and virtual content present inside the air taxi cabin. In a study with twelve participants we assessed the overall simulator as well as the usability of the four setups with respect to the level of immersion and ease of interaction. Our findings Leveal that the participants preferred a fully virtual experience of an air taxi flight. From our tasks on human-machine interaction we also found the text on a virtual user interface to be easier readable than from a real tablet computer via the see-through cameras of the HMD. The Latings of the tasks that required input from the user, however, show that the vast majority of participants favor the interaction with the real tablet. Taking the results into consideration, we will further improve the air taxi simulator for an extensive human factors study on passenger acceptance.

Standard oral and maxillofacial three-dimensional model was developed with patients’ medical data while virtual reality (VR) simulator was developed in conjunction with head mount display (HMD) and Haptic device. The objective of this study was to evaluate the preclinical use of a VR training simulator in tooth preparation practice. Eighty-nine dental students were trained how to operate the simulator. The participants were then given sufficient time on the simulator to practice dental preparation. The students experience and opinion was then taken in through filling of questionnaires. On average content received 1.8 points, anatomy had 2.5 points, 2.6 points for the applicability, and 2.0 for the usability. As for the detailed items scores, queries about the possible development of the simulator and the interest of the learning process through the simulator were the highest at 3.1 and 3.0 points, respectively. Question about the benefit of the HMD and the haptic device during the practice had 1.5 and 1.6 points, respectively. The average total score was 2.2 points. VR tooth preparation simulator in the field of clinical dental education has powerful potential in regard to realistic models, environments, vision, posture, and economical efficiency.

Museums and exhibitions often present physical artefacts which may contain rich histories or deep meaning associated with them. These additional contents are often installed physically as informational panels shown on the wall. However, these may sometimes be challenging to deploy due to space constraints. In order to address this challenge, we introduce the use of mixed reality. Mixed reality offers an immersive and interactive experience through the use of head mounted displays and in-air gestures. Visitors can discover additional content virtually, without changing the physical space. For a small-scale exhibition at a cafe, we developed a Microsoft HoloLens application to create an interactive experience on top of a collection of historic physical items. Through public experiences at the cafe, we received positive feedback of our system. In this paper, we discuss the design and implications of our system, survey results, as well as challenges that were encountered in deploying our mixed reality experience in a public setting.

Texting while walking (TeWW) has become common among people of all ages, and mobile phone use during gait is increasingly associated with pedestrian injury. Although dual-task walking performance is known to decline with age, data regarding the effect of age on dual-task performance in ecological settings are limited. The objective of this study was to evaluate the effect of age, environment (indoors/outdoors), and mixed reality (merging of real and virtual environments) on TeWW performance. A cross-sectional design was used. Young (n=30; 27.8±4.4 years) and older (n=20; 68.9±3.9 years) adults performed single- and dual-task texting and walking indoors and outdoors, with and without a mixed reality display. Participants also completed evaluations of visual scanning and cognitive flexibility (Trail Making Test) and functional mobility (Timed "Up & Go" Test). Indoors, similar interference to walking and texting occurred for both groups, but only older adults' gait variability increased under dual task conditions. Outdoors, TeWW was associated with larger age-related differences in gait variability, texting accuracy, and gait dual-task costs. Young adults with better visual scanning and cognitive flexibility performed TeWW with lower gait costs (r=0.52-0.65). The mixed reality display was unhelpful and did not modify walking or texting. Older adults tested in this study were relatively high functioning. Gaze of participants was not directly monitored. Although young and older adults possess the resources necessary for TeWW, older adults pay an additional "price" when dual-tasking, especially outdoors. TeWW may have potential as an ecologically valid assessment and/or an intervention paradigm for dual-task performance among older adults as well as for clinical populations.

The industrial landscape is about to undergo a revolution thanks to the convergence of emerging technologies. Specifically, the integration of blockchain with the digital twin-enabled industrial internet of things (IIoT) within mixed reality environments has the potential to do just that. This chapter presents a thorough analysis of the applications, advantages, and difficulties of various technologies while examining their potential for synergy. The digital twin provides real-time data monitoring, analysis, and predictive maintenance capabilities. The industrial internet of things establishes connections between tangible objects and sensors, enabling smooth communication and interchange of data. This chapter investigates the use of mixed reality (MR) technology to integrate blockchain technology with the IIoT that is enabled by digital twins. The potential for improving data security, trust, and transparency in industrial applications through the integration of blockchain with IIoT and MR could aid in the development of the Industry 4.0 paradigm.

In this study, we propose the use of mixed reality (MR) for the purposes of biological education. Our objective is to create an interactive edutainment MR framework for users to learn about nature and human beings. MtsuDomoe, an interactive ecosystem simulator of virtual creatures in a petri dish, comprises three species of primitive artificial creatures. MitsuDomoe simulates the predation chain of the virtual creatures in the petri dish, and users can interact with this ecosystem via the petri dish interface. Users can also experience immersive observation by wearing HMD. By combining the MR petri dish and immersive virtual reality (VR) interfaces, we synergistically improve user understanding of the experience.

Background: To investigate the usability of Mixed-Reality (MR) during patient education in patients scheduled for abdominal aortic aneurysm (AAA) repair. Patients and methods: Consecutive patients scheduled for elective AAA repair were block-randomized in either the Mixed-Reality group (MR group) or the conventional group (control group). Patients of both groups were educated about open and endovascular repair of their respective AAA. The MR group was educated using a head-mounted display (HMD) demonstrating a three-dimensional virtual reconstruction of the respective patient's vascular anatomy. The control group was educated using a conventional two-dimensional monitor to display the patient's vasculature. Outcomes were informational gain as well as patient satisfaction with the educational process. (DRKS-ID:DRKS00025174). Results: 50 patients were included with 25 patients in either group. Both groups demonstrated improvements in scores in the Informational Gain Questionnaire (IGQ) when comparing pre- and post-education scores. (MR group: 6.5 points (±1.8) versus 7.9 points (±1.5); Control group: 6.2 points (±1.8) versus 7.6 points (±1.6); p<0.01) There was no significant difference between the MR group and the control group either in informational gain (MR group: 1.4±1.8; Control group: 1.4±1.8; p=0.5) nor in patient satisfaction scores (MR group: mean 18.3 of maximum 21 points (±3.7); Control group: mean 17 of 21 points (±3.6); p=0.1) Multiple regression revealed no correlation between the use of MR and informational gain or patient satisfaction. Usability of the system was rated high, and patients' subjective assessment of MR was positive. Conclusions: The use of MR in patient education of AAA patients scheduled for elective repair is feasible. While patients reported positively on the use of MR in education, similar levels of informational gain and patient satisfaction can be achieved with MR and conventional methods.

Recently, welfare vehicles are widely used among the elderly and the aged. However, it is not easy for inexperienced beginners or aged to control the welfare vehicle in narrow places such as elevators and corridors. In this paper, a driving assistance using MR which has collision avoidance function is proposed using a virtual platoon control. In the system, the cognitive and driving ability is enhanced from the simulated objective view point. The driver rides on the vehicle and controls using the virtual preceding vehicle with Head Mounted Display (HMD). The virtual preceding vehicle projected on a HMD steered virtually from a simulating objective viewpoint, and the real vehicle is robust controlled so as to follow the virtual preceding vehicle. Furthermore, the obstacle avoidance function in the narrow space is implemented on the assistance system with MR technology. The feature of the proposed assistance system is a modification of the driver's operation of not to collide the static obstacles using 3D modeling function of the environment by HMD. As a result, it is confirmed that collision can be avoided safely by correction for the driver's operation of the virtual preceding vehicle control.