Multi-Party Mixed Reality Interaction for Earth Sciences Education

Extended-Reality Technologies: An Overview of Emerging Applications in Medical Education and Clinical Care.

With Generation Z becoming the primary group of nursing learners and the increasing shortage of nursing education resources, augmented reality and mixed reality based on head-mounted displays are being used more and more in nursing education. However, the current application landscape and the proper usage of these concepts remain unclear. Therefore, this study aims to conduct a scoping review to explore the current applications of head-mounted display-based augmented reality and mixed reality in nursing education and to clarify the definitions and usage of the concepts of augmented reality and mixed reality, ultimately providing directions for future applications and research. Based on the five-stage framework and PRISMA-ScR guidelines, a comprehensive collection and summarization of evidence regarding the application of head-mounted display-based augmented reality and mixed reality in nursing education were conducted. The databases retrieved include CNKI, Wanfang Database, VIP, CBM, PubMed, Cochrane Library, Embase, CINAHL, Web of Science, Scopus, IEEE Xplore, ACM Digital Library, and Ei. The languages of the included literature are Chinese and English. The retrieval was up to October 2024. A total of 44 studies were included in this review, covering three types of head-mounted displays: immersive head-mounted displays, smart glasses, and smartphone-based head-mounted displays. The main application areas were skills training and knowledge acquisition. Most of the studies were feasibility studies, though they also included some efficacy studies and research on personal experiences. In addition, these studies often employed vague or inconsistent definitions of augmented reality and mixed reality. Despite various explorations in the application of head-mounted display-based augmented reality and mixed reality in nursing education, there is still room for improvement in the relevant theory and utilization of this technology. In the future, interventions should use the three dimensions (observation of reality, real - virtual interaction, and fidelity of virtuality) described in detail, rather than simply employing the concepts of augmented reality or mixed reality. Efforts should be concentrated on developing and implementing head-mounted displays combined with other technologies that boast enhanced performance and cost - effectiveness, and further validating their effectiveness.

Evaluation of cognitive load for a mixed reality anatomy application

A Mixed Reality (MR) application using an optical see-through headset was developed to assess short-term spatial memory. A study with 29 participants was conducted. Data from this study were compared to two previous studies using mobile Augmented Reality (AR) and Virtual Reality (VR) with headsets. When comparing the three technologies (MR, AR, VR) for the performance variables, there were no statistically significant differences for either the total number of correctly placed objects or the total number of attempts. However, the MR application required more time than the AR and VR applications in the evaluation phase and more time than the VR application in the learning phase. Our arguments for the longer time are the novelty of the MR application for the participants and the characteristics of the applications. The key results from the MR study include the following: (1) the objects used in the MR application were correctly positioned on a map, which implies that the memory acquired with the MR application is effectively transferred to the user's mental map; (2) for the performance variables, there were no significant differences in the results by gender; (3) and the usability rating decreased with computer experience. The results show that the MR application is effective for spatial memory assessment and was well rated by the participants. The three technologies, along with suitable hardware, are effective for spatial memory assessment. However, MR using optical see-through headsets offers advantages over mobile AR and VR using headsets, discussed in this publication.

The perceptual behavior of consumers on a product displayed in the market has a vital role in analyzing the importance given to that product. Therefore, various strategies have been developed to understand this consumer behavior in the selection of products. Immersive technologies like virtual, augmented, and mixed reality are among them. With the foremost feature of immersion in the virtual world and interaction of users with virtual objects, virtual reality, and augmented reality have unlocked their potential in research and a user-friendly tool for analyzing consumer behavior. In addition to these technologies, mixed reality also has a significant role in investigating consumer behavior. Studies on immersive technologies in food applications are vast, hence this review focuses on the applications of virtual, augmented, and mixed reality in the food selection behavior of consumers. The behavioral studies are elicited to develop new products based on consumer needs, to understand the shopping behavior in supermarkets for real-time usage, and to know the influence of emotions in a selection of products. The findings suggest that virtual, augmented, and mixed reality induce immersion of the users in food selection behavioral studies. Information on the technological advancements in the tools used for bringing immersion and interaction are discussed for its futuristic applications in food. Though immersive technology gives users a realistic virtual environment experience, its application in food systems is in the budding stage. More research on human response studies would contribute to its innovative and inevitable application in the future.

This paper shows our work in progress towards an easy to use development environment for Mixed Reality (MR) Applications. We argue that development of MR applications is a collaboration between interaction designers who know about user requirements, and expert developers who know the intricacies of MR development. This collaboration should be supported by tools that aid both roles and ease their communication. We also argue that real MR development should allow easy migration from one hardware setup to another, since hardware greatly varies in these type of applications, and it is important to fit a solution to the particular user's requirements and context.We show the foundational concepts in our work and current Integrated Development Environment (IDE) implementation. This work is based on InTml, a domain specific language for MR applications, and Eclipse, an open source, general purpose IDE.

This study aims to explore the application of mixed reality in experimental safety teaching, designing and implementing a mixed reality-based experimental teaching method to enhance the effectiveness of experiments and user experience. Through field visits to design laboratories and expert interviews, this research analyzes the existing problems in design experimental teaching and real user needs. It proposes a multi-level, multi-dimensional mixed reality innovation design experiment platform construction method from teaching to dialogue, from human-computer interaction to perception, from creation to manufacturing, and from communication to thinking. Finally, this study uses the intelligent cockpit interface experience perception mixed reality experimental platform as an example to confirm that mixed reality technology can provide an immersive learning experience. This research provides theoretical support and practical reference for the application of mixed reality in experimental safety teaching, and offers prospects for future teaching design and research.

With tremendous research, innovation in human computer interaction, infographics and greater accessibility of visualization technologies i.e., augmented reality; virtual reality and mixed reality is to find new possibilities for its applications. The ability to transform the user's perception and interaction with his environment has increased his popularity of use in all spheres of life. This study analyses the literature on augmented reality, virtual reality and mixed reality applications in order to find the research gaps in these fields. This review study has taken 103 papers of 2010-2021 into account from SCOPUS, SCI and IEEE databases. The results of the analysis can provide some useful insights regarding potential and use of augmented reality, virtual reality and mixed reality in the different areas of application.

In proposing a modern approach to process industry operational integrity management, a question that often surfaces is "What is the most complex activity within operational integrity management that affects performance, productivity and efficiency?". There are variety of answers to this question, one answer being "Human limitations in conceiving data to make informed decisions". The objective of this presentation is to outlines the key benefits of mixed reality platforms in presenting data to managers, operators and maintenance personnel working in an Oil & Gas facility. Process industry in the past did not generate as much data as it does today. With the amount of data generated every day, it is necessary to innovate so as to analyze data and visually represent them thereby providing a "total immersive knowledge". In light of the recent developments in mixed reality platforms, it is now possible to use this technology to visually present data focused around the needs of humans. Using mixed reality, one can merge the view of real equipment of a plant with a virtual world of data, simulation and instructions, such that both the physical and digital world co-exist. Few of the applications of this technology in the Oil & Gas industry are 1) providing safe operations, maintenance and repair instructions real time 2) monitor, record and analyze data real time during inspections 3) visualize data outside the control room 4) effectively train personnel during visits 5) analyze trends and predict outcomes 6) share data with managers, vendors for efficient decisions. The presentation outlines the key benefits of mixed reality platforms such as Microsoft Hololens in presenting data to managers, operators and maintenance personnel working in an Oil & Gas facility. The presentation will also discuss findings of a sponsored pilot project where mixed reality was implemented. At the end, the presentation will also raise the current challenges and possible solutions in successfully implementing such technologies.

As technology advances, mobile devices have gradually turned into wearable devices. Furthermore, virtual reality (VR), augmented reality (AR), and mixed reality (MR) are being increasingly applied in medical fields such as medical education and training, surgical simulation, neurological rehabilitation, psychotherapy, and telemedicine. Research results demonstrate the ability of VR, AR, and MR to ameliorate the inconveniences that are often associated with traditional medical care, reduce incidents of medical malpractice caused by unskilled operations, and reduce the cost of medical education and training. What is more, the application of these technologies has enhanced the effectiveness of medical education and training, raised the level of diagnosis and treatment, improved the doctor-patient relationship, and boosted the efficiency of medical execution. The present study introduces VR, AR, and MR applications in medical practice and education with the aim of helping health professionals better understand the applications and use these technologies to improve the quality of medical care.

A preliminary clinical report of implant placements with 2 immersive reality technologies is described: augmented reality with head mounted display and mixed reality with a tablet PC. Both immersive realities are promising and could facilitate innovative dental applications. However, mixed reality requires further development for clinical optimization.

The availability of skilled workers has become a severe issue for production companies in developed countries like Germany in recent years. Specialists for maintenance tasks are in strong demand and their workforce should be steered towards the most challenging tasks in maintenance. In this case study we analyze, whether less skilled or not specifically instructed workers can perform routine maintenance tasks without being directly supported by experts. In our setup ten participants try to handle a maintenance routine checklist in three different ways: paper-based printouts, on a mobile tablet, and with a Microsoft HoloLens 2 Mixed Reality (MR) device. The results show that workers fail to complete the task with a mere paper-based approach, but are able to succeed both with the mobile tablet and MR application. Quantitative time measurements and qualitative analyses indicate that the mobile tablet app was easier to learn, did not impair the worker as much as the HoloLens and worked more stable. However, the HoloLens solution showed advantages as hands-free and spatial guiding device, which could be important aspects when better MR hardware will be available and MR interaction principles become common knowledge.

The priority of modern dentistry is to keep patients’ teeth for as long as possible. Tooth extraction is a procedure performed as a last resort when conservative methods and endodontic surgery procedures have not brought the expected results. As a consequence, the number of patients in dental offices, who require first and repeated endodontic treatment, is increasing. Thus, the development of new technologies in the medical industry, including microscopy, computer tomography (CT), as well as diode and neodymium-YAG-erbium lasers, enables dentists to increase the percentage of successful treatments. Moreover, mixed reality (MR) is a very new technology, in which the 3D view can help plan or simulate various types of tasks before they will be carried out in real life. In dentistry, 3D holography can be applied to display CT data to plan endodontic treatment. The most important element in effective root canal treatment is the precise imaging of the root canal. The CT scans allow dentists to view the anatomy of the patient’s tooth with much higher precision and understanding than using 2D radiography (RTG-radiographic photo) pictures. Recently, the development of new 3D technologies allows dentists to obtain even more data from existing CT scans. In this paper, the CT scan data were applied to generate patient teeth in 3D and simulate the view of the root canal’s anatomy in MR devices, i.e., Microsoft HoloLens 2. Using DICOM RAW data from the CT exam, we generated a 3D model of the jaw with a tooth. In the next step, the crown of the tooth was removed in a similar way to how a dentist would do this using a dental handpiece. Furthermore, all root canals were cleaned of everything inside. This way we achieved empty tunnels, namely root canals. Finally, we added appropriate lighting, similar to the type of lighting that dentists use. The proposed approach enables to display of the root canals in the same way as during the endodontic procedure using a microscope. It allows for the visualization of the root canal and changing its direction, in which dimensional accuracy is crucial. It turns out that mixed reality can be considered a complementary method to the traditional approach, which reduces the amount of time for the root canal treatment procedure by up to 72.25%, depending on the complexity of the case, and increases its effectiveness. Thus, the mixed reality-based system can be considered an effective tool for planning dental treatment.

This article aims to assist readers in understanding the current status of studies on the subject by providing a descriptive bibliometric analysis of publications on virtual reality (VR), augmented reality (AR), and mixed reality (MR) technologies in cultural heritage. A bibliometric analysis of 1214 publications in this discipline in the Scopus database between 2014 and the beginning of June 2024 was performed. We used VOSviewer and Bibliometrix as the analysis tools in this investigation. The outcome of this study provides a detailed overview of the descriptive bibliometric analysis based on seven categories, including the annual count of articles and citations, the most productive author, the primary affiliation, the publication source, and the subject areas. The contribution of this research lies in offering valuable insights for practitioners and researchers, helping them make informed decisions on the use of immersive technologies, for example, VR, AR, and MR, in the context of cultural heritage.

Augmented Reality (AR) and Mixed Reality (MR) are new emerging technologies that are becoming increasingly popular. Because these technologies are so new, there is a lack of standards and consistency in application and hardware design. This can make it difficult to learn and frustrating for users. One way to standardized design and enhance the usability of a product is through the use of heuristic evaluations. General heuristics, such as Nielsen’s 10 or Schneiderman’s 8 usability heuristics have been used to evaluate these technologies. These heuristics are a useful starting point because they bring attention to many crucial aspects of the usability of a product. However, additional aspects that could alter the users’ experience may not be assessed due to the uniqueness of AR and MR. There are very few validated AR and MR heuristics in the literature for practitioners to use. The purpose of this study was to create and validate a heuristic checklist that can be used to assess and inform design changes that influence the user experience of an AR or MR application and/or device. We followed an established and comprehensive 8-stage methodology developed by Quiñones, Rusu, & Rusu to create and validate our AR and MR usability heuristic checklist [4]. This included a search and summary of the current literature, formally defining heuristics based on this literature search, and validating through heuristic evaluations, expert reviews, and user testing. Our final revised heuristic checklist included 11 heuristics and 94 checklist items that encompasses usability aspects for AR and MR technologies. KeywordsHuman factorsUser experienceAugmented RealityMixed RealityUsabilityDesign guidelines