Human Interface Guidelines for Interaction Zones in AR

Extended-Reality Technologies: An Overview of Emerging Applications in Medical Education and Clinical Care.

The perceptual behavior of consumers on a product displayed in the market has a vital role in analyzing the importance given to that product. Therefore, various strategies have been developed to understand this consumer behavior in the selection of products. Immersive technologies like virtual, augmented, and mixed reality are among them. With the foremost feature of immersion in the virtual world and interaction of users with virtual objects, virtual reality, and augmented reality have unlocked their potential in research and a user-friendly tool for analyzing consumer behavior. In addition to these technologies, mixed reality also has a significant role in investigating consumer behavior. Studies on immersive technologies in food applications are vast, hence this review focuses on the applications of virtual, augmented, and mixed reality in the food selection behavior of consumers. The behavioral studies are elicited to develop new products based on consumer needs, to understand the shopping behavior in supermarkets for real-time usage, and to know the influence of emotions in a selection of products. The findings suggest that virtual, augmented, and mixed reality induce immersion of the users in food selection behavioral studies. Information on the technological advancements in the tools used for bringing immersion and interaction are discussed for its futuristic applications in food. Though immersive technology gives users a realistic virtual environment experience, its application in food systems is in the budding stage. More research on human response studies would contribute to its innovative and inevitable application in the future.

A Mixed Reality (MR) application using an optical see-through headset was developed to assess short-term spatial memory. A study with 29 participants was conducted. Data from this study were compared to two previous studies using mobile Augmented Reality (AR) and Virtual Reality (VR) with headsets. When comparing the three technologies (MR, AR, VR) for the performance variables, there were no statistically significant differences for either the total number of correctly placed objects or the total number of attempts. However, the MR application required more time than the AR and VR applications in the evaluation phase and more time than the VR application in the learning phase. Our arguments for the longer time are the novelty of the MR application for the participants and the characteristics of the applications. The key results from the MR study include the following: (1) the objects used in the MR application were correctly positioned on a map, which implies that the memory acquired with the MR application is effectively transferred to the user's mental map; (2) for the performance variables, there were no significant differences in the results by gender; (3) and the usability rating decreased with computer experience. The results show that the MR application is effective for spatial memory assessment and was well rated by the participants. The three technologies, along with suitable hardware, are effective for spatial memory assessment. However, MR using optical see-through headsets offers advantages over mobile AR and VR using headsets, discussed in this publication.

With Generation Z becoming the primary group of nursing learners and the increasing shortage of nursing education resources, augmented reality and mixed reality based on head-mounted displays are being used more and more in nursing education. However, the current application landscape and the proper usage of these concepts remain unclear. Therefore, this study aims to conduct a scoping review to explore the current applications of head-mounted display-based augmented reality and mixed reality in nursing education and to clarify the definitions and usage of the concepts of augmented reality and mixed reality, ultimately providing directions for future applications and research. Based on the five-stage framework and PRISMA-ScR guidelines, a comprehensive collection and summarization of evidence regarding the application of head-mounted display-based augmented reality and mixed reality in nursing education were conducted. The databases retrieved include CNKI, Wanfang Database, VIP, CBM, PubMed, Cochrane Library, Embase, CINAHL, Web of Science, Scopus, IEEE Xplore, ACM Digital Library, and Ei. The languages of the included literature are Chinese and English. The retrieval was up to October 2024. A total of 44 studies were included in this review, covering three types of head-mounted displays: immersive head-mounted displays, smart glasses, and smartphone-based head-mounted displays. The main application areas were skills training and knowledge acquisition. Most of the studies were feasibility studies, though they also included some efficacy studies and research on personal experiences. In addition, these studies often employed vague or inconsistent definitions of augmented reality and mixed reality. Despite various explorations in the application of head-mounted display-based augmented reality and mixed reality in nursing education, there is still room for improvement in the relevant theory and utilization of this technology. In the future, interventions should use the three dimensions (observation of reality, real - virtual interaction, and fidelity of virtuality) described in detail, rather than simply employing the concepts of augmented reality or mixed reality. Efforts should be concentrated on developing and implementing head-mounted displays combined with other technologies that boast enhanced performance and cost - effectiveness, and further validating their effectiveness.

With tremendous research, innovation in human computer interaction, infographics and greater accessibility of visualization technologies i.e., augmented reality; virtual reality and mixed reality is to find new possibilities for its applications. The ability to transform the user's perception and interaction with his environment has increased his popularity of use in all spheres of life. This study analyses the literature on augmented reality, virtual reality and mixed reality applications in order to find the research gaps in these fields. This review study has taken 103 papers of 2010-2021 into account from SCOPUS, SCI and IEEE databases. The results of the analysis can provide some useful insights regarding potential and use of augmented reality, virtual reality and mixed reality in the different areas of application.

Underground spaces are rapidly expanding, with their functions increasingly extending to commercial, entertainment, and residential areas. However, the inherent characteristics of closure and complexity in these environments significantly impact user experience and safety. An experimental system was developed to explore human perception in underground spaces. The system consisted of three main components: a mixed reality (MR) device, a set of wearable biosensors, and a data platform. Typical elements of the underground environment were built by digital tools and overlaid onto real-world physical space using the MR device. Scenarios, such as fire hazards or earthquakes, were simulated based on physical mechanism and introduced into the MR environment. During these simulations, human responses were monitored via wearable biosensors. All data were transmitted to and stored on the data platform, where analysis revealed correlations between human responses and environmental factors, providing insights for the optimization of underground spaces. A fire drill scenario was studied as the application of this system. Smoke is generated by mixed-reality device according to the data derived from Fire Dynamics Simulator(FDS), after which the participant was asked to find the exit under low visibility. Meanwhile, the response of participants, including electroencephalogram(EEG), electrodermal activity(EDA) and photoplethysmographic(PPG), was collected for further analysis. It was indicated that the system could effectively build a vivid experimental scenario with less modeling work and higher movement freedom than virtual reality(VR) and provided a more immersive experience than augmented reality(AR). The human physiological signal can be recorded and utilized as a supplement for subjective response in the future.

As technology advances, mobile devices have gradually turned into wearable devices. Furthermore, virtual reality (VR), augmented reality (AR), and mixed reality (MR) are being increasingly applied in medical fields such as medical education and training, surgical simulation, neurological rehabilitation, psychotherapy, and telemedicine. Research results demonstrate the ability of VR, AR, and MR to ameliorate the inconveniences that are often associated with traditional medical care, reduce incidents of medical malpractice caused by unskilled operations, and reduce the cost of medical education and training. What is more, the application of these technologies has enhanced the effectiveness of medical education and training, raised the level of diagnosis and treatment, improved the doctor-patient relationship, and boosted the efficiency of medical execution. The present study introduces VR, AR, and MR applications in medical practice and education with the aim of helping health professionals better understand the applications and use these technologies to improve the quality of medical care.

Augmented Reality (AR) and Mixed Reality (MR) are new emerging technologies that are becoming increasingly popular. Because these technologies are so new, there is a lack of standards and consistency in application and hardware design. This can make it difficult to learn and frustrating for users. One way to standardized design and enhance the usability of a product is through the use of heuristic evaluations. General heuristics, such as Nielsen’s 10 or Schneiderman’s 8 usability heuristics have been used to evaluate these technologies. These heuristics are a useful starting point because they bring attention to many crucial aspects of the usability of a product. However, additional aspects that could alter the users’ experience may not be assessed due to the uniqueness of AR and MR. There are very few validated AR and MR heuristics in the literature for practitioners to use. The purpose of this study was to create and validate a heuristic checklist that can be used to assess and inform design changes that influence the user experience of an AR or MR application and/or device. We followed an established and comprehensive 8-stage methodology developed by Quiñones, Rusu, & Rusu to create and validate our AR and MR usability heuristic checklist [4]. This included a search and summary of the current literature, formally defining heuristics based on this literature search, and validating through heuristic evaluations, expert reviews, and user testing. Our final revised heuristic checklist included 11 heuristics and 94 checklist items that encompasses usability aspects for AR and MR technologies. KeywordsHuman factorsUser experienceAugmented RealityMixed RealityUsabilityDesign guidelines

This article aims to assist readers in understanding the current status of studies on the subject by providing a descriptive bibliometric analysis of publications on virtual reality (VR), augmented reality (AR), and mixed reality (MR) technologies in cultural heritage. A bibliometric analysis of 1214 publications in this discipline in the Scopus database between 2014 and the beginning of June 2024 was performed. We used VOSviewer and Bibliometrix as the analysis tools in this investigation. The outcome of this study provides a detailed overview of the descriptive bibliometric analysis based on seven categories, including the annual count of articles and citations, the most productive author, the primary affiliation, the publication source, and the subject areas. The contribution of this research lies in offering valuable insights for practitioners and researchers, helping them make informed decisions on the use of immersive technologies, for example, VR, AR, and MR, in the context of cultural heritage.

BackgroundExtended reality, which encompasses virtual reality (VR), augmented reality (AR), and mixed reality (MR), is increasingly used in medical education. Studies assessing the effectiveness of these new educational modalities should measure relevant outcomes using outcome measurement tools with validity evidence.ObjectiveOur aim is to determine the choice of outcomes, measurement instruments, and the use of measurement instruments with validity evidence in randomized controlled trials (RCTs) on the effectiveness of VR, AR, and MR in medical student education.MethodsWe conducted a systematic mapping review. We searched 7 major bibliographic databases from January 1990 to April 2020, and 2 reviewers screened the citations and extracted data independently from the included studies. We report our findings in line with the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines.ResultsOf the 126 retrieved RCTs, 115 (91.3%) were on VR and 11 (8.7%) were on AR. No RCT on MR in medical student education was found. Of the 115 studies on VR, 64 (55.6%) were on VR simulators, 30 (26.1%) on screen-based VR, 9 (7.8%) on VR patient simulations, and 12 (10.4%) on VR serious games. Most studies reported only a single outcome and immediate postintervention assessment data. Skills outcome was the most common outcome reported in studies on VR simulators (97%), VR patient simulations (100%), and AR (73%). Knowledge was the most common outcome reported in studies on screen-based VR (80%) and VR serious games (58%). Less common outcomes included participants’ attitudes, satisfaction, cognitive or mental load, learning efficacy, engagement or self-efficacy beliefs, emotional state, competency developed, and patient outcomes. At least one form of validity evidence was found in approximately half of the studies on VR simulators (55%), VR patient simulations (56%), VR serious games (58%), and AR (55%) and in a quarter of the studies on screen-based VR (27%). Most studies used assessment methods that were implemented in a nondigital format, such as paper-based written exercises or in-person assessments where examiners observed performance (72%).ConclusionsRCTs on VR and AR in medical education report a restricted range of outcomes, mostly skills and knowledge. The studies largely report immediate postintervention outcome data and use assessment methods that are in a nondigital format. Future RCTs should include a broader set of outcomes, report on the validity evidence of the measurement instruments used, and explore the use of assessments that are implemented digitally.

Questionnaires are used for evaluating virtual, augmented, and mixed reality (MR) in healthcare. Utilizing the perfect one to meet the needs is a challenge for researchers and examiners. Hence, this study aimed to review and determine the frequently used questionnaires for evaluating mentioned services. The PubMed and Web of science databases were searched for conducting this review. Original observational and interventional studies that used a referenced questionnaire to evaluate virtual reality (VR), augmented reality (AR), and MR in healthcare were included. Data were analyzed using descriptive statistics. Totally, 179 studies were included in this research. The questionnaires are mostly used for evaluating usability (39%) and cybersickness (37.5%). System Usability Scale (28%), Simulator Sickness Questionnaire (27.5%), and Presence Questionnaire (9.5%) were the most used questionnaires. The use of special, need-based, and targeted questionnaires to evaluate the various outcomes of VR, AR, and MR can help in better evaluation of these services.

Traditional Mixed Reality(MR) and Augmented Reality(AR) devices support a wide gamut of sensors, but the limited computational resource onboard such devices make advanced tasks difficult. With the introduction of Edge computing and more advanced Edge hardware, we are no longer bound to just the onboard processor or the Cloud as our only source of computational power. In our work, we introduce the use of Edge with MR devices to provide a cooperative perception capability to the MR device. We base our approach on the portability and low latency of the Edge. Through our prototype system, we demonstrate the potential of devices and evaluate the performance and feasibility through real world trials. Our evaluation proves that the system is capable of supporting cooperative perception tasks.

The priority of modern dentistry is to keep patients’ teeth for as long as possible. Tooth extraction is a procedure performed as a last resort when conservative methods and endodontic surgery procedures have not brought the expected results. As a consequence, the number of patients in dental offices, who require first and repeated endodontic treatment, is increasing. Thus, the development of new technologies in the medical industry, including microscopy, computer tomography (CT), as well as diode and neodymium-YAG-erbium lasers, enables dentists to increase the percentage of successful treatments. Moreover, mixed reality (MR) is a very new technology, in which the 3D view can help plan or simulate various types of tasks before they will be carried out in real life. In dentistry, 3D holography can be applied to display CT data to plan endodontic treatment. The most important element in effective root canal treatment is the precise imaging of the root canal. The CT scans allow dentists to view the anatomy of the patient’s tooth with much higher precision and understanding than using 2D radiography (RTG-radiographic photo) pictures. Recently, the development of new 3D technologies allows dentists to obtain even more data from existing CT scans. In this paper, the CT scan data were applied to generate patient teeth in 3D and simulate the view of the root canal’s anatomy in MR devices, i.e., Microsoft HoloLens 2. Using DICOM RAW data from the CT exam, we generated a 3D model of the jaw with a tooth. In the next step, the crown of the tooth was removed in a similar way to how a dentist would do this using a dental handpiece. Furthermore, all root canals were cleaned of everything inside. This way we achieved empty tunnels, namely root canals. Finally, we added appropriate lighting, similar to the type of lighting that dentists use. The proposed approach enables to display of the root canals in the same way as during the endodontic procedure using a microscope. It allows for the visualization of the root canal and changing its direction, in which dimensional accuracy is crucial. It turns out that mixed reality can be considered a complementary method to the traditional approach, which reduces the amount of time for the root canal treatment procedure by up to 72.25%, depending on the complexity of the case, and increases its effectiveness. Thus, the mixed reality-based system can be considered an effective tool for planning dental treatment.

Three-dimensional (3D) imaging and image reconstruction play a prominent role in the diagnosis, treatment planning, and post-therapeutic monitoring of patients with congenital heart disease. More interactive and realistic medical experiences take advantage of advanced visualization techniques like augmented, mixed, and virtual reality. Further, 3D printing is now used in medicine. All these technologies improve the understanding of the complex morphologies of congenital heart disease. In this review article, we describe the technical advantages and disadvantages of various advanced visualization techniques and their medical applications in the field of congenital heart disease. In addition, unresolved issues and future perspectives of these evolving techniques are described.

This paper shows our work in progress towards an easy to use development environment for Mixed Reality (MR) Applications. We argue that development of MR applications is a collaboration between interaction designers who know about user requirements, and expert developers who know the intricacies of MR development. This collaboration should be supported by tools that aid both roles and ease their communication. We also argue that real MR development should allow easy migration from one hardware setup to another, since hardware greatly varies in these type of applications, and it is important to fit a solution to the particular user's requirements and context.We show the foundational concepts in our work and current Integrated Development Environment (IDE) implementation. This work is based on InTml, a domain specific language for MR applications, and Eclipse, an open source, general purpose IDE.