Cross-Device Platform for Collaborative and Immersive Experiences in Mixed Reality

Extended-Reality Technologies: An Overview of Emerging Applications in Medical Education and Clinical Care.

The perceptual behavior of consumers on a product displayed in the market has a vital role in analyzing the importance given to that product. Therefore, various strategies have been developed to understand this consumer behavior in the selection of products. Immersive technologies like virtual, augmented, and mixed reality are among them. With the foremost feature of immersion in the virtual world and interaction of users with virtual objects, virtual reality, and augmented reality have unlocked their potential in research and a user-friendly tool for analyzing consumer behavior. In addition to these technologies, mixed reality also has a significant role in investigating consumer behavior. Studies on immersive technologies in food applications are vast, hence this review focuses on the applications of virtual, augmented, and mixed reality in the food selection behavior of consumers. The behavioral studies are elicited to develop new products based on consumer needs, to understand the shopping behavior in supermarkets for real-time usage, and to know the influence of emotions in a selection of products. The findings suggest that virtual, augmented, and mixed reality induce immersion of the users in food selection behavioral studies. Information on the technological advancements in the tools used for bringing immersion and interaction are discussed for its futuristic applications in food. Though immersive technology gives users a realistic virtual environment experience, its application in food systems is in the budding stage. More research on human response studies would contribute to its innovative and inevitable application in the future.

Underground spaces are rapidly expanding, with their functions increasingly extending to commercial, entertainment, and residential areas. However, the inherent characteristics of closure and complexity in these environments significantly impact user experience and safety. An experimental system was developed to explore human perception in underground spaces. The system consisted of three main components: a mixed reality (MR) device, a set of wearable biosensors, and a data platform. Typical elements of the underground environment were built by digital tools and overlaid onto real-world physical space using the MR device. Scenarios, such as fire hazards or earthquakes, were simulated based on physical mechanism and introduced into the MR environment. During these simulations, human responses were monitored via wearable biosensors. All data were transmitted to and stored on the data platform, where analysis revealed correlations between human responses and environmental factors, providing insights for the optimization of underground spaces. A fire drill scenario was studied as the application of this system. Smoke is generated by mixed-reality device according to the data derived from Fire Dynamics Simulator(FDS), after which the participant was asked to find the exit under low visibility. Meanwhile, the response of participants, including electroencephalogram(EEG), electrodermal activity(EDA) and photoplethysmographic(PPG), was collected for further analysis. It was indicated that the system could effectively build a vivid experimental scenario with less modeling work and higher movement freedom than virtual reality(VR) and provided a more immersive experience than augmented reality(AR). The human physiological signal can be recorded and utilized as a supplement for subjective response in the future.

Lower body tracking and user experience analysis are ongoing challenges in immersive technologies and biomechanics research, respectively. Particularly, in immersive technologies like mixed reality, there are limited research applications in user comfort and experience, and locomotion. In this paper, we aimed to conduct a cross-disciplinary research (by combining the elements of Biomechanics and human–computer interaction) in understanding the impact of user experience and immersive technologies induced symptoms in virtual and mixed reality environments for obstacle clearance tasks. For this, we created obstacle clearance applications for Oculus Quest and Microsoft HoloLens. Also, we conducted a user study with 12 participants under two conditions (virtual reality and mixed reality) in obstacle clearance behavior of varying heights. The results from our study indicated that the levels of immersion and enjoyment were significantly higher in virtual reality compared to mixed reality. In terms of virtual reality-induced symptoms, the participants found the mixed reality environment to be slightly more comfortable than virtual reality.

BackgroundExtended reality, which encompasses virtual reality (VR), augmented reality (AR), and mixed reality (MR), is increasingly used in medical education. Studies assessing the effectiveness of these new educational modalities should measure relevant outcomes using outcome measurement tools with validity evidence.ObjectiveOur aim is to determine the choice of outcomes, measurement instruments, and the use of measurement instruments with validity evidence in randomized controlled trials (RCTs) on the effectiveness of VR, AR, and MR in medical student education.MethodsWe conducted a systematic mapping review. We searched 7 major bibliographic databases from January 1990 to April 2020, and 2 reviewers screened the citations and extracted data independently from the included studies. We report our findings in line with the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines.ResultsOf the 126 retrieved RCTs, 115 (91.3%) were on VR and 11 (8.7%) were on AR. No RCT on MR in medical student education was found. Of the 115 studies on VR, 64 (55.6%) were on VR simulators, 30 (26.1%) on screen-based VR, 9 (7.8%) on VR patient simulations, and 12 (10.4%) on VR serious games. Most studies reported only a single outcome and immediate postintervention assessment data. Skills outcome was the most common outcome reported in studies on VR simulators (97%), VR patient simulations (100%), and AR (73%). Knowledge was the most common outcome reported in studies on screen-based VR (80%) and VR serious games (58%). Less common outcomes included participants’ attitudes, satisfaction, cognitive or mental load, learning efficacy, engagement or self-efficacy beliefs, emotional state, competency developed, and patient outcomes. At least one form of validity evidence was found in approximately half of the studies on VR simulators (55%), VR patient simulations (56%), VR serious games (58%), and AR (55%) and in a quarter of the studies on screen-based VR (27%). Most studies used assessment methods that were implemented in a nondigital format, such as paper-based written exercises or in-person assessments where examiners observed performance (72%).ConclusionsRCTs on VR and AR in medical education report a restricted range of outcomes, mostly skills and knowledge. The studies largely report immediate postintervention outcome data and use assessment methods that are in a nondigital format. Future RCTs should include a broader set of outcomes, report on the validity evidence of the measurement instruments used, and explore the use of assessments that are implemented digitally.

VR/MR systems integrated with heat transfer simulation for training of thermoforming: A multicriteria decision-making user study

Immersive technologies offer an alternative means of presenting and communicating data derived from BIM modeling. Augmented, virtual, and mixed reality technologies allow architects to visualize and interact with digital data, providing a simulation of their physical presence in an architectural structure, and thus a more realistic and interactive approach to various stages of project activities. In fact, immersive technologies can be used throughout the entire life cycle of an architectural project: planning, design, construction, and operation. The aim of research is to identify and systematize data on computer tools using immersive technologies and the prospects for its development in architectural activities. The scientific paper describes three groups of software products based on immersive technologies with BIM modeling support. Each of the groups is based on a particular immersive technology: augmented reality (AR), virtual reality (VR), or mixed reality (MR). The main difference between these technologies is the level of immersion in the virtual environment: AR overlays computer-generated content on the real world; VR content is 100% digital; MR is a digital overlay that allows virtual 3D elements to interact with the real environment. Each group contains different software products: the first group is mostly characterized by mobile applications, since the physical world is supplemented with digital content through the screens of smartphones, tablets, and other compact equipment; the second group includes various plug-ins that integrate into professional tools and improve them in terms of visualization in a virtual environment and the possibility of its research; The third group with mixed reality support is smaller in number of programs, but it is the most promising for the future development of the architectural profession, as it provides an integrated approach to architectural activity by expanding the professional tools.

The article considers the possibility of using virtual and mixed reality in studying the geometry of the crystal lattice in the framework of the course of solid state physics. The application of mixed and virtual reality technologies in the study of various branches of physics is receiving more and more attention. Virtual reality headsets were used in practical and laboratory classes, where students had to create crystal structures and carry out modeling of various processes. When studying complex spatial objects of crystallography, virtual reality technology allowed students to familiarize themselves with the object of study in more detail, understand the structure of crystals and, accordingly, complete the task in a timely manner. At the same time, the prolonged use of virtual reality headsets led to rapid fatigue of students.

Service prototyping is an innovative iterative process envisioned to enhance the service development process while refining the anticipated service experience. Immersive technologies, such as: Virtual Reality (VR), Augmented Reality (AR), and Mixed Reality (MR), in service prototyping have the potential to enhancing the co-creation of service ideas. It is intended to transform intangible service aspects into an experience, even before the service exists. However, there is a lack of research studies comparing different forms of service prototype. Such studies would help to find out the most appropriate Service Prototype (SP) form for exploring, communicating and evaluating new service ideas. Several SP experiment sessions were conducted in France and Germany within an academic context in 2018 to compare different performance factors of conventional Service Prototypes (CSP) versus Immersive Service Prototypes (ISP). The participants have to disassemble and then reassemble a simple three-part mechanical element with the aid of four different SP forms. This paper presents the results of the experiment sessions, involving 38 participants, conducted at Furtwangen University campus in Germany. These results reveal that participants preferred ISP forms rather than CSP forms. However, it also confirms that there are still some difficulties in applying and using VR or AR devices.

Questionnaires are used for evaluating virtual, augmented, and mixed reality (MR) in healthcare. Utilizing the perfect one to meet the needs is a challenge for researchers and examiners. Hence, this study aimed to review and determine the frequently used questionnaires for evaluating mentioned services. The PubMed and Web of science databases were searched for conducting this review. Original observational and interventional studies that used a referenced questionnaire to evaluate virtual reality (VR), augmented reality (AR), and MR in healthcare were included. Data were analyzed using descriptive statistics. Totally, 179 studies were included in this research. The questionnaires are mostly used for evaluating usability (39%) and cybersickness (37.5%). System Usability Scale (28%), Simulator Sickness Questionnaire (27.5%), and Presence Questionnaire (9.5%) were the most used questionnaires. The use of special, need-based, and targeted questionnaires to evaluate the various outcomes of VR, AR, and MR can help in better evaluation of these services.

Virtual and mixed reality technology has advanced tremendously over the past several years. This nascent medium has the potential to transform how people communicate over distance, train for unfamiliar tasks, operate in challenging environments, and how they visualize, interact, and make decisions based on complex data. At the same time, the marketplace has experienced a proliferation of network-connected devices and generalized sensors that are becoming increasingly accessible and ubiquitous. As the Internet of Things expands to encompass a predicted 50 billion connected devices by 2020, the volume and complexity of information generated in pervasive and virtualized environments will continue to grow exponentially. The convergence of these trends demands a theoretically grounded research agenda that can address emerging challenges for human-information interaction (HII). Virtual and mixed reality environments can provide controlled settings where HII phenomena can be observed and measured, new theories developed, and novel algorithms and interaction techniques evaluated. In this paper, we describe the intersection of pervasive computing with virtual and mixed reality, identify current research gaps and opportunities to advance the fundamental understanding of HII, and discuss implications for the design and development of cyber-human systems for both military and civilian use.

Three-dimensional (3D) imaging and image reconstruction play a prominent role in the diagnosis, treatment planning, and post-therapeutic monitoring of patients with congenital heart disease. More interactive and realistic medical experiences take advantage of advanced visualization techniques like augmented, mixed, and virtual reality. Further, 3D printing is now used in medicine. All these technologies improve the understanding of the complex morphologies of congenital heart disease. In this review article, we describe the technical advantages and disadvantages of various advanced visualization techniques and their medical applications in the field of congenital heart disease. In addition, unresolved issues and future perspectives of these evolving techniques are described.

Do VR and AR versions of an immersive cultural experience engender different user experiences?

Collaboration and guidance are key aspects of many software tasks. In traditional desktop software, such aspects are well supported through built-in collaboration functions or general-purpose techniques such as screen and video sharing. In Mixed Reality environments, where users carry out actions in a three-dimensional space, collaboration and guidance may also be required. However, other users may or may not be using the same Mixed Reality interface. Users may not have access to the same information, the same visual representation, or the same interaction affordances. These asymmetries make communication and collaboration between users harder. To address asymmetries in Mixed Reality environments, we introduce Interactive Mixed-Dimensional Media. In these media, the visual representation of information streams can be changed between 2D and 3D. Different representations can be chosen automatically, based on context, or through associated interaction techniques that give users control over exploring spatial, temporal, and dimensional levels of detail. This ensures that any information or interaction makes sense across different dimensions, interfaces and spaces. We have deployed these techniques in three different contexts: mixed-reality telepresence for physical task instruction, video-based instruction for VR tasks, and live interaction between a VR user and a non-VR user. From these works, we show that Mixed Reality environments that provide users with interactive mixed-dimensional media interfaces improve performance and user experience in collaboration and guidance tasks.

This abstract is a companion piece to the ninth iteration of the “Me and My VE” interactive session. The purpose of this alternative format session is to allow human factors researchers to demonstrate how they use virtual reality, mixed reality, and virtual environments to support end-user research. The session begins with each presenter providing a brief introduction of their virtual reality, mixed reality, or virtual environment application. After this introduction, attendees engage with the presenters and their virtual environments in an interactive demonstration period. This year’s demonstrations include applications ranging from designing virtual environments for accessibility to using virtual and extended reality to support medical and aviation training, content validation, and human-computer interaction studies. The goal of the session is to allow attendees to experience how human factors professionals use virtual environments to support human factors-oriented research and to learn about the exciting work being conducted in this field.