Mixed reality merchandising: bricks, clicks – and mix

This paper shows our work in progress towards an easy to use development environment for Mixed Reality (MR) Applications. We argue that development of MR applications is a collaboration between interaction designers who know about user requirements, and expert developers who know the intricacies of MR development. This collaboration should be supported by tools that aid both roles and ease their communication. We also argue that real MR development should allow easy migration from one hardware setup to another, since hardware greatly varies in these type of applications, and it is important to fit a solution to the particular user's requirements and context.We show the foundational concepts in our work and current Integrated Development Environment (IDE) implementation. This work is based on InTml, a domain specific language for MR applications, and Eclipse, an open source, general purpose IDE.

Mixed reality technology has provided more possibilities for users to interact with both the virtual and real worlds. By integrating virtual elements with real scenes, mixed reality offers a novel immersive experience. Mixed reality also introduces complex perceptual and cognitive challenges when processing conflicting information from the virtual and real worlds. This study investigates the impact of color mismatches on response time in a mixed reality scenario. We recruited 15 participants to identify the font color of words while ignoring their meaning. The experiment included three levels of perceptual-conceptual matching. We performed a global comparison across all conditions, a within-group comparison, and a between-group comparison. Results show that in the mixed reality scenario, the processing of visual information relies on low-level physical properties (e.g., brightness and hue) and is modulated by high-level cognitive factors (e.g., conflict processing and expectation). These findings indicate that visual mismatches in mixed reality can significantly affect response time, particularly when perceptual salience is low and different from cognitive expectations. The results have implications for the design of mixed reality interfaces in complex environments where users must process information from the virtual and real worlds, such as head-up displays, warehouse management systems, and industrial instructional overlays.

With Generation Z becoming the primary group of nursing learners and the increasing shortage of nursing education resources, augmented reality and mixed reality based on head-mounted displays are being used more and more in nursing education. However, the current application landscape and the proper usage of these concepts remain unclear. Therefore, this study aims to conduct a scoping review to explore the current applications of head-mounted display-based augmented reality and mixed reality in nursing education and to clarify the definitions and usage of the concepts of augmented reality and mixed reality, ultimately providing directions for future applications and research. Based on the five-stage framework and PRISMA-ScR guidelines, a comprehensive collection and summarization of evidence regarding the application of head-mounted display-based augmented reality and mixed reality in nursing education were conducted. The databases retrieved include CNKI, Wanfang Database, VIP, CBM, PubMed, Cochrane Library, Embase, CINAHL, Web of Science, Scopus, IEEE Xplore, ACM Digital Library, and Ei. The languages of the included literature are Chinese and English. The retrieval was up to October 2024. A total of 44 studies were included in this review, covering three types of head-mounted displays: immersive head-mounted displays, smart glasses, and smartphone-based head-mounted displays. The main application areas were skills training and knowledge acquisition. Most of the studies were feasibility studies, though they also included some efficacy studies and research on personal experiences. In addition, these studies often employed vague or inconsistent definitions of augmented reality and mixed reality. Despite various explorations in the application of head-mounted display-based augmented reality and mixed reality in nursing education, there is still room for improvement in the relevant theory and utilization of this technology. In the future, interventions should use the three dimensions (observation of reality, real - virtual interaction, and fidelity of virtuality) described in detail, rather than simply employing the concepts of augmented reality or mixed reality. Efforts should be concentrated on developing and implementing head-mounted displays combined with other technologies that boast enhanced performance and cost - effectiveness, and further validating their effectiveness.

PurposeThe purpose of this study is to explore the effectiveness of mixed reality (MR) technology in enhancing library tours for first-year students at a university academic library. It specifically aims to investigate whether MR tours can reduce library anxiety, improve knowledge acquisition and increase motivation when compared to traditional library tours.Design/methodology/approachThis study uses a quasi-experimental research design, comparing two groups of first-year students (n = 96). One group (n = 48) experiences a MR library tour using the HoloLens 2 device, while the other group (n = 48) participates in a traditional library tour led by library staff. The participants’ library anxiety, knowledge acquisition and motivation are measured using relevant questionnaires before and after the tours.FindingsThe findings indicate that while both MR and traditional tours are effective in imparting knowledge, there is no statistically significant difference in overall knowledge acquisition. However, the MR tour significantly enhances students’ perceived competence, interest and effort, providing higher engagement and motivation. Traditional tours, on the other hand, are more effective in reducing library anxiety, particularly in relation to interactions with librarians.Originality/valueThis study highlights the potential of MR technology to enhance library tours by balancing immersive experiences with ease of use. MR bridges the gap between virtual reality’s immersion and the accessibility, lower cost of content development and simplicity of application usage. It aligns with trends in academic libraries by offering high engagement without common health issues like motion sickness. However, while MR applications can be developed at a relatively lower cost, the high cost of MR equipment remains a limitation for institutions. Despite this, MR presents a promising solution for improving student engagement and learning, with the potential to become more accessible as hardware costs decrease.

Historic preservationists are increasingly focusing their attention on the use of digital technologies to advance the field. These tools equip experts with the ability to capture, process, produce, and reproduce digital data of cultural heritage for documentation, interpretation, restoration and conservation purposes. In recent years, researchers have experimented with Mixed Reality (MR) applications to preserve cultural heritage in a non-contact and reversible manner. By allowing experts to intervene without altering extant physical objects, MR opens up countless possibilities for preservation strategies once the physical range of actions have been exhausted. However, more importantly, by mixing virtual and physical worlds, MR offers new possibilities for preserving historical transformation itself. In this paper, I will argue that MR provides the best way of preserving the authenticity of architectural objects by capturing the historical transformations that they endure.

Fractions are one of the most complex and challenging notions for children and can often lead to frustration and a revulsion for mathematics in general. In this article, we present the Magic Cauldron, a Mixed Reality (MR) application, designed to help children apprehend fractions in a fun and interactive way. The proposed solution is a digital extension to a board game, called the Potion Workshop that is used to introduce fractions in more than 2000 schools in France. We put together a team, composed of the mathematics didacticians who designed the Potion Workshop, several teachers who use this game in their class, a multimedia designer and computer scientists, in order to create a MR game that would tackle several of the key notions that are still hard to grasp. In this article, we present the Design-Based method followed by this team. It offers insights on how to implicate non-computer scientists in the design of complex custom MR interactions. Through several cycles of collaborative design, involving three teachers and their students and the development of three prototypes, this method allowed us to produce a truly original MR application.

Mixed reality refers to the integration of virtual reality into the real-world environment. This digital content can be interacted with in real time. The emergence of mixed reality technology has been made possible by the introduction of head-mounted displays, which are being utilized across multiple surgical specialties. In upper extremity surgery, mixed reality has widespread applications in trauma, corrective surgery, arthroplasty, arthroscopy, and oncology. Preoperatively, mixed reality allows for complex 3D planning. Intraoperatively, surgeons can access this 3D data in a sterile environment. While in its infant stages, mixed reality is likely to become a powerful tool for intraoperative guidance and navigation. Mixed reality can change the paradigm of communication, as it allows the sharing of visual data from the surgeon's perspective, enabling remote assistance and participation.

Current mixed reality (MR) designs predominantly prioritise functionality and usability, often overlooking individual's diverse value needs. To create more meaningful MR experiences, this paper aims to address this gap by exploring how human values can be integrated into MR design. We propose a values-based design process and evaluate it through three design workshops considering a remote collaborative learning scenario. By comparing our approach to an existing MR application for collaborative learning, we demonstrate how embedding human values into MR design can lead to more ethical and human-centred outcomes. Our findings contribute to advancing the design and application of MR technologies to be more aligned with people's diverse needs and values.

This study aims to compare the effectiveness of mixed reality, pen and paper, and digital note-taking in the context of educational excursions. A mixed reality headset was used to take notes in mixed reality, mobile phones for taking notes digitally, as well as pen and paper paired with a clipboard for taking notes in longhand. The study was motivated by the potential affordances that mixed reality brings to note-taking, such as integrating the real world alongside digital note-taking platforms in an immersive manner. However, there are also downsides, such as the bulkiness of the headset. While existing research has covered virtual reality and augmented reality within classroom settings, there remains a gap in current research in understanding the application of mixed reality in contexts outside of the classroom, such as in field-based learning. To address this, this study aims to conduct preliminary research on how note-taking in mixed reality compares with two other traditional methods in a new context to understand the potential of mixed reality and feasibility in note-taking and learning. Educational excursions were chosen as it was surmised that the affordances of mixed reality headsets are suited to contexts of learning during which the note-taking platform needs to be mobile. The modalities of information students could encounter in a field trip were recognised as physical, textual, and video. For physical information in field trips, our data shows that mixed reality note-taking was more effective in both extracting greater amounts of content in note form, as well as improving knowledge retention. For textual information, mixed reality note-taking was also more effective in extracting greater amounts of content in note form, but pen and paper note-taking was the more efficacious method in the organisation of notes. Mixed reality has the potential to be integrated into field trips, but should not entirely replace traditional methods of note-taking, especially as feedback from participants show that the headsets can cause discomfort. As the comfort of wearables improves, more research can be carried out to continue this work on the potential of mixed reality in collaborative learning.

Now, the Industry 4.0 has changed the way works the all process in industry and engineering. These changes affect in dramatic way in the training and learning worker's process. Either because the workers need to learn new technologies or skill, or because appear new technology that change the learning process in the very important way. The mixed reality applications like a tools int the worker's training process stay at risk because the actual tools to create mixed reality experiences are very complex and expensive. DesignMe-MR is a toolbox that aim to simplify the creation process of the mixed reality learning experiences to achieve decrease the cost to make the widespread implementation of mixed reality as a tool for operator training in any industry or engineering. DesignMe-MR use the mixed reality to help the mixed reality learning experiences creation process, making it a very useful tool for creators. The tests show a high level of user satisfaction in the application of mixed reality both in the content creation process and in the execution of the resulting training

A Mixed Reality (MR) application using an optical see-through headset was developed to assess short-term spatial memory. A study with 29 participants was conducted. Data from this study were compared to two previous studies using mobile Augmented Reality (AR) and Virtual Reality (VR) with headsets. When comparing the three technologies (MR, AR, VR) for the performance variables, there were no statistically significant differences for either the total number of correctly placed objects or the total number of attempts. However, the MR application required more time than the AR and VR applications in the evaluation phase and more time than the VR application in the learning phase. Our arguments for the longer time are the novelty of the MR application for the participants and the characteristics of the applications. The key results from the MR study include the following: (1) the objects used in the MR application were correctly positioned on a map, which implies that the memory acquired with the MR application is effectively transferred to the user's mental map; (2) for the performance variables, there were no significant differences in the results by gender; (3) and the usability rating decreased with computer experience. The results show that the MR application is effective for spatial memory assessment and was well rated by the participants. The three technologies, along with suitable hardware, are effective for spatial memory assessment. However, MR using optical see-through headsets offers advantages over mobile AR and VR using headsets, discussed in this publication.

ARTIST is a research approach introducing novel methods for real-time multi-entity interaction between human and non-human entities, to create reusable and optimized Mixed Reality (MR) experiences with low-effort, towards a Shared MR Experiences Ecosystem (SMRE2). As a result, ARTIST delivers high quality MR experiences, facilitating the interaction between a variety of entities which interact in a virtual and symbiotic way within a mega, virtual and fully-experiential world. Specifically, ARTIST aims to develop novel methods for low-effort (code-free) implementation and deployment of open and reusable MR content, applications and tools, introducing the novel concept of an Experience as a Trajectory (EaaT). In addition, ARTIST will provide tools for the tracking, monitoring and analysis of user behaviour and their interaction with the environment and with other users, towards optimizing MR experiences by recommending their reconfiguration, dynamically (at run-time) or statically (at development time). Finally, it will provide tools for synthesizing experiences into new mega and still reconfigurable EaaTs, enhancing them at the same time using semantically integrated related data/information available in disparate and heterogeneous resources.

This study aims to explore the application of mixed reality in experimental safety teaching, designing and implementing a mixed reality-based experimental teaching method to enhance the effectiveness of experiments and user experience. Through field visits to design laboratories and expert interviews, this research analyzes the existing problems in design experimental teaching and real user needs. It proposes a multi-level, multi-dimensional mixed reality innovation design experiment platform construction method from teaching to dialogue, from human-computer interaction to perception, from creation to manufacturing, and from communication to thinking. Finally, this study uses the intelligent cockpit interface experience perception mixed reality experimental platform as an example to confirm that mixed reality technology can provide an immersive learning experience. This research provides theoretical support and practical reference for the application of mixed reality in experimental safety teaching, and offers prospects for future teaching design and research.

Figure 1:DRoom is a gamified demonstration testbed for our Real Haptics technology, a novel method for interacting in Mixed Reality. On the left side we see the physical setup, on the right side the virtual experienceTypical applications in mixed reality use a simulated mechanism for manipulating objects in the virtual environment, through VR controllers held in user’s hands, without real touch. As an alternative, within our Distributed Reality concept [3] we propose Real Haptics, a novel interaction method based on two key ideas: providing real user embodiment in the virtual spaces by segmenting user real hands and allowing the real manipulation of physical objects, which in the virtual scene may preserve or change their real appearance. To evaluate the concept, we deployed a functional prototype with Unity, using color-based algorithms for object segmentation [2] and Aruco library for object tracking [1]. Based on this platform we built a gamified testbed: a five-minute escape room game, which 53 users played twice, once with Real Haptics and once with purely virtual objects and avatars using VR controllers. After each run, users answered a survey derived from standard questionnaires measuring presence, embodiment and quality of experience. This information plus some quantitative data related to performance, validated our hypothesis that Real Haptics significantly improves presence or embodiment with respect to the counterpart virtual solution. We foresee our technology to provide a significant advance in the control of virtual environments, with a first potential focus on training applications.

Evaluation of cognitive load for a mixed reality anatomy application