RobotQuest: A Robotic Game Based on Projected Mixed Reality and Proximity Interaction

Extended-Reality Technologies: An Overview of Emerging Applications in Medical Education and Clinical Care.

Blowing in the wind: Increasing social presence with a virtual human via environmental airflow interaction in mixed reality

A mixed reality environment for collaborative product design and development

The ability to interact with virtual objects using gestures would allow users to improve their experience in Mixed Reality (MR) environments, especially when they use AR headsets. Today, MR head-mounted displays like the HoloLens integrate hand gesture based interaction allowing users to take actions in MR environments. However, the proposed interactions remain limited. In this paper, we propose to combine a Leap Motion Controller (LMC) with a HoloLens in order to improve gesture interaction with virtual objects. Two main issues are presented: an interactive calibration procedure for the coupled HoloLens-LMC device and an intuitive hand-based interaction approach using LMC data in the HoloLens environment. A set of first experiments was carried out to evaluate the accuracy and the usability of the proposed approach.

Developing a vehicle is always a long and complex task. This is especially true for autonomous cars. Tasks performed by the driver are taken over by the vehicle and must be performed with maximum reliability. Developing these systems is a difficult task, especially due to limited testing capabilities. Testing a vehicle in a closed environment is safe and controlled, but the variety of test scenarios is limited. By using mixed reality environments, one can create a diverse environment around a real test vehicle, with traffic, obstacles, and unexpected situations. The real movement of the test vehicle allows testing decision and motion planning level vehicular functions. Information from the virtual world can be considered as input to the vehicle sensor. Mixed reality or digital twin simulation environments can greatly assist the autonomous vehicle development process and also serve as a basis for validation procedures for such systems. This article introduces a mixed reality simulation environment that integrates a real test vehicle into a virtual environment, can handle other real and virtual obstacles, contains traffic simulations, and visualizes all of these.

Digital hydraulics is a discrete technology that integrates advanced dynamic system controls, digital electronics, and machine learning to enhance fluid power systems’ performance, overall efficiency, and controllability. A mechanically actuated inline three-piston variable displacement digital pump was previously proposed and designed. The inline three-piston pump incorporates complex mechanical and hydraulic subsystems and highly coupled mechanisms. The complexity of the utilized subsystems poses challenges when assessing the viability of the conceptual design. Therefore, this work focuses on designing, developing, and implementing a collaborative virtual platform involving a digitized module showcasing the internal mechanical structure of the digital pump utilizing mixed reality (MR) technology. MR technology is acknowledged as the forthcoming evolution of the human–machine interface in the real–virtual environment utilizing computers and wearables. This technology permits running simulations that examine the complexity of highly coupled systems, like the digital pump, where understanding the physical phenomenon is far too intricate. The developed MR platform permits multiple users to collaborate in a synchronized immersive MR environment to study and analyze the applicability of the pump’s design and the adequacy of the operated mechanisms. The collaborative MR platform was designed and developed on the Unity game engine, employing Microsoft Azure and Photon Unity Networking to set up the synchronized MR environment. The platform involves a fully interactive virtual module on the digital pump design, developed in multiple stages using Microsoft’s Mixed Reality Tool Kit (MRTK) for Unity and deployed in the synchronized MR environment through a HoloLens 2 MR headset. A research study involving 71 participants was carried out at Purdue University. The study’s objective was to explore the impact of the collaborative MR environment on understanding the complexity and operation of the digital pump. It also sought to assess the effectiveness of MR in facilitating collaboration among fluid power stakeholders in a synchronized digital reality setting to study, diagnose, and control their complex systems. Surveys were designed and completed by all 71 participants after experiencing the MR platform. The results indicate that approximately 75% of the participants expressed positive attitudes toward their overall MR platform experience, with particular appreciation for its immersive nature and the synchronized collaborative environment it provided. More than 70% of the participants agreed that the pump’s collaborative MR platform was essential for studying and understanding the complexity and intricacy of the digital pump’s mechanical structure. Overall, the results demonstrate that the MR platform effectively facilitates the visualization of the complex pump’s internal structure, inspection of the assembly of each of the involved subsystems, and testing the applicability of the complicated mechanisms.

Recently, smart phones have become a shining star in the mobile phone market, and the services provided by smart phones have significantly increased due to the advancements in hardware development and the great variety of user interfaces. Also, the stress of modern life has given more and more importance to the recreation and game industries. Therefore, this study aims to integrate a smart phone with a digital game so that the player can have the feel of real actions and enjoy the interaction during the game. By combining a digital game with a physical device, i.e., a mobile phone, the game player is provided with a richer gaming experience in which the mobile phone serves more than as a phone, and the game is more than tapping at a keyboard or clicking a mouse. The digital game in this study is about fishing. The fishing games on the market are typically played with a keyboard or a mouse, which nevertheless makes the games quite different from the real fishing experience. For example, the existing fishing games can hardly simulate the action of casting a fishing rod or reeling in the fish on the hook. In this study, a simulated setting is designed to mimic real situations so that digital interaction between the human and the computer in a human-computer interface can be assessed. A digital interactive game including both hardware and software is designed in this study, and whether the interactivity and the human-computer interface of the interactive game are acceptable by players is studied. According to questionnaire survey results, most players give positive evaluation to the combination of the mobile phone (i.e., hardware) and the digital game (i.e., software), and the more interactive the human-computer interface is, the stronger the interest and the greater the pleasure a player will feel in playing the game. Suggestions on how to improve the hardware and software design are also provided.

Mixed reality offers new potentials for social interaction experiences with virtual agents. In addition, it can be used to experiment with the design of physical robots. However, while previous studies have investigated comfortable social distances between humans and artificial agents in real and virtual environments, there is little data with regards to mixed reality environments. In this paper, we conducted an experiment in which participants were asked to walk up to an agent to ask a question, in order to investigate the social distances maintained, as well as the subject's experience of the interaction. We manipulated both the embodiment of the agent (robot vs. human and virtual vs. physical) as well as closed vs. open posture of the agent. The virtual agent was displayed using a mixed reality headset. Our experiment involved 35 participants in a within-subject design. We show that, in the context of social interactions, mixed reality fares well against physical environments, and robots fare well against humans, barring a few technical challenges.

Mixed Reality (MR) is the next evolution of human interacting with the computer as MR has the ability to combine the physical environment and digital environment and making them coexist with each other. Interaction is still a huge research area in Augmented Reality (AR) but very less in MR, this is due to current advanced MR display techniques still not robust and intuitive enough to let the user to naturally interact with 3D content. New techniques on user interaction have been widely studied, the advanced technique in interaction when the system able to invoke more than one input modalities. Multimodal interaction undertakes to deliver intuitive multiple objects manipulation with gestures. This paper discusses the multimodal interaction technique using gesture and speech which the proposed experimental setup to implement multimodal in the MR interface. The real hand gesture is combined with speech inputs in MR to perform spatial object manipulations. The paper explains the implementation stage that involves interaction using gesture and speech inputs to enhance user experience in MR workspace. After acquiring gesture input and speech commands, spatial manipulation for selection and scaling using multimodal interaction has been invoked, and this paper ends with a discussion.

We visualize life-size sequential photographs of sports activities in a mixed reality (MR) environment. Wearing a video-see-through head-mounted display, an observer records the motions of a player using a handheld camera. Our system then places billboards in the three-dimensional MR space, on which the sequential photographs of the player’s motion are presented at life-size. In a user study, we found that the observers perceived the size of the motions more accurately than when viewing sequential photographs on a monitor display.

This paper focuses on an emerging research area represented by robotic gaming and aims to explore the design space of interactive games that combine commercial-off-the-shelf robots and mixed reality. To this purpose, a software platform is developed which allows players to interact with both physical elements and virtual content projected on the ground. A game is then created to show designers how to maximize opportunities offered by such a technology and to build playful experiences.

This study reviews existing literature on how video games are being used to convey contemporary political discourse. Digital games, as meaningful cultural artifacts, have become a communication medium in their own right. They can serve as social mirrors, framing contemporary reality through metaphors that represent and recreate transcendent events or social facts through immersive experiences. Likewise, video games have played a significant role in shaping our current politics and culture. This article seeks to answer the research question: How has contemporary political discourse been explored through digital games in academic literature? To do so, we conducted a systematic literature review following the SALSA (search, appraisal, synthesis, and analysis) framework. We identified <em>N</em> = 25 journal articles written in English and Spanish, published between January 2013 and September 2023. We found that first-person shooter games were the most frequently discussed game genre in the academic literature, followed by newsgames. We propose a new method for categorizing political messages in digital games, which we have called the PRICE dimensions model (participation, representation, ideology, conflict, and education). The studied papers were classified into five main thematic groups: (a) video games as a tool for digital propaganda; (b) video games aiming to raise awareness of political issues; (c) games and gamification elements for radicalization; (d) game design that justifies, minimizes, or downplays violence; and (e) players’ role in conveying political messages.

Lower body tracking and user experience analysis are ongoing challenges in immersive technologies and biomechanics research, respectively. Particularly, in immersive technologies like mixed reality, there are limited research applications in user comfort and experience, and locomotion. In this paper, we aimed to conduct a cross-disciplinary research (by combining the elements of Biomechanics and human–computer interaction) in understanding the impact of user experience and immersive technologies induced symptoms in virtual and mixed reality environments for obstacle clearance tasks. For this, we created obstacle clearance applications for Oculus Quest and Microsoft HoloLens. Also, we conducted a user study with 12 participants under two conditions (virtual reality and mixed reality) in obstacle clearance behavior of varying heights. The results from our study indicated that the levels of immersion and enjoyment were significantly higher in virtual reality compared to mixed reality. In terms of virtual reality-induced symptoms, the participants found the mixed reality environment to be slightly more comfortable than virtual reality.

Augmented Reality and Mixed Reality technologies make us feel the real world and the virtual world are mixed with each other through visual matching. However, existing mixed reality contents using these technologies have a limitation that interact only with virtual objects and cannot interact with real objects. If a dynamic real object need to be influenced by a virtual object, or vice versa, illusion of Mixed Reality can be broken by the incongruity of the interaction. If these limitations are improved, real objects, which were always static, can operate dynamically and interact with virtual objects, so becomes difficult to distinguish between real objects and virtual objects. In this paper, we propose extended interactions in a mixed reality environment that can give illusion which is difficult to distinguish between real objects and virtual objects. And to confirm this, we implement authoring environment based Hololens.

A relevant literature review suggests that today’s children are increasingly immersing themselves in ubiquitous technologies, including interactive media and digital games. Therefore, this research uses valid measures to investigate the primary school students’ motivations toward playing educational games, at home and at school. The study was carried out amongst year-3 students in a small European state. The findings reported that there were strong correlations between the students’ perceived usefulness of the educational games and their behavioral intention to use them for their learning. The results also indicated that there was no significant relationship between the perceived ease of gameplay and the children’s enjoyment in engaging with the school’s digital games. To the best of our knowledge, there is no other study in academia that has explored the children’s technology acceptance,normative pressures and their intrinsic motivations to use digital learning games in the context of primary education. Therefore, this contribution opens future research avenues, as this study can be replicated in other contexts.