Head-mounted display with mid-air tactile feedback

Extended-Reality Technologies: An Overview of Emerging Applications in Medical Education and Clinical Care.

An increasing number of consumer-oriented head-mounted displays (HMD) for augmented and virtual reality (AR/VR) are capable of finger and hand tracking. We report on the accuracy of off-the-shelf VR and AR HMDs when used for touch-based tasks such as pointing or drawing. Specifically, we report on the finger tracking accuracy of the VR head-mounted displays Oculus Quest, Vive Pro and the Leap Motion controller, when attached to a VR HMD, as well as the finger tracking accuracy of the AR head-mounted displays Microsoft HoloLens 2 and Magic Leap. We present the results of two experiments in which we compare the accuracy for absolute and relative pointing tasks using both human participants and a robot. The results suggest that HTC Vive has a lower spatial accuracy than the Oculus Quest and Leap Motion and that the Microsoft HoloLens 2 provides higher spatial accuracy than Magic Leap One. These findings can serve as decision support for researchers and practitioners in choosing which systems to use in the future.

Since the metaverse was introduced, virtual reality (VR) head-mounted displays (HMDs) are increasingly being used for VR gaming, sports and 3D films. Additionally, the Pancake optical module has made HMDs smaller and lighter, which greatly increased its comfortability. However, during real use, there will inevitably be a lot of user head mouvements, such as shaking, lowering and raising, etc., which will cause certain displacement of the HMDs due to its inertia. Then the relative displacement of HMDs may affect the pressure and pressure-bearing area of the head, which leads to user discomfort.Currently, the research on the comfort of HMDs primarily focuses on the static scenarios without mouvement. This paper investigated the relative displacement and user comfort of the HMDs in motion. The main purpose of this study is to determine the relationship between HMDs' displacement and the contact area in order to provide design guidances and insights for HMDs, particularly the binding area. This paper also proposed an effective experimental method for measuring the relative displacement of HMDs, which can be used to assess the dynamic stability.A simple VR environment was bulit to guide the participant’s head movements by moving circular lightened dots, for simulating the head mouvements in daily life. At the same time, a wearable ruler sticker was designed to measure the displacement of the HMDs relative to the head.The experimenter took measurements of participants’ heads and assisted them to get familiar with the HMDs, and then pasted ruler stickers on the participants’ faces. The participants put on the HMD and adjust it to a comfortable position. Then they completed the head movement under the guidance of the VR environment and repeated it for 5 times. The same procedure was repeated after changing the contact area of the binding with 40mm*40mm foam cotton pieces. The HMD’s position was recorded before and after the mouvement.The results shows that, the HMDs exhibit a significant displacement relative to the head, after performing typical head motions, which causes some slight discomfort. Meanwhile, after doing a data correlation with SPSS, it can be found that the contact area of the binding is one of the main reasons for HMDs’ displacement. And the displacement can be greatly reduced and the pressure comfort rarely changes when the contact area of the bindings is increased, which has a positive impact on the overall comfort evaluation. Besides, we believe that in addition to the contact area, the mass distribution, pressure, and materials also have an effect on the relative displacement of HMDs, which might be the focus of future research. In future, new prototypes will be developed to increase the accuracy of environmental variables control and the evaluation of different factors.

Background. Robotic-assisted thoracic surgery (RATS) is now standard for lung cancer treatment, offering advantages over traditional methods. However, RATS’s minimally invasive approach poses challenges like limited visibility and tactile feedback, affecting surgeons’ navigation through com-plex anatomy. To enhance preoperative familiarization with patient-specific anatomy, we devel-oped a virtual reality (VR) surgical navigation system. Using head-mounted displays (HMDs), this system provides a comprehensive, interactive view of the patient’s anatomy pre-surgery, aiming to improve preoperative simulation and intraoperative navigation. Methods. We integrated 3D data from preoperative CT scans into Perspectus VR Education software, displayed via HMDs for in-teractive 3D reconstruction of pulmonary structures. This detailed visualization aids in tailored preoperative resection simulations. During RATS, surgeons access these 3D images through Tile-ProTM multi-display for real-time guidance. Results. The VR system enabled precise visualization of pulmonary structures and lesion relations, enhancing surgical safety and accuracy. The HMDs offered true 3D interaction with patient data, facilitating surgical planning. Conclusions. VR sim-ulation with HMDs, akin to a robotic 3D viewer, offers a novel approach to developing robotic surgical skills. Integrated with routine imaging, it improves preoperative planning, safety, and accuracy of anatomical resections. This technology particularly aids in lesion identification in RATS, optimizing surgical outcomes.

Virtual reality technologies, including head-mounted displays (HMD), can provide benefits to psychological research by combining high degrees of experimental control with improved ecological validity. This is due to the strong feeling of being in the displayed environment (presence) experienced by VR users. As of yet, it is not fully explored how using HMDs impacts basic perceptual tasks, such as object perception. In traditional display setups, the congruency between background environment and object category has been shown to impact response times in object perception tasks. In this study, we investigated whether this well-established effect is comparable when using desktop and HMD devices. In the study, 21 participants used both desktop and HMD setups to perform an object identification task and, subsequently, their subjective presence while experiencing two-distinct virtual environments (a beach and a home environment) was evaluated. Participants were quicker to identify objects in the HMD condition, independent of object-environment congruency, while congruency effects were not impacted. Furthermore, participants reported significantly higher presence in the HMD condition.

Immersive virtual reality (VR) emerges as a promising research and clinical tool. However, several studies suggest that VR induced adverse symptoms and effects (VRISE) may undermine the health and safety standards, and the reliability of the scientific results. In the current literature review, the technical reasons for the adverse symptomatology are investigated to provide suggestions and technological knowledge for the implementation of VR head-mounted display (HMD) systems in cognitive neuroscience. The technological systematic literature indicated features pertinent to display, sound, motion tracking, navigation, ergonomic interactions, user experience, and computer hardware that should be considered by the researchers. Subsequently, a meta-analysis of 44 neuroscientific or neuropsychological studies involving VR HMD systems was performed. The meta-analysis of the VR studies demonstrated that new generation HMDs induced significantly less VRISE and marginally fewer dropouts. Importantly, the commercial versions of the new generation HMDs with ergonomic interactions had zero incidents of adverse symptomatology and dropouts. HMDs equivalent to or greater than the commercial versions of contemporary HMDs accompanied with ergonomic interactions are suitable for implementation in cognitive neuroscience. In conclusion, researchers' technological competency, along with meticulous methods and reports pertinent to software, hardware, and VRISE, are paramount to ensure the health and safety standards and the reliability of neuroscientific results.

Virtual reality (VR) technology offers vision researchers the opportunity to conduct immersive studies in simulated real-world scenes. However, an accurate colour calibration of the VR head mounted display (HMD), both in terms of luminance and chromaticity, is required to precisely control the presented stimuli. Such a calibration presents significant new challenges, for example, due to the large field of view of the HMD, or the software implementation used for scene rendering, which might alter the colour appearance of objects. Here, we propose a framework for calibrating an HMD using an imaging colorimeter, the I29 (Radiant Vision Systems, Redmond, WA, USA). We examine two scenarios, both with and without using a rendering software for visualisation. In addition, we present a colour constancy experiment design for VR through a gaming engine software, Unreal Engine 4. The colours of the objects of study are chosen according to the previously defined calibration. Results show a high-colour constancy performance among participants, in agreement with recent studies performed on real-world scenarios. Our studies show that our methodology allows us to control and measure the colours presented in the HMD, effectively enabling the use of VR technology for colour vision research.

Virtual Reality (VR) has expanded from the training and high-end entertainment world into the homes of everyday users through mobile head mounted displays (HMDs). Currently, several different interaction methods for mobile VR HMDs exist, including remotes, external controllers, motion controllers, touch, and free form hand tracking. The goal of this study was to evaluate the impact that these interaction methods have on specifically mobile VR HMD user experience, including performance, user preference, and enjoyment. It was hypothesized that interaction methods would vary significantly in enjoyment, preference, and performance as many factors such as ease of use and familiarity can influence these factors. To evaluate the interaction methods, participants performed trials of a matching task utilizing menus in a VR environment in which the participants selected the features of an object (e.g., shape, color, pattern) from a menu to the right of the object. The Leap Motion had usability issues that impacted it’s performance and enjoyment scores resulting in it being the poorest performer of the group. Overall the gamepad was most preferred and performed the best due to its ease of use and familiarity, but the touchpad and leap motion offered immersive aspects. These results highlight the importance of ease of use in interaction methods and the impact it has on overall user experience. This paper will discuss these results and implications for design of interaction methods and associated VR HMD controllers.

BackgroundVideo-assisted thoracoscopic surgery (VATS) has become a standard approach for the treatment of lung cancer. However, its minimally invasive nature limits the field of view and reduces tactile feedback. These limitations make it vital that surgeons thoroughly familiarize themselves with the patient’s anatomy preoperatively. We have developed a virtual reality (VR) surgical navigation system using head-mounted displays (HMD). The aim of this study was to investigate the potential utility of this VR simulation system in both preoperative planning and intraoperative assistance, including support during thoracoscopic sublobar resection.MethodsThree-dimensional (3D) polygon data derived from preoperative computed tomography data was loaded into BananaVision software developed at Colorado State University and displayed on an HMD. An interactive 3D reconstruction image was created, in which all the pulmonary structures could be individually imaged. Preoperative resection simulations were performed with patient-individualized reconstructed 3D images.ResultsThe 3D anatomic structure of pulmonary vessels and a clear vision into the space between the lesion and adjacent tissues were successfully appreciated during preoperative simulation. Surgeons could easily evaluate the real patient’s anatomy in preoperative simulations to improve the accuracy and safety of actual surgery. The VR software and HMD allowed surgeons to visualize and interact with real patient data in true 3D providing a unique perspective.ConclusionsThis initial experience suggests that a VR simulation with HMD facilitated preoperative simulation. Routine imaging modalities combined with VR systems could substantially improve preoperative planning and contribute to the safety and accuracy of anatomic resection.

With the development of virtual reality (VR) technology, various VR devices are flooding the market. VR head-mounted displays (HMDs) sales are overgrowing as the primary medium for experiencing VR games. Therefore, exploring user concerns about HMDs is significant, enabling developers to design better VR devices and VR games. In this work, we conduct an empirical study of 7,413 VR games and 176,428 player reviews collected from three platforms (i.e., Oculus, Steam, and Viveport). First, we investigate the evolutionary trends of HMDs based on VR games. Subsequently, we extract nine topics (e.g., cybersickness, performance, and privacy) related to HMDs from game reviews that users are concerned about, analyze the emotional trends, and explore the voices of players. We find that users complain less about the cybersickness and immersion of HMDs and more about other topics (e.g., performance, tracking, and compatibility). In addition, more users have expressed concerns about privacy in recent years.

BackgroundApathy is a frequent and underrecognized neurological disorder symptom. Reduced goal-directed behavior caused by apathy is associated with poor outcomes for older adults in residential aged care. Recommended nonpharmacological treatments include person-centered therapy using information and communication technology. Virtual reality (VR) in the form of head-mounted displays (HMDs) is a fully immersive technology that provides access to a wide range of freely available content. The use of VR as a therapy tool has demonstrated promise in the treatment of posttraumatic stress disorder and anxiety. In addition, VR has been used to improve conditions including depression, anxiety, cognitive function, and balance in older adults with memory deficits, Alzheimer disease, and Parkinson disease. Research using VR for the symptoms of apathy in older adults living in residential aged care facilities is limited.ObjectiveThis study aims to examine whether using HMDs as a tool for reminiscence therapy improves the symptoms of apathy compared with using a laptop computer and physical items with older adults living in residential aged care.MethodsIn this multisite trial, 43 participants were allocated to one of three groups: reminiscence therapy intervention using VR in the form of HMDs, reminiscence therapy using a laptop computer supplemented by physical items if required (active control), and a usual care (passive control) group. The primary outcome was apathy, and the secondary outcomes included cognition and depression. The side effects of using HMDs were also measured in the VR group.ResultsMixed model analyses revealed no significant group interaction over time in outcomes between the VR and laptop groups (estimate=−2.24, SE 1.89; t40=−1.18; P=.24). Pooled apathy scores in the two intervention groups compared with the passive control group also revealed no significant group interaction over time (estimate=−0.26, SE 1.66; t40=−0.16; P=.88). There were no significant secondary outcomes. Most participants in the VR group stated that they would prefer to watch content in VR than on a flat screen (Χ22=11.2; P=.004), side effects from HMD use were negligible to minimal according to the Simulator Sickness Questionnaire cutoff scores.ConclusionsAlthough there were no significant results in outcome measures, this study found that participants engaged in the research and enjoyed the process of reminiscing using both forms of technology. It was found that VR can be implemented in an aged care setting with correct protocols in place. Providing residents in aged care with a choice of technology may assist in increasing participation in activities. We cannot dismiss the importance of immediate effects while the therapy was in progress, and this is an avenue for future research.Trial RegistrationAustralian New Zealand Clinical Trials Registry ACTRN12619001510134; https://www.anzctr.org.au/Trial/Registration/TrialReview.aspx?id=378564.International Registered Report Identifier (IRRID)RR2-DOI: 10.1136/bmjopen-2020-046030

Virtual reality (VR) technology has great potential in practical courses. This study aims to explain the benefits of virtual reality (VR) in the transformation of practical courses learning and student experience. This study will take physics courses as an example. Two independent studies were conducted, and college students were divided into two groups, using a computer all-in-one machine and a head-mounted display (HMD) device. Students using head-mounted display (HMD) reported higher levels of presence, engagement and empathy, compared to computer all-in-one, and had better mastery of knowledge than students using computer all-in-one. These findings suggest that virtual reality (VR) has a positive effect on student emotions in practical courses like physics. Moreover, head-mounted display (HMD) virtual reality (VR) devices have the potential to provide students with new immersive experiences, with cognitive and emotional benefits.

The Development of Virtual Reality as a Tool to Investigate Eating Behavior

Virtual reality (VR) is being used increasingly in many fields of medicine, including rehabilitation. Both 2D and 3D virtual environments (VEs) can be viewed either through a head mounted display (HMD) or on a screen (computer monitor and rear projection system, SPS). However, the question of whether the medium through which the environment is viewed affects motor performance has not been addressed. The objective of our study was to determine whether movement patterns were different when movements were performed in a 3D fully immersive VE viewed via an HMD or SPS. Two groups of subjects were recruited (stroke, healthy). They performed pointing movements to targets placed in the ipsilateral, central and contralateral arm workspaces in a VE. The VE, designed to resemble the interior of an elevator, was viewed via an HMD or a SPS. Arm motor impairment and spasticity were evaluated in both groups of subjects. The kinematics of the pointing movements were recorded using an optical tracking system (Optotrak Certus, 100 Hz, 6 markers). Arm motor performance (speed, precision and trajectory straightness) and movement quality outcomes (elbow and shoulder ranges of motion and trunk forward displacement) were analyzed using 2 way ANOVAs. Preliminary results suggest that the control group had straighter movements and used more shoulder flexion as compared to the stroke group. When the VE was viewed via both media, there were no differences in terms of endpoint precision and speed, elbow and shoulder ranges of motion and trunk forward displacement in both groups. Both groups reported that they completely enjoyed performing the movements when viewing them via both media. All subjects in the control group and 80% of subjects in the stroke group reported that VE was engaging, that it felt real and that the movements performed were similar to those made in the physical world. The results of this study have implications for the design of rehabilitation applications using VR aimed at improving arm motor activity and function.

In this paper, we propose a holographic three-dimensional (3D) head-mounted display based on 4K-spatial light modulators (SLMs). This work is to overcome the limitation of stereoscopic 3D virtual reality and augmented reality head-mounted display. We build and compare two systems using 2K and 4K SLMs with pixel pitches 8.1 μm and 3.74 μm, respectively. One is a monocular system for each eye, and the other is a binocular system using two tiled SLMs for two eyes. The viewing angle of the holographic head-mounted 3D display is enlarged from 3.8 ∘ to 16.4 ∘ by SLM tiling, which demonstrates potential applications of true 3D displays in virtual reality and augmented reality.