A desktop VR prototype for industrial training applications

Purpose Immersive Virtual Reality (VR) systems allow for highly repetitive tasks to be performed within a virtual environment that increases practice in home environments. VR can increase access to rehabilitation by reducing access barriers. However, rehabilitation outcomes between immersive VR systems and conventional physical rehabilitation are not well understood. The purpose of this case study was to assess the use of a custom clinically based VR simulation for testing gross hand dexterity with an individual with chronic stroke. Materials and methods The participant performed the box and blocks test (BBT) in an immersive VR environment and a physical environment. Three trials of the BBT were performed with their less-affected and affected hands each in both environments while measuring cortical activity using fNIRS. Rests were given between trials and environment conditions. Results Our results show that there was no statistical difference in the number of blocks moved between the physical and VR BBT for both the affected and less-affected hands. Furthermore, our results also indicate no statistically significant difference between the physical BBT and VR BBT conditions on contralateral motor cortex activation, suggesting that cortical involvement is comparable between physical and VR conditions. Conclusions These results suggest that an immersive VR system may be able to elicit functional and motor cortex activations that are comparable to the conventional physical BBT. Importantly, these findings highlights the potential benefits of VR therapy as a remote therapy intervention and/or to increase the effectiveness and practicality of current in-person rehabilitation programs. Implications for rehabilitation These findings highlight the potential benefits of immersive virtual reality as a remote therapy intervention. Immersive virtual reality use has potential benefits to increase the effectiveness and practicality of current in-person rehabilitation programs.

Extended-Reality Technologies: An Overview of Emerging Applications in Medical Education and Clinical Care.

Several experiments have provided evidence that ego-centric distances are perceived as compressed in immersive virtual environments relative to the real world. The principal factors responsible for this phenomenon have remained largely unknown. However, recent experiments suggest that when the virtual environment (VE) is an exact replica of a user's real physical surroundings, the person's distance perception improves. Furthermore, it has been shown that when users start their virtual reality (VR) experience in such a virtual replica and then gradually transition to a different VE, their sense of presence in the actual virtual world increases significantly. In this case the virtual replica serves as a transitional environment between the real and virtual world.In this paper we examine whether a person's distance estimation skills can be transferred from a transitional environment to a different VE. We have conducted blind walking experiments to analyze if starting the VR experience in a transitional environment can improve a person's ability to estimate distances in an immersive VR system. We found that users significantly improve their distance estimation skills when they enter the virtual world via a transitional environment.

Virtual reality applications seek to fully immerse participants into their virtual world experience. The investigation of how stimuli on the different senses influence the users is therefore crucial. As navigation is one of the most ubiquitous tasks in virtual environments, studying the influence of haptics on user presence is a necessity for future applications. This work presents an empirical study on the role of haptics during travel in a desktop virtual environment. Three techniques were compared in respect to task performance, perceived task performance, perceived presence and mental and physical workload. While our results indicate that haptics has a positive influence on participant’s perceived presence and performance, his total workload remains constant. Furthermore, we show that these findings apply to both experienced and unexperienced virtual environment users.

Though virtual reality (VR) has been advanced to certain levels of maturity in recent years, the general public, especially the population of the blind and visually impaired (BVI), still cannot enjoy the benefit provided by VR. Current VR accessibility applications have been developed either on expensive head-mounted displays or with extra accessories and mechanisms, which are either not accessible or inconvenient for BVI individuals. In this paper, we present a mobile VR app that enables BVI users to access a virtual environment on an iPhone in order to build their skills of perception and recognition of the virtual environment and the virtual objects in the environment. The app uses the iPhone on a selfie stick to simulate a long cane in VR, and applies Augmented Reality (AR) techniques to track the iPhone’s real-time poses in an empty space of the real world, which is then synchronized to the long cane in the VR environment. Due to the use of mixed reality (the integration of VR & AR), we call it the Mixed Reality cane (MR Cane), which provides BVI users auditory and vibrotactile feedback whenever the virtual cane comes in contact with objects in VR. Thus, the MR Cane allows BVI individuals to interact with the virtual objects and identify approximate sizes and locations of the objects in the virtual environment. We performed preliminary user studies with blind-folded participants to investigate the effectiveness of the proposed mobile approach and the results indicate that the proposed MR Cane could be effective to help BVI individuals in understanding the interaction with virtual objects and exploring 3D virtual environments. The MR Cane concept can be extended to new applications of navigation, training and entertainment for BVI individuals without more significant efforts.

In the recent past, efforts have been made to adopt immersive virtual reality (IVR) systems as a means for design reviews in factory layout planning. While several solutions for this scenario have been developed, their integration into existing planning workflows has not been discussed yet. From our own experience of developing such a solution, we conclude that the use of IVR systems-like CAVEs-is rather disruptive to existing workflows. One major reason for this is that IVR systems are not available everywhere due to their high costs and large physical footprint. As a consequence, planners have to travel to sites offering such systems which is especially prohibitive as planners are usually geographically dispersed. In this paper, we present a concept for integrating IVR systems into the factory planning process by means of a 3D collaborative virtual environment (3DCVE) without disrupting the underlying planning workflow. The goal is to combine non-immersive and IVR systems to facilitate collaborative walkthrough sessions. However, this scenario poses unique challenges to interactive collaborative work that to the best of our knowledge have not been addressed so far. In this regard, we discuss approaches to viewpoint sharing, telepointing and annotation support that are geared towards distributed heterogeneous 3DCVEs.

This talk will concentrate on how some of the recent advances in computer graphics and vision fit into a visual effects pipeline and their application for digitizing props. The production of movies often requires digitizing physical properties of various props. At Weta Digital we model, texture, and shade thousands of props in immense detail. We present a vision-based pipeline for digitizing props that requires far less user input than previous methods. This vision based pipeline fully automates the process of creating virtual cameras, reference geometry, and texture maps, which is then used by our artists to create the final assets at a much faster rate.

Recent advances in computer graphics and interactive techniques have increased the visual quality and flexibility of Augmented Reality (AR) applications. This, in turn has increased the viability of applying AR to educational exhibits for use in Science Centres, Museums, Libraries and other education centres. This article outlines a selection of five projects developed at the Human Interface Technology Laboratory in New Zealand (HIT Lab NZ) that have explored different techniques for applying AR to educational exhibits.These exhibits have received very positive feedback and appear to have educational benefits involving spatial, temporal and contextual conceptualisation and provide kinaesthetic, explorative and knowledge-challenging stimulus. The controls available to a user of turning a page, moving an AR marker, moving their head and moving a slider provide sufficient freedom to create many interaction scenarios that can serve educative outcomes. While the use of virtual media provides many advantages, creating new content is still quite difficult, requiring specialist software and skills. Useability observations are shared.

Novel view synthesis of remote sensing scenes is of great significance for scene visualization, human-computer interaction, and various downstream applications. Despite the recent advances in computer graphics and photogrammetry technology, generating novel views is still challenging particularly for remote sensing images due to its high complexity, view sparsity and limited view-perspective variations. In this paper, we propose a novel remote sensing view synthesis method by leveraging the recent advances in implicit neural representations. Considering the overhead and far depth imaging of remote sensing images, we represent the 3D space by combining implicit multiplane images (MPI) representation and deep neural networks. The 3D scene is reconstructed under a self-supervised optimization paradigm through a differentiable multiplane renderer with multi-view input constraints. Images from any novel views thus can be freely rendered on the basis of the reconstructed model. As a by-product, the depth maps corresponding to the given viewpoint can be generated along with the rendering output. We refer to our method as Implicit Multiplane Images (ImMPI). To further improve the view synthesis under sparse-view inputs, we explore the learning-based initialization of remote sensing 3D scenes and proposed a neural network based Prior extractor to accelerate the optimization process. In addition, we propose a new dataset for remote sensing novel view synthesis with multi-view real-world google earth images. Extensive experiments demonstrate the superiority of the ImMPI over previous state-of-the-art methods in terms of reconstruction accuracy, visual fidelity, and time efficiency. Ablation experiments also suggest the effectiveness of our methodology design. Our dataset and code can be found at https://github.com/wyc-Chang/ImMPI

An immersive type virtual reality (VR) system has been used as one of the visualization techniques for analyzing various complicated phenomena in a computational science. On the other hand, the volume rendering method is widely used for the intuitively understanding of 3-dimensionaly distributed physical quantities. Thus, it is very useful to apply the volume rendering method to the immersive VR system. However, a plane slice sampling method conventionally used in the volume rendering method has a problem that the quality of visualized images deteriorates in the immersive VR system. In this research and development, a sphere slice sampling method is applied to resolve this problem. In this presentation, the comparison between these two methods about the quality of image and the display speed has been performed. Further, the comparison between the hardware volume rendering method and software volume rendering method has been performed.

How can we increase awareness and understanding of other cultures using interactive online digital visualizations of past civilizations? In order to answer the above question, this chapter first questions the success of current virtual environments and asks whether they are capable of producing a platform that supports the experience and understanding of place-inscribed culture. To answer this question, a typology of successful virtual environment applications is suggested. These include online communities, virtual realityVirtual environmentVirtual Reality (VR) travel websites, game engines, virtual reality used for training, and virtual reality used as part of therapeutic means to either relax or divert patients, or to cure phobiasPhobias. Impeding the success of virtual environments in terms of technological constraints, lack of evaluation techniques and results, and a lack of content-specific applications that best utilize desktop computing capabilities and respond to user needs. It suggests that much more work needs to be done on not just usable but also useful content. Specifically, there is a large gap in knowledge on what might constitute useful and appropriate virtual travel environments, and how contextual interaction may affect our cultural understanding in these environments.

Finding enjoyable and effective long-term approaches to rehabilitation for improving the upper limb (UL) function of people with multiple sclerosis (MS) is challenging. Virtual reality (VR) with its engagement, interaction, and real time feedback could be a solution, with immersive VR systems having preliminary evidence in stroke to be more effective in improving UL outcomes in comparison to non-immersive VR. However, immersive VR research is limited within the realm of MS rehabilitation. Therefore, the main aim of the PhD was to explore immersive VR as an approach for UL rehabilitation for people with MS (pwMS). This was achieved in four stages: 1) a systematic review of the current evidence for VR in UL rehabilitation in MS; 2) predevelopment focus groups to determine requirements for the development of bespoke immersive VR games; 3) software development of bespoke VR games to potentially improve UL function for pwMS; and 4) a feasibility study of the games and immersive VR for pwMS’ UL rehabilitation. The systematic review initially yielded ten articles with varying study design, VR intervention and UL outcome measures used. There were some statistically significant improvements after VR interventions in outcomes such as in nine-hole peg test (NHPT), hand grip strength and active range of motion. There were however conflicting results regarding if VR was more effective than conventional approaches. Whilst the review demonstrated there were positive results for VR improving UL function, most of the studies had small sample sizes and were inadequately powered, and the level of immersion on effectiveness in MS remains unknown. The predevelopment focus groups recruited ten pwMS and eight specialist clinicians to explore their views on immersive VR as an approach to UL rehabilitation and determine requirements to develop VR games for this purpose. PwMS commonly reported that their UL problems interfered with activities of daily living and resulted in the loss of meaningful hobbies such as writing. Many pwMS neglected UL exercise and found strategies for adapting to their UL impairments. Similarly, clinicians stated UL rehabilitation was neglected within their service and that it was challenging to find interesting treatment strategies. VR was suggested by both participant groups as a potential solution, as it was convenient for pwMS to access and could provide a more engaging and disguised approach to exercise. Although, there were shared concerns about cybersickness and disengagement while using VR approaches. Finally, both groups agreed games should be meaningful and adaptable for users but suggested different VR activities, with clinicians suggesting games directly reflecting activities of daily living and pwMS suggesting more abstract activities. Recommendations were made for the development of VR rehabilitation games which could be personalised and customisable for the varying abilities of pwMS. Using theme-based content analysis and requirement analysis, requirements for the software development of immersive VR games were created and developed using Unity game engine. Three games, “Piano”; “Catch-a-falling-star” and “VR Whack-a-mole” which targeted dexterity and strength for pwMS were developed, with different difficulty levels implemented to adapt to varying ability of pwMS. The final stage of this PhD was a randomised, controlled feasibility study to investigate an intervention of immersive VR involving the three co-produced games for pwMS’ UL rehabilitation. Participants were randomised to either an intervention group consisting of 30 minutes of VR co-produced games, twice/week, for eight weeks, or to a control group of usual care. Participants were recruited from the NHS and third sector organisations. Nineteen pwMS were recruited (intervention n=11, control n=8), with a recruitment rate of 3.2 participants/month. Only minor adverse events were reported (n=9), such as fatigue and UL pain. Adherence was good with 87% (±12%) session completed, and two dropouts were reported in the control group. There were positive trends for improving NHPT and hand grip strength, but there was an observed ceiling effect in the Action Research Arm Test. Participants reported high levels of satisfaction and good usability of the co-produced VR games, as measured by the USE questionnaire. Thematic analysis revealed participants enjoyed the distraction and atmospheres that immersive VR provided, and believed the games were fit for purpose with the majority of participants reporting improvements in their UL function. Participants suggested more variety and progression of exercises for future development. Overall, these studies show that immersive VR has the potential to be beneficial for UL rehabilitation and is feasible and safe for pwMS. Therefore, future work should refine VR games for long-term application and explore the clinical and cost effectiveness of this approach.

The thesis presents the research of inventing a novel and natural spatial human-system interface – MagicPad that is to be used in a fully immersive virtual environment. With the advance in Virtual Reality (VR) technology, modern computer systems are capable of immersing users into a synthetic environment. Instead of sitting in front of the monitor, virtual objects are surrounding the users in an immersive VR system and therefore the traditional human-computer interfaces such as mouse and keyboard are no longer appropriate / fit for such purpose. Since then, researchers have been actively seeking new means of human-computer interaction in the three-dimensional space. The objective of this research is to innovate, design and build a novel spatial user interface for immersive VR systems by investigating and identifying the core elements of an intuitive and effective spatial user interface. The research methodology adopted is based on the quantitative research methods where the perception and interaction of human subjects with a synthetic virtual environment such user interfaces are collected to investigate the performance and effectiveness of the spatial user interface. The key elements of an intuitive and effective spatial user interface are first identified and the design concept of the MagicPad user interface prototype is then established. In order to realize the overall design and study the concept of the spatial human-system interface, an immersive Virtual Reality system imseCAVE, which serves as the test bed of the MagicPad user interface is designed and built from scratch. In this research, the MagicPad interface is implemented and quantitative user evaluations with the imseCAVE are carried out to draw conclusion and future research directions in the field of Virtual Reality and 3D User Interfaces.

In this paper, a collision contact modeling method for virtual reality environments is presented. Models in virtual environments are typically represented using triangle meshes. Most traditional collision contact simulations require the surface information of rigid bodies. This makes traditional collision contact simulations inapplicable for most virtual environments. This paper presents methods to model the behavior of colliding objects in virtual environments using triangle mesh representations of the objects. RAPID™ is used to obtain the approximate collision contact information between individual triangle meshes. Impulse calculations are combined with the collision detection to simulate post-collision behavior of objects. A three-point method is introduced to calculate the final resting place of objects or calculate unstable equilibrium and further dynamics of objects. These methods have been integrated with the Virtual Assembly Design Environment (VADE) to enhance VADE’s existing dynamics simulation capabilities. A series of test cases are presented and analyzed to validate these new methods.

The effects of the presence of others during a rowing exercise in a virtual reality environment