Human gait recognition for virtual environments exploration

Three-dimensional sound's effectiveness in virtual reality (VR) environments has been widely studied. However, due to the big differences between VR and augmented reality (AR) systems in registration, calibration, perceptual difference of immersiveness, navigation, and localization, it is important to develop new approaches to seamlessly register virtual 3-D sound in AR environments and conduct studies on 3-D sound's effectiveness in AR context. In this paper, we design two experimental AR environments to study the effectiveness of 3-D sound both quantitatively and qualitatively. Two different tracking methods are applied to retrieve the 3-D position of virtual sound sources in each experiment. We examine the impacts of 3-D sound on improving depth perception and shortening task completion time. We also investigate its impacts on immersive and realistic perception, different spatial objects identification, and subjective feeling of human presence and collaboration. Our studies show that applying 3-D sound is an effective way to complement visual AR environments. It helps depth perception and task performance, and facilitates collaborations between users. Moreover, it enables a more realistic environment and more immersive feeling of being inside the AR environment by both visual and auditory means. In order to make full use of the intensity cues provided by 3-D sound, a process to scale the intensity difference of 3-D sound at different depths is designed to cater small AR environments. The user study results show that the scaled 3-D sound significantly increases the accuracy of depth judgments and shortens the searching task completion time. This method provides a necessary foundation for implementing 3-D sound in small AR environments. Our user study results also show that this process does not degrade the intuitiveness and realism of an augmented audio reality environment

Smart building is a rising interdisciplinary research field that aims to improve the monitoring, management, and maintenance of buildings. The authors present an innovative solution for combining building information modeling (BIM) data with ambient information collected by heterogeneous devices deployed in the building. To collect environmental information, they exploit a distributed software architecture. It enables interoperability between heterogeneous data sources--either physical devices, such as sensor nodes, or third-party software such as Archibus--where building information resides. On top of this infrastructure, they developed an Android-based application that presents environmental building information integrated with BIM data in an augmented and virtual reality environment. The proposed solution gives users awareness about building conditions and energy consumption.

This paper presents a study investigating the influence of look-alike avatars on students' perceptions of teaching effectiveness in three-dimensional virtual and augmented reality environments. This study investigated three avatar representations: i) look-alike avatar of the instructor; ii) stick avatar; and iii) video recording of the instructor. Eighteen participants were asked to rank the three avatar representations, as well as the immersion experience of the teaching simulation on the virtual and augmented reality displays. The result of this study suggests that look-alike avatars can be used to represent an instructor in virtual environments.

BackgroundIt is crucial to use innovative tools to gain clinical reasoning skills through experiential learning in pharmacy education, and one of the most effective tools used in this is virtual reality. However, the lack of research that empirically demonstrates the instructional principles of virtual environment design, especially in the clinical setting, has resulted in a significant gap in the current literature.AimIn this study, it was aimed to determine the instructional design principles of a virtual environment for the clinical pharmacy.MethodThe study was conducted as a design‐based research. The data of the study related to three components of the developed virtual reality environment in pharmacy education (VRPE). These components, scenario development interface‐scenario development interface, virtual environment, and scenario, were collected through interviews, observation, document review, and a system usability scale. Instructional design principles for the VRPE were determined through analysis of the collected data.ResultsThe prominent instructional design principles of the VRPE were revealed as immersion, interaction, diversity, flow, aesthetics‐ergonomic‐universal design, informing, and retest availability. The significance of the study was the adoption of a holistic perspective in the design of a virtual reality environment for pharmacy education. In addition, one of the most critical features of the study was eliminating the expertise barrier in virtual environment design. Thus, it was concluded that the VRPE is an extraordinary tool that can be used for pharmacy education to promote experiential learning.

The effects of the presence of others during a rowing exercise in a virtual reality environment

Extended-Reality Technologies: An Overview of Emerging Applications in Medical Education and Clinical Care.

Virtual reality environments are increasingly being used for upper limb rehabilitation in poststroke patients. Our goal was to determine if arm reaching movements made in a 2-D video-capture virtual reality environment are similar to those made in a comparable physical environment. We compared arm and trunk kinematics for reaches made with the right, dominant arm to three targets (14 trials per target) in both environments by 16 adults with right poststroke hemiparesis and by eight healthy age-matched controls. Movement kinematics were recorded with a three-camera optoelectronic system at 100 samples/s. Reaching movements made by both control and stroke subjects were affected by viewing the targets in the video-capture 2-D virtual environment. Movements were slower, shorter, less straight, less accurate and involved smaller ranges of shoulder and elbow joint excursions for target reaches in the virtual environment compared to the physical environment in all subjects. Thus, there was a decrease in the overall movement quality for movements made in the 2-D virtual environment. This suggests that 2-D video-capture virtual reality environments should be used with caution when the goal of the rehabilitation program is to improve the quality of movement patterns of the upper limb.

Pain and discomfort are perceptible during many medical procedures. In the past, drugs have been the conventional means to alleviate pain, but in many instances, medications by themselves do not provide optimal results. Current advances are being made to control pain by integrating both the science of pain medications and the science of the human mind. Various psychological techniques, including distraction by virtual reality environments and the playing of video games, are being employed to treat pain. In virtual reality environments, an image is provided for the patient in a realistic, immersive manner devoid of distractions. This technology allows users to interact at many levels with the virtual environment, using many of their senses, and encourages them to become immersed in the virtual world they are experiencing. When immersion is high, much of the user's attention is focused on the virtual environment, leaving little attention left to focus on other things, such as pain. In this way virtual reality provides an effective medium for reproducing and/or enhancing the distractive qualities of guided imagery for the majority of the population who cannot visualize successfully.

Decision making is vital in complex sporting tasks but is difficult to test and train. New technologies such as virtual and augmented reality offer novel opportunities for improving decision making, yet it remains unclear whether training gains using these new approaches will improve decision making on-field. To clarify the potential benefits, a clear conceptualization of decision making is required, particularly for invasive team sports such as football, basketball and field hockey, where decisions are complex with many possible options offered. Therefore, the aim of this position paper is to establish a framework for the design of virtual and augmented environments that help invasive team sport athletes to train their decision-making capacities. To achieve this, we propose a framework for conceptualising ‘natural’ decision making within the performance environment in invasive team sports that views decision making as a continuous cyclical process where the ball carrier interacts with teammates to create ‘windows of opportunity’, and where skilled decision makers often delay decisions to create time, and in turn new opportunities, rather than necessarily selecting the first option available to them. Within the framework, we make a distinction between decision making and anticipation, proposing that decision making requires a series of on-going anticipatory judgments. Based on the framework, we subsequently highlight the consequences for testing and training decision making using virtual and augmented reality environments, in particular outlining the technological challenges that need to be overcome for natural decision making to be represented within virtual and augmented environments.

Advanced fire control technologies that utilize computer vision-guided target recognition will enable dismounted soldiers with augmented reality displays, such as the integrated visual augmentation system, enhanced situational awareness. Here we describe a virtual reality framework and environment for the design and evaluation of computer vision algorithms and augmented reality interfaces intended to enhance dismounted soldier situational awareness. For training models, synthetic image datasets of targets in virtual environments can be generated in tandem with neural network learning. To evaluate models under simulated operational environments, a dismounted soldier combat scenario was developed. Trained models are used to process input from a “virtual camera” in-line with a rifle-mounted telescopic sight. Augmented reality overlays are projected over the sight’s optics, modeling the function of current state-of-the-art holographic displays. To assess the impact of these capabilities on situational awareness, performance metrics and physiological monitoring were integrated into the system. To investigate how sensors beyond visible wavelength optical imaging may be leveraged to enhance this capability, particularly in degraded visual environments, the virtual camera framework was extended to introduce methods for simulating multispectral infrared imaging. Thus, this virtual reality framework provides a platform for evaluating multispectral computer vision algorithms under simulated operational conditions, as well as iteratively refining the design of augmented reality displays. Improving the design of these components in virtual reality provides a rapid and cost-effective method for refining specifications and capabilities toward a field-deployable system.

While thorough documentation has always been a crucial feature of life cycle monitoring and assessment, different forms of documentation have not always been well integrated. Thorough documentation can include 2D images, 3D models, architectural drawings, reports from past interventions, current monitoring data, among other elements. In life-cycle monitoring, it is important that all of these types of documentation are well integrated and organized. The work presented here reflects a methodology and digital workflow for organizing and integrating existing documentation and data about a structure into a virtual reality environment. Through presenting data about a structure in an intuitive, organized, and interactive way, a virtual environment will foster communication among different groups working on a project, and greatly help the engineers to quickly and comprehensively assess the condition of the structure. The virtual environment proposed here uses 360 degree spherical imaging (Ricoh Theta) and virtual environment software (Kolor Panotour Pro) to aid a user in virtually experiencing a structure accompanied by its associated data. This project was implemented on Streicker Bridge, a 350 foot pedestrian bridge, in Princeton, NJ, USA. Since construction began in 2009, this bridge has been monitored using fiber-optic strain sensors. This data is incorporated into the virtual environment to highlight the possibilities of this method for life-cycle management. The lessons learned will be recommendations for how virtual environments can be used to enhance current practices and visualization methods for long-term monitoring and assessment.

This paper presents the theoretical foundation, practical implementation, and empirical evaluation of a glove for interaction with 3-D virtual environments. At the dawn of the “Spatial Computing Era”, where users continuously interact with 3-D Virtual and Augmented Reality environments, the need for a practical and intuitive interaction system that can efficiently engage 3-D elements is becoming pressing. Over the last few decades, there have been attempts to provide such an interaction mechanism using a glove. However, glove systems are currently not in widespread use due to their high cost and, we propose, due to their inability to sustain high levels of performance under certain situations. Performance deterioration has been observed due to the distortion of the local magnetic field caused by ordinary ferromagnetic objects present near the glove’s operating space. There are several areas where reliable hand-tracking gloves could provide a next generation of improved solutions, such as American Sign Language training and automatic translation to text and training and evaluation for activities that require high motor skills in the hands (e.g., playing some musical instruments, training of surgeons, etc.). While the use of a hand-tracking glove toward these goals seems intuitive, some of the currently available glove systems may not meet the accuracy and reliability levels required for those use cases. This paper describes our concept of an interaction glove instrumented with miniature magnetic, angular rate, and gravity (MARG) sensors and aided by a single camera. The camera used is an off-the-shelf red, green, and blue–depth (RGB-D) camera. We describe a proof-of-concept implementation of the system using our custom “GMVDK” orientation estimation algorithm. This paper also describes the glove’s empirical evaluation with human-subject performance tests. The results show that the prototype glove, using the GMVDK algorithm, is able to operate without performance losses, even in magnetically distorted environments.

This paper describes the development of a virtual reality environment which facilitates the design of spherical four-bar mechanisms. The virtual environment allows the user to naturally interact with the input data and specify the design parameters while operating in a three-dimensional environment. We see this development as a logical extension of existing graphics-based spatial design software. The need for a three-dimensional design space is driven by the difficulty in specifying design inputs and constraints for a spatial problem using a two-dimensional interface. In addition, once the mechanism has been created, the virtual environment provides the opportunity for the user to visually verify that the mechanism will perform the desired three-dimensional motion.

The traditional rubber hand illusion is a psychological experiment where participants are under the illusion that a rubber hand is part of their own body. This paper examines the use of real, virtual and augmented reality environments for identifying the elements that influence body ownership in healthy participants. Compared to the classical experiment where a plastic rubber hand was used, a realistic 3D representation was chosen to create the same illusion this time in both immersive virtual reality and augmented reality. Experiments were performed on 30 volunteers undergoing testing session composed of three stages. Participants were asked to complete two different questionnaires, one measuring their cognitive workload and another one regarding their experience with the rubber hand illusion. In addition, EEG signals of the individuals were recorded, resulting in 90 electroencephalogram datasets. Results indicate correlations between ownership statements with beta and gamma electroencephalogram bands in premotor cortex activity. Link between higher gamma production in ventral premotor area during the illusion was established in previous studies.

The virtual reality Walking Corsi Test