Crossmodal perception in immersive environments

Extended-Reality Technologies: An Overview of Emerging Applications in Medical Education and Clinical Care.

The perception of rotation gain, defined as a modification of the virtual rotation with respect to the real rotation, has been widely studied to determine detection thresholds and widely applied to redirected navigation techniques. In contrast, Field of View (FoV) restrictions have been explored in virtual reality as a mitigation strategy for motion sickness, although they can alter user’s perception and navigation performance in virtual environments. This paper explores whether the use of dynamic FoV manipulations, referred also as vignetting, could alter the perception of rotation gains during virtual rotations in virtual environments (VEs). We conducted a study to estimate and compare perceptual thresholds of rotation gains while varying the vignetting type (no vignetting, horizontal and global vignetting) and the vignetting effect (luminance or blur). 24 Participants performed 60 or 90\(^\circ \) virtual rotations in a virtual forest, with different rotation gains applied. Participants have to choose whether or not the virtual rotation was greater than the physical one. Results showed that the point of subjective equality was different across the vignetting types, but not across the vignetting effect or the turns. Subjective questionnaires indicated that vignetting seems less comfortable than the baseline condition to perform the task. We discuss the applications of such results to improve the design of vignetting for redirection techniques.KeywordsVirtual RealityPerceptionRotation gainsVignetting

Motion perception in immersive virtual environments significantly differs from the real world. For example, previous work has shown that users tend to underestimate travel distances in virtual environments (VEs). As a solution to this problem, researchers proposed to scale the mapped virtual camera motion relative to the tracked real-world movement of a user until real and virtual motion are perceived as equal, i.e., real-world movements could be mapped with a larger gain to the VE in order to compensate for the underestimation. However, introducing discrepancies between real and virtual motion can become a problem, in particular, due to misalignments of both worlds and distorted space cognition. In this paper, we describe a different approach that introduces apparent self-motion illusions by manipulating optic flow fields during movements in VEs. These manipulations can affect self-motion perception in VEs, but omit a quantitative discrepancy between real and virtual motions. In particular, we consider to which regions of the virtual view these apparent self-motion illusions can be applied, i.e., the ground plane or peripheral vision. Therefore, we introduce four illusions and show in experiments that optic flow manipulation can significantly affect users' self-motion judgments. Furthermore, we show that with such manipulations of optic flow fields the underestimation of travel distances can be compensated.

For humans to effectively interact with their environment, it is important for the visual system to determine the absolute size and distance of objects. Previous experiments performed in full-cue, real-world environments have demonstrated that blind walking to targets serves as an accurate indication of distance perception, up to about 25 m. In contrast, the same task performed in virtual environments (VEs) using head-mounted displays shows significant underestimation in walking. To date, blind walking is the only visually directed action task that has been used to evaluate distance perception in VEs beyond reaching distances. The possible influence of the response measure itself on absolute distance perception in virtual environments is currently an open question. Blind walking involves locomotion and the egocentric updating of the environment with one's own movement. We compared this measure to blind throwing, a task that involves the initiation of a movement directed by vision, but no further interaction within the environment. Both throwing and walking were compressed in the VE but accurate in the real world. We suggest that distance compression found in VEs may be a result of a general perceptual origin rather than specific to the response measure.

We investigate the detection thresholds of tactile perception in virtual environments. We estimate the absolute detection thresholds for electrovibration stimuli under different parameters of excitation signals (waveform and frequency). Experimental results and subjective comments suggest that participants’ tactile sensitivities to electrovibration stimuli are decreased in virtual environments. The frequency significantly affects the absolute detection thresholds, and lower voltages can be perceived under 60Hz for the square wave than the sinusoidal wave. Our findings provide a foundation to future studies aiming at modelling and simulating desired tactile perception of objects in virtual reality.

In this paper, we propose a holographic three-dimensional (3D) head-mounted display based on 4K-spatial light modulators (SLMs). This work is to overcome the limitation of stereoscopic 3D virtual reality and augmented reality head-mounted display. We build and compare two systems using 2K and 4K SLMs with pixel pitches 8.1 μm and 3.74 μm, respectively. One is a monocular system for each eye, and the other is a binocular system using two tiled SLMs for two eyes. The viewing angle of the holographic head-mounted 3D display is enlarged from 3.8 ∘ to 16.4 ∘ by SLM tiling, which demonstrates potential applications of true 3D displays in virtual reality and augmented reality.

This exegesis will investigate the influence of audio stimuli and its relationship with visual perception, attention and hierarchical flow within visual communication - utilising a heuristic form of inquiry. New terminology of black and white thoughts will also be explored to analyse logical and lateral thinking when applied to problem solving. By utilising visual abstraction, cross-modal perception and iconic-coupling techniques I wish to explore how one can create stronger engaging experiences. This supports the research of Menon and Levitin (2005) and Ramachandran and Hirstein (1999) in the way the brain responds and rewards when resolving problems and when listening to music. By offering more rewards, we can create an environment that encourages stronger engagement. When an artificial synaesthetic experience is generated by iconic-coupling and cross -modal binding processes, a deeper correlation between auditory and visual elements may result. Integration of specific iconic-coup ling within the design process of the following two works - 'Inner sense ' and 'Elements' - creates an environment that would support active engagement with the spectator; increasing the chances of visual influence and memory retention. By utilising the powerful emotive feelings generated by music, a more personal emotivememory connection is formed when combined with vision. This artificial synaesthetic experience allows for a deeper cross-modal connection drawing from a lifetime's worth of experiences that shape our understanding and stimulate our imagination.

Virtual Reality is becoming more popular over the years because it allows the user to be the main actor in another environment and interact with it in real time. New interaction methods are being studied, like tangible interfaces, but there is little work done related to small distances when grabbing objects through a virtual environment. This study is important because, in our perspective, interaction in virtual reality will be at arms reach, meaning that the user will interact within very close distances (under 1 m). In this paper, the research team further evaluate distance perception using gender, the presence of avatar and height (fixed or personalised). The sample consisted of 64 participants (32 females and 32 males) evenly distributed between all four conditions (8 males and 8 females for each condition). Results revealed that gender does have an impact on small distance estimation; height does not have an impact on distance estimation; and avatar does make a difference when trying to grab a real object through the virtual environment.

Virtual reality (VR) is becoming increasingly integrated into people’s lives. However, current applications are usually limited to a certain fixed location. With the introduction of wireless and mobile head mounted displays (HMD) there is no longer a need for a workstation. In this way it is possible to introduce VR experiences in public spaces. Some airlines have already started to use VR as an alternative entertainment system on flights. The company Inflight VR offers a complete system with various applications approved for in flight VR [2]. However, VR in an aircraft is a controlled space compared to other public transport systems. Flixbus in the US also offers FlixVR, a long distance bus experience, where passengers can experience different games and cinematic experiences [1]. It seems that VR is on the rise in the transportation system. The use of VR in any kind of transport brings new challenges. Especially in comparison to a controlled environment of a laboratory or living room. In this work we are focusing our evaluation on VR scenarios in public transport systems: planes, train, metro, tram or as a co-driver in a car. We analyse different seating situations for each type of transportation and its implications to VR applications. We present an analysis of possible scenarios including their challenges and constraints for interaction, space and navigation for public seated VR.

In this paper, we propose a technique that simplifies pseudo-gustatory systems by using the cross-modal effect between vision and olfaction to change only the visual and olfactory stimuli without altering the ingredients of the food. Conventional pseudo-gustatory simulations require one olfactory stimulus for each flavor. In contrast, we hypothesize that the cross-modal effect between vision and olfaction in a pseudo-gustatory presentation can be utilized to reduce the required number of olfactory stimuli, and verify this hypothesis using a pseudo-gustatory display with our proposed method. This pseudo-gustatory display uses the visual-olfactory cross-modal effect and the key scent components decided on the basis of similarity analysis of olfactory perception. We also investigate how users perceive the taste of a drink under various visual and olfactory conditions, both with and without the visual-olfactory interactions. Our results indicate that the cross-modal effect between vision and olfaction can effectively simplify pseudo-gustatory simulations.

Downstairs resident classification characteristics for upstairs walking vibration noise in an apartment building under virtual reality environment

Purpose This pilot study aimed to (a) investigate opportunities for immersive Virtual Reality (VR) technology in communication, physical, and visual rehabilitation by examining the interaction of people without disabilities in a range of structured virtual environments; and (b) validate research protocols that might be used to evaluate the physical, visual, and verbal interaction of users in virtual worlds, and their safety while using the technology. Methods Thirteen adults identifying as people without disability were exposed to VR via a head-mounted display. A video-review method was used to qualitatively code and analyse each participant’s communication, movement, orientation, and support needs. Results All participants oriented to their virtual environments sufficiently to use applications. Their spoken language was effective for interaction, although unconventional social behaviours were also observed. Two participants reported minor adverse reactions consistent with mild cybersickness. Conclusion The results provide insight into the types of environments and characters that support the greatest communicative, physical, and visual interaction in immersive VR. The tested protocols are useful to assess safety when using VR, and to observe communicative, physical, and visual interaction with virtual environments and characters. Implications for future research and use of VR with people with communication, physical and visual disability are discussed.

Virtual reality (VR) is being used increasingly in many fields of medicine, including rehabilitation. Both 2D and 3D virtual environments (VEs) can be viewed either through a head mounted display (HMD) or on a screen (computer monitor and rear projection system, SPS). However, the question of whether the medium through which the environment is viewed affects motor performance has not been addressed. The objective of our study was to determine whether movement patterns were different when movements were performed in a 3D fully immersive VE viewed via an HMD or SPS. Two groups of subjects were recruited (stroke, healthy). They performed pointing movements to targets placed in the ipsilateral, central and contralateral arm workspaces in a VE. The VE, designed to resemble the interior of an elevator, was viewed via an HMD or a SPS. Arm motor impairment and spasticity were evaluated in both groups of subjects. The kinematics of the pointing movements were recorded using an optical tracking system (Optotrak Certus, 100 Hz, 6 markers). Arm motor performance (speed, precision and trajectory straightness) and movement quality outcomes (elbow and shoulder ranges of motion and trunk forward displacement) were analyzed using 2 way ANOVAs. Preliminary results suggest that the control group had straighter movements and used more shoulder flexion as compared to the stroke group. When the VE was viewed via both media, there were no differences in terms of endpoint precision and speed, elbow and shoulder ranges of motion and trunk forward displacement in both groups. Both groups reported that they completely enjoyed performing the movements when viewing them via both media. All subjects in the control group and 80% of subjects in the stroke group reported that VE was engaging, that it felt real and that the movements performed were similar to those made in the physical world. The results of this study have implications for the design of rehabilitation applications using VR aimed at improving arm motor activity and function.

Ask the Expert: Anxiety Disorders Virtual Reality Therapy

Virtual reality (VR) enables the study of visually-guided behavior in situations that are otherwise not amenable to laboratory research. An example is a fire evacuation scenario, in which building occupants have to find their way to safety through smoke and flames, identify safe egress routes, and avoid threats. First, however, it must be determined whether evacuation behavior in VR is comparable to that in a real environment. We are thus initially studying participant responses to a fire alarm in matched virtual and real environments. The present experiment investigates how the decision to evacuate is influenced by visual cues about a neighbor’s behavior. In the Control group, a lone participant is exposed to a fire alarm while performing a bogus task. In the Passive group, a participant performs the same task with a confederate who ignores the fire alarm. In the Active group, the confederate immediately leaves the room when the fire alarm sounds. The gender of the participant is crossed with the gender of the confederate. 75 participants experience the scenario in a real environment (25 per group), and 75 others in a matched virtual environment presented in a wireless head mounted display (HMD), with avatars of the confederates. In the real environment, two-thirds of participants in the Control group evacuated the room. In the Active group, nearly all did so, whereas in the Passive group all but one did. More female than male participants evacuated in the Control group, but there were no other gender effects. Data collection in the virtual environment is in progress, and will be completed by spring. The findings thus far demonstrate a very strong influence of neighbor behavior on a participant’s evacuation decision. The results will provide answers to the much-debated question of whether evacuation behavior is similar in virtual and real environments. Meeting abstract presented at VSS 2015