Implementation and Evaluation of Touch-based Interaction Using Electrovibration Haptic Feedback in Virtual Environments

In this demonstration, we introduce Face/On, an embedded feedback device that leverages the contact area between the user's face and a virtual reality (VR) head-mounted display (HMD) to provide rich haptic feedback in virtual environments (VEs). Head-worn haptic feedback devices have been explored in previous work to provide directional cues via grids of actuators and localized feedback on the users' skin. Most of these solutions were immersion breaking due to their encumbering and uncomfortable design and build around a single actuator type, thus limiting the overall fidelity and flexibility of the haptic feedback. We present Face/On, a VR HMD face cushion with three types of discreetly embedded actuators that provide rich haptic feedback without encumbering users with invasive instrumentation on the body. By combining vibro-tactile and thermal feedback with electrical muscle stimulation (EMS), Face/On can simulate a wide range of scenarios and benefit from synergy effects between these feedback types.

The ability to recognize and classify human gestures bears a huge potential of improving haptic feedback in virtual environments: when it is possible to predict how a user wants to explore or manipulate his environment both the visual and the auditive feedback may be adapted in order to enhance the immersiveness of haptic displays. Within this work a software approach is suggested that allows for such a real-time classification of gestures in a continuous data stream. A visually based tracking system is used to record the hand movements, while Hidden Markov models are applied to analyze the data. After presenting the methodological background and the software implementation, the outcome of an evaluation study is discussed. It reveals satisfying results of gesture classification and, what is particularly important, the recognition is fast enough to enable multi-modal haptic feedback.KeywordsGesture recognitionHidden Markov modelMulti-modal haptic feedbackVirtual reality

The usability of virtual reality (VR) controller interfaces are often complex and difficult for first time users. Most controllers provide minimal feedback which relegates the potential for heightened interaction and feedback within virtual experiences. This research explores how haptic technology systems partnered with VR can deliver immersive interactions between user and virtual environment (VE). This research involves the development of a haptic glove interface prototype that incorporates a force feedback and vibrotactile feedback system. It focuses on determining a workflow that communicates in real-time user interaction and environmental feedback using Unreal Engine and the produced haptic glove system. Testing and calibrating the prototype feedback system provided a baseline for developers to rationalise and improve accuracy of current real-time virtual feedback systems. The evaluation of this research in industry unfolds new technical knowledge for implementing a wider range of haptic technologies within VR. This further development would involve reviewing the usability and interaction standards for VR users in the design process.

Haptic interaction in slow virtual environments (VEs) can become unstable due to the phase lag introduced in the control loop by the slow update rate of the VE. Increasing the physical damping and/or limiting the contact stiffness rendered to users can mitigate the destabilizing effect of the low VE update rate. However, large physical damping and compliant virtual contacts decrease the sense of presence in VEs, especially during interaction with rigid virtual objects. To increase the maximum virtual contact stiffness that can be rendered to users without increasing the interface damping, this paper proposes a control strategy based on multirate wave communications between a haptic interface and a VE updated at a slow and fixed rate. The multirate wave communications are shown to be guaranteed passive only if the decrease of the wave sampling rate at the connection between the haptic interface and the VE does not cause aliasing. Therefore, an antialiasing low-pass filter is placed before the wave rate drop in the communications. The passivity condition is verified analytically and numerically for multirate haptic interaction in VEs with various contact stiffnesses and update rates. The transparency of haptic interaction in slow VEs to which users connect via passive multirate wave communications is investigated analytically in the frequency domain. Experiments validate that passive multirate wave communications can render stiffer contact in slow VEs than conventional direct coupling, and illustrate the destabilizing effect of the aliasing caused by the sampling rate drop in the communications.

Sound is an important, but often neglected, component for creating a self-motion illusion (vection) in Virtual Reality applications, for example, motion simulators. Apart from auditory motion cues, sound can provide contextual information representing self-motion in a virtual environment. In two experiments we investigated the benefits of hearing an engine sound when presenting auditory (Experiment 1) or auditory-vibrotactile (Experiment 2) virtual environments inducing linear vection. The addition of the engine sound to the auditory scene significantly enhanced subjective ratings of vection intensity in Experiment 1 and vection onset times but not subjective ratings in Experiment 2. Further analysis using individual imagery vividness scores showed that this disparity between vection measures was created by participants with higher kinesthetic imagery. On the other hand, for participants with lower kinesthetic imagery scores, the engine sound enhanced vection sensation in both experiments. A high correlation with participants' kinesthetic imagery vividness scores suggests the influence of a first person perspective in the perception of the engine sound. We hypothesize that self-motion sounds (e.g., the sound of footsteps, engine sound) represent a specific type of acoustic body-centered feedback in virtual environments. Therefore, the results may contribute to a better understanding of the role of self-representation sounds (sonic self-avatars), in virtual and augmented environments.

The performance levels of human subjects in size identification and size discrimination experiments in both real and virtual environments are presented. The virtual environments are displayed with a PHANToM desktop three degree-of-freedom haptic interface. Results indicate that performance of the size identification and size discrimination tasks in the virtual environment is comparable to that in the real environment, implying that the haptic device does a good job of simulating reality for these tasks. Additionally, performance in the virtual environment was measured at below maximum machine performance levels for two machine parameters. The tabulated scores for the perception tasks in a sub-optimal virtual environment were found to be comparable to that in the real environment, supporting previous claims that haptic interface hardware may be able to convey, for these perceptual tasks, sufficient perceptual information to the user with relatively low levels of machine quality in terms of the following parameters: maximum endpoint force and maximum virtual surface stiffness. Results are comparable to those found for similar experiments conducted with other haptic interface hardware, further supporting this claim. Finally, it was found that varying maximum output force and virtual surface stiffness simultaneously does not have a compounding effect that significantly affects performance for size discrimination tasks.

Current immersive Virtual Reality (VR) system strategies do not fully support dynamic Human Computer Interaction (HCI) and since there is a growing need for better immersion, due consideration should be given to integrate additional modalities for improved HCI. While feedback in Virtual Environments (VE) is predominantly provided to the user through the visual and auditory channels, additional modalities such as haptics can increase the sense of presence and efficiency in VE simulations. Haptic interfaces can enhance the VE interaction by enabling users to “touch” and “feel” virtual objects that are simulated in the environment. This paper examines the reasons behind its integration based on the limitations of present immersive projection system.

The haptic interface or force feedback device improves the quality of human computer interaction by accommodating the sense of touch in a computer simulation. A haptic device enables a two way communication between the user and the host computer. The applications of force feedback device cover a wide area such as in medical training, systems for the visually impaired, engineering research, and creative art-work. The role of haptic feedback varies depending on the applications in question and the devices used. This includes haptic as a teaching tool, and a communication aid. Despite the benefits of haptic feedback, its deployment to support interactions in a virtual environment is still slow. This is due to the high cost of the haptic devices in the market. We address this issue by presenting a conceptual framework in designing and implementing a cost-effective haptic device; building on similar concepts obtained from existing technologies. The device is developed using those gadgets that are mainly available in-house. It is projected that haptic feedback generated from the device could benefit teaching abstract concepts to students.

In an initial study, we characterized the properties of human reach motion in the presence or absence of visuo-haptic feedback in real and Immersive Virtual Environments (IVEs) or virtual reality within a participant’s maximum arm reach. Our goal is to understand how physical reaching actions to the perceived location of targets in the presence or absence of visuo-haptic feedback are different between real and virtual viewing conditions. Typically, participants reach to the perceived location of objects in the three-dimensional (3D) environment to perform selection and manipulation actions during 3D interaction in applications such as virtual assembly or rehabilitation. In these tasks, participants typically have distorted perceptual information in the IVE as compared to the real world, in part due to technological limitations such as minimal visual field of view, resolution, latency, and jitter. In an empirical evaluation, we asked the following questions: (i) how do the perceptual differences between virtual and real world affect our ability to accurately reach to the locations of 3D objects, and (ii) how do the motor responses of participants differ between the presence or absence of visual and haptic feedback? We examined factors such as velocity and distance of physical reaching behavior between the real world and IVE, both in the presence or absence of visuo-haptic information. The results suggest that physical reach responses vary systematically between real and virtual environments, especially in situations involving the presence or absence of visuo-haptic feedback. The implications of our study provide a methodological framework for the analysis of reaching motions for selection and manipulation with novel 3D interaction metaphors and to successfully characterize visuo-haptic versus non-visuo-haptic physical reaches in virtual and real-world situations.

The complex nature of virtual environments customarily hinders users to interact in a natural, intuitive and optimal way. Different user characteristics are hardly taken into account when designing 3D user interfaces for virtual environments. We envision that user interaction in virtual environments can be enhanced by integrating adaptation and personalization into 3D user interfaces. Through our research, we aim to provide adaptive and personalized 3D user interfaces for enhancing user interaction in virtual environments. The establishment of a user model becomes an important first step to facilitate adaptation and personalization to the user. In order to partly construct the user model, we carried out an experiment on 3D target acquisition task with four user groups (differing in experience level and gender). In this paper, we present a general user model that will enable first-time users to benefit instantly from adaptation and personalization in virtual environments.Keywordsvirtual environmentsadaptive 3D interfacesuser model

Interactive Virtual Environments (VEs) have the potential to increase student interest in soil science. Accordingly a bespoke ‘soil atlas’ was created using Java3D as an interactive 3D VE, to show soil information in the context of (and as affected by) the over-lying landscape. To display the below-ground soil characteristics, four sets of artistic illustrations were produced, each set showing the effects of soil organic-matter density and water content on fungal density, to determine potential for visualisations and interactivity in stimulating interest in soil and soil illustrations, interest being an important factor in facilitating learning. The illustrations were created using 3D modelling packages, and a wide range of styles were produced. This allowed a preliminary study of the relative merits of different artistic styles, scientific-credibility, scale, abstraction and ‘realism’ (e.g. photo-realism or realism of forms), and any relationship between these and the level of interest indicated by the study participants in the soil visualisations and VE. The study found significant differences in mean interest ratings for different soil illustration styles, as well as in the perception of scientific-credibility of these styles, albeit for both measures there was considerable difference of attitude between participants about particular styles. There was also found to be a highly significant positive correlation between participants rating styles highly for interest and highly for scientific-credibility. There was furthermore a particularly high interest rating among participants for seeing temporal soil processes illustrated/animated, suggesting this as a particularly promising method for further stimulating interest in soil illustrations and soil itself.

Vibrotactile feedback in virtual motor learning: A systematic review

Selection of key visual cues in real and virtual environments for assembly tasks

In recent years, there has been a growing interest in developing force feedback interfaces that allow people to access information presented in 3D virtual reality environments (VEs). One of the major application fields is medicine. Several cases exist where standard medical treatment cannot be applied to patients for several reasons, such as distance from hospital, lack of specialized doctor, and lack of medical equipment. The use of haptic‐enabled virtual environments provides an alternative solution in a variety of such cases, thus enabling assessment and treatment of patients. Furthermore, haptic‐enabled virtual environments provide a nonhazardous means of treatment for several disorders like phobia treatment, where the patient has to be exposed to the phobia object. Training of disabled people, like the visually impaired, is another important application of haptic‐enabled VEs. This chapter presents an extended description of haptic virtual environments and the major methodologies and techniques used in medical applications. Further, it demonstrates indicatively a number of applications in order to illustrate the applicability of VEs in several areas of medical treatment and training.

BackgroundPattern recognition technology allows for more intuitive control of myoelectric prostheses. However, the need to collect electromyographic data to initially train the pattern recognition system, and to re-train it during prosthesis use, adds complexity that can make using such a system difficult. Although experienced clinicians may be able to guide users to ensure successful data collection methods, they may not always be available when a user needs to (re)train their device.MethodsHere we present an engaging and interactive virtual reality environment for optimal training of a myoelectric controller. Using this tool, we evaluated the importance of training a classifier actively (i.e., moving the residual limb during data collection) compared to passively (i.e., maintaining the limb in a single, neutral orientation), and whether computational adaptation through serious gaming can improve performance.ResultsWe found that actively trained classifiers performed significantly better than passively trained classifiers for non-amputees (P < 0.05). Furthermore, collecting data passively with minimal instruction, paired with computational adaptation in a virtual environment, significantly improved real-time performance of myoelectric controllers.ConclusionThese results further support previous work which suggested active movements during data collection can improve pattern recognition systems. Furthermore, adaptation within a virtual guided serious game environment can improve real-time performance of myoelectric controllers.