Haptic-Visual Interaction Training System for Refueling Aircraft

There is an inherent trade-off between stability and transparency in haptic interaction systems when interacting with virtual environments. Therefore, no perfect transparency can be displayed with absolute stability. For meaningful interactions, anyhow, some level of transparency is necessary for providing realistic feelings. On the other hand, robust stability must absolutely be guaranteed against any virtual environments because unstable behaviors disrupt contact realism completely and/or may also injure human operators. In order to increase magnitude transparency of the haptic interaction system, this paper proposes a way of systematic inclusion of Coulomb friction component in the previous force bounding approach. In the inclusion, two type of force bounding approach are derived. The less conservative condition can generate significantly higher magnitude transparency in terms of the initial contact crispness as well as the steady-state contact force for very stiff virtual objects. However, there occur some contact oscillations to diminish contact realism due to the energy accumulation during free motion. In order to avoid contact oscillations, a more conservative condition is proposed for systematically removing the past accumulated energy. The proposed algorithm is also compared with the other similar algorithm. Comprehensive experimental results are presented to show the effectiveness of the proposed approaches.

The design of virtual environments (VEs) can strongly influence users’ emotions. These VEs are also an important aspect of immersive Virtual Reality (VR) exergames – training system that can inspire athletes to train in a highly motivated way and achieve a higher training intensity. VR-based training and rehabilitation systems can increase a user’s motivation to train and to repeat physical exercises. The surrounding VE can potentially predominantly influence users’ motivation and hence potentially even physical performance. Besides providing potentially motivating environments, physical training can be enhanced by gamification. However, it is unclear whether the surrounding VE of a VR-based physical training system influences the effectiveness of gamification. We investigate whether an emotional positive or emotional negative design influences the sport performance and interacts with the positive effects of gamification. In a user study, we immerse participants in VEs following either an emotional positive, neutral, or negative design and measure the duration the participants can hold a static strength-endurance exercise. The study targeted the investigation of the effects of (1) emotional VE design as well as the (2) presence and absence of gamification. We did not observe significant differences in the performance of the participants independent of the conditions of VE design or gamification. Gamification caused a dominating effect on emotion and motivation over the emotional design of the VEs, thus indicating an overall positive impact. The emotional design influenced the participants’ intrinsic motivation but caused mixed results with respect to emotion. Overall, our results indicate the importance of using gamification and support the commonly used emotional positive VEs for physical training. They further indicate that the design space could also include other emotional directions of VE design.

This paper presents the creation of an assembly simulation environment with multisensory feedback (auditory and visual), and the evaluation of the effects of auditory and visual feedback on the task performance in the context of assembly simulation in a virtual environment (VE). This VE experimental system platform brings together complex technologies such as constraint-based assembly simulation, optical motion tracking technology, and real time 3D sound generation technology around a virtual reality workbench and a common software platform. A peg-in-a-hole and a Sener electronic box assembly task have been used as the task cases to conduct the human factor experiment, using sixteen participants. Both objective performance data (i.e., task completion time, TCT; and human performance error rate, HPER) and subjective opinions (i.e., questionnaires) on the utilization of auditory and visual feedback in a virtual assembly environment (VAE) have been gathered from the experiment. Results showed that the introduction of auditory and/or visual feedback into VAE did improve the assembly task performance. They also indicated that integrated feedback (auditory plus visual) offered better assembly task performance than either feedback used in isolation. Most participants preferred integrated feedback to either individual feedback (auditory or visual) or no feedback. The participants' comments demonstrated that nonrealistic or inappropriate feedback had a negative effect on the task performance, and easily made them frustrated.

In most applications of virtual environments (VEs), like training and design evaluation, a good sense of orientation is needed in the VE. Orientation performance when moving around in the real world relies on visual as well as proprioceptive feedback. However, the navigation metaphors which are used to move around the VE often lack proprioceptive feedback. Furthermore, the visual feedback in a VE is often relatively poor compared to the visual feedback available in the real world. Therefore, we have quantified the influence of visual and proprioceptive feedback on orientation performance in VEs. Subjects were immersed in a virtual forest and were asked to turn specific angles using three navigation metaphors, differing in the kind of proprioceptive feedback which is provided (no proprioceptive feedback, vestibular feedback, and vestibular and kinesthetic feedback). The results indicate that the most accurate turn performance is found when kinesthetic feedback is present, in a condition where subjects use their legs to turn around. This indicates that incorporating this kind of feedback in navigation metaphors is quite beneficial. Orientation on only the visual component is most inaccurate, leading to progressively larger undershoots for larger angles.

Virtual environments are often used in pre-wearing training and assessment of prosthetic control abilities. Here, we developed a virtual prosthetic hand training platform for evaluation of closed-loop control of grasp force. Biorealistic controllers emulated a pair of antagonistic muscles that actuated the thumb and index fingers of the hand. Surface electromyographic (sEMG) signals from a pair of antagonistic residual muscles drove the biorealistic controllers. Tactile forces from fingertip sensors were conveyed to amputees through evoked tactile sensations (ETS) elicited at the projected finger map (PFM) areas of the stump. A forearm amputee subject participated in force tracking or holding tasks using the virtual hand with residual muscle EMGs, or the contralateral intact hand. Root-mean-square error (RMSE) was used as outcome measure of motor performance. Results in this subject showed that the biorealistic controller enabled the virtual hand to track and maintain grasping forces. The best performance in both tasks was achieved by the contralateral intact hand with visual feedback. The roles of visual or tactile feedback in force tracking or maintaining were also assessed with the virtual hand. For force holding task, hybrid tactile and visual feedback with biorealistic control had a better performance than single visual or tactile feedback in terms of RMSE, success rate, and force variability. While in the force pursuing task, tactile feedback did not seem to add visual feedback in following the target force. The study suggests that training may be required for a novel virtual hand user to perceive and integrate multiple modalities of feedback information, so as to optimize the closed-loop control ability.

The effects of the presence of others during a rowing exercise in a virtual reality environment

This paper proposes a new method of object manipulation in a Virtual Environment that uses a visual and haptic feedback based manipulation aid. If the displayed position of the object is constrained on a face of another object in the virtual environment, a user can place the virtual object at a precise position as easily as in a real environment. The visual feedback based manipulation aid constrains the motion of the virtual object as if it were moved in a real environment. Furthermore, in simultaneous correspondence with the visual constraints, the haptic feedback based manipulation aid constrains the motion of the user’s hand in the real environment. The proposed method introduces a magnetic metaphor: the user has the sensation of manipulating a virtual object on a rigid surface without using a large and powerful robot arm and the time consuming simulation of detailed physical phenomena. The properties of this manipulation aid based on two kinds of feedback are discussed. Experimental results show the effectiveness of our proposed method.

The emergence of low cost hap tic devices and the evolution of technologies for developing virtual reality environments for web have motivated the development of tools that allow the use of hap tic systems together with applications and web browsers. The availability of web hap tic systems results on the possibility of multisensory experiences for internet users. This paper analyzes technologies for integrating hap tic and web systems, exposing the features of each tool in order to find the best strategy to incorporate virtual touch in Museu3l by using the hap tic technology, so that it becomes one of the first virtual museums that makes hap tic interaction with a work of art in a web visit possible.

We show a novel encountered‐type haptic system, H‐Wall, to enable haptic feedback using a manipulator of seven degrees of freedom suitable for simulating indoor virtual reality environments, which are characterized and confined by a set of vertical walls and revolving doors. At runtime, our system tracks hand motion using a red–green–blue depth sensor and locates its configuration. Then, the robotic manipulator plans a trajectory for the end effector, attached to a rectangular rigid board, to make contact with the hand to deliver a sense of touch as long as the perceived hand contact force is substantial. The force feedback is generated in a passive sense for static walls that the rigid board, corresponding to a vertical wall, holds its position as long as the perceived hand contact force is substantial. For a revolving door, the force feedback is generated in an active sense based on impedance control. In order to address the issue of limited workspace, we also propose a new reachability map, calledper‐plane reachability map, that is optimized to answer whether passive haptic feedback can be generated by a manipulator when the user touches a vertical wall at a given orientation. We successfully demonstrate our system to provide an illusion to the user in a virtual environment with touch sensation to the surrounding environment.

In this paper we report on a pilot study conducted to identify tasks and understand scale and spatial resolution limits for the design of virtual engineering laboratories. The virtual environment is based on using the Oculus Rift SDK2 headset combined with the Leap Motion Controller (LMC) mounted on the headset. We first discuss a functional break-down structure to identify typical tasks conducted in a laboratory. From this functional breakdown basic gestures are identified to assist in virtual laboratory operations. Using this breakdown as a guide we developed two virtual environments. The first familiarizes the user with basic capabilities of the Rift and LMC. While the second environment was developed to measure scale and spatial effects of interfaces. This data will be used to guide the development of virtual laboratories environments for use in engineering degree programs and improve testing of these and other interfaces.

Some of our recent observations suggest that mental rotation may be important for reduction of motion sickness in microgravity as well as in the microgravity simulator. Therefore, we suggest that development of the ability to perform mental rotation may be important for adaptation to many virtual environments. Training virtual environment operators to perform mental rotation may enhance operator performance both by increasing their ability to "locomote in" and manipulate that environment and by reducing motion sickness associated with transitions between virtual and normal environments.

Egocentric distances in virtual environments are commonly underperceived by up to 50% of the intended distance. However, a brief period of interaction in which participants walk through the virtual environment while receiving visual feedback can dramatically improve distance judgments. Two experiments were designed to explore whether the increase in postinteraction distance judgments is due to perception-action recalibration or the rescaling of perceived space. Perception-action recalibration as a result of walking interaction should only affect action-specific distance judgments, whereas rescaling of perceived space should affect all distance judgments based on the rescaled percept. Participants made blind-walking distance judgments and verbal size judgments in response to objects in a virtual environment before and after interacting with the environment through either walking (Experiment 1) or reaching (Experiment 2). Size judgments were used to infer perceived distance under the assumption of size-distance invariance, and these served as an implicit measure of perceived distance. Preinteraction walking and size-based distance judgments indicated an underperception of egocentric distance, whereas postinteraction walking and size-based distance judgments both increased as a result of the walking interaction, indicating that walking through the virtual environment with continuous visual feedback caused rescaling of the perceived space. However, interaction with the virtual environment through reaching had no effect on either type of distance judgment, indicating that physical translation through the virtual environment may be necessary for a rescaling of perceived space. Furthermore, the size-based distance and walking distance judgments were highly correlated, even across changes in perceived distance, providing support for the size-distance invariance hypothesis.

In snowboard training, it is especially important to learn proper weight transfer and to perceive a gap between the perceived movement and the actual body movement. However, previous training systems don’t have sufficient function, and the effectiveness of each feedback on weight transfer, which includes not only the front and back but also left and right has not been reported. In this study, we propose and develop a real-time snowboard training system for a novice using visual and auditory feedback, and confirm its basic performance and effectiveness of each feedback. This system mainly consists of four components: (1) a center of pressure (COP) detection device, (2) a posture estimation device, (3) a visual feedback device, (4) an auditory feedback device. We experimented to confirm the feasibility to enhance motor learning by using the developed system and to evaluate the effectiveness of each feedback on weight transfer, which includes not only the front and back but also left and right. In conclusion, we developed a real-time snowboard training system for a novice using visual and auditory feedback and confirmed that our system could assist the proper weight transfer. Furthermore, in short-term learning, we confirmed that the effectiveness of each feedback, and visual feedback or visual and auditory feedback is the best way to enhance motor learning.

A new collision detection method for CSG-represented objects in virtual manufacturing

This paper presents a control method using an adjusting output-limiter for stable haptic rendering in a virtual environment. In a simulation of force-reflecting interaction with deformable objects in a virtual environment, a quick computation of the accurate impedance of deformable objects is rare. This is particularly true when physics-based models, such as tensor-mass models or mass-spring models, are used. The problem is aggravated if the simulation involves changes in the geometry and/or impedance of the deformation model, such as cutting or suturing. The proposed control method guarantees stable haptic interactions with deformable objects of unknown and/or varying impedance. The method is based on the time-domain passivity theorem and the two-port network model. The controller adjusts the maximum permissible force to guarantee the passivity of the haptic system at every sampling instant. The controller notes only the magnitude of the reflective force, and does not depend on properties of the employed force model. This allows the proposed control method applicable to haptic systems involving deformable objects with unknown, nonlinear, and/or time-varying impedance. Designs of the controllers are presented for impedance-type and admittance-type haptic systems. The method is also extended for multiple degrees-of-freedom.