Tactile feedback with adaptive controller for a force-reflecting haptic display. 2. Improvements and evaluation

Stroke is a worldwide health problem causing loss of sensory and motor functions. Biofeedback plays a significant role in the rehabilitation process of stroke patients where various haptic feedback methods and devices are used. Such methods and devices can greatly improve the biofeedback provided during the rehabilitation process. This work proposes a soft monolithic haptic feedback device that can be directly manufactured using a low-cost additive manufacturing technology known as fused deposition modelling (FDM) that employ soft and flexible materials with low elastic moduli. A single unit of the soft haptic feedback device proposed was analyzed using finite element modeling (FEM). The experimental mechanical deformation and force output of the device are presented. A conductive resistive material was used to directly 3D print the sensory component of the device that provides force generated by the soft haptic feedback device when it is in direct contact with the human skin. A thermoplastic poly(urethane) (TPU) was employed to 3D print the actuation unit of the soft haptic feedback device.

Auditory models that are capable of achieving human performance in tasks related to speech perception would provide a basis for realizing effective speech processing systems. Saving bits in speech coders, for example, relies on a perceptual tolerance to acoustic deviations from the original speech. Perceptual invariance to adverse signal conditions (noise, microphone and channel distortions, room reverberations) and to phonemic variability (due to nonuniqueness of articulatory gestures) may provide a basis for robust speech recognition. A state-of-the-art auditory model that simulates, in considerable detail, the outer parts of the auditory periphery up through the auditory nerve level is described. Speech information is extracted from the simulated auditory nerve firings, and used in place of the conventional input to several speech coding and recognition systems. The performance of these systems improves as a result of this replacement, but is still short of achieving human performance. The shortcomings occur, in particular, in tasks related to low bit-rate coding and to speech recognition. Since schemes for low bit-rate coding rely on signal manipulations that spread over durations of several tens of ms, and since schemes for speech recognition rely on phonemic/articulatory information that extend over similar time intervals, it is concluded that the shortcomings are due mainly to perceptually related rules over durations of 50-100 ms. These observations suggest a need for a study aimed at understanding how auditory nerve activity is integrated over time intervals of that duration. The author discusses preliminary experimental results that confirm human usage of such integration, with different integration rules for different time-frequency regions depending on the phoneme-discrimination task. >

Interaction with computers and virtual environments is almost totally based on visual displays, along with auditory feedback. Somesthetic feedback, a necessary feature in all other skilled activity, is usually absent. Haptic interfaces attempt to solve this problem by coupling the human sense of touch with a computer-generated world. Haptics is the science of applying tactile sensation and kinesthesis to interaction with computer applications. By using special input/output devices (joysticks, data gloves, or other devices), users can receive feedback from computer applications in the form of sensations felt in the hand or other parts of the body. These sensations include forces, vibrations and/or motions. The aim of this tutorial is to provide participants with an overview of the technical aspects of haptic feedback and provide information about different haptic feedback devices and applications of these devices using virtual reality and computer simulations for motor rehabilitation, learning and teleoperation. The tutorial will include opportunities for hands-on practice with haptic devices and different environments. The haptic devices available for hands-on practice will include the Novint Falcon, and the SensAble OMNI devices.

Modeling Human Performance in Complex Tracking Tasks with a Two-Level Control Model

Haptic feedback is important to improve the operability of virtual objects. However, AR systems with haptic feedback are still not many. Moreover, users cannot perform tasks that need large workspaces in those systems because of physical constraints from haptic feedback devices. In this paper, we combined a wearable haptic device and a marker-less AR technology to develop an AR system in which users can perform activities that need large workspaces. Differences of the operability of virtual objects in an assembly task of large machines are examined to evaluate effects of the haptic feedback in this system. The result indicates that haptic feedback is effective and users can perform activities that need large space in this system.

This paper presents the development of a compact tactile display and its integration in teleoperation. The system's operation is based on the display of surface shape to an area of the fingertip through a 4 × 4 array of tactors moving perpendicularly to the skin surface. The tactors are spring loaded and are actuated remotely by dc motors through a flexible tendon transmission. This novel implementation of conventional actuation principles achieves a compact design with superior performance compared to devices of a similar footprint, demonstrating an excellent combination of tactor spatiotemporal resolution, force, and amplitude. The display's ergonomic design and high performance make it suitable for integration on haptic devices for tactile feedback in VR and in Teleoperation. This paper presents the design, control, and performance of the tactile display and of the transmission system. It also demonstrates its integration on an Omega7 force feedback device for the teleoperation of an LWR KUKA manipulator. An experiment is presented where users teleoperated the stylus of the robot in a 3D contour following task with and without tactile feedback. In this experiment, force feedback from the slave is fused with model-based local tactile feedback. Subjects' performances indicate an improvement in teleoperation when both tactile and force feedback are present.

Virtual environments are often used in pre-wearing training and assessment of prosthetic control abilities. Here, we developed a virtual prosthetic hand training platform for evaluation of closed-loop control of grasp force. Biorealistic controllers emulated a pair of antagonistic muscles that actuated the thumb and index fingers of the hand. Surface electromyographic (sEMG) signals from a pair of antagonistic residual muscles drove the biorealistic controllers. Tactile forces from fingertip sensors were conveyed to amputees through evoked tactile sensations (ETS) elicited at the projected finger map (PFM) areas of the stump. A forearm amputee subject participated in force tracking or holding tasks using the virtual hand with residual muscle EMGs, or the contralateral intact hand. Root-mean-square error (RMSE) was used as outcome measure of motor performance. Results in this subject showed that the biorealistic controller enabled the virtual hand to track and maintain grasping forces. The best performance in both tasks was achieved by the contralateral intact hand with visual feedback. The roles of visual or tactile feedback in force tracking or maintaining were also assessed with the virtual hand. For force holding task, hybrid tactile and visual feedback with biorealistic control had a better performance than single visual or tactile feedback in terms of RMSE, success rate, and force variability. While in the force pursuing task, tactile feedback did not seem to add visual feedback in following the target force. The study suggests that training may be required for a novel virtual hand user to perceive and integrate multiple modalities of feedback information, so as to optimize the closed-loop control ability.

In a collaborative scenario, the communication between humans and robots is a fundamental aspect to achieve good efficiency and ergonomics in the task execution. A lot of research has been made related to enabling a robot system to understand and predict human behaviour, allowing the robot to adapt its motion to avoid collisions with human workers. Assuming the production task has a high degree of variability, the robot’s movements can be difficult to predict, leading to a feeling of anxiety in the worker when the robot changes its trajectory and approaches since the worker has no information about the planned movement of the robot. Additionally, without information about the robot’s movement, the human worker cannot effectively plan own activity without forcing the robot to constantly replan its movement. We propose a novel approach to communicating the robot’s intentions to a human worker. The improvement to the collaboration is presented by introducing haptic feedback devices, whose task is to notify the human worker about the currently planned robot’s trajectory and changes in its status. In order to verify the effectiveness of the developed human-machine interface in the conditions of a shared collaborative workspace, a user study was designed and conducted among 16 participants, whose objective was to accurately recognise the goal position of the robot during its movement. Data collected during the experiment included both objective and subjective parameters. Statistically significant results of the experiment indicated that all the participants could improve their task completion time by over 45% and generally were more subjectively satisfied when completing the task with equipped haptic feedback devices. The results also suggest the usefulness of the developed notification system since it improved users’ awareness about the motion plan of the robot.

Soft robotics and pneumatic actuation present opportunities for lightweight wearable haptic devices that provide distributed touch feedback to the skin. Ideally, such devices would be easily donned and doffed, since permanent coverage of a large area of the skin is undesirable. Here we present the design and evaluation of a concept device called HapWRAP: a growing haptic device constructed from flexible low density polyethylene. Controlled air flow through tubes and pouches allows HapWRAP to grow out of a compact housing unit and provide a combination of directional and force feedback to a user. When activated, HapWRAP grows up and around the forearm; its loops form a temporary sleeve. After growth, pneumatic actuators inflate and deflate to stimulate mechanoreceptors in the skin at distinguishable locations. This paper describes the design and manufacturing of HapWRAP, reports its performance metrics, and tests its suitability as a haptic feedback device. Participants were able to interpret force and direction cues from HapWRAP with 92.5% accuracy. These findings suggest that HapWRAP can be successfully used for applications where both force and direction cues are necessary.

The tactile sensation is the human interact with machines in the form of felt sensations in the hand or other parts of body skin. This paper presents a film type actuator for haptic feedback devices, based on electrostatic actuation of plasticized cellulose acetate stacked membranes actuator, which is fabricated and evaluated for suitability of haptic feedback devices. The stacked membranes actuator shows many promising properties such as high transparency, lightweight, wide range of actuation frequency and high vibration acceleration for instance. The actuator structure shows intense vibration acceleration due to several layers of stacked membranes associated with the intensified electrostatic attraction force between chargeable cellulose acetate membranes. Experiment for measuring vibrational acceleration was conducted over a wide frequency range and actuation voltage to prove actuator’s great potential application as tactile actuator of haptic feedback devices. In addition, the operating principle, fabrication method and performance measurements are explained in details.

Haptic technologies and applications have received enormous attention in the last decade. The incorporation of haptic modality into multimedia applications adds excitement and enjoyment to an application. It also adds a more natural feel to multimedia applications, that otherwise would be limited to vision and audition, by engaging as well the user's sense of touch, giving a more intrinsic feel essential for ambient intelligent applications. However, the improvement of an application's Quality of Experience (QoE) by the addition of haptic feedback is still not completely understood. The research presented in this paper focuses on the effect of haptic feedback and what it potentially adds to the experience of the user as opposed to the traditional visual and auditory feedback. In essence, it investigates certain issues regarding stylus-based haptic education applications and haptic-enhanced entertainment videos. To this end, we used two haptic applications: the haptic handwriting learning tool to experiment with force feedback haptic interaction and the tactile YouTube application for tactile haptic feedback. In both applications, our analysis shows that the addition of haptic feedback will increase the QoE in the absence of fatigue or discomfort for this category of applications. This implies that the incorporation of haptic modality (both force feedback as well as tactile feedback) has positively contributed to the overall QoE for the users.

Increasingly higher demands are being made on the capacity‐limited cognitive capabilities of human operators as they strive to maintain situation awareness (i.e., understanding “what is going on”) and performance in complex tasks. In the current study we asked whether: (a) training administered via a mobile phone‐based app could improve multitasking and (b) improved multitasking in the app would generalize to improved performance and situation awareness in a simulated air traffic control task (ATC). Participants completed the ATC task before and after multiple sessions of app‐based multitasking training or control training. Multitasking on the app improved across training sessions. However, this did not lead to improved performance or situation awareness, or workload reduction, relative to control training on the ATC task. These outcomes indicate that app‐based multitasking training based on repetition of a single training task will not necessarily yield generalizable benefits to human performance in other complex dynamic tasks.

A key factor for reliable object manipulation is the tactile information provided by the skin of our hands. As this sensory information is so essential in our daily life it should also be provided during teleoperation of robotic devices or in the control of myoelectric prostheses. It is well-known that feeding back the tactile information to the user can lead to a more natural and intuitive control of robotic devices. However, in some applications it is difficult to use the hands as natural feedback channels since they may already be overloaded with other tasks or, e.g., in case of hand prostheses not accessible at all. Many alternatives for tactile feedback to the human hand have already been investigated. In particular, one approach shows that humans can integrate uni-directional (normal) force feedback at the toe into their sensorimotor-control loop. Extending this work, we investigate the human's capability to discriminate spatial forces at the bare front side of their toe. A state-of-the-art haptic feedback device was used to apply forces with three different amplitudes—2 N, 5 N, and 8 N—to subjects' right big toes. During the experiments, different force stimuli were presented, i.e., direction of the applied force was changed, such that tangential components occured. In total the four directions up (distal), down (proximal), left (medial), and right (lateral) were tested. The proportion of the tangential force was varied corresponding to a directional change of 5° to 25° with respect to the normal force. Given these force stimuli, the subjects' task was to identify the direction of the force change. We found the amplitude of the force as well as the proportion of tangential forces to have a significant influence on the success rate. Furthermore, the direction right showed a significantly different successrate from all other directions. The stimuli with a force amplitude of 8 N achieved success rates over 89% in all directions. The results of the user study provide evidence that the subjects were able to discriminate spatial forces at their toe within defined force amplitudes and tangential proportion.

The haptic interface or force feedback device improves the quality of human computer interaction by accommodating the sense of touch in a computer simulation. A haptic device enables a two way communication between the user and the host computer. The applications of force feedback device cover a wide area such as in medical training, systems for the visually impaired, engineering research, and creative art-work. The role of haptic feedback varies depending on the applications in question and the devices used. This includes haptic as a teaching tool, and a communication aid. Despite the benefits of haptic feedback, its deployment to support interactions in a virtual environment is still slow. This is due to the high cost of the haptic devices in the market. We address this issue by presenting a conceptual framework in designing and implementing a cost-effective haptic device; building on similar concepts obtained from existing technologies. The device is developed using those gadgets that are mainly available in-house. It is projected that haptic feedback generated from the device could benefit teaching abstract concepts to students.

The use of haptic technologies has recently become immensely essential in Human-Computer Interaction to improve user experience and performance. With the introduction of tactile feedback on a touchscreen device, commonly known as surface haptics, several applications and interaction paradigms have become a reality. However, the effects of tactile feedback on the preference of 2D images in visuo-tactile exploration task on touchscreen devices remain largely unknown. In this study, we investigated differences of preference score (the tendency of participants to like/dislike a 2D image based on its visual and tactile properties), reach time, interaction time, and response time under four conditions of feedback: no tactile feedback, high-quality of tactile information (sharp tactile texture), low-quality of tactile information (blurred tactile texture), and incorrect tactile information (mismatch tactile texture). The tactile feedback is rendered in the form of roughness that is simulated by modulating the friction between the finger and the surface and is derived from the 2D image. Thirty-six participants completed visuo-tactile exploration tasks for a total of 36 trials (3 2D images × 4 tactile textures × 3 repetitions). Results showed that the presence of tactile feedback enhanced users’ preference (tactile feedback conditions were rated significantly higher than the no tactile feedback condition for preference regardless of the quality/correctness of tactile feedback). This finding is also supported through results from self-reporting where 88.89% of participants preferred to experience the 2D image with tactile feedback. Additionally, the presence of tactile feedback resulted in significantly larger interaction time and response time compared to the no tactile feedback condition. Furthermore, the quality and correctness of tactile information significantly impacted the preference rating (sharp tactile textures were rated statistically higher than blurred tactile and mismatched tactile textures). All of these findings demonstrate that tactile feedback plays a crucial role in users’ preference and thus motivates further the development of surface haptic technologies.