Ferro tale

In this study we compared tactile and visual feedbacks for the motor imagery-based brain–computer interface (BCI) in five healthy subjects. A vertical green bar from the center of the fixing cross to the edge of the screen was used as visual feedback. Vibration motors that were placed on the forearms of the right and the left hands and on the back of the subject’s neck were used as tactile feedback. A vibration signal was used to confirm the correct classification of the EEG patterns of the motor imagery of right and left hand movements and the rest task. The accuracy of recognition in the classification of the three states (right hand movement, left hand movement, and rest) in the BCI without feedback exceeded the random level (33% for the three states) for all the subjects and was rather high (67.8% ± 13.4% (mean ± standard deviation)). Including the visual and tactile feedback in the BCI did not significantly change the mean accuracy of recognition of mental states for all the subjects (70.5% ± 14.8% for the visual feedback and 65.9% ± 12.4% for the tactile feedback). The analysis of the dynamics of the movement imagery skill in BCI users with the tactile and visual feedback showed no significant differences between these types of feedback. Thus, it has been found that the tactile feedback can be used in the motor imagery-based BCI instead of the commonly used visual feedback, which greatly expands the possibilities of the practical application of the BCI.

Virtual environments are often used in pre-wearing training and assessment of prosthetic control abilities. Here, we developed a virtual prosthetic hand training platform for evaluation of closed-loop control of grasp force. Biorealistic controllers emulated a pair of antagonistic muscles that actuated the thumb and index fingers of the hand. Surface electromyographic (sEMG) signals from a pair of antagonistic residual muscles drove the biorealistic controllers. Tactile forces from fingertip sensors were conveyed to amputees through evoked tactile sensations (ETS) elicited at the projected finger map (PFM) areas of the stump. A forearm amputee subject participated in force tracking or holding tasks using the virtual hand with residual muscle EMGs, or the contralateral intact hand. Root-mean-square error (RMSE) was used as outcome measure of motor performance. Results in this subject showed that the biorealistic controller enabled the virtual hand to track and maintain grasping forces. The best performance in both tasks was achieved by the contralateral intact hand with visual feedback. The roles of visual or tactile feedback in force tracking or maintaining were also assessed with the virtual hand. For force holding task, hybrid tactile and visual feedback with biorealistic control had a better performance than single visual or tactile feedback in terms of RMSE, success rate, and force variability. While in the force pursuing task, tactile feedback did not seem to add visual feedback in following the target force. The study suggests that training may be required for a novel virtual hand user to perceive and integrate multiple modalities of feedback information, so as to optimize the closed-loop control ability.

This paper describes development of an activity based, multimodal balance rehabilitation training device. Various sensors can be used, including a force plate, inertial sensors, and depth sensing cameras, and various combinations of visual, auditory and tactile feedback can be configured depending on the rehabilitation task and activity. Tactile feedback is presented via a lightweight belt that is worn on the torso. Generally, visual feedback is only needed at the start of rehabilitation training (task orientation) while tactile feedback may be used to augment balance control. Tactile feedback can be configured as a cue that certain movement targets or limits have been reached or as an immediate indicator of the variance in postural sway. Tactile feedback allows the subject to naturally concentrate on the functional rehabilitation task, and is less reliant on visual or verbal cues.

Virtual reality can help to learn basic laparoscopic tasks. However, no haptic feedback, which alerts for tissue slippage, is provided by most simulators, although, it might be of influence for the decrease of errors. This study explored whether visual or tactile feedback can be used to alert the surgeon of tissue slippage. Twenty-four participants performed a laparoscopic grasping task and where provided with either visual or tactile feedback about tissue slippage. The reaction time with the visual feedback was compared to the reaction time with tactile feedback signal. The results showed that when tissue slippage is simulated, tactile feedback shows significant faster reaction times (269ms) than visual feedback signals (398ms).

Fitts’ law models the relationship between amplitude, precision, and speed of rapid movements. It is widely used to quantify performance in pointing tasks, study human-computer interaction, and generally to understand perceptual-motor information processes, including research to model performance in isometric force production tasks. Applying Fitts’ law to an isometric grip force task would allow for quantifying grasp performance in rehabilitative medicine and may aid research on prosthetic control and design. We examined whether Fitts’ law would hold when participants attempted to accurately produce their intended force output while grasping a manipulandum when presented with images of various everyday objects (we termed this the implicit task). Although our main interest was the implicit task, to benchmark it and establish validity, we examined performance against a more standard visual feedback condition via a digital force-feedback meter on a video monitor (explicit task). Next, we progressed from visual force feedback with force meter targets to the same targets without visual force feedback (operating largely on feedforward control with tactile feedback). This provided an opportunity to see if Fitts’ law would hold without vision, and allowed us to progress toward the more naturalistic implicit task (which does not include visual feedback). Finally, we changed the nature of the targets from requiring explicit force values presented as arrows on a force-feedback meter (explicit targets) to the more naturalistic and intuitive target forces implied by images of objects (implicit targets). With visual force feedback the relation between task difficulty and the time to produce the target grip force was predicted by Fitts’ law (average r2 = 0.82). Without vision, average grip force scaled accurately although force variability was insensitive to the target presented. In contrast, images of everyday objects generated more reliable grip forces without the visualized force meter. In sum, population means were well-described by Fitts’ law for explicit targets with vision (r2 = 0.96) and implicit targets (r2 = 0.89), but not as well-described for explicit targets without vision (r2 = 0.54). Implicit targets should provide a realistic see-object-squeeze-object test using Fitts’ law to quantify the relative speed-accuracy relationship of any given grasper.

Humans systematically adapt their strategies for closed-loop control based on visual feedback according to the dynamics of the system. Tactile feedback is a key element in many human–machine interfaces, but it is not known if and how well human control adapts to changes in system dynamics when information about the system state is provided using this type of feedback. In this study, 11 participants tracked a pseudorandom trajectory with a virtual, position- or velocity-controlled plant using a joystick. Visual or electrotactile feedback provided the instantaneous error between the target and generated trajectory. Frequency-domain system identification indicated that human control adapted in similar ways to the different control modes (i.e., position/velocity control) for both feedback modalities. For the plant dynamics modeled as gain and integrator, the human controller behaved as a low-pass filter and gain, respectively (under the assumption of quasi-linear behavior). However, while tracking quality was largely similar for both control modes with visual feedback, velocity control enabled substantially worse control with electrotactile feedback compared to position control. Furthermore, for both control modes, the crossover frequency of open-loop transfer functions was lower for electrotactile feedback (0.9 and 1.1 rad/s) than for visual feedback (1.5 and 1.7 rad/s) indicating limited control bandwidth. To summarize, closed-loop control based on electrotactile feedback enables natural adaptations in human control strategy, which is encouraging for tactile feedback-controlled human–machine interfaces, but the lower control bandwidth and lower tracking quality with velocity control may impose functional limitations.

Haptic technologies and applications have received enormous attention in the last decade. The incorporation of haptic modality into multimedia applications adds excitement and enjoyment to an application. It also adds a more natural feel to multimedia applications, that otherwise would be limited to vision and audition, by engaging as well the user's sense of touch, giving a more intrinsic feel essential for ambient intelligent applications. However, the improvement of an application's Quality of Experience (QoE) by the addition of haptic feedback is still not completely understood. The research presented in this paper focuses on the effect of haptic feedback and what it potentially adds to the experience of the user as opposed to the traditional visual and auditory feedback. In essence, it investigates certain issues regarding stylus-based haptic education applications and haptic-enhanced entertainment videos. To this end, we used two haptic applications: the haptic handwriting learning tool to experiment with force feedback haptic interaction and the tactile YouTube application for tactile haptic feedback. In both applications, our analysis shows that the addition of haptic feedback will increase the QoE in the absence of fatigue or discomfort for this category of applications. This implies that the incorporation of haptic modality (both force feedback as well as tactile feedback) has positively contributed to the overall QoE for the users.

In the present study, the function of the somatosensory feedback system in postural control was investigated. For the sake of simplicity, the present study considered only balancing in the anteroposterior direction using the ankle strategy, in which the ankle moment is mainly used to maintain balance. To suppress the vestibular and visual feedback paths, a subject stood on a force-measuring platform with a fixed back support. Because the subject's body was immovable under these conditions, the subject controlled a computer model that simulated the subject's load at the ankles. Information about the sway angle of the model was fed through the somatosensory feedback path. Frequency response functions of the ankle moment in response to the sway angle were calculated. The experimental results suggest that the human somatosensory feedback system has derivative characteristics and, consequently, can maintain an upright posture by itself. The results were compared with those of previous studies on vestibular and visual feedback systems. The comparison reveals that subject-to-subject variance in the somatosensory system is significantly smaller than that in the other systems. This may indicate that the somatosensory feedback is the most automatic of the systems and plays a dominant role when a subject maintains an upright posture using the ankle strategy.

We investigated the role of visual feedback in human upright posture control. To make the role of vision clear, we compared the visual, vestibular and somatosensory feedback systems in their ability to perceive movement and suppress short- and long-term sway. First, we measured thresholds for the perception of movement Then effects of each feedback system to suppress short- and long-term sway were analyzed through posture control experiments, in which the available sensory input was limited. The visual and somatosensory thresholds were small ( 0.5 degree). The results of the posture control experiments suggested that the visual system contributed to suppression of sway in the frequency range below 0.4 Hz and the system had a minor effect in frequencies above 0.5 Hz. These results support our previous finding that the visual system does not allow a normal subject to maintain an upright posture by itself. The somatosensory system, on the other hand, suppressed body sway around 0.2 and 0.9 Hz. The higher frequency range might enable the system to maintain an upright posture by itself.

Tactile feedback is pivotal for grasping and manipulation in humans. Providing functionally effective sensory feedback to prostheses users is an open challenge. Past paradigms were mostly based on vibro- or electrotactile stimulations. However, the tactile sensitivity on the targeted body parts (usually the forearm) is greatly less than that of the hand/fingertips, restricting the amount of information that can be provided through this channel. Visual feedback is the most investigated technique in motor learning studies, where it showed positive effects in learning both simple and complex tasks; however, it was not exploited in prosthetics due to technological limitations. Here, we investigated if visual information provided in the form of augmented reality (AR) feedback can be integrated by able-bodied participants in their sensorimotor control of a pick-and-lift task while controlling a robotic hand. For this purpose, we provided visual continuous feedback related to grip force and hand closure to the participants. Each variable was mapped to the length of one of the two ellipse axes visualized on the screen of wearable single-eye display AR glasses. We observed changes in behavior when subtle (i.e., not announced to the participants) manipulation of the AR feedback was introduced, which indicated that the participants integrated the artificial feedback within the sensorimotor control of the task. These results demonstrate that it is possible to deliver effective information through AR feedback in a compact and wearable fashion. This feedback modality may be exploited for delivering sensory feedback to amputees in a clinical scenario.

Tactile feedback plays an important role in hand manipulation, especially in the grasping process which is one of the major functions of the hand. However, few commercially available prosthetic hands or hand motor function rehabilitation systems are equipped with tactile feedback. The absence of suitable tactile feedback modules leads to an inferior rehabilitation performance with a large burden on user training and compromised usability. Thus, it is challenging and essential to integrate a proper tactile feedback module with the existing hand rehabilitation systems to achieve a better control performance and accelerate the rehabilitation process. This paper focuses on the implementation and evaluation of the electrotactile feedback (EF) enhanced rehabilitation system. A virtual hand rehabilitation platform is proposed comprising an surface electromyography (sEMG) acquisition module, an electrotactile stimulation module, a virtual environment with sEMG-driven humanlike hand and numerical feedbacks of grasping force and fingertip deformation, where a closed-loop control is formed. Three different feedback conditions including visual feedback (VF), EF, and no feedback (NF) are compared based on the proposed platform. Experiments were conducted on 10 able-bodied subjects, and multiple quantitative metrics for the rehabilitation performance evaluation including training burden estimation and success rate (SR) of tasks were adopted. Results indicate that the integration of EF is helpful to both reduce the rehabilitation duration and improve the virtual grasping SR in comparison with the NF condition while possessing a better practicality over VF. Note to Practitioners —This paper is motivated by the problem of hand grasp control for rehabilitation purposes, but it also applies to other hand motor function rehabilitation process. Existing hand motor function rehabilitation approaches generally lack a proper feedback and rely on the heavy burden during user training and the users experience. This paper suggests incorporating EF to improve the efficiency and efficacy of the rehabilitation process. The electrical stimulation is driven by the myoelectric-sensing-based force estimation and encoded in a manual scheme to fit each individual involved. In our work, a virtual hand rehabilitation platform is implemented to verify the feasibility of EF in reducing the burden of user training and improving the rehabilitation performance, which allows the expanding of the current system into a broader spectrum of motor function rehabilitation applications. Experiments on able-bodied subjects suggest that the EF in the proposed virtual hand rehabilitation platform is feasible, but it has not been tested on the limb-impaired subjects and confined to a manual encoding of electrical stimulation. In the future research, the design of a general EF enhanced hand rehabilitation platform with a standardized stimulation parameter optimization will be addressed and further validated on the subjects with limb impairments and amputation.

Vitreo-retinal surgery encompasses the surgical procedures performed on the vitreous humor and the retina. A procedure typically consists of the removal of the vitreous humor, the peeling of a membrane and/or the repair of a retinal detachment. Operations are performed with needle shaped instruments which enter the eye through surgeon made scleral openings. An instrument is moved by hand in four degrees of freedom (three rotations and one translation) through this opening. Two rotations (? and ? ) are for a lateral instrument tip movement. The other two DoFs (z and ?) are the translation and rotation along the instrument axis. Actuation of for example a forceps can be considered as a fifth DoF. Characteristically, the manipulation of delicate, micrometer range thick intraocular tissue is required. Today, eye surgery is performed with a maximum of two instruments simultaneously. The surgeon relies on visual feedback only, since instrument forces are below the human detection limit. A microscope provides the visual feedback. It forces the surgeon to work in a static and non ergonomic body posture. Although the surgeon’s proficiency improves throughout his career, hand tremor may become a problem around his mid-fifties. Robotically assisted surgery with a master-slave system enhances dexterity. The slave with instrument manipulators is placed over the eye. The surgeon controls the instrument manipulators via haptic interfaces at the master. The master and slave are connected by electronic hardware and control software. Implementation of tremor filtering in the control software and downscaling of the hand motion allow prolongation of the surgeon’s career. Furthermore, it becomes possible to do tasks like intraocular cannulation which can not be done by manually performed surgery. This thesis focusses on the master console. Eye surgery procedures are observed in the operating room of different hospitals to gain insight in the requirements for the master. The master console as designed has an adjustable frame, a 3D display and two haptic interfaces with a coarse adjustment arm each. The console is mounted at the head of the operating table and is combined with the slave. It is compact, easy to place and allows the surgeon to have a direct view on and a physical contact with the patient. Furthermore, it fits in today’s manual surgery arrangement. Each haptic interface has the same five degrees of freedom as the instrument inside the eye. Through these interfaces, the surgeon can feel the augmented instrument forces. Downscaling of the hand motion results in a more accurate instrument movement compared to manually performed surgery. Together with the visual feedback, it is like the surgeon grasps the instrument near the tip inside the eye. The similarity between hand motion and motion of the instrument tip as seen on the display results in an intuitive manipulation. Pre-adjustment of the interface is done via the coarse adjustment arm. Mode switching enables to control three or more instruments manipulators with only two interfaces. Two one degree of freedom master-slave systems with force feedback are built to derive the requirements for the haptic interface. Hardware in the loop testing provides valuable insights and shows the possibility of force feedback without the use of force sensors. Two five DoF haptic interfaces are realized for bimanual operation. Each DoF has a position encoder and a force feedback motor. A correct representation of the upscaled instrument forces is only possible if the disturbance forces are low. Actuators are therefore mounted to the fixed world or in the neighborhood of the pivoting point for a low contribution to the inertia. The use of direct drive for ' and and low geared, backdriveable transmissions for the other three DoFs gives a minimum of friction. Disturbance forces are further minimized by a proper cable layout and actuator-amplifier combinations without torque ripple. The similarity in DoFs between vitreo-retinal eye surgery and minimally invasive surgery (MIS) enables the system to be used for MIS as well. Experiments in combination with a slave robot for laparoscopic and thoracoscopic surgery show that an instrument can be manipulated in a comfortable and intuitive way. User experience of surgeons and others is utilized to improve the haptic interface further. A parallel instead of a serial actuation concept for the ' and DoFs reduces the inertia, eliminates the flexible cable connection between frame and motor and allows that the heat of the motor is transferred directly to the frame. A newly designed z-?? module combines the actuation and suspension of the hand held part of the interface and has a three times larger z range than in the first design of the haptic interface.

The neuroimaging of humans using 7T magnetic resonance imaging (MRI) has been conducted using phased array (PA) coils with different numbers of receiving channels. PA coils with a high number of channels may offer parallel imaging (PI) with a high reduction (R)-factor, which is enabled via under-sampling and coil geometry (g) factor, increasing the radiofrequency signal sensitivity provided by a small coil. The goals of this study were to assess and validate the coil performance of PA coils with different numbers of receiver (Rx)-channels in and to propose the coil selection guidelines by visualizing 7T brain images. The combined magnetic flux density (||B1||) distributions of four configurations of PA coils—4-, 8-, 12-, and 16-channel Rx-only mode under the local transmit (Tx) mode of birdcage coils—were evaluated using electromagnetic (EM) calculations. These four configurations of PA coils and a local Tx coil were designed and built for a 7T MRI experiment. For 7T brain imaging experiments, all PA coils with (w/) and without (w/o) R-factors were compared in terms of signal-to-noise ratio (SNR) and spatial noise variation (SNV). EM simulation results clearly demonstrated that PA coils with a high number of Rx channels showed more homogeneously distributed ||B1|| fields than a PA coils with a low number of Rx coils. The results of this study demonstrate that a collection of smaller surface coils can contribute to high RF signal sensitivity in terms of the anatomical coverage of the brain and may facilitate PI. With further improvement in coil technology, researchers and clinicians will be provided with PA coils with different numbers of channels, which can ensure the optimum SNR and PI benefits for 7T brain MR imaging.

Self-face recognition shares brain regions active during proprioceptive illusion in the right inferior fronto-parietal superior longitudinal fasciculus III network

Tactile feedback has the capability of reducing the workload on the visual channel, during visual feedback in brain-computer interfaces (BCIs). It is requisite to analyse the brain signals corresponding to the tactile stimulations. This work is aimed at analysing the brain signals while the users are vibrotactually stimulated. The brain signals are acquired non-invasively by electroencephalography (EEG), while brushless coin-type vibration motors are actuated in particular patterns to convey the object shape information on subjects' skin surface in form of vibrations. The acquired EEG signals are pre-processed to eliminate the effect of various types of noises and to extract the EEG signals corresponding to relevant frequency bands. Adaptive autoregressive (AAR) parameters are extracted from the pre-processed EEG signals and are finally classified by Naive Bayesian $(NB)$ approach, in order to recognize the vibratotactually stimulated object shapes from brain signals. In addition to the classifier output, subjects' verbal responses about the object shape they perceived are also noted for validation. Three successive sessions of shape recognition from vibrotactile pattern show an improvement in EEG classification accuracy from 63.75% to 74.37%, and also depicted learning of the stimulus from subjects' psychological response which is observed to increase from 75% to 95%. This observation substantiates the learning of vibrotactile stimulation in user over the sessions which in turn increases the system efficacy.