Bone Conduction as Sensory Feedback Interface: A Preliminary Study

In this paper, we discuss the potential contribution of affective touch to the user experience and robot performance in human-robot interaction, with an in-depth look into upper-limb prosthesis use as a well-suited example. Research on providing haptic feedback in human-robot interaction has worked to relay discriminative information during functional activities of daily living, like grasping a cup of tea. However, this approach neglects to recognize the affective information our bodies give and receive during social activities of daily living, like shaking hands. The discussion covers the emotional dimensions of affective touch and its role in conveying distinct emotions. In this work, we provide a human needs-centered approach to human-robot interaction design and argue for an equal emphasis to be placed on providing affective haptic feedback channels to meet the social tactile needs and interactions of human agents. We suggest incorporating affective touch to enhance user experience when interacting with and through semi-autonomous systems such as prosthetic limbs, particularly in fostering trust. Real-time analysis of trust as a dynamic phenomenon can pave the way towards adaptive shared autonomy strategies and consequently enhance the acceptance of prosthetic limbs. Here we highlight certain feasibility considerations, emphasizing practical designs and multi-sensory approaches for the effective implementation of affective touch interfaces.

Osseoperception is the sensation arising from the mechanical stimulation of a bone-anchored prosthesis. Here we show that not only touch, but also hearing is involved in this phenomenon. Using mechanical vibrations ranging from 0.1 to 6 kHz, we performed four psychophysical measures (perception threshold, sensation discrimination, frequency discrimination and reaction time) on 12 upper and lower limb amputees and found that subjects: consistently reported perceiving a sound when the stimulus was delivered at frequencies equal to or above 400 Hz; were able to discriminate frequency differences between stimuli delivered at high stimulation frequencies (~1500 Hz); improved their reaction time for bimodal stimuli (i.e. when both vibration and sound were perceived). Our results demonstrate that osseoperception is a multisensory perception, which can explain the improved environment perception of bone-anchored prosthesis users. This phenomenon might be exploited in novel prosthetic devices to enhance their control, thus ultimately improving the amputees’ quality of life.

Simulations of low-frequency vibration pattern at the inner ear for activation of the vestibular system.

BackgroundConsistency is one of the most important concerns to convey stable artificially induced sensory feedback. However, the constancy of perceived sensations cannot be guaranteed, as the artificially evoked sensation is a function of the interaction of stimulation parameters. The hypothesis of this study is that the selected stimulation parameters in multi-electrode cutaneous stimulation have significant impacts on the perception threshold.MethodsThe investigated parameters included the stimulated location, the number of active electrodes, the number of pulses, and the interleaved time between a pair of electrodes. Biphasic, rectangular pulses were applied via five surface electrodes placed on the forearm of 12 healthy subjects.ResultsOur main findings were: 1) the perception thresholds at the five stimulated locations were significantly different (p < 0.0001), 2) dual-channel simultaneous stimulation lowered the perception thresholds and led to smaller variance in perception thresholds compared to single-channel stimulation, 3) the perception threshold was inversely related to the number of pulses, and 4) the perception threshold increased with increasing interleaved time when the interleaved time between two electrodes was below 500 μs.ConclusionsTo maintain a consistent perception threshold, our findings indicate that dual-channel simultaneous stimulation with at least five pulses should be used, and that the interleaved time between two electrodes should be longer than 500 μs. We believe that these findings have implications for design of reliable sensory feedback codes.

Conclusions: The present study showed that elevation of bone-conduction (BC) thresholds at low frequencies might be a characteristic audiometric finding in cases with otitis media with effusion (OME) with high pepsinogen (PG) concentrations. Objective: The objective of this study was to investigate whether there is any characteristic audiometric finding in adult cases with otitis media with high PG compared to those with low PG. Methods: Twenty-four adult patients with unilateral OME of undetermined etiology and high PG concentrations (> 500 ng/ml) in their middle ear effusions (high PG group) were selected. The air-conduction and BC thresholds of pure tone audiometry were compared between the affected and healthy ears. Results were compared to those in 23 patients with low PG concentrations (< 50 ng/ml; low PG group). Results: The average BC difference in the threshold at 0.25 kHz between the affected ear and the healthy ear was significantly higher in the high PG group than in the low PG group, with a significantly higher proportion of patients in the high PG group having BC thresholds at 0.25 kHz in the affected ear that were ≥ 15 dB higher than in the healthy ear.

Chapter 5 - Sensory feedback for upper limb prostheses

Electrotactile stimulation has been suggested as a modality for providing sensory feedback in upper limb prostheses. This study investigates the multiday variability of subdermal and surface stimulation. Electrical stimulation was delivered using either surface or fine wire electrodes placed right under the skin in eight amputees for seven consecutive days. The variability of psychophysical measurements, including detection threshold (DT), pain threshold (PT), dynamic range (DR), just noticeable difference (JND), Weber fraction (WF) and quality of evoked sensations, was evaluated using the coefficient of variation (CoV). In addition, the systematic change in the mean of the parameters across days was assessed in both stimulation modalities. In the case of DT, PT, DR, and perceived intensity at 100 Hz, the CoV of surface stimulation was significantly smaller than that of subdermal stimulation. Only PT showed a significant systematic change in the mean value across days for both modalities. The outcome of this study has implications for the choice of modality in delivering sensory feedback, though the significance of the quantified variability needs to be evaluated using usability tests with user feedback.

ABSTRACTDespite progress in research and media attention on active upper limb prostheses, presently the most common commercial upper limb prosthetic devices are not fundamentally different from solutions offered almost one century ago. Limited information transfer for both control and sensory-motor integration and challenges in socket technology have been major obstacles. By analysing the present state-of-the-art and academic achievements, we provide our opinion on the future of upper limb prostheses. We believe that surgical procedures for muscle reinnervation and osseointegration will become increasingly clinically relevant; muscle electrical signals will remain the main clinical means for prosthetic control; and chronic electrode implants, first in muscles (control), then in nerves (sensory feedback), will become viable clinical solutions. After decades of suspended clinically relevant progress, it is foreseeable that a new generation of upper limb prostheses will enter the market in the near future based on such advances, thereby offering substantial clinical benefit for patients.

BackgroundRecent studies have shown that neural stimulation can be used to provide artificial sensory feedback to amputees eliciting sensations referred on the amputated hand. The temporal properties of the neural stimulation modulate aspects of evoked sensations that can be exploited in a bidirectional hand prosthesis.MethodsWe previously collected evidence that the derivative of the amplitude of the stimulation (intra-digit temporal dynamics) allows subjects to recognize object compliance and that the time delay among stimuli injected through electrodes implanted in different nerves (inter-digit temporal distance) allows to recognize object shapes. Nevertheless, a detailed characterization of the subjects’ sensitivity to variations of intra-digit temporal dynamic and inter-digit temporal distance of the intraneural tactile feedback has not been executed. An exhaustive understanding of the overall potentials and limits of intraneural stimulation to deliver sensory feedback is of paramount importance to bring this approach closer and closer to the natural situation. To this aim, here we asked two trans-radial amputees to identify stimuli with different temporal characteristics delivered to the same active site (intra-digit temporal Dynamic Recognition (DR)) or between two active sites (inter-digit Temporal distance Recognition (TR)). Finally, we compared the results achieved for (simulated) TR with conceptually similar experiments with real objects with one subject.ResultsWe found that the subjects were able to identify stimuli with temporal differences (perceptual thresholds) larger than 0.25 s for DR and larger than 0.125 s for TR, respectively. Moreover, we also found no statistically significant differences when the subjects were asked to identify three objects during simulated ‘open-loop’ TR experiments or real ‘closed-loop’ tests while controlling robotic hand.ConclusionsThis study is a new step towards a more detailed analysis of the overall potentials and limits of intraneural sensory feedback. A full characterization is necessary to develop more advanced prostheses capable of restoring all lost functions and of being perceived more as a natural limb by users.

Surgical Considerations for an Osseointegrated Steady State Implant (OSIA2®) in Children.

This paper discusses the joint venture between the German Aerospace Center (DLR) Institute of Robotics and Mechatronics and the German robot manufacturer, KUKA Robotics GmbH, for the successful transfer of the knowledge in robot dynamics and control, human robot interaction, and sensory feedback as originally developed for the remote control of space robots to industrial robotics. This technology transfer venture aims to promote more industrial sensory feedback applications with particular focus on the fusion of vision and force, while merging service and industrial robots into the future. As such, the aim is to transfer new robonaut technologies and ultra-lightweight, torque-controlled, kinematically redundant arms with multifingered articulated hands into the terrestrial robot industry.

One of the most common tasks of a robot companion in the home is communication. In order to initiate an information exchange with its human partner, the robot needs to attract the attention of the human. This paper presents results of two user studies ( $$\mathrm{N}=12$$ ) to evaluate the effectiveness of unimodal and multimodal communication cues for attracting attention. Results showed that unimodal communication cues which involve sound generate the fastest reaction times. Contrary to expectations, multimodal communication cues resulted in longer reaction times with respect to the unimodal communication cue that produced the shortest reaction time.

Previous research has shown that design features that support privacy are essential for new technologies looking to gain widespread adoption. As such, privacy-sensitive design will be important for the adoption of social robots, as they could introduce new types of privacy risks to users. In this paper, we report findings from our preliminary study on users' perceptions and attitudes toward privacy in human-robot interaction, based on interviews that we conducted about a workplace social robot.

Somatosensory neuroprostheses exploit invasive and non-invasive feedback technologies to restore sensorimotor functions lost to disease or trauma. These devices use electrical stimulation to communicate sensory information to the brain. A sensation characterization procedure is thus necessary to determine the appropriate stimulation parameters and to establish a clear personalized map of the sensations that can be restored. Several questionnaires have been described in the literature to collect the quality, type, location, and intensity of the evoked sensations, but there is still no standard psychometric platform. Here, we propose a new psychometric system containing previously validated questionnaires on evoked sensations, which can be applied to any kind of somatosensory neuroprosthesis. The platform collects stimulation parameters used to elicit sensations and records subjects' percepts in terms of sensation location, type, quality, perceptual threshold, and intensity. It further collects data using standardized assessment questionnaires and scales, performs measurements over time, and collects phantom limb pain syndrome data. The psychometric platform is user-friendly and provides clinicians with all the information needed to assess the sensory feedback. The psychometric platform was validated with three trans-radial amputees. The platform was used to assess intraneural sensory feedback provided through implanted peripheral nerve interfaces. The proposed platform could act as a new standardized assessment toolbox to homogenize the reporting of results obtained with different technologies in the field of somatosensory neuroprosthetics.

Recent advancements in spiking neural networks (SNNs) have highlighted their advantages, including energy efficiency, real-time processing, and compatibility with neuromorphic hardware. These features make SNNs particularly well-suited for human-robot interaction (HRI) in rehabilitative wearable robotics, where real-time adaptability and low power consumption are essential. However, there is still a lack of comprehensive reviews on SNNs’ application to HRI. This paper addresses this gap by providing a detailed overview of the latest advancements in SNNs from the perspective of embodied intelligence in rehabilitative wearable robots. We systematically examine recent progress in SNNs, including spiking neuron models, encoding methods, and learning mechanisms. These advancements are then analyzed with a focus on HRI, addressing specific challenges in rehabilitative wearable robots from three key perspectives: human motion decoding, robotic control, and neuromorphic implementation for embedded systems. By reviewing current research, this paper highlights the potential benefits and limitations of SNNs in achieving embodied intelligence and identifies crucial areas for further investigation, offering new insights and directions for their future applications in rehabilitative wearable robotics.