Visual feedback of intonation I: Effectiveness and induced practice behavior.

Effects of auditory and visual feedback on remote pilot manual flying performance

The effects of feedback to assist elementary school students classified as either normally achieving (NA) or with a mathematics learning disability (MLD) in acquiring the fact series of 0 to 9 for the operations of addition, subtraction, multiplication, and division were examined in Study 1. The acquisition of each fact series was facilitated by immediate, but not by delayed feedback, the latter of which was no more effective than control procedures. The students with math disabilities were tested with either delayed feedback or a Scantron form in Study 1, then participated in Study 2, in which they were provided with feedback from either an educator or the Immediate Feedback Assessment Technique (IF AT). The beneficial effects of immediate feedback reported in Study 1 were replicated and extended during maintenance which continued for as many as 25 sessions. The effects of auditory feedback provided by an educator and visual feedback provided by the IF AT were compared with the effects of combined auditory and visual feedback provided by the Write-Say method in Study 3. The integrated presentation of auditory and visual feedback was no more effective than the use of either modality, separately. The comparable effectiveness of feedback by an educator and by the IF AT, and the nonsynergistic effects of combining auditory with visual feedback, suggests that a response medium such as the IF AT has considerable adjunctive potential to assist in the instruction of elementary school students with special learning needs

Motor and visual influences on auditory neural processing during speaking and listening

Long-term effects of auditory kinetic feedback on relearned symmetrical body weight distribution while rising and sitting down were studied in stroke patients. Thirty patients were re-tested on average 33 months after having trained with and without auditory feedback. Body weight distribution on the legs was measured with two force plates. At re-test there was a decrease from 48% to 39% of body weight distribution (%BWD) on the paretic leg in rising and in sitting down in the patients in the auditory feedback group (p < 0.001). In the control group the decrease was from 44 to 39% BWD on the paretic leg (p < 0.05) in rising and from 44 to 42% BWD (n.s.) in sitting down. The symmetrical body-weight distribution, acquired after auditory feedback training, was not consistent over time. Movement time, however, was significantly reduced in the auditory feedback group (p < 0.05). Possible reasons for the findings are discussed.

Slip feedback is an important cue in everyday object manipulation, but it is generally missing in tele-operation systems. To test the usefulness of simple, abstract types of feedback that warn the user about slip events, we tested the effect of auditory and haptic vibration feedback in a tele-operation task. Participants were asked to hold an object in a remote robot hand, and the force profiles that they exerted in response to slip events were measured. Haptic feedback did not significantly change the response characteristics, but auditory feedback did significantly improve response latency. A small but significant difference between haptic and auditory reaction times (60 ms) found in our control experiment might explain the difference between the feedback types.

This study examined the effects on driving, usability and subjective workload of performing music selection tasks using a touch screen interface. The benefits of providing visual and/or auditory feedback was also explored. Thirty participants performed music selection tasks with a touch screen interface while driving, with four forms of feedback. The music selection tasks significantly increased subjective workload and degraded performance on a range of driving measures. The provision of any form of feedback did not significantly affect driving performance, usability or subjective workload. Results suggest that touch screens may not be a suitable input device for navigating scrollable lists.

Effects of auditory feedback on movements with two-segment sequence and eye-hand coordination: Using a short auditory contact cue

Effect of an auditory feedback substitution, tactilo-kinesthetic, or visual feedback on kinematics of pouring water from kettle into cup

Cognitive training is effective at retaining cognitive function and delaying decline for typically ageing older adults, individuals with mild cognitive impairment, and persons with dementia. Technological resources can address limiting factors that inhibit engagement and access to this treatment. We investigated how a socially assistive robot-facilitated memory task with sensory feedback was received by older adults. The impact of unimodal and multimodal administration of auditory and haptic feedback using two robot embodiments (Pepper and Nao) was evaluated in terms of user performance, usability, and workload. In contrast to sensory feedback research, auditory feedback resulted in significantly higher task accuracy. This was, however, supported by previous work from neurological literature. Auditory feedback also received significantly higher usability, and this preference was validated by qualitative feedback from participants. Regardless of robotic embodiment, this study demonstrates an advantage for auditory feedback (over haptic and multimodal) in cognitive training activities for older adults.

Concurrent augmented feedback has been shown to be less effective for learning simple motor tasks than for complex tasks. However, as mostly artificial tasks have been investigated, transfer of results to tasks in sports and rehabilitation remains unknown. Therefore, in this study, the effect of different concurrent feedback was evaluated in trunk-arm rowing. It was then investigated whether multimodal audiovisual and visuohaptic feedback are more effective for learning than visual feedback only. Naïve subjects (N = 24) trained in three groups on a highly realistic virtual reality-based rowing simulator. In the visual feedback group, the subject's oar was superimposed to the target oar, which continuously became more transparent when the deviation between the oars decreased. Moreover, a trace of the subject's trajectory emerged if deviations exceeded a threshold. The audiovisual feedback group trained with oar movement sonification in addition to visual feedback to facilitate learning of the velocity profile. In the visuohaptic group, the oar movement was inhibited by path deviation-dependent braking forces to enhance learning of spatial aspects. All groups significantly decreased the spatial error (tendency in visual group) and velocity error from baseline to the retention tests. Audiovisual feedback fostered learning of the velocity profile significantly more than visuohaptic feedback. The study revealed that well-designed concurrent feedback fosters complex task learning, especially if the advantages of different modalities are exploited. Further studies should analyze the impact of within-feedback design parameters and the transferability of the results to other tasks in sports and rehabilitation.

Force-input interfaces are now attracting research interest and feedback schemes that yield high accurate operation are being investigated. Extrinsic feedback can help the user to operate a device with high accuracy and/or efficiency. However, for learning force control, the effectiveness of multimodal augmented feedback has not been investigated enough. This paper focuses on foam objects for learning force control and describes an experiment conducted to verify the effectiveness of augmented visual and auditory feedback. In the experiment, participants apply pushing force using their thumb to a foam cube under different augmented feedback conditions: 1 visual feedback, 2 auditory feedback and 3 visual-auditory feedback. Training and retention tests are conducted. The results show that just five minutes of practice similar to playing a musical instrument can enhance force reproduction skills and make subsequent force inputs more accurate.

This paper investigates the effect of adaptive auditory feedback on continued player performance for stroke patients in a reaction time tablet game. The feedback sound pitch followed a saw-tooth shape that cumulatively increased for fast reaction times and flying back to the base pitch after a single slow reaction time. The analysis was based on data obtained in a field trial with lesion patients during their regular rehabilitation. The auditory feedback events were categorized by feedback type (positive/negative) and the associated pitch change of either high or low magnitude. Both feedback type and magnitude significantly affected on players performance. Negative feedback improved patients reaction times in subsequent hits by 0.42 s and positive feedback impaired their performance by 0.15 s.

ABSTRACTNormal auditory feedback contributes to moment-to-moment control of speech production. Effects of auditory feedback’s absence on hearing-impaired individuals are widely documented, but auditory perturbation has not been investigated. Our objective was to evaluate the effect of delayed auditory feedback (DAF) and frequency altered feedback (FAF) on speech production among prelingual cochlear implant (CI) users and normal hearing (NH) individuals, to evaluate CI users’ reliance on auditory feedback. Twenty young adults (10 CI, 10 NH), without developmental and cognitive impairments, participated in the study. Under variable auditory feedback conditions, speech production (spontaneous or reading aloud) was measured using speech rate, percentage of interruptions, fundamental frequency (F0), and relative intensity. Results showed that (1) both DAF and FAF caused slower speech rates and more interruptions while reading aloud, with DAF having larger effect; (2) altered feedback produced no differences between groups, except an increase in F0 for CI users during DAF; and (3) CI users’ ability to understand speech via phone and without lip-reading was positively correlated with performance under DAF. These findings suggest that auditory perturbation similarly affects speech production among prelingual CI users and NH individuals, indicating CI users depend on auditory feedback to the same degree as normal hearing individuals.

Auditory feedback may provide the nervous system with valuable temporal (e. g., footstep sounds) and spatial (e.g., external reference sounds) information that can assist in the control of upright walking. As such, hearing loss may directly contribute to declines in mobility among older adults. Our purpose was to examine the impact of auditory feedback on the control of walking in older adults. Twenty older adults (65–86 years) with no diagnosed hearing loss walked on a treadmill for three sound conditions: Baseline, Ear Plugs, and White Noise. We hypothesized that in response to reduced temporal auditory feedback during the Ear Plugs and White Noise conditions, participants would adapt shorter and faster steps that are traditionally believed to increase mechanical stability. This hypothesis was not supported. Interestingly, we observed increases in step length (p = 0.047) and step time (p = 0.026) during the Ear Plugs condition vs. Baseline. Taking longer steps during the Ear Plugs condition may have increased ground reaction forces, thus allowing participants to sense footsteps via an occlusion effect. As a follow-up, we performed a Pearson's correlation relating the step length increase during the Ear Plugs condition to participants' scores on a clinical walking balance test, the Functional Gait Assessment. We found a moderate negative relationship (rho = −0.44, p = 0.055), indicating that participants with worse balance made the greatest increases in step length during the Ear Plugs condition. This trend suggests that participants may have actively sought auditory feedback with longer steps, sacrificing a more mechanically stable stepping pattern. We also hypothesized that reduced spatial localization feedback during the Ear Plugs and White Noise conditions would decrease control of center of mass (COM) dynamics, resulting in an increase in lateral COM excursion, lateral margin of stability, and maximum Lyapunov exponent. However, we found no main effects of auditory feedback on these metrics (p = 0.580, p = 0.896, and p = 0.056, respectively). Overall, these results suggest that during a steady-state walking task, healthy older adults can maintain walking control without auditory feedback. However, increases in step length observed during the Ear Plugs condition suggest that temporal auditory cues provide locomotor feedback that becomes increasingly valuable as balance deteriorates with age.

Visual feedback gait retraining has been reported to successfully reduce impact loading in runners, even when the runners were distracted. However, auditory feedback is more feasible in real life application. Hence, this study compared the peak positive acceleration (PPA), vertical average (VALR) and instantaneous (VILR) loading rate during distracted running before and after a course of auditory feedback gait retraining in 16 runners. The runners were asked to land with softer footfalls with and without auditory feedback. Low or high sound pitch was generated according to the impact of particular footfall, when compared with the preset target. Runners then received a course of auditory gait retraining, and after the gait retraining, runners completed a reassessment. Runners before gait retraining exhibited lower PPA, VALR and VILR with augmented auditory feedback (p<0.049). We found a reduction in PPA, VALR and VILR after gait retraining, regardless of the presence of feedback (p<0.018). However, runners after gait retraining did not demonstrate further reduction in PPA and VALR with auditory feedback (p>0.104). A small effect of auditory feedback on VILR in runners after gait retraining was observed (p=0.032). Real time auditory feedback gait retraining is effective in impact loading reduction, even when the runners were distracted.