Time-dependent Neural Processing of Auditory Feedback during Voice Pitch Error Detection

ERP correlates of language-specific processing of auditory pitch feedback during self-vocalization

The Sense of Agency (SoA) refers to the subjective experience of having control over our own actions and their outcomes. SoA is experienced when there is a match between the predicted and actual sensory outcomes of an intended motor action. Thus, receiving sensory feedback related to motor actions is essential for the formation of the SoA. As a consequence, individuals with sensory loss due to neuropathies or limb impairments might experience reduced SoA. Therefore, incorporating artificial sensory feedback in assistive technologies (e.g. prosthetics, exoskeletons, sensory substitution systems) is crucial for patients to properly experience SoA and thereby enhance the effective use of such technologies. In this study, with the use of neuro-cognitive measures, we validated the effect of a vibrotactile sensory feedback system on SoA for lower-limb movements. Healthy participants (n = 25) performed a stepping task while receiving vibrotactile sensory feedback and, in a control condition, auditory feedback. We employed SoA measures to validate the sensory feedback system, in particular the N100 ERP component of the electroencephalography (EEG) signal as an implicit neural measure of SoA, temporal interval estimation tasks as an implicit behavioural measure, and SoA questionnaires as an explicit measure. Statistical analyses, including repeated-measures ANOVA and t-tests, were conducted to examine the effects of externally generated versus self-generated stimuli on N100 modulation, temporal interval estimations, and SoA questionnaire results. Our results showed a significant modulation of the N100 component for self-generated, relative to externally generated, sensory feedback. The direction of the effect was opposite for vibrotactile and auditory sensory feedback, with enhancement in vibrotactile and attenuation in auditory modality. In contrast, no notable effects were found in interval estimation tasks or explicit agency measures, pointing to a possible dissociation between implicit and explicit measures of SoA and indicating higher sensitivity of implicit neural measures. Sensory processing of the vibrotactile sensory feedback is sensitive to the difference between outcomes that are actively generated compared to passively received, as indexed by the modulation in amplitude of the N100 component. This is an indication that vibrotactile sensory feedback is capable of eliciting sense of agency. Additionally, we found that modulation of sensory processing is more sensitive than temporal interval estimates to self- vs. externally-generated outcomes. Therefore, we suggest using it as a metric for the design and validation of assistive technologies.

Across the animal kingdom, neural responses in the auditory cortex are suppressed during vocalization, and humans are no exception. A common hypothesis is that suppression increases sensitivity to auditory feedback, enabling the detection of vocalization errors. This hypothesis has been previously confirmed in non-human primates, however a direct link between auditory suppression and sensitivity in human speech monitoring remains elusive. To address this issue, we obtained intracranial electroencephalography (iEEG) recordings from 35 neurosurgical participants during speech production. We first characterized the detailed topography of auditory suppression, which varied across superior temporal gyrus (STG). Next, we performed a delayed auditory feedback (DAF) task to determine whether the suppressed sites were also sensitive to auditory feedback alterations. Indeed, overlapping sites showed enhanced responses to feedback, indicating sensitivity. Importantly, there was a strong correlation between the degree of auditory suppression and feedback sensitivity, suggesting suppression might be a key mechanism that underlies speech monitoring. Further, we found that when participants produced speech with simultaneous auditory feedback, posterior STG was selectively activated if participants were engaged in a DAF paradigm, suggesting that increased attentional load can modulate auditory feedback sensitivity.

During vocalization, auditory feedback is used to attain and maintain a desired fundamental frequency (F0). The use of auditory feedback to control vocalization has been studied using adaptation studies, in which an F0 feedback alteration is maintained across many trials, and using midutterance perturbations, where a brief F0 change is introduced during an ongoing utterance. Participants vocalized during two blocks. In an adaptation block, a one semitone F0 shift was maintained for 40 trials, and midutterance perturbations were introduced by removing the feedback alteration. In a control block, a midutterance perturbation was introduced in trials, which were either unshifted or shifted randomly by one semitone at before the onset of vocalization (similar to the adaptation trials). F0 compensation to midutterance perturbations was identical in all conditions, and was always smaller than the compensation to a shift at utterance onset. These results are explained by a change in the control strategy at uttera...

Neural correlates of impaired vocal feedback control in post-stroke aphasia

When a speaker's voice returns to one's own ears with a 200-ms delay, the delay causes the speaker to speak less fluently. This phenomenon is called a delayed auditory feedback (DAF) effect. To investigate neural mechanisms of speech processing through the DAF effect, we conducted a functional magnetic resonance imaging (fMRI) experiment, in which we designed a paradigm to explore the conscious overt-speech processing and the automatic overt-speech processing separately, while reducing articulatory motion artifacts. The subjects were instructed to (1) read aloud visually presented sentences under real-time auditory feedback (NORMAL), (2) read aloud rapidly under real-time auditory feedback (FAST), (3) read aloud slowly under real-time auditory feedback (SLOW), and (4) read aloud under DAF (DELAY). In the contrasts of DELAY-NORMAL, DELAY-FAST, and DELAY-SLOW, the bilateral superior temporal gyrus (STG), the supramarginal gyrus (SMG), and the middle temporal gyrus (MTG) showed significant activation. Moreover, we found that the STG activation was correlated with the degree of DAF effect for all subjects. Because the temporo-parietal regions did not show significant activation in the comparisons among NORMAL, FAST, and SLOW conditions, we can exclude the possibility that its activation is due to speech rates or enhanced attention to altered speech sounds. These results suggest that the temporo-parietal regions function as a conscious self-monitoring system to support an automatic speech production system.

This experiment used a two-by-two factorial design to investigate the effects of two levels of oral practice and two types of auditory feedback upon reading times and number of speech disfluencies for 60 normally fluent speakers. During tests under delayed auditory feedback, subjects who had experienced six previous oral reading trials on a standard passage under normal auditory feedback made significantly fewer articulation errors and had significantly shorter reading times than subjects without prior reading practice. Tests of practiced subjects vs nonpracticed subjects under normal auditory feedback showed no significant differences in articulation errors or reading times. It was suggested that functional properties of auditory feedback cues were temporarily changed by oral reading practice.

Zebra finches memorize a tutor song as juveniles, and then faithfully reproduce this song for the remainder of their lives. This thesis investigates how the intricate and concerted activity of ensembles of neurons in the zebra finch brain gives rise to this complex behavior. The first section of the thesis demonstrates how neurons in RA, a pre-motor song control nucleus, coordinate their activity to generate the instantaneous spectral and temporal structure seen in the song. Similar sounds are produced by entirely different ensembles of RA neurons. Additionally, rapidly changing patterns of RA neural activity are capable of generating constant acoustic outputs. This is a new form of neural coding, previously unobserved in any other system. This unusual neural representation provides a concise account of an unexplained aspect of song learning. The second part of the thesis develops a new, noninvasive method of altering the auditory feedback heard by singing zebra finches. This type of feedback perturbation causes the normally stable songs of adult birds to deteriorate, indicating that auditory feedback is essential to song control throughout life. Restoration of the normal auditory feedback heard by these birds enables the gradual recovery of their original songs, demonstrating that a memory of the original song persists in the song control system despite the destabilization of the behavior. The song is thus maintained by an active feedback control process. The final project investigates the role of nucleus LMAN in real-time song control via auditory feedback. Neurons in nucleus LMAN were previously believed to compare auditory feedback to a memorized song template. The result of this comparison was thought to be used as an error-correction signal to update the motor control program. By recording from individual LMAN neurons while simultaneously manipulating the auditory feedback heard by the singing bird, it is shown that during singing, rather than auditory feedback, LMAN processes an efference copy of the motor commands used to generate the bird's song. This suggests a different model of the system, in which the efference copy is the basis of the real-time error-correction signal. Taken together, these three projects significantly expand our knowledge of the neural mechanisms responsible for the generation of birdsong.

Feedback-based error monitoring processes during musical performance: An ERP study

Spatiotemporal source analysis of vocalization-associated magnetic fields

Self-monitoring for speech errors: Two-stage detection and repair with and without auditory feedback

BackgroundSelf-agency is the experience of being the agent of one’s own thoughts and motor actions. The intact experience of self-agency is necessary for successful interactions with the outside world (i.e., reality monitoring). Reality monitoring is the ability to distinguish internally self-generated information from outside reality (externally-derived information). We found that healthy control (HC) participants recruit medial prefrontal cortex (mPFC) during encoding of self-generated information, which is also activated during accurate retrieval of self-generated information. By contrast, patients with schizophrenia (SZ) have specific self-agency impairments and do not show mPFC activation during encoding or retrieval of self-generated information. These findings indicate that SZ may rely more on environmental externally-derived information, rather than on internal self-generated information, to guide reality monitoring. Here, we relate the experience of self-agency during a lower-level speech feedback monitoring (i.e., monitoring what we hear ourselves say) to our higher-level cognitive reality monitoring task. We examine whether the sense of self-agency during speech feedback monitoring and reality monitoring are driven by the same fundamental mechanism that we hypothesize underlies the capacity to experience self-agency—the ability to make reliable predictions about the outcomes of self-generated actions.MethodsDuring speech feedback monitoring we assess self-agency by altering environmental auditory feedback so that participants listen to a perturbed version of their own speech. When subjects hear minimal perturbations in their auditory feedback while speaking, they make corrective responses, indicating that they judge the perturbations as errors in their speech output. These corrective responses are modulated by subjects’ reliance on internal predictions about the outcome of their speech output; the more subjects rely on their internal predictions, i.e. their sense of self-agency, the less they rely on external auditory feedback, resulting in smaller corrective responses. Thus, subjects who produced smaller corrective responses manifested an enhanced sense of self-agency in that they relied more on their internal predictions to generate their own actions (i.e., their speech output).ResultsWe found that self-agency judgments in the reality-monitoring task were higher in people who had smaller corrective responses (p=.05) during minimal speech perturbations of their auditory feedback. These results provide support for a unitary process for the experience of self-agency resulting from the ability to reliably predict the outcomes of self-generated actions, that governs low-level speech control and higher level reality monitoring.DiscussionThese findings have important therapeutic implications in SZ, suggesting that the more participants rely on internal predictions to guide their actions, the smaller their corrective responses in their speech output, and the more likely they are to make correct judgments of self-agency during reality monitoring. In conclusion, these findings, therefore, indicate that reality monitoring and speech monitoring paradigms provide quick and robust markers of the experience of self-agency, indicating which subjects followed their internal predictions to guide their own actions. Together, these findings have important therapeutic implications for potentiating improvements in self-agency judgments not only in HC, but in patients with schizophrenia who suffer from critical self-agency impairments.

Effector-independent brain network for auditory-motor integration: fMRI evidence from singing and cello playing

Two auditory feedback perturbation experiments were conducted to examine the nature of control of the first two formants in vowels. In the first experiment, talkers heard their auditory feedback with either F1 or F2 shifted in frequency. Talkers altered production of the perturbed formant by changing its frequency in the opposite direction to the perturbation but did not produce a correlated alteration of the unperturbed formant. Thus, the motor control system is capable of fine-grained independent control of F1 and F2. In the second experiment, a large meta-analysis was conducted on data from talkers who received feedback where both F1 and F2 had been perturbed. A moderate correlation was found between individual compensations in F1 and F2 suggesting that the control of F1 and F2 is processed in a common manner at some level. While a wide range of individual compensation magnitudes were observed, no significant correlations were found between individuals' compensations and vowel space differences. Similarly, no significant correlations were found between individuals' compensations and variability in normal vowel production. Further, when receiving normal auditory feedback, most of the population exhibited no significant correlation between the natural variation in production of F1 and F2.

The auditory system consists of the ascending and descending (corticofugal) ­systems. The corticofugal system forms multiple feedback loops with the ascending system, so that the neural mechanisms for auditory signal processing cannot be fully understood without the exploration of the function of the corticofugal system. Corticofugal modulation (changes in response properties evoked by the corticofugal system) of subcortical neurons for auditory signal processing is one of the most important functions of the auditory cortex in the cerebrum.KeywordsAuditory CortexGood FrequencyCochlear MicrophonicCentral Auditory SystemCollicular NeuronThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.