The cortical computations underlying feedback control in vocal production

Spoken language exists because of a remarkable neural process. Inside a speaker's brain, an intended message gives rise to neural signals activating the muscles of the vocal tract. The process is remarkable because these muscles are activated in just the right way that the vocal tract produces sounds a listener understands as the intended message. What is the best approach to understanding the neural substrate of this crucial motor control process? One of the key recent modeling developments in neuroscience has been the use of state feedback control (SFC) theory to explain the role of the CNS in motor control. SFC postulates that the CNS controls motor output by (1) estimating the current dynamic state of the thing (e.g., arm) being controlled, and (2) generating controls based on this estimated state. SFC has successfully predicted a great range of non-speech motor phenomena, but as yet has not received attention in the speech motor control community. Here, we review some of the key characteristics of speech motor control and what they say about the role of the CNS in the process. We then discuss prior efforts to model the role of CNS in speech motor control, and argue that these models have inherent limitations – limitations that are overcome by an SFC model of speech motor control which we describe. We conclude by discussing a plausible neural substrate of our model.

Neurobiology of speaking and stuttering

Background : Ziegler (2003) reviews neural and behavioural evidence to support a task-dependent model of motor control, whereby motor speech disorders represent impairment of a sensory-minor system specialised for speech. Based on this approach, Ziegler argues against the examination of volitional or novel nonspeech motor activities to gain insight into motor control for speech. Aims : The primary objective of this paper is to discuss Ziegler's (2003) conceptualisation of the task-dependent model and present an alternative integrative model. In the latter, speech and volitional nonspeech motor control are integrated into the functioning of a more general motor system where neural and behavioural systems demonstrate areas of overlap. Studies of the nervous system, the evolutionary foundations of speech in great apes, behaviour, and motor learning are presented to support an integrative model of motor control. Main Contribution : Neurological and evolutionary evidence strongly suggest that neural networks are flexible, multifaceted, multifunctional and overlapping in function. It is highly likely that a higher-level behaviour like speech involves networks that are similarly multifunctional and overlapping with other motor functions. Ziegler's concept of task-dependent motor control may relate as much to parameters such as complexity, familiarity, and automaticity of task performance as to the nonpeech-speech distinction. Studies are reviewed that support the inclusion of nonspeech motor tasks in assessment of the disordered speech motor control system. Specific nonspeech tasks clearly facilitate differential diagnosis and provide insight into the functioning breakdown of the motor system for related but more complex speech behaviours. Finally, motor learing studies are discussed with particular reference to how these might inform models of motor control. Conclusions : This response cautions against the seemingly premature acceptance of a model assuming separate sensory motor systems for volitional nonspeech motor activities and speech. Continued research, without the limitations imposed by such an assumption, is required to enhance understanding of component parts of complex behaviours and how those components interact in normal and damaged systems at different levels of a performance. These data have the potential to form the basis for new, more effective approaches to treatment of motor speech disorders.

A treatment system for reducing stuttering uses an auditory feedback modification technique to train the stutterer's speech motor control system to be more stable. The auditory feedback modification is based on a model of speech motor control in the human brain that incorporates a variation of observer-based control and Smith prediction. In addition, the Kalman gain of the model is set by comparing the speech muscular control signals sent to the person's vocal tract to the corresponding auditory speech sounds the person actually hears. It is believed that the speech motor control system of a stutterer has set the Kalman gain too high, thereby creating an unstable control system that in turn causes stuttering. A feedback modifier feeds back the stutterer's speech to the stutterer with a perturbation that is small enough to pass a validator function that is believed to be part of the speech motor control system. The small perturbations increase the difference between the target speech and the fed back speech, which is believed to cause the speech motor control system to decrease the Kalman gain. Over a treatment program, it is believed that the perturbations "train" or adapt the stutterer's speech motor control system so as to reduce the Kalman gain, which is further believed to persist when the auditory feedback is discontinued.

This paper investigates the issue of decentralized control for interconnected semi-Markovian systems with partially accessible transition rates (TRs). Firstly, a dynamic system model with a memory event-triggered mechanism (METM) is designed, which can effectively improve the fault tolerance of the event-triggering mechanism by employing the historical trigger data. Then a state feedback control model with dynamic METM is constructed, in which the semi-Markovian parameters with completely unknown and partially known transition probabilities are considered. Some sufficient conditions that insure the stochastic stability of the interconnected semi-Markovian systems can be obtained by utilizing the Lyapunov function and suitable model transformations method. Meanwhile, the parameters and the controller gain matrices of dynamic METM are also solved simultaneously by applying the linear matrix inequalities (LMIs). Finally, a simulation example is given to verify the effectiveness of the proposed method.

Many voice disorders are the result of intricate neural and/or biomechanical impairments that are poorly understood. The limited knowledge of their etiological and pathophysiological mechanisms hampers effective clinical management. Behavioral studies have been used concurrently with computational models to better understand typical and pathological laryngeal motor control. Thus far, however, a unified computational framework that quantitatively integrates physiologically relevant models of phonation with the neural control of speech has not been developed. Here, we introduce LaDIVA, a novel neurocomputational model with physiologically based laryngeal motor control. We combined the DIVA model (an established neural network model of speech motor control) with the extended body-cover model (a physics-based vocal fold model). The resulting integrated model, LaDIVA, was validated by comparing its model simulations with behavioral responses to perturbations of auditory vocal fundamental frequency (fo) feedback in adults with typical speech. LaDIVA demonstrated capability to simulate different modes of laryngeal motor control, ranging from short-term (i.e., reflexive) and long-term (i.e., adaptive) auditory feedback paradigms, to generating prosodic contours in speech. Simulations showed that LaDIVA’s laryngeal motor control displays properties of motor equivalence, i.e., LaDIVA could robustly generate compensatory responses to reflexive vocal fo perturbations with varying initial laryngeal muscle activation levels leading to the same output. The model can also generate prosodic contours for studying laryngeal motor control in running speech. LaDIVA can expand the understanding of the physiology of human phonation to enable, for the first time, the investigation of causal effects of neural motor control in the fine structure of the vocal signal.

Effective human speech requires the neural integration of ongoing vocal production with the auditory and somatosensory feedback signals that are produced. We are using invasive electrophysiology techniques in patient volunteers undergoing neurosurgical treatment in order to gain insights into these mechanisms and underlying neural circuits. By using multi-contact electrode arrays chronically implanted over the perisylvian temporal lobe auditory cortex (e.g. area PLST) and the inferior frontal gyrus (IFG), we can examine local field potentials and frequency-specific responses from cortical areas important for both vocal production and speech sound processing. Our initial studies have found that during self-vocalization, focal areas within higher order auditory cortex on the superior temporal gyrus (STG) show response modulation compared to the responses of the same areas during passive listening. Manipulation of the auditory feedback that a speaker receives during vocalization (e.g. pitch-shifted or delayed auditory feedback) leads to further modulation of these PLST sites. Measures of functional connectivity including electrical stimulation tract tracing or phase-synchrony analysis demonstrate that portions of PLST are functionally connected to regions of the IFG. These findings support forward models for vocal control in which efference copies of premotor cortex activity modulate sub-regions of auditory cortex.

Traditionally, psycholinguistic models and models of speech motor control have approached the word production process from different angles and have remained, to a large extent, separate from one another. Ultimately, however, the process entails the entire pathway from semantic processing to articulation: producing a word from meaning requires activating a concept, retrieving the word, selecting its segments, ordering those segments into the right sequence, and finally articulating them with the correct timing. This chapter reviews the neural basis of word production from the complementary perspectives of the psycholinguistic literature and the speech motor control literature.

Neural correlates of impaired vocal feedback control in post-stroke aphasia

In this paper, we propose an adaptive fuzzy output feedback control method for trajectory tracking control problem for robotic systems. Using Lyapunov method, we first develop a stable adaptive fuzzy state feedback control algorithm by assuming that the systems output and its derivatives are available for feedback control design. The algorithm combines fuzzy systems with robust adaptive controller. The fuzzy system approximates the certainty equivalent (CE)-based optimal controller while robustifying adaptive control term is used to cope with uncertainties that appeared from the effect of external disturbance, fuzzy approximation errors and other modeling errors. Then, an output feedback form of the position-velocity (state feedback) controller is proposed where unknown velocity signal is replaced by the output of model-free linear estimator. We show via asymptotic analysis that the tracking performance of the output feedback design can recover the performance achieved under the state feedback control design. Finally, the proposed method is implemented and evaluated on a 2-DOF robotic system to demonstrate the theoretical development for the real-time applications.

Reliability and Representational Bandwidth in the Auditory Cortex

Speech production involves a combination of feedforward and sensory feedback-based control mechanisms. The latter have been characterized in experiments involving real-time perturbations during speech. Unexpected perturbations of auditory feedback result in corrective motor responses with a minimum latency of approximately 100 ms after perturbation onset. These responses have been shown for both pitch and formant frequency perturbations, and the responsible neural circuitry includes portions of the superior temporal gyrus and ventral premotor cortex (vPMC). Corrective responses to somatosensory perturbations (such as a downward force applied to the jaw) occur approximately 50 ms after perturbation onset and are mediated by ventral somatosensory cortex and vPMC. The degree to which an individual weights auditory versus somatosensory feedback varies substantially. Such differences rely in part on differences in sensory acuity, e.g., a speaker with relatively poor hearing is likely to rely more heavily on somatosensory feedback control mechanisms than auditory feedback control mechanisms. Additionally, reliance on feedback control may be modulated to compensate for impairments in the feedforward system for speech, e.g., adults who stutter show a reduced reliance on auditory feedback control compared to fluent speakers, perhaps because auditory feedback can have a deleterious effect on speech initiation in stuttering.

This paper evaluates, through simulations, the performance and robustness of grid-forming (GFM) inverter-based resources (IBRs) using state feedback and conventional proportional-integral (PI) controllers, both in the dq reference frame. The objective is to investigate whether a standardized control strategy among paralleled IBRs can improve system stability and contribute to the ongoing energy transition, where the penetration of renewable energy sources (RES) enabled by power electronics is steadily increasing. A linear quadratic regulator (LQR)-based state feedback control model is derived and implemented, and its performance under parallel operation is compared with the most commonly used control strategy for GFM inverters. The state feedback approach exhibits better performance under the studied conditions, demonstrating robustness under variations in grid parameters. Moreover, the IBRs show to be interoperable, capable of operating in parallel without interfering with each other. Notably, this stability is achieved without the need for auxiliary techniques such as virtual impedance.

Modulation of vocal pitch is a key speech feature that conveys important linguistic and affective information. Auditory feedback is used to monitor and maintain pitch. We examined induced neural high gamma power (HGP) (65-150 Hz) using magnetoencephalography during pitch feedback control. Participants phonated into a microphone while hearing their auditory feedback through headphones. During each phonation, a single real-time 400 ms pitch shift was applied to the auditory feedback. Participants compensated by rapidly changing their pitch to oppose the pitch shifts. This behavioral change required coordination of the neural speech motor control network, including integration of auditory and somatosensory feedback to initiate change in motor plans. We found increases in HGP across both hemispheres within 200 ms of pitch shifts, covering left sensory and right premotor, parietal, temporal, and frontal regions, involved in sensory detection and processing of the pitch shift. Later responses to pitch shifts (200-300 ms) were right dominant, in parietal, frontal, and temporal regions. Timing of activity in these regions indicates their role in coordinating motor change and detecting and processing of the sensory consequences of this change. Subtracting out cortical responses during passive listening to recordings of the phonations isolated HGP increases specific to speech production, highlighting right parietal and premotor cortex, and left posterior temporal cortex involvement in the motor response. Correlation of HGP with behavioral compensation demonstrated right frontal region involvement in modulating participant's compensatory response. This study highlights the bihemispheric sensorimotor cortical network involvement in auditory feedback-based control of vocal pitch.

Across the animal kingdom, neural responses in the auditory cortex are suppressed during vocalization, and humans are no exception. A common hypothesis is that suppression increases sensitivity to auditory feedback, enabling the detection of vocalization errors. This hypothesis has been previously confirmed in non-human primates, however a direct link between auditory suppression and sensitivity in human speech monitoring remains elusive. To address this issue, we obtained intracranial electroencephalography (iEEG) recordings from 35 neurosurgical participants during speech production. We first characterized the detailed topography of auditory suppression, which varied across superior temporal gyrus (STG). Next, we performed a delayed auditory feedback (DAF) task to determine whether the suppressed sites were also sensitive to auditory feedback alterations. Indeed, overlapping sites showed enhanced responses to feedback, indicating sensitivity. Importantly, there was a strong correlation between the degree of auditory suppression and feedback sensitivity, suggesting suppression might be a key mechanism that underlies speech monitoring. Further, we found that when participants produced speech with simultaneous auditory feedback, posterior STG was selectively activated if participants were engaged in a DAF paradigm, suggesting that increased attentional load can modulate auditory feedback sensitivity.