Speech motor learning without hearing

Auditory speech perception enables listeners to access phonological categories from speech sounds. During speech production and speech motor learning, speakers' experience matched auditory and somatosensory input. Accordingly, access to phonetic units might also be provided by somatosensory information. The present study assessed whether humans can identify vowels using somatosensory feedback, without auditory feedback. A tongue-positioning task was used in which participants were required to achieve different tongue postures within the /e, ε, a/ articulatory range, in a procedure that was totally nonspeech like, involving distorted visual feedback of tongue shape. Tongue postures were measured using electromagnetic articulography. At the end of each tongue-positioning trial, subjects were required to whisper the corresponding vocal tract configuration with masked auditory feedback and to identify the vowel associated with the reached tongue posture. Masked auditory feedback ensured that vowel categorization was based on somatosensory feedback rather than auditory feedback. A separate group of subjects was required to auditorily classify the whispered sounds. In addition, we modeled the link between vowel categories and tongue postures in normal speech production with a Bayesian classifier based on the tongue postures recorded from the same speakers for several repetitions of the /e, ε, a/ vowels during a separate speech production task. Overall, our results indicate that vowel categorization is possible with somatosensory feedback alone, with an accuracy that is similar to the accuracy of the auditory perception of whispered sounds, and in congruence with normal speech articulation, as accounted for by the Bayesian classifier.

Talkers are known to compensate only partially for experimentally-induced changes to their auditory feedback. In a typical experiment, talkers might hear their FI feedback shifted higher (so that/epsilon/ sounds like /ae/, for example), and compensate by lowering FI in their subsequent speech by about a quarter of that distance. Here, we sought to characterize and understand partial compensation by examining how talkers respond to each step on a staircase of increasing shifts in auditory feedback. Subjects wore an apparatus which altered their real time auditory feedback. They were asked to repeat visually-presented hVd stimulus words while feedback was altered stepwise over the course of 360 trials. We used a novel analysis method to calculate each subject's compensation at each compensation step relative to their baseline. Results demonstrated that subjects compensated more for small feedback shifts than for larger shifts. We suggest that this pattern is consistent with vowel targets that incorporate auditory and somatosensory information, and a speech motor control system that is driven by differential weighting of auditory and somatosensory feedback.

A cochlear implant (CI) is a hearing device introduced in the 1980s for profoundly deaf subjects who gained little or no benefit from powerful hearing aids. This device comprises an electrode array inserted in the cochlea, connected to an internal receiver, and an externally worn speech processor. The CI transforms acoustic signals into electrical currents which directly stimulate the auditory nerve. Since the early 1990s, cochlear implantation in children has been developing rapidly. Although it is still difficult to predict how a child will perform with a cochlear implant, the success of cochlear implantation can no longer be denied. In this paper, some recent papers and reports, and the results of the various Nijmegen cochlear implant studies, are reviewed. Issues about selection, examinations, surgery and the outcome are discussed. Overall, our results were comparable with those of other authors. It can be concluded that cochlear implantation is an effective treatment for postlingually deaf as well as prelingually (congenital or acquired) deaf children with profound bilateral sensorineural deafness.

Speech perception relies on auditory and visual cues and there are strong links between speech perception and production. We aimed to evaluate the role of auditory and visual modalities on speech perception and production in adults with impaired hearing or sight versus those with normal hearing and sight. We examined speech perception and production of three isolated vowels (/i/, /y/, /u/), which were selected based on their different auditory and visual perceptual saliencies, in 12 deaf adults who used one or two cochlear implants (CIs), 14 congenitally blind adults, and 16 adults with normal sight and hearing. The results showed that the deaf adults who used a CI had worse vowel identification and discrimination perception and they also produced vowels that were less typical or precise than other participants. They had different tongue positions in speech production, which possibly partly explains the poorer quality of their spoken vowels. Blind individuals had larger lip openings and smaller lip protrusions for the rounded vowel and unrounded vowels, compared to the other participants, but they still produced vowels that were similar to those produced by the adults with normal sight and hearing. In summary, the deaf adults, even though they used CIs, had greater difficulty in producing accurate vowel targets than the blind adults, whereas the blind adults were still able to produce accurate vowel targets, even though they used different articulatory strategies.

ABSTRACTNormal auditory feedback contributes to moment-to-moment control of speech production. Effects of auditory feedback’s absence on hearing-impaired individuals are widely documented, but auditory perturbation has not been investigated. Our objective was to evaluate the effect of delayed auditory feedback (DAF) and frequency altered feedback (FAF) on speech production among prelingual cochlear implant (CI) users and normal hearing (NH) individuals, to evaluate CI users’ reliance on auditory feedback. Twenty young adults (10 CI, 10 NH), without developmental and cognitive impairments, participated in the study. Under variable auditory feedback conditions, speech production (spontaneous or reading aloud) was measured using speech rate, percentage of interruptions, fundamental frequency (F0), and relative intensity. Results showed that (1) both DAF and FAF caused slower speech rates and more interruptions while reading aloud, with DAF having larger effect; (2) altered feedback produced no differences between groups, except an increase in F0 for CI users during DAF; and (3) CI users’ ability to understand speech via phone and without lip-reading was positively correlated with performance under DAF. These findings suggest that auditory perturbation similarly affects speech production among prelingual CI users and NH individuals, indicating CI users depend on auditory feedback to the same degree as normal hearing individuals.

By 24 mo of age, most typically developing infants with normal hearing successfully transition to the production of words that can be understood about 50% of the time. This study compares early phonological development in children with and without hearing loss to gain a clearer understanding of the effects of hearing loss in early-identified children. A secondary goal was to identify measures of early phonetic development that are predictors of later speech production outcomes. The vocalizations and early words of 21 infants with normal hearing and 12 early-identified infants with hearing loss were followed longitudinally over a period of 14 mo (from 10 to 24 mo of age). Thirty-minute mother-child interaction samples were video recorded at 6- to 8-wk intervals in a laboratory playroom setting. Vocalizations produced at 16 and 24 mo were categorized according to communicative intent and recognizable words versus other types. Groups were compared on the structural complexity of words produced at 24 mo of age. Parent report measures of vocabulary development were collected from 10 to 30 mo of age, and Goldman-Fristoe Test of Articulation scores at 36 mo were used in regression analyses. Both groups increased the purposeful use of voice between 16 and 24 mo of age. However, at 24 mo of age, the toddlers with hearing loss produced significantly fewer words that could be recognized by their mothers. Their samples were dominated by unintelligible communicative attempts at this age. In contrast, the samples from normal hearing children were dominated by words and phrases. At 24 mo of age, toddlers with normal hearing were more advanced than those with hearing loss on seven measures of the structural complexity of words. The children with normal hearing attempted more complex words and productions were more accurate than those of children with hearing loss. At 10 to 16 mo of age, the groups did not differ significantly on parent-report measures of receptive vocabulary. However, the hearing loss group was much slower to develop expressive vocabulary and demonstrated larger individual differences than the normal hearing group. Six children identified as atypical differed from all other children in vowel accuracy and complexity of word attempts. However, both atypical infants and typical infants with hearing loss were significantly less accurate than normal hearing infants in consonant and word production. Early measures of syllable production predicted unique variance in later speech production and vocabulary outcomes. The transition from babble to words in infants with hearing loss appears to be delayed but parallel to that of infants with normal hearing. These delays appear to exert significant influences on expressive vocabulary development. Parents may appreciate knowing that some children with hearing loss may develop early vocabulary at a slower rate than children with normal hearing. Clinicians should monitor landmarks from babble onset through transitions to words. Indicators of atypical development were delayed and/or limited use of syllables with consonants, vowel errors and limited production of recognizable words.

Verbal working memory is poorer for children with hearing loss than for peers with normal hearing (NH), even with cochlear implantation and early intervention. Poor verbal working memory can affect academic performance, especially in higher grades, making this deficit a significant problem. This study examined the stability of verbal working memory across middle childhood, tested working memory in adolescents with NH or cochlear implants (CIs), explored whether signal enhancement can improve verbal working memory, and tested two hypotheses proposed to explain the poor verbal working memory of children with hearing loss: (a) Diminished auditory experience directly affects executive functions, including working memory; (b) degraded auditory inputs inhibit children's abilities to recover the phonological structure needed for encoding verbal material into storage. Fourteen-year-olds served as subjects: 55 with NH; 52 with CIs. Immediate serial recall tasks were used to assess working memory. Stimuli consisted of nonverbal, spatial stimuli and four kinds of verbal, acoustic stimuli: nonrhyming and rhyming words, and nonrhyming words with two kinds of signal enhancement: audiovisual and indexical. Analyses examined (a) stability of verbal working memory across middle childhood, (b) differences in verbal and nonverbal working memory, (c) effects of signal enhancement on recall, (d) phonological processing abilities, and (e) source of the diminished verbal working memory in adolescents with cochlear implants. Verbal working memory remained stable across middle childhood. Adolescents across groups performed similarly for nonverbal stimuli, but those with CIs displayed poorer recall accuracy for verbal stimuli; signal enhancement did not improve recall. Poor phonological sensitivity largely accounted for the group effect. The central executive for working memory is not affected by hearing loss or cochlear implantation. Instead, the phonological deficit faced by adolescents with CIs denigrates the representation in storage and augmenting the signal does not help.

Acoustic and perceptual appraisal of speech production in pediatric cochlear implant users

Children who acquire language through a cochlear implant (CI) provide a unique opportunity to study the physical and cognitive bases of speech perception, as the target language these children must acquire is the same as for children without hearing loss, but the input to the nervous system from a CI is very different from normal hearing. Psycholinguistic models of speech perception in adults with normal hearing propose that the phonological lexicon is organized in a phonetic similarity space based on linguistic features and phonemes, and that these units are important for spoken word recognition. Although the physical signal received through a CI is different from aurally perceived speech, we propose that the same abstract units of language are nonetheless a part of the linguistic knowledge of children who learn spoken language through CIs. In this paper, we model performance by children with CIs on tests of open-set word recognition using their performance on closed-set tests of feature identification. Our model of word recognition does not employ any free parameters, and therefore is highly constrained. Modeling the cognitive process of spoken word recognition on the basis of feature identification performance is important to both theoretical and clinical research. The model of word recognition we propose is appropriate for both listeners who use CIs and those with normal hearing. It is therefore a theoretical model of the process of spoken word recognition in general, as well one way to address the question of whether the cognitive process of spoken word recognition is the same for both CI users and those with normal hearing. Comparing observed and predicted open-set word recognition performance from closed-set identification tasks may be particularly useful clinically for the pediatric CI population. If differences between observed and predicted performance can be attributed to particular components of the model, then it may be possible to provide intervention that is maximally beneficial to the child.

Advances in hearing science have dramatically reduced the age of identification of hearing loss (HL) and improved intervention strategies, especially amplification methods. As a result, children with HL have earlier, better auditory access. How this early auditory experience affects language and speech abilities is not fully understood. This study addresses one aspect of speech production looking closely at point vowel duration. Although typically longer in children with HL and younger children, vowel duration has not been examined in early identified children with HL, and possible influence of device type (cochlear implant or hearing aids) is unclear. This study examines point vowel duration at four‐ and five‐year‐olds among children with normal hearing, cochlear implants, and hearing aids. Children produced /ae, ɑ, i, u/ in words modeled by the experimenter in a live‐voice, listen‐and‐repeat format. As expected, results indicate that children with HL and younger children produced longer vowels. Results a...

Normal-hearing (NH) infants participate in social exchanges soon after birth. What does vocal turn-taking look like in children with hearing loss after cochlear implantation? The study examined the prevalence and temporal structure of vocal turns during spontaneous interactions between mothers and their children with cochlear implants (CIs) over the first year after implantation compared to interactions between mothers and children with normal hearing. Mothers’ play with children with CIs (n = 12) were recorded at 3 (mean age 18.3 mo) and 9 (mean age 27.5 mo) months post CI. Mothers with age-matched hearing children (n = 12) were recorded at the corresponding time points. The CI group initially differed from the NH group in several ways (i.e., fewer vocal turns, more simultaneous speech, and longer between-speaker pauses) but progressed to NH levels by 9 months post CI, demonstrating the positive effects of CIs. Dyadic effects were also observed in the timing of mothers’ responses, which were related to those of their children. However, children with CIs continued to show an atypical pattern in the relative timing of between- vs within-speaker pauses across both test sessions, indicating a potentially protracted time course for the influence of CIs on dyadic interactions.

Is has been claimed that auditory feedback mechanisms enable monitoring and calibration of feedforward commands in speech production. Therefore, lack of auditory feedback may interfere with adequate compensation strategies in perturbed situations. This study investigates the effect of hearing status and a lip tube perturbation on vowel production. Eleven normal-hearing controls, and seventeen cochlear implant (CI) users (7 prelingually, 10 postlingually) were recorded during the production of the vowel /u/. Acoustic and articulatory analyses were conducted with and without a 15-mm-diam tube inserted between the lips. Recording sessions were also made before and after the perturbation, with and without auditory feedback. Deaf participants' auditory feedback was provided by the CI and interrupted by switching off their implant devices. Separate analyses were conducted on the acoustic parameters (first formant, second formant, and fundamental frequency) and on articulatory parameters. Results revealed a main effect of group and an interaction between condition and hearing status. Together, results suggest that auditory feedback plays an important role in speech compensation.

New Multi-Center Clinical Trial to Address Cochlear Implantation of Children with Asymmetric Hearing Loss or Single-Sided Deafness

Speech production involves a combination of feedforward and sensory feedback-based control mechanisms. The latter have been characterized in experiments involving real-time perturbations during speech. Unexpected perturbations of auditory feedback result in corrective motor responses with a minimum latency of approximately 100 ms after perturbation onset. These responses have been shown for both pitch and formant frequency perturbations, and the responsible neural circuitry includes portions of the superior temporal gyrus and ventral premotor cortex (vPMC). Corrective responses to somatosensory perturbations (such as a downward force applied to the jaw) occur approximately 50 ms after perturbation onset and are mediated by ventral somatosensory cortex and vPMC. The degree to which an individual weights auditory versus somatosensory feedback varies substantially. Such differences rely in part on differences in sensory acuity, e.g., a speaker with relatively poor hearing is likely to rely more heavily on somatosensory feedback control mechanisms than auditory feedback control mechanisms. Additionally, reliance on feedback control may be modulated to compensate for impairments in the feedforward system for speech, e.g., adults who stutter show a reduced reliance on auditory feedback control compared to fluent speakers, perhaps because auditory feedback can have a deleterious effect on speech initiation in stuttering.

This study investigates the production of voiceless alveolar and postalveolar fricatives and affricates by bilingual and monolingual children with hearing loss who use cochlear implants (CIs) and their peers with normal hearing (NH). Fifty-four children participated in our study, including 12 Spanish-English bilingual CI users (M = 6;0 [years;months]), 12 monolingual English-speaking children with CIs (M = 6;1), 20 bilingual children with NH (M = 6;5), and 10 monolingual English-speaking children with NH (M = 5;10). Picture elicitation targeting /s/, /tʃ/, and /ʃ/ was administered. Repeated-measures analyses of variance comparing group means for frication duration, rise time, and centroid frequency were conducted for the effects of CI use and bilingualism. All groups distinguished the target sounds in the 3 acoustic parameters examined. Regarding frication duration and rise time, the Spanish productions of bilingual children with CIs differed from their bilingual peers with NH. English frication duration patterns for bilingual versus monolingual CI users also differed. Centroid frequency was a stronger place cue for children with NH than for children with CIs. Patterns of fricative and affricate production display effects of bilingualism and diminished signal, yielding unique patterns for bilingual and monolingual CI users.