Individual differences in speech and nonspeech processing among normal‐hearing subjects

As reading is inherently a multisensory, audiovisual (AV) process where visual symbols (i.e., letters) are connected to speech sounds, the question has been raised whether individuals with reading difficulties, like children with developmental dyslexia (DD), have broader impairments in multisensory processing. This question has been posed before, yet it remains unanswered due to (a) the complexity and contentious etiology of DD along with (b) lack of consensus on developmentally appropriate AV processing tasks. We created an ecologically valid task for measuring multisensory AV processing by leveraging the natural phenomenon that speech perception improves when listeners are provided visual information from mouth movements (particularly when the auditory signal is degraded). We designed this AV processing task with low cognitive and linguistic demands such that children with and without DD would have equal unimodal (auditory and visual) performance. We then collected data in a group of 135 children (age 6.5-15) with an AV speech perception task to answer the following questions: (1) How do AV speech perception benefits manifest in children, with and without DD? (2) Do children all use the same perceptual weights to create AV speech perception benefits, and (3) what is the role of phonological processing in AV speech perception? We show that children with and without DD have equal AV speech perception benefits on this task, but that children with DD rely less on auditory processing in more difficult listening situations to create these benefits and weigh both incoming information streams differently. Lastly, any reported differences in speech perception in children with DD might be better explained by differences in phonological processing than differences in reading skills. RESEARCH HIGHLIGHTS: Children with versus without developmental dyslexia have equal audiovisual speech perception benefits, regardless of their phonological awareness or reading skills. Children with developmental dyslexia rely less on auditory performance to create audiovisual speech perception benefits. Individual differences in speech perception in children might be better explained by differences in phonological processing than differences in reading skills.

Studies of speech processing have generally made the implicit assumption that typically developed speakers can distinguish all sounds of their mother tongue in perception and production. As such, individual differences in speech processing is usually studied with speakers differing in training/experience (Strait & Kraus, 2011), or populations with developmental disorders (Facoetti et al., 2010), and few investigations have been conducted without the effects from training/experience among typically-developed individuals. However, sociolinguists have long recognized that native speakers vary in their ability to discriminate speech sounds in their language, and enormous variability exists especially during a sound change in progress.
\n
\nTaking the opportunity of an on-going tone merging in Hong Kong Cantonese, this thesis aims to systematically investigate individual differences of native speech perception, production, and their relationships with cognitive functions among typically-developed speakers. Three participant groups were recruited, who presented respectively the pattern of good perception and good production of all Cantonese tones [+Per+Pro], that of good perception of all tones but poor production of specifically the T2/T5 distinction [+Per-Pro], and that of poor perception and production of specifically the T2/T5 distinction [-Per-Pro]. Behavioral and neural measures of tone perception included reaction time, discrimination sensitivity index, and components of event-related potentials (ERPs) – the mismatch negativity (MMN), P3a, and rise time of amplitude envelope. Acoustic measurements were used to evaluate tone production in terms of both pitch and amplitude rise time. Components of attention and working memory in auditory and visual modalities are assessed with published cognitive test batteries.
\n
\nThe results show that, apart from the expected differences in accuracy and discrimination sensitivity of tone perception, both [+Per-Pro] and [-Per-Pro] took significantly longer to discriminate between tones than [+Per+Pro]. As for the performance in production, besides the differences in pitch offset, both [+Per-Pro] and [-Per-Pro] showed decreased differentiation in rise time between the two rising tones in production, compared with [+Per+Pro].With respect to the brain responses reflected in the MMN and P3a to pitch deviations among tones, [-Per-Pro] showed smaller and slower responses than one or both of the other two groups, but [+Per+Pro] and [+Per-Pro] did not differ from each other. However, both [+Per-Pro] and [-Per-Pro] showed weaker neural responses compared with [+Per+Pro] to the rise time of T5. In addition, [-Per-Pro] was poorer in tasks pertaining to the ability of attention switching/shifting regardless of modality than one or both of the other two groups, but [+Per+Pro] and [+Per-Pro] did not differ from each other.
\n
\nFurther correlation and regression analyses reveal that both pitch contour/height and rise time contributed to distinctive perception and production of rising tones, measures of perception (behavioral and neural) and production were correlated with each other, and attentional shifting in visual and auditory modalities significantly predicted performances of discrimination and production. Taken together, the findings of the present study suggest that attentional switching influences the quality of phonological representations, resulting in different degrees of distinctiveness in perception and production among typically-developed individuals.

Understanding speech in demanding environments is essential for daily communication. Previous research has shown that even highly proficient bilinguals may experience greater difficulty than monolinguals in understanding speech in noise. In the present study we further address this issue by examining the effects of varying task demand, fatigue, and practice on speech perception by bilinguals. One group of monolingual English listeners and three groups of Spanish–English bilinguals (bilingual since childhood, bilingual since teenager, and bilingual since adulthood) listened to nonsense syllables presented in noise and at increasing presentation rates. Listeners twice completed the two speech perception tasks on two days of testing. On one day the speech tasks were preceded by and on the other day followed by approximately 40 minutes of testing on nonspeech auditory tasks. Monolingual and bilingual listeners’ overall performance, their performance across the two days of testing (practice effect), and their performance across test orders (fatigue effect) will be compared. Preliminary results from two monolingual and two highly proficient bilinguals (bilingual since age 16 or earlier) show similar overall performance on both speech tasks and no effect of fatigue (test order), but suggest a greater practice effect for bilinguals’ perception of syllables in noise.

Purpose The extant literature suggests that individual differences in speech perception can be linked to broad receptive language phenotype. For example, a recent study found that individuals with a smaller receptive vocabulary showed diminished lexically guided perceptual learning compared to individuals with a larger receptive vocabulary. Here, we examined (a) whether such individual differences stem from variation in reliance on lexical information or variation in perceptual learning itself and (b) whether a relationship exists between lexical recruitment and lexically guided perceptual learning more broadly, as predicted by current models of lexically guided perceptual learning. Method In Experiment 1, adult participants (n = 70) completed measures of receptive and expressive language ability, lexical recruitment, and lexically guided perceptual learning. In Experiment 2, adult participants (n = 120) completed the same lexical recruitment and lexically guided perceptual learning tasks to provide a high-powered replication of the primary findings from Experiment 1. Results In Experiment 1, individuals with weaker receptive language ability showed increased lexical recruitment relative to individuals with higher receptive language ability; however, receptive language ability did not predict the magnitude of lexically guided perceptual learning. Moreover, the results of both experiments converged to show no evidence indicating a relationship between lexical recruitment and lexically guided perceptual learning. Conclusion The current findings suggest that (a) individuals with weaker language ability demonstrate increased reliance on lexical information for speech perception compared to those with stronger receptive language ability; (b) individuals with weaker language ability maintain an intact perceptual learning mechanism; and, (c) to the degree that the measures used here accurately capture individual differences in lexical recruitment and lexically guided perceptual learning, there is no graded relationship between these two constructs.

Smith and Lewicki, Neural Comp. 17, 19–45, 2005a; Adv. Neural. Inf. Process. Syst. 17, 1289–1296, 2005b; Nature, 439, 7079, 2006, demonstrated that mammalian hearing follows an efficient coding principle (Barlow, Sensory Communications, 217–234, 1961; Atick, Network, 3(2), 213–251, 1992; Simoncelli and Olshausen, Ann. Rev. Neurosci., 24, 1193–1216, 2001; Laughlin and Sejnowski, ‘‘Communications in Neuronal Networks,’’ Science, 301, 1870–1874 2003). Auditory neurons efficiently code for natural sounds in the environment, maximizing information rate while minimizing coding cost (Shannon, Science, 270, 303–304, 1948). Applying the same analysis to speech coding suggests that speech acoustics are optimally adapted to the mammalian auditory code (Smith and Lewicki, Neural Comp. 17, 19–45, 2005a; Adv. Neural. Inf. Process. Syst. 17, 1289–1296, 2005b; Nature, 439, 7079, 2006). The present work applies this efficient coding theory to the problem of speech perception in individuals using cochlear implants (CI), for which there exist vast individual differences in speech perception and spectral resolution (Zeng et al., Auditory Prostheses and Electric Hearing, 20, 1–14, 2004). A machine-learning method for CI filterbank design based on the efficient-coding hypothesis is presented. Further, a pair of experiments to evaluate this approach using noise-excited vocoder speech (Shannon et al., Bell Systems Technical Journal 27, 379–423, 623–656, 1995) is described. Participants’ recognition of continuous speech and isolated syllables is significantly more accurate for speech filtered through the theoretically-motivated efficient-coding filterbank relative to the standard cochleotopic filterbank, particularly for speech transients. These findings offer insight in CI design and provide behavioral evidence for efficient coding in human perception.

The function of rapid perceptual learning for speech in adult listeners is poorly understood. On the one hand, perceptual learning of speech results in rapid and long-lasting improvements in the perception of many types of distorted and degraded speech signals. On the other hand, this learning is highly specific to stimuli that were encountered during its acquisition. Therefore, it is unclear whether past perceptual learning could support future speech perception under ecological conditions. Here, we hypothesize that rapid perceptual learning is a resource that is recruited when new speech challenges are encountered and used to support perception under those specific conditions. We review three lines of evidence related to aspects of this hypothesis – about the specificity of learning, the general nature of associations between rapid perceptual learning and speech perception and rapid perceptual-learning in populations with poor perception under adverse conditions.

Cochlear implantation constitutes a successful therapy of inner ear deafness, with the majority of patients showing good outcomes. There is, however, still some unexplained variability in outcomes with a number of cochlear-implant (CI) users, showing major limitations in speech comprehension. The current study used a multimodal diagnostic approach combining single-photon emission computed tomography (SPECT) and electroencephalography (EEG) to examine the mechanisms underlying speech processing in postlingually deafened CI users (N = 21). In one session, the participants performed a speech discrimination task, during which a 96-channel EEG was recorded and the perfusions marker 99mTc-HMPAO was injected intravenously. The SPECT scan was acquired 1.5 h after injection to measure the cortical activity during the speech task. The second session included a SPECT scan after injection without stimulation at rest. Analysis of EEG and SPECT data showed N400 and P600 event-related potentials (ERPs) particularly evoked by semantic violations in the sentences, and enhanced perfusion in a temporo-frontal network during task compared to rest, involving the auditory cortex bilaterally and Broca’s area. Moreover, higher performance in testing for word recognition and verbal intelligence strongly correlated to the activation in this network during the speech task. However, comparing CI users with lower and higher speech intelligibility [median split with cutoff + 7.6 dB signal-to-noise ratio (SNR) in the Göttinger sentence test] revealed for CI users with higher performance additional activations of parietal and occipital regions and for those with lower performance stronger activation of superior frontal areas. Furthermore, SPECT activity was tightly coupled with EEG and cognitive abilities, as indicated by correlations between (1) cortical activation and the amplitudes in EEG, N400 (temporal and occipital areas)/P600 (parietal and occipital areas) and (2) between cortical activation in left-sided temporal and bilateral occipital/parietal areas and working memory capacity. These results suggest the recruitment of a temporo-frontal network in CI users during speech processing and a close connection between ERP effects and cortical activation in CI users. The observed differences in speech-evoked cortical activation patterns for CI users with higher and lower speech intelligibility suggest distinct processing strategies during speech rehabilitation with CI.

The present study examined whether abnormal cerebral control was evident in the lip movements of adult stutterers (AS) compared to non-stutterers (ANS), as demonstrated during speech and non-speech tasks. Electromyographic (EMG) activity during production of words with initial /f/ and /p/ phonemes, single-sentence production and lip pursing were measured. Differences between AS and ANS were evident in all tasks. For AS, the highest EMG amplitude was in the region of the left lower lip, which is indicative of greater right hemisphere participation. However, in ANS the right lower lip displayed the highest EMG amplitude, suggesting greater left hemisphere participation. Further, AS showed greater asynchronous lip activity than ANS for all tasks. These results support the hypotheses of reversed lateralization for speech and non-speech processing and reduced coordination of speech musculature in AS.

In order to investigate the relation between reading accuracy and speech processing, 20 children from grades 2 and 3 who were skilled in reading were compared with 20 less skilled readers on a speech perception and production task. The two groups of readers were indistinguishable in their production of the two-syllable words dippy, deepy, tippy, and teepy and in their perception of the stop consonants /d/ versus /t/. Less skilled readers were significantly less accurate than the skilled readers in a vowel identification task involving the lax and tense high vowels /i/ and /i/. The error pattern for vowel identification was similar across groups, with both groups making fewer errors when short and longer segments were alternated. The results imply that vowel phonemes are less securely represented in the perceptual system of less skilled readers than are consonant phonemes. In addition, the results raise the possibility that a selective perceptual impairment underlies at least some of the phonemic awareness problems that have been associated with poor reading.

Previous studies of the perception of time-varying sinusoidal (TVS) sentences suggested that amplitude comodulation may serve as a grouping cue in speech perception [T. D. Carrell, J. Acoust. Soc. Am. Suppl. 1 86, S102 (1989)]. The purpose of the present investigation was to further explore that idea and to evaluate whether the effects observed in TVS studies are related to other temporally based phenomena such as comodulation masking release (CMR). If a common process underlies the perception of amplitude-modulated speech and nonspeech stimuli, then grouping effects should occur in speech perception in a manner similar to that observed in nonspeech studies when the amplitude modulation characteristics are similar. Examined in the present study was the effect of amplitude modulation rate on intelligibility of TVS sentences as compared with rate effects in CMR phenomena. Amplitude modulation rates were 5, 10, 20, 50, 100, and 200 Hz. Although the results suggested that comodulation is important in speech perception, effective rates may be different from those observed in nonspeech tasks. These findings are important for our understanding of auditory object formation of both speech and nonspeech stimuli.

This study explored auditory speech processing and comprehension abilities in 5–8-year-old monolingual Hungarian children with functional articulation disorders (FADs) and their typically developing peers. Our main hypothesis was that children with FAD would show co-existing auditory speech processing disorders, with different levels of these skills depending on the nature of the receptive processes. The tasks included (i) sentence and non-word repetitions, (ii) non-word discrimination and (iii) sentence and story comprehension. Results suggest that the auditory speech processing of children with FAD is underdeveloped compared with that of typically developing children, and largely varies across task types. In addition, there are differences between children with FAD and controls in all age groups from 5 to 8 years. Our results have several clinical implications.

BackgroundIn our Dynamic Pathways, account, we hypothesized that childhood stuttering reflects an impairment in speech sensorimotor control that is conditioned by cognitive, linguistic, and emotional factors. The purpose of this study was to investigate potential differences in levels of sympathetic arousal during performance of speech and non-speech tasks between children who do and do not stutter.MethodsSeventy-two preschool-aged children participated in the study, 47 children who stutter (CWS; 38 boys) and 25 children who do not stutter (CWNS; 18 boys). We recorded skin conductance and blood pulse volume (BPV) signals, indices of sympathetic arousal, during higher/lower load speech tasks (structured sentence production and picture description) and non-speech tasks (jaw wagging and forceful blowing). We included a measure that reflects children’s attitudes about their communication skills and a parent-report assessment of temperament.ResultsWe found no significant differences between preschool CWS and CWNS in phasic skin conductance response amplitude or frequency, BPV, and pulse rate for any of the experimental tasks. However, compared to CWNS, CWS had, on average, significantly higher skin conductance levels (SCL), indexing slowly changing tonic sympathetic activity, across both speech and non-speech experimental conditions. We found distinctive task-related profiles of sympathetic arousal in both groups of preschool children. Most children produced the highest levels of sympathetic arousal in the physically demanding blowing task rather than in speech, as seen in previous studies of adults. We did not find differences in temperament between the two groups of preschool children nor a relationship among behavioral indices of temperament and communication attitude and physiological measures of sympathetic arousal.ConclusionWe did not find that atypically high, speech-related sympathetic arousal is a significant factor in early childhood stuttering. Rather, CWS had higher, on average, task-related tonic SCLs across speech and non-speech tasks. A relationship among behavioral measures of temperament and physiological measures of sympathetic arousal was not confirmed. Key questions for future experiments are how the typical coupling of sympathetic and speech sensorimotor systems develops over childhood and adolescence and whether task related developmental profiles follow a different course in children who continue to stutter.

Temporal order judgment (TOJ) in dichotic listening can be a difficult task. Previous experiments that used two speech stimuli on each trial (S/S) obtained sizable error rates when subjects were required to report which ear led (TOJ-by-ear). When subjects were required to identify the leading stimulus (TO J-by-stimulus), the error rate increased substantially. Apparently, the two speech stimuli were competing for analysis by the same processor, and so were overloading it. The present experiment used the same TOJ tasks, but presented a speech and a nonspeech stimulus on each trial (S/NS). The error rate was comparable to that of S/S for TO J-by-ear, but did not increase for TO J-by-stimulus. This would be expected if the speech and nonspeech stimuli are being sent to different processors, each of which performs its analysis without interference from the other. The interpretation of the data given here is consistent with the results of standard identification experiments reported elsewhere: when asked to identify both stimuli on each dichotic trial, subjects made many errors on S/S, while performance was virtually error-free on S/NS.

Audiovisual Integration of Speech in a Bistable Illusion

Speech production difficulties are apparent in people who stutter (PWS). PWS also have difficulties in speech perception compared to controls. It is unclear whether the speech perception difficulties in PWS are independent of, or related to, their speech production difficulties. To investigate this issue, functional MRI data were collected on 13 PWS and 13 controls whilst the participants performed a speech production task and a speech perception task. PWS performed poorer than controls in the perception task and the poorer performance was associated with a functional activity difference in the left anterior insula (part of the speech motor area) compared to controls. PWS also showed a functional activity difference in this and the surrounding area [left inferior frontal cortex (IFC)/anterior insula] in the production task compared to controls. Conjunction analysis showed that the functional activity differences between PWS and controls in the left IFC/anterior insula coincided across the perception and production tasks. Furthermore, Granger Causality Analysis on the resting-state fMRI data of the participants showed that the causal connection from the left IFC/anterior insula to an area in the left primary auditory cortex (Heschl’s gyrus) differed significantly between PWS and controls. The strength of this connection correlated significantly with performance in the perception task. These results suggest that speech perception difficulties in PWS are associated with anomalous functional activity in the speech motor area, and the altered functional connectivity from this area to the auditory area plays a role in the speech perception difficulties of PWS.