The Psychophysics of Speech Perception

The present study provides an elaboration of the "double-weak" theory of speech perception proposed by Nearey (1990, 1991). In this framework, the objects of speech perception (and production) are viewed as neither primarily auditory nor primarily gestural; rather, they are abstract, symbolic elements lawfully constrained to map onto relatively simple (but not entirely transparent) patterns in both domains. Speech cannot be understood unless both articulation and acoustics are considered: Many production strategies appear to be directed at achieving acoustically-oriented goals, yet most context effects in speech perception seem to be motivated by the consequences of gestural overlap. The double-weak framework suggests that speech perception and speech production are less-than-perfect inverses of each other. Despite long-term accommodation of each for the demands of the other, real-time production and perception may operate as autonomous subsystems. A family of perceptual models is discussed that provides varying degrees of approximation to "ideal solutions" (in the sense of minimizing error rate) to classifying production data exhibiting contextual interactions. Members of this family that provide substantial, yet incomplete, compensation for the consequences of gestural overlap appear to be adequate to account for the results of many speech perception experiments. Such partial perceptual compensation allows, in principle, for the kind of imperfect "error correction" discussed by Ohala (1981, 1990) in conjunction with hypo- and hypercorrection phenomena.

That early word learning and speech perception skills have different sensitive periods and that very early implantation may affect later vocabulary outcomes more than speech perception outcomes. Several studies have found that deaf children who receive cochlear implants before 3 years of age tend to have better speech perception outcomes than children implanted later. Recent studies have not found age-at-implantation effects on speech perception or central auditory processing among children implanted younger than 2 years, suggesting that there may be a sensitive period for speech perception skills that closes by around 3 years of age. There has been very little work investigating possible sensitive periods for other language skills, such as the ability to learn words. Recent work suggests the possibility that the development of word-learning skills may have an earlier sensitive period than the development of speech perception skills. Assess speech perception and vocabulary outcomes in children implanted before 13 months of age and in children implanted between 16 and 23 months of age. Children implanted during the first year of life had better vocabulary outcomes than children implanted during the second year of life. However, earlier implanted children did not show better speech perception outcomes than later implanted children. There may be an earlier sensitive period for developing the ability to associate the sound patterns of words to their referents than for developing speech perception and central auditory processing skills.

The perception of any given sound is influenced by surrounding sounds. When successive sounds differ in their spectral compositions, these differences may be perceptually magnified, resulting in spectral contrast effects (SCEs). For example, listeners are more likely to perceive /ɪ/ (low F1) following sentences with higher F1 frequencies; listeners are also more likely to perceive /ɛ/ (high F1) following sentences with lower F1 frequencies. Previous research showed that SCEs for vowel categorization were attenuated when sentence contexts were spoken by different talkers [Assgari and Stilp. (2015). J. Acoust. Soc. Am. 138(5), 3023-3032], but the locus of this diminished contextual influence was not specified. Here, three experiments examined implications of variable talker acoustics for SCEs in the categorization of /ɪ/ and /ɛ/. The results showed that SCEs were smaller when the mean fundamental frequency (f0) of context sentences was highly variable across talkers compared to when mean f0 was more consistent, even when talker gender was held constant. In contrast, SCE magnitudes were not influenced by variability in mean F1. These findings suggest that talker variability attenuates SCEs due to diminished consistency of f0 as a contextual influence. Connections between these results and talker normalization are considered.

The phonetic distinction between the stop consonant /b/ and semivowel /w/ is cued primarily by duration of the initial formant transition. Syllable duration, however, is a context that influences human adults' and infants' perception of this distinction. For example, when overall syllable duration is long (296 ms), infants show poor discrimination of syllables with transition durations of 16 and 40 ms, but good discrimination of syllables with transition durations of 40 and 64 ms. However, when overall syllable duration is short (80 ms), the opposite is true [P. D. Eimas and J. L. Miller, Science 209, 1140–1141 (1980)]. The current study addresses whether rhesus and Japanese macaques' perception of the /b-w/ distinction is also influenced by syllable duration: Six CV syllables used in the Eimas and Miller (1980) study were presented to five macaques using a same-different discrimination paradigm. Results showed that macaques' pattern of discrimination was similar to the pattern shown by infants, thus supporting an auditory-based mechanism for the effect of syllable duration in phonetic perception. [Work supported by NSF and NIH.]

All perception takes place in context. Recognition of a given speech sound is influenced by the acoustic properties of surrounding sounds. When the spectral composition of earlier (context) sounds (e.g., more energy at lower first formant [F1] frequencies) differs from that of a later (target) sound (e.g., vowel with intermediate F1), the auditory system magnifies this difference, biasing target categorization (e.g., towards higher-F1 /ɛ/). Historically, these studies used filters to force context sounds to possess desired spectral compositions. This approach is agnostic to the natural signal statistics of speech (inherent spectral compositions without any additional manipulations). The auditory system is thought to be attuned to such stimulus statistics, but this has gone untested. Here, vowel categorization was measured following unfiltered (already possessing the desired spectral composition) or filtered sentences (to match spectral characteristics of unfiltered sentences). Vowel categorization was biased in both cases, with larger biases as the spectral prominences in context sentences increased. This confirms sensitivity to natural signal statistics, extending spectral context effects in speech perception to more naturalistic listening conditions. Importantly, categorization biases were smaller and more variable following unfiltered sentences, raising important questions about how faithfully experiments using filtered contexts model everyday speech perception.

To determine the diagnostic yield of the nerve-muscle-skin (NMS) biopsy in paediatric neurology practice. A consecutive series of 98 paediatric NMS biopsies done 1989-1994 retrospectively reviewed in the context of pre-biopsy clinical and laboratory parameters. Bivariate associations based on chi-square test. Unconfounded associations between pre-biopsy variables and positive diagnostic yield (PDY) assessed by multiple logistic regression. Fifty seven out of 98 patients central (global delay, seizures, abnormal CNS imaging) process; 41/98 patients peripheral (motor delay, weakness) process, electromyography-nerve conduction studies (EMG-NCS) 87/98 cases; abnormal 43/87. Positive diagnostic yield (PDY) in 42/98 (43%) biopsies. Statistically significant bivariate associations between PDY and pre-biopsy; age, presenting symptom, developmental delay, weakness, reflexes, CPK, lactate, EMG-NCS and process. Unconfounded associations demonstrated with PDY and age, reflexes and process. The presence of a peripheral process or an abnormal EMG-NCS strongly predictive of PDY: 34/41 (83%) peripheral process cases had PDY, 32/40 (80%) abnormal EMG-NCS cases had PDY, and 29/31 (93.5%) peripheral process and abnormal EMG-NCS cases had PDY. Abnormal EMG-NCS with central process improved PDY to 3/9 (33%) from 4/37 (11%) for normal EMG-NCS. NMS biopsy is a valuable diagnostic tool, particularly in the context of a suspected peripheral process or a central processes with an abnormal EMG-NCS.

A number of recent studies have examined the effects of phonological variation on the perception of speech. These studies show that both the lexical representations of words and the mechanisms of lexical access are organized so that natural, systematic variation is tolerated by the perceptual system, while a general intolerance of random deviation is maintained. Lexical abstraction distinguishes between phonetic features that form the invariant core of a word and those that are susceptible to variation. Phonological inference relies on the context of surface changes to retrieve the underlying phonological form. In this article we present a model of these processes in speech perception, based on connectionist learning techniques. A simple recurrent network was trained on the mapping from the variant surface form of speech to the underlying form. Once trained, the network exhibited features of both abstraction and inference in its processing of normal speech, and predicted that similar behavior will be found in the perception of nonsense words. This prediction was confirmed in subsequent research (Gaskell & Marslen‐Wilson, 1994).

Although cochlear implant patients are achieving increasingly higher levels of performance, speech perception in noise continues to be problematic. The newest generations of implant speech processors are equipped with preprocessing and/or external accessories that are purported to improve listening in noise. Most speech perception measures in the clinical setting, however, do not provide a close approximation to real-world listening environments. To assess speech perception for adult cochlear implant recipients in the presence of a realistic restaurant simulation generated by an eight-loudspeaker (R-SPACE) array in order to determine whether commercially available preprocessing strategies and/or external accessories yield improved sentence recognition in noise. Single-subject, repeated-measures design with two groups of participants: Advanced Bionics and Cochlear Corporation recipients. Thirty-four subjects, ranging in age from 18 to 90 yr (mean 54.5 yr), participated in this prospective study. Fourteen subjects were Advanced Bionics recipients, and 20 subjects were Cochlear Corporation recipients. Speech reception thresholds (SRTs) in semidiffuse restaurant noise originating from an eight-loudspeaker array were assessed with the subjects' preferred listening programs as well as with the addition of either Beam preprocessing (Cochlear Corporation) or the T-Mic accessory option (Advanced Bionics). In Experiment 1, adaptive SRTs with the Hearing in Noise Test sentences were obtained for all 34 subjects. For Cochlear Corporation recipients, SRTs were obtained with their preferred everyday listening program as well as with the addition of Focus preprocessing. For Advanced Bionics recipients, SRTs were obtained with the integrated behind-the-ear (BTE) mic as well as with the T-Mic. Statistical analysis using a repeated-measures analysis of variance (ANOVA) evaluated the effects of the preprocessing strategy or external accessory in reducing the SRT in noise. In addition, a standard t-test was run to evaluate effectiveness across manufacturer for improving the SRT in noise. In Experiment 2, 16 of the 20 Cochlear Corporation subjects were reassessed obtaining an SRT in noise using the manufacturer-suggested "Everyday," "Noise," and "Focus" preprocessing strategies. A repeated-measures ANOVA was employed to assess the effects of preprocessing. The primary findings were (i) both Noise and Focus preprocessing strategies (Cochlear Corporation) significantly improved the SRT in noise as compared to Everyday preprocessing, (ii) the T-Mic accessory option (Advanced Bionics) significantly improved the SRT as compared to the BTE mic, and (iii) Focus preprocessing and the T-Mic resulted in similar degrees of improvement that were not found to be significantly different from one another. Options available in current cochlear implant sound processors are able to significantly improve speech understanding in a realistic, semidiffuse noise with both Cochlear Corporation and Advanced Bionics systems. For Cochlear Corporation recipients, Focus preprocessing yields the best speech-recognition performance in a complex listening environment; however, it is recommended that Noise preprocessing be used as the new default for everyday listening environments to avoid the need for switching programs throughout the day. For Advanced Bionics recipients, the T-Mic offers significantly improved performance in noise and is recommended for everyday use in all listening environments.

Speech perception is influenced by context. This influence can help to alleviate issues that arise from the extreme acoustic variability of speech. Two examples of contextual influences are talker normalization and spectral contrast effects (SCEs). Talker normalization occurs when listeners hear different talkers causing speech perception to be slower and less accurate. SCEs occur when spectral characteristics change from context sentences to target vowels and speech perception is biased by that change. It has been demonstrated that SCEs are restrained when contexts are spoken by different talkers (Assgari & Stilp, 2015). However, what about hearing different talkers restrains these effects was not entirely clear. In addition, while these are both considered contextual influences on speech perception, they have never been formally related to each other. The series of studies reported here served two purposes. First, these studies sought to establish why hearing different talkers restrained SCEs. Results indicate that variability in pitch (as measured by fundamental frequency), a primary acoustic cue to talker changes, restricts the influence of spectral changes on speech perception. Second, these studies attempted to relate talker normalization and SCEs by measuring them concurrently. Talker normalization (as measured by response times) and SCEs were evident in the same task suggesting that they act on speech perception at the same time. Further, these measures of talker normalization were shown to be influenced by f0 variability suggesting that SCEs and talker normalization are both related to f0 variability. However, no relationship between individual’s SCEs and response times was found. Possible reasons why f0 variability may restrain context effects are discussed.

The question of hemispheric lateralization of neural processes is one that is pertinent to a range of subdisciplines of cognitive neuroscience. Language is often assumed to be left-lateralized in the human brain, but there has been a long running debate about the underlying reasons for this. We addressed this problem with fMRI by identifying the neural responses to amplitude and spectral modulations in speech and how these interact with speech intelligibility to test previous claims for hemispheric asymmetries in acoustic and linguistic processes in speech perception. We used both univariate and multivariate analyses of the data, which enabled us to both identify the networks involved in processing these acoustic and linguistic factors and to test the significance of any apparent hemispheric asymmetries. We demonstrate bilateral activation of superior temporal cortex in response to speech-derived acoustic modulations in the absence of intelligibility. However, in a contrast of amplitude-modulated and spectrally modulated conditions that differed only in their intelligibility (where one was partially intelligible and the other unintelligible), we show a left dominant pattern of activation in STS, inferior frontal cortex, and insula. Crucially, multivariate pattern analysis showed that there were significant differences between the left and the right hemispheres only in the processing of intelligible speech. This result shows that the left hemisphere dominance in linguistic processing does not arise because of low-level, speech-derived acoustic factors and that multivariate pattern analysis provides a method for unbiased testing of hemispheric asymmetries in processing.

Recent experiments show that nonspeech sounds can shift the perceptual boundaries between the phonetic categories of speech [e.g., Holt, Psych. Sci. 16, 305–312 (2005)]. The current experiment investigates whether nonspeech sounds will also affect the speed of access to words. Subjects received prime-target pairs in which the target was one of the three words deed, dad, and dude, preceded either by tone complexes which matched the frequencies of the first two formants of each target or by a neutral tone midway between the ranges of F1 and F2. Every prime was paired with every target in a word-identification task. Results demonstrated a priming effect of the nonspeech sounds on acoustically similar speech. Significant differences were found in reaction time, with subjects responding faster when targets were preceded by matched primes than when they were preceded by mismatched or neutral primes. These results support a role for general auditory processes in speech perception and lexical access. [Work supported by NIDCD.]

The paper presents a concept of lifelong plasticity of peripheral vision. Central vision processing is accepted as critical and irreplaceable for normal perception in humans. While peripheral processing chiefly carries information about motion stimuli features and redirects foveal attention to new objects, it can also take over functions typical for central vision. Here I review the data showing the plasticity of peripheral vision found in functional, developmental, and comparative studies. Even though it is well established that afferent projections from central and peripheral retinal regions are not established simultaneously during early postnatal life, central vision is commonly used as a general model of development of the visual system. Based on clinical studies and visually deprived animal models, I describe how central and peripheral visual field representations separately rely on early visual experience. Peripheral visual processing (motion) is more affected by binocular visual deprivation than central visual processing (spatial resolution). In addition, our own experimental findings show the possible recruitment of coarse peripheral vision for fine spatial analysis. Accordingly, I hypothesize that the balance between central and peripheral visual processing, established in the course of development, is susceptible to plastic adaptations during the entire life span, with peripheral vision capable of taking over central processing.

The ability to perceive speech is related to the listener's ability to differentiate among frequencies (i.e., spectral resolution). Cochlear implant (CI) users exhibit variable speech-perception and spectral-resolution abilities, which can be attributed in part to the extent of electrode interactions at the periphery (i.e., spatial selectivity). However, electrophysiological measures of peripheral spatial selectivity have not been found to correlate with speech perception. The purpose of this study was to evaluate auditory processing at the periphery and cortex using both simple and spectrally complex stimuli to better understand the stages of neural processing underlying speech perception. The hypotheses were that (1) by more completely characterizing peripheral excitation patterns than in previous studies, significant correlations with measures of spectral selectivity and speech perception would be observed, (2) adding information about processing at a level central to the auditory nerve would account for additional variability in speech perception, and (3) responses elicited with spectrally complex stimuli would be more strongly correlated with speech perception than responses elicited with spectrally simple stimuli. Eleven adult CI users participated. Three experimental processor programs (MAPs) were created to vary the likelihood of electrode interactions within each participant. For each MAP, a subset of 7 of 22 intracochlear electrodes was activated: adjacent (MAP 1), every other (MAP 2), or every third (MAP 3). Peripheral spatial selectivity was assessed using the electrically evoked compound action potential (ECAP) to obtain channel-interaction functions for all activated electrodes (13 functions total). Central processing was assessed by eliciting the auditory change complex with both spatial (electrode pairs) and spectral (rippled noise) stimulus changes. Speech-perception measures included vowel discrimination and the Bamford-Kowal-Bench Speech-in-Noise test. Spatial and spectral selectivity and speech perception were expected to be poorest with MAP 1 (closest electrode spacing) and best with MAP 3 (widest electrode spacing). Relationships among the electrophysiological and speech-perception measures were evaluated using mixed-model and simple linear regression analyses. All electrophysiological measures were significantly correlated with each other and with speech scores for the mixed-model analysis, which takes into account multiple measures per person (i.e., experimental MAPs). The ECAP measures were the best predictor. In the simple linear regression analysis on MAP 3 data, only the cortical measures were significantly correlated with speech scores; spectral auditory change complex amplitude was the strongest predictor. The results suggest that both peripheral and central electrophysiological measures of spatial and spectral selectivity provide valuable information about speech perception. Clinically, it is often desirable to optimize performance for individual CI users. These results suggest that ECAP measures may be most useful for within-subject applications when multiple measures are performed to make decisions about processor options. They also suggest that if the goal is to compare performance across individuals based on a single measure, then processing central to the auditory nerve (specifically, cortical measures of discriminability) should be considered.

There is a lack of high level of evidence studies comparing the effect of different treatment options for single-sided deafness (SSD). To determine the effect of a cochlear implant (CI), bone conduction device (BCD), contralateral routing of signals hearing aid (CROS), and no treatment on speech perception in noise outcomes in patients with SSD. In this single-center randomized clinical trial, adult patients with SSD were randomized into 3 groups: CI; a trial period with first a BCD on a headband and then a CROS; or a trial period with first a CROS and then a BCD on a headband. After the trial periods, patients opted for a BCD, CROS, or no treatment. Measurements were completed at baseline and at 3, 6, 12, and 24 months of follow-up. Data were collected from July 2014 to October 2021, and data were analyzed from December 2022 to May 2023. CI, BCD then CROS, or CROS then BCD. The primary outcome was the speech reception threshold in noise (SRTn), measured with speech and noise from the front (S0N0), speech directed to the poor ear and noise to the better ear (SpeNbe), and vice versa (SbeNpe). The secondary outcome was disease-specific quality of life (QOL). Of 120 included patients, 60 (50.0%) were female, and the mean (SD) age at inclusion was 53.0 (12.1) years. At the start of follow-up, 28 patients received a CI, 25 a BCD, 34 a CROS, and 26 chose no treatment. At 24 months, the CI group had significantly better speech perception in noise scores than the BCD group (difference in SRTn: SbeNpe, -4.7 dB; 95% confidence interval, -6.6 to -3.0; SpeNbe, -2.2 dB; 95% confidence interval, -4.6 to -1.1), the CROS group (difference in SRTn: S0N0, -1.3 dB; 95% confidence interval, -1.7 to -0.2; SbeNpe, -5.3 dB; 95% confidence interval, -6.0 to -3.1), and the no treatment group (difference in SRTn: SpeNbe, -6.3 dB; 95% confidence interval, -7.5 to -4.9). Compared with the no treatment group, the BCD and CROS groups showed significantly better (difference in SRTn for SpeNbe, -4.1 dB [95% confidence interval, -5.2 to -1.5] and -4.1 dB [95% confidence interval, -6.1 to -3.3], respectively) or worse (difference for SbeNpe, 4.0 dB [95% confidence interval, 2.6 to 6.2] and 4.6 dB [95% confidence interval, 2.8 to 5.7], respectively) speech perception in noise. Self-reported speech perception abilities were significantly better for the CI group compared with the BCD, CROS, and no treatment groups. In this randomized clinical trial, the CI group outperformed the BCD, CROS, and no treatment groups in terms of speech perception in noise and disease-specific QOL in patients with SSD after 24 months of follow-up. These results indicate that patients with SSD can partially regain the advantages of binaural hearing after treatment. trialregister.nl Identifier: NL4457.

Objective: We investigated the factors influencing speech perception in babble for 5-year-old children with hearing loss who were using hearing aids (HAs) or cochlear implants (CIs). Design: Speech reception thresholds (SRTs) for 50% correct identification were measured in two conditions – speech collocated with babble, and speech with spatially separated babble. The difference in SRTs between the two conditions give a measure of binaural unmasking, commonly known as spatial release from masking (SRM). Multiple linear regression analyses were conducted to examine the influence of a range of demographic factors on outcomes. Study sample: Participants were 252 children enrolled in the Longitudinal Outcomes of Children with Hearing Impairment (LOCHI) study. Results: Children using HAs or CIs required a better signal-to-noise ratio to achieve the same level of performance as their normal-hearing peers but demonstrated SRM of a similar magnitude. For children using HAs, speech perception was significantly influenced by cognitive and language abilities. For children using CIs, age at CI activation and language ability were significant predictors of speech perception outcomes. Conclusions: Speech perception in children with hearing loss can be enhanced by improving their language abilities. Early age at cochlear implantation was also associated with better outcomes.