Intelligibility of speech produced during simultaneous communication

Competitive state-of-the-art automatic pathological speech intelligibility measures typically rely on regression training on a large number of features, require a large amount of healthy speech training data, or are applicable only to phonetically balanced scenarios where healthy and pathological speakers utter the same utterances. As a result, their performance in unseen data is unsatisfactory, and they cannot be used in low-resource languages or in phonetically unbalanced scenarios. To overcome these drawbacks, we propose a subspace-based intelligibility (SBI) measure. The SBI measure operates based on the hypothesis that dominant spectral patterns of pathological speech differ from intelligible speech (where the pathological and intelligible speech signals do not need to match in phonetic content), with the difference increasing as pathological speech intelligibility decreases. The SBI measure uses a minimal number of speech recordings to compute dominant spectral basis vectors spanning intelligible and pathological speech. The subspaces spanned by the intelligible and pathological spectral basis vectors are compared to each other through a subspace distance measure, which is directly used (i.e., without any training) as the pathological speech intelligibility estimate. Exploiting psychoacoustic evidence on the importance of spectral modulation cues to the perceived speech intelligibility and clinical evidence on the degradation of these cues in pathological speech, we show that the power of the proposed SBI measure lies in capturing the effect of spectral modulation degradation. To be able to additionally track possible degradations in the temporal structure of the pathological speech signal, we also propose two extensions of the SBI measure by incorporating short-time temporal information. Experimental results for different languages and speech pathologies show that the proposed intelligibility measures yield high and significant correlations with subjective intelligibility ratings, while not requiring any regression training or a large number of healthy speech recordings and being applicable to phonetically unbalanced scenarios.

Neural encoding of spectro-temporal cues at slow and near speech-rate in cochlear implant users

Speech intelligibility and prosody production in children with cochlear implants

The purpose of this study was twofold: (a) to determine the extent to which 4-channel, slow-acting wide dynamic range amplitude compression (WDRC) can counteract the perceptual effects of reduced auditory dynamic range and (b) to examine the relation between objective measures of speech intelligibility and categorical ratings of speech quality for sentences processed with slow-acting WDRC. Multiband expansion was used to simulate the effects of elevated thresholds and loudness recruitment in normal hearing listeners. While some previous studies have shown that WDRC can improve both speech intelligibility and quality, others have found no benefit. The current experiment shows that moderate amounts of compression can provide a small but significant improvement in speech intelligibility, relative to linear amplification, for simulated-loss listeners with small dynamic ranges (i.e., flat, moderate hearing loss). This benefit was found for speech at conversational levels, both in quiet and in a background of babble. Simulated-loss listeners with large dynamic ranges (i.e., sloping, mild-to-moderate hearing loss) did not show any improvement. Comparison of speech intelligibility scores and subjective ratings of intelligibility showed that listeners with simulated hearing loss could accurately judge the overall intelligibility of speech. However, in all listeners, ratings of pleasantness decreased as the compression ratio increased. These findings suggest that subjective measures of speech quality should be used in conjunction with either objective or subjective measures of speech intelligibility to ensure that participant-selected hearing aid parameters optimize both comfort and intelligibility.

The objective of this paper was to determine how phoneme identification was affected by the cochlear location and spacing of the electrodes in cochlear implant listeners. Subjects were initially programmed with the full complement of 20 active electrodes, in which each electrode was assigned to represent the output of one filter in the normal SPEAK processor. In the present study several four-electrode processors were constructed by assigning the output of more than one filter to a single electrode. In all conditions speech sounds were still analyzed into 20 frequency bands and processed according to the usual SPEAK processing strategy, but the location and spacing of the four stimulated electrode pairs were varied systematically. In Experiment I, the spacing between stimulated electrodes was fixed at 3.75 mm and the cochlear location of the four electrode pairs was shifted from the most-apical position up to 3.0 mm toward the base in 0.75 mm steps. In Experiment II, the spatial separation between the four electrode pairs (each bipolar-plus-one) was systematically changed from 1.5 mm to 4.5 mm while holding the most apical active electrode fixed. In Experiment III, the spacing of active electrodes was varied to represent equal tonotopic spacing to equal linear frequency intervals between pairs. Recognition of medial vowels and consonants was measured in three subjects with these custom four-electrode speech processors. In Experiment I, results showed that both vowel and consonant recognition were best when the electrodes were in the most apical locations. In Experiment II, best speech recognition occurred when electrode pairs were separated by 3 to 3.75 mm. In Experiment III, both vowel and consonant recognition scores decreased when the spacing of electrode pairs was changed from equal tonotopic spacing to equal linear frequency intervals. Overall, vowel and consonant recognition were best at the most apical electrode locations and when the spacing of electrodes matched the frequency intervals of the analysis filters. Consonant recognition was relatively robust to alterations in electrode location and spacing. The best vowel scores with four-electrode speech processors were about 10 percentage lower than scores obtained with the full 20-electrode speech processors. However, the best consonant scores with four-electrode speech processors were similar to those obtained with the full 20-electrode speech processors. Information transmission analysis revealed that temporal envelope cues (voicing and manner) were not strongly affected by changes in electrode location and spacing, whereas spectral cues, as represented by vowel recognition and consonantal place of articulation, were strongly affected. Both spectral and temporal phoneme cues were strongly affected by the degree of tonotopic warping, created by altering both the location and spacing of the activated electrodes. The cochlear location and spacing of the activated electrodes had a clear effect on phoneme recognition. Temporal cues were less affected by tonotopic shifts or linear tonotopic stretching or shrinking, but were susceptible to nonlinear tonotopic warping. Spectral cues were sensitive to all tonotopic manipulations: shifting, linear stretching, and nonlinear warping. However, the present experiments could not differentiate whether the optimal mapping between analysis frequency bands and stimulation electrodes was determined by the normal acoustic tonotopic pattern or by the pattern learned from experience with the 20-electrode implant.

For this project, different large lecture halls were analyzed using various speech intelligibility measurement techniques. The goal was to determine how well these measurements correlate with each other and convey speech intelligibility. Objective speech intelligibility and other related acoustical measurements were made throughout each hall at varying distances from the speaker. These measurements were repeated in each hall in a standardized manner, allowing the results from the different halls to be compared effectively. These objective measurements were related to the subjective analysis of the hall to determine which measurements indicate an appropriate speech intelligibility level for that space. Background mechanical system noise was introduced into some of the halls to determine the effect on speech intelligibility. Some of the halls tested were modeled in an architectural acoustics computer program to determine how well the actual measurements correlate with the predicted measurements. Measurement techniques and results of the study will be presented.

When locating a sound source, listeners are expected to rely more on spectral cues for elevation and temporal cues for azimuth. This work investigates whether sound localization ability can be predicted by individually-measured use of spectral and temporal cues. Participants were older adults with sensorineural hearing loss. Sound sources for the localization task were created in a virtual room, using a mix of virtual and physical loudspeakers in the front hemifield of the listener. Sources were distributed evenly between ±90 deg azimuth and ±20 deg elevation. The signal was a 1 s broadband 4-Hz amplitude-modulated noise. To assess the relationship between localization ability and use of spectral and temporal cues, listeners performed cue weighting and cue discrimination tasks. In the cue weighting task, listeners identified synthetic speech sounds that varied spectro-temporally. The cue discrimination task measured the smallest detectable difference in either spectral or temporal information among the same set of ambiguous speech sounds. Together, these tasks form a cue profile which identifies whether the listener relies to a greater extent on temporal or spectral cues. Older hearing-impaired listeners varied in their ability to localize sounds. Localization results will be discussed in the context of cue profiles and audiometry. [Work supported by NIH.]

Sensory systems constantly adapt their responses to the current environment. In hearing, adaptation may facilitate communication in noisy settings, a benefit frequently (but controversially) attributed to the medial olivocochlear reflex (MOCR) enhancing the neural representation of speech. Here, we show that human listeners (N = 14; five male) recognize more words presented monaurally in ipsilateral, contralateral, and bilateral noise when they are given some time to adapt to the noise. This finding challenges models and theories that claim that speech intelligibility in noise is invariant over time. In addition, we show that this adaptation to the noise occurs also for words processed to maintain the slow-amplitude modulations in speech (the envelope) disregarding the faster fluctuations (the temporal fine structure). This demonstrates that noise adaptation reflects an enhancement of amplitude modulation speech cues and is unaffected by temporal fine structure cues. Last, we show that cochlear implant users (N = 7; four male) show normal monaural adaptation to ipsilateral noise. Because the electrical stimulation delivered by cochlear implants is independent from the MOCR, this demonstrates that noise adaptation does not require the MOCR. We argue that noise adaptation probably reflects adaptation of the dynamic range of auditory neurons to the noise level statistics.SIGNIFICANCE STATEMENT People find it easier to understand speech in noisy environments when they are given some time to adapt to the noise. This benefit is frequently but controversially attributed to the medial olivocochlear efferent reflex enhancing the representation of speech cues in the auditory nerve. Here, we show that the adaptation to noise reflects an enhancement of the slow fluctuations in amplitude over time that are present in speech. In addition, we show that adaptation to noise for cochlear implant users is not statistically different from that for listeners with normal hearing. Because the electrical stimulation delivered by cochlear implants is independent from the medial olivocochlear efferent reflex, this demonstrates that adaptation to noise does not require this reflex.

Speech intelligibility is usually assessed using subjective tests, but several objective measures of speech intelligibility have also been proposed. This study compares intelligibility of IEEE sentences in collocated speech-shaped noise, recorded through a low-cost amplifier and three current-issue NHS hearing aids with single-channel noise reduction settings switched on and off. Results from twenty-one normal hearing listeners indicates that single-channel noise reduction algorithms can significantly improve intelligibility of noisy speech (p < 0.05), but no differences can be seen between different hearing aid models currently available on the NHS. The low-cost, off-the shelf hearing amplifier performed significantly worse than the NHS prescribed devices. Three objective speech intelligibility measures were applied to the recordings. None of the objective metrics was found to accurately predict the outcomes seen in this study for all hearing aid conditions, though general trends can be predicted (r = 0.83 overall, but for any discrete signal-to-noise ratio across all conditions, maximum r = 0.64). Machine-learning approaches were used to refit parameters from the best performing of these metrics. Although performance was improved, problems in predicting particular conditions remain, suggesting that objective measures do not encompass all relevant factors. [Action on Hearing Loss (England/Wales Registered Charity Number 207720).]

Cochlear implants provide users with limited spectral and temporal information. In this study, the amount of spectral and temporal information was systematically varied through simulations of cochlear implant processors using a noise-excited vocoder. Spectral information was controlled by varying the number of channels between 1 and 16, and temporal information was controlled by varying the lowpass cutoff frequencies of the envelope extractors from 1 to 512 Hz. Consonants and vowels processed using those conditions were presented to seven normal-hearing native-English-speaking listeners for identification. The results demonstrated that both spectral and temporal cues were important for consonant and vowel recognition with the spectral cues having a greater effect than the temporal cues for the ranges of numbers of channels and lowpass cutoff frequencies tested. The lowpass cutoff for asymptotic performance in consonant and vowel recognition was 16 and 4 Hz, respectively. The number of channels at which performance plateaued for consonants and vowels was 8 and 12, respectively. Within the above-mentioned ranges of lowpass cutoff frequency and number of channels, the temporal and spectral cues showed a tradeoff for phoneme recognition. Information transfer analyses showed different relative contributions of spectral and temporal cues in the perception of various phonetic/acoustic features.

The present study investigated the development of audiovisual comprehension skills in prelingually deaf children who received cochlear implants. We analyzed results obtained with the Common Phrases (Robbins et al., 1995) test of sentence comprehension from 80 prelingually deaf children with cochlear implants who were enrolled in a longitudinal study, from pre-implantation to 5 years after implantation. The results revealed that prelingually deaf children with cochlear implants performed better under audiovisual (AV) presentation compared with auditory-alone (A-alone) or visual-alone (V-alone) conditions. AV sentence comprehension skills were found to be strongly correlated with several clinical outcome measures of speech perception, speech intelligibility, and language. Finally, pre-implantation V-alone performance on the Common Phrases test was strongly correlated with 3-year postimplantation performance on clinical outcome measures of speech perception, speech intelligibility, and language skills. The results suggest that lipreading skills and AV speech perception reflect a common source of variance associated with the development of phonological processing skills that is shared among a wide range of speech and language outcome measures.

The purpose of this study was to describe normative and validation data on the Intelligibility in Context Scale (ICS; McLeod, Harrison, & McCormack, 2012c) for English-speaking children. The ICS is a 7-item, parent-report measure of children's speech intelligibility with a range of communicative partners. Data were collected from the parents of 803 Australian English-speaking children ranging in age from 4;0 (years;months) to 5;5 (37.0% were multilingual). The mean ICS score was 4.4 (SD = 0.7) out of a possible total score of 5. Children's speech was reported to be most intelligible to their parents, followed by their immediate family, friends, and teachers; children's speech was least intelligible to strangers. The ICS had high internal consistency (α = .94). Significant differences in scores were identified on the basis of sex and age but not on the basis of socioeconomic status or the number of languages spoken. There were significant differences in scores between children whose parents had concerns about their child's speech (M = 3.9) and those who did not (M = 4.6). A sensitivity of .82 and a specificity of .58 were established as the optimal cutoff. Test-retest reliability and criterion validity were established for 184 children with a speech sound disorder. There was a significant low correlation between the ICS mean score and percentage of phonemes correct (r = .30), percentage of consonants correct (r = .24), and percentage of vowels correct (r = .30) on the Diagnostic Evaluation of Articulation and Phonology (Dodd, Hua, Crosbie, Holm, & Ozanne, 2002). Thirty-one parents completed the ICS related to English and another language spoken by their child with a speech sound disorder. The significant correlations between the scores suggest that the ICS may be robust between languages. This article provides normative ICS data for English-speaking children and additional validation of the psychometric properties of the ICS. The robustness of the ICS was suggested because mean ICS scores were not affected by socioeconomic status, number of languages spoken, or whether the ICS was completed in relation to English or another language. The ICS is recommended as a screening measure of children's speech intelligibility.

Verifying two commercial software implementations of impulse-response-based speech intelligibility measurements

Word segmentation, detecting word boundaries in continuous speech, is a critical aspect of language learning. Previous research in infants and adults demonstrated that a stream of speech can be readily segmented based solely on the statistical and speech cues afforded by the input. Using functional magnetic resonance imaging (fMRI), the neural substrate of word segmentation was examined on-line as participants listened to three streams of concatenated syllables, containing either statistical regularities alone, statistical regularities and speech cues, or no cues. Despite the participants' inability to explicitly detect differences between the speech streams, neural activity differed significantly across conditions, with left-lateralized signal increases in temporal cortices observed only when participants listened to streams containing statistical regularities, particularly the stream containing speech cues. In a second fMRI study, designed to verify that word segmentation had implicitly taken place, participants listened to trisyllabic combinations that occurred with different frequencies in the streams of speech they just heard ("words," 45 times; "partwords," 15 times; "nonwords," once). Reliably greater activity in left inferior and middle frontal gyri was observed when comparing words with partwords and, to a lesser extent, when comparing partwords with nonwords. Activity in these regions, taken to index the implicit detection of word boundaries, was positively correlated with participants' rapid auditory processing skills. These findings provide a neural signature of on-line word segmentation in the mature brain and an initial model with which to study developmental changes in the neural architecture involved in processing speech cues during language learning.

The contribution of visual information to the perception of speech in noise with and without informative temporal fine structure