Speech enhancement using noise estimation based on dynamic quantile tracking for hearing impaired listeners

Developing an International Core Outcome Set for SSD Interventions

Background and Aim: Many older adults experience difficulty in speech perception in noisy environments and fast speech. Speech perception is dependent on bottom-up and top-down auditory processing information. This study aimed to investigate the relationship between Interaural Time Difference (ITD) and speech processing speed in older adults with difficulty in speech perception in noise. Methods: In this study, 36 Iranian older adults with normal hearing (23 men and 13 women) aged 65–75 years who had complaints about the difficulty with speech perception in noise, participated. The ITD test with two stimuli (high-pass and low-pass noise) was used to asess the auditory localization ability. Time-Compressed Speech Test (TCST) and reaction time test were used to measure their speech processing speed. Pearson correlation test was performed to examine the relationship between ITD and speech processing speed. Results: The auditory localization errors were more significant for ITDs with low-pass noise than high-pass noise. The results showed a negative correlation between localization errors and the TCST score at time compression ratios of 40% (p=0.005) and 60% (p=0.002). The highest correlation was observed between ITD (low-pass noise) and the TCST score at the time compression ratio of 60% in the right ear (r=–0.66). The mean ITD errors were not significantly correlated to the reaction time test score (p>0.05). Conclusion: Word recognition in the elderly declines with increasing speech rate. The findings of this study can be used in clinical practice for consultation and rehabilitation of older adults with communication difficulties. Keywords: Sound localization; speed processing; speech perception; elderly

Background: Regarding the strong auditory spatial plasticity capability of the central auditory system and the effect of short-term and long-term rehabilitation programs in elderly people, it seems that an auditory spatial training can help this population in informational masking release and better track speech in noisy environments. The main purposes of this study are developing an informational masking measurement test and an auditory spatial training program. Protocol: This study will be conducted in two parts. Part 1: develop and determine the validity of an informational masking measurement test by recruiting two groups of young (n=50) and old (n=50) participants with normal hearing who have no difficulty in understanding speech in noisy environments. Part 2 (clinical trial): two groups of 60-75-year-olds with normal hearing, who complain about difficulty in speech perception in noisy environments, will participate as control and intervention groups to examine the effect of auditory spatial training. Intervention: 8 sessions of auditory spatial training. The informational masking measurement test and Speech, Spatial and Qualities of Hearing Scale will be compared before intervention, immediately after intervention, and one month after intervention between the two groups. Discussion: Since auditory training programs do not deal with informational masking release, an auditory spatial training will be designed, aiming to improve hearing in noisy environments for elderly populations. Trial registration: Iranian Registry of Clinical Trials (IRCT20190118042404N1) on 25th February 2019.

<p style="text-align: justify;"><span lang="EN-US">Objectives</span><span lang="EN-US">. Children with Autism Spectrum Disorder (ASD) often face difficulties in speech perception, particularly in noisy environments. These problems are associated with central auditory processing disorders. FM systems, which improve the signal-to-noise ratio, can be used to enhance speech perception. The aim of this study was to investigate the effects of FM systems when working with children with ASD in noisy conditions and their use in a school setting. Methods. The pilot study was conducted in two stages. In the first stage, the “Words in Noise” test and the “Sentence Repetition” task were used to assess children’s ability to recognize speech in noisy conditions with and without the use of FM systems. Fourteen children with ASD and 14 typically developing peers participated in the experiment. In the second stage, FM systems were studied in real school environments with 10 elementary school students with autism. Changes in auditory abilities were assessed using the L.I.F.E.-R scale. Results. In the first stage, it was found that children with ASD recognized significantly fewer words in noisy conditions compared to their typically developing peers. In the “Sentence Repetition” task, the use of FM systems improved the performance of children with ASD from 58.3% to 76.9% (p=0.0005). In the second stage, most participants showed minimal changes in auditory ability scores on the L.I.F.E.-R scale: the average score before using FM systems was 54.9, and after using them it was 57</span><span lang="EN-US">.4 (p=0.2322). However, several students showed improvements of 8–13 points, reflecting individual variability in the effect of using FM systems. Discussion. FM systems have demonstrated their effectiveness in improving speech perception in noisy environments for children with ASD. However, the integration of FM systems into the school process requires further research on their effectiveness in real conditions, as well as adaptations to minimize discomfort for children and improve interaction with teachers and tutors.</span></p>

The cochlea plays a key role in the transmission from acoustic vibration to neural stimulation upon which the brain perceives the sound. A cochlear implant (CI) is an auditory prosthesis to replace the damaged cochlear hair cells to achieve acoustic-to-neural conversion. However, the CI is a very coarse bionic imitation of the normal cochlea. The highly resolved time-frequency-intensity information transmitted by the normal cochlea, which is vital to high-quality auditory perception such as speech perception in challenging environments, cannot be guaranteed by CIs. Although CI recipients with state-of-the-art commercial CI devices achieve good speech perception in quiet backgrounds, they usually suffer from poor speech perception in noisy environments. Therefore, noise suppression or speech enhancement (SE) is one of the most important technologies for CI. In this study, we introduce recent progress in deep learning (DL), mostly neural networks (NN)-based SE front ends to CI, and discuss how the hearing properties of the CI recipients could be utilized to optimize the DL-based SE. In particular, different loss functions are introduced to supervise the NN training, and a set of objective and subjective experiments is presented. Results verify that the CI recipients are more sensitive to the residual noise than the SE-induced speech distortion, which has been common knowledge in CI research. Furthermore, speech reception threshold (SRT) in noise tests demonstrates that the intelligibility of the denoised speech can be significantly improved when the NN is trained with a loss function bias to more noise suppression than that with equal attention on noise residue and speech distortion.

Sensorineural hearing impairment is associated with increased intraspeech spectral masking and results in degraded speech perception in noisy environment due to increased masking. Speech enhancement using spectral subtraction can be used for suppressing the external noise. Multiband frequency compression of the complex spectral samples has been reported to reduce the effects of increased intraspeech masking. A combination of these techniques is implemented for real-time processing for improving speech perception by persons with moderate sensorineural loss. For reducing computational complexity and memory requirement, spectral subtraction is carried out using a cascaded-median based estimation of the noise spectrum without voice activity detection. Multi-band frequency compression, based on auditory critical bandwidths, is carried out using fixed-frame processing along with least-squares error based signal estimation to reduce the processing delay. To reduce computational complexity the two processing stages share the FFT based analysis-synthesis. The processing is implemented and tested for satisfactory operation, with sampling frequency of 10 kHz, 25.6 ms window with 75% overlap, using a 16-bit fixed-point DSP processor. The real-time operation is achieved with signal delay of approximately 36 ms and using about one-third of the computing capacity of the processor.

Background Speech perception in noisy environments is significantly influenced by a listener’s ability to segregate competing auditory streams, a process in which fundamental frequency (F0) plays a crucial role, particularly under informational masking conditions. F0 differences, quantified in semitone shifts, facilitate stream segregation by enhancing perceptual distinctiveness between target and masker signals. Although this phenomenon has been extensively studied in various languages, there is limited research on its effects in Malayalam, a Dravidian language characterized by rich prosodic features. The present study investigated how semitone-based variations in F0 affect the perception of low-frequency PB Malayalam words in the presence of gender-specific speech maskers. Twenty-seven young adults (13 males, 14 females) aged 18 to 25 years, all native Malayalam speakers with normal hearing sensitivity, participated in the study. An adaptive SNR-50 paradigm was employed to assess speech perception across eight experimental conditions involving four semitone shifts (0, 4, 8, and 12 ST) and two-talker babble maskers of either male or female voices. Stimuli were presented binaurally, and participants completed a closed-set image selection task to indicate their responses. Results Significant main effects were observed for masker gender, with better performance in the different-gender (male) masker condition. Although semitone shifts alone did not yield significant differences, a significant interaction between semitone and masker gender was noted. Performance was consistently highest at 0 ST across both masker types. Conclusion These findings suggest that while increased F0 separation can aid speech perception, voice naturalness at 0 ST may offer greater perceptual stability. Results have implications for understanding pitch-based stream segregation in Malayalam and improving clinical speech perception strategies.

Speech perception in noisy environments is a common challenge among older adults, even for those with clinically normal hearing. Cognitive decline may be one of the contributing factors, and, as such, auditory-cognitive training may enhance speech perception in these conditions. This study aims to determine if auditory-cognitive training can improve speech-in-noise listening in normal-hearing, older adults using neural and behavioral measures, supplemented with comparisons across younger and older adults. Neural responses were obtained using magnetoencephalography (MEG) while participants listened to long, narrative passages (60 s) under four noise conditions. Neural measures employed reverse correlation using encoding and decoding models, via the temporal response function (TRF) framework, to predict neural responses and reconstruct stimulus features, respectively, with the boosting algorithm to enforce sparsity. Behavioral measures, such as working memory (reading span; RSPAN), speech perception in noise (SPIN), and nonlinguistic auditory stream segregation (stochastic figure-ground; SFG) showed improvement post-training, along with neural and subjective ratings for listening effort. Additionally, auditory-cognitive training may enhance the neural contrast between the selectively attended and unattended stimulus reconstructions, and pre-training SFG performance may predict the extent of this neuroplasticity change. These results provide promising, additional insight into the effects of auditory-cognitive training, both perceptually and neurally.

If speech is reproduced in an acoustically disturbed environment via a loudspeaker, the perception in terms of intelligibility or listening effort may be impaired. Typical situations arise in connection with mobile phones or public address systems. The location of the listener is called the near-end while the speech signal is received from the far-end. Usually, the acoustical background noise cannot be influenced, but speech perception may be enhanced by digital pre-processing of the loudspeaker signal. This pre-processing technique is called Near-End Listening Enhancement (NELE). The received signal is filtered adaptively, by taking the instantaneous characteristics of the background noise into account and by exploiting psychoacoustic properties of the auditory system such as the masking threshold. The increase of the loudspeaker volume is often not allowed or very limited, to avoid injury of hearing or damages of the loudspeaker and because of the listening comfort. In this contribution, state-of-the-art NELE algorithms operating in the short-term frequency domain are reviewed and novel time domain approaches are presented. Furthermore, the differing NELE constraints of public address systems and mobile phones are discussed.

This study investigates the effectiveness of Deep Neural Network (DNN)-generated virtual humans in enhancing audiovisual speech perception in noisy environments, with a focus on using neural measures to quantify these effects. Lip movements are essential for speech comprehension, especially when auditory cues are degraded by noise. Traditional recording methods produce high-quality audiovisual materials but are resource intensive. This research explores the use of DNN avatars as a promising alternative, utilizing a commercially available tool to create realistic virtual humans. The study included both simple sentences and a short story to improve ecological validity.Eleven young, normal-hearing participants proficient in Flemish-Dutch listened to semantically meaningful sentences and a short story with various speaker types: a female FACS avatar, male and female DNN avatars, and a video of a human male speaker. The study included behavioral measures which consisted of an adaptive recall procedure and an adaptive rate procedure and electrophysiological measures consisting of neural tracking.Findings in the adaptive recall procedure showed consistent audiovisual benefits, with the human speaker offering the greatest benefit (−4.75 dB SNR), followed by the DNN avatar (−4.00 dB SNR) and the FACS avatar (−1.55 dB SNR). Additionally in the adaptive rate procedure, the DNN avatar improved speech intelligibility, with average SRTs enhancing from −7.17 dB SNR (audio-only) to −9.02 dB SNR (audiovisual). The results from the neural tracking procedure indicated that most participants experienced audiovisual benefits, particularly in the −9 dB SNR range, revealing that audiovisual cues provided by DNN avatars can enhance speech perception, validating these avatars as effective tools for studying audiovisual effects when using both behavioral measures and electrophysiological measures.

In this paper, we present a new Speech Enhancement (SE) technique capable of running on a smartphone, as an assistive device for hearing aids (HAs). The developed method incorporates the coherence between the speech and noise signals to obtain a SE gain function which is used in conjunction with the gain function obtained by Spectral Subtraction using adaptive gain averaging. SE using coherence based gain function is found to suppress the background noise well, while inducing speech distortion. On the other hand, SE using Spectral Subtraction improves speech quality with tolerable speech distortion, but introduces background musical noise for certain noise types. The weighted fusion of the two gain functions strikes a balance between noise suppression and speech distortion. Also it allows the user to control the weighting factor based on the noisy environment and their comfort level of hearing. The developed method is computationally fast and operates in real-time. The proposed method was evaluated for machinery, babble, and car noise types, using both objective and subjective measures for both quality and intelligibility of the enhanced speech. The results show significant improvements in comparison with stand-alone Spectral Subtraction with weighted gain averaging SE methods.

Cochlear implants (CIs) provide an opportunity for the hearing impaired to perceive sound through electrical stimulation of the hearing (cochlear) nerve. However, there is a high risk of losing a patient’s natural hearing during CI surgery, which has been shown to reduce speech perception in noisy environments as well as music appreciation. This is a major barrier to the adoption of CIs by the hearing impaired. Electrocochleography (ECochG) has been used to detect intra-operative trauma that may lead to loss of natural hearing. There is early evidence that ECochG can enable early intervention to save natural hearing of the patient. However, detection of trauma by observing changes in the ECochG response is typically carried out by a human expert. Here, we discuss a method of automating the analysis of cochlear responses during CI surgery. We establish, using historical patient data, that the proposed method is highly accurate (∼94% and ∼95% for sensitivity and specificity respectively) when compared to a human expert. The automation of real-time cochlear response analysis is expected to improve the scalability of ECochG and improve patient safety.

Auditory temporal processing, reading, and phonological awareness among aging adults

In many deep learning based speech enhancement frameworks, the deep neural network (DNN) is usually trained under the mean square error (MSE) criterion to estimate the potential clean amplitudes. Yet, the MSE criterion is not directly connected with speech perception, which potentiates alternative training targets for further improvements in speech quality and intelligibility. This paper introduces maximizing the correlation between the enhanced speech and the reference, so as to further improve the speech quality and intelligibility of the enhanced speech synchronously. Based on the short-time Fourier transform amplitude spectra, the correlations are firstly calculated between the envelops along the time and frequency respectively. Then, the time and frequency envelop correlations are incorporated into the time-frequency correlation (TFC). Finally, the loss function is defined as 1 minus the TFC. In experiment, several loss functions containing the time and frequency envelop correlations are investigated. The system trained with the TFC loss outperforms the baseline trained with the MSE loss with promising improvements in both speech quality and intelligibility. Phoneme recognition results also confirm the effectiveness of maximizing the correlation for speech enhancement.

Background and Objectives：In this study, we evaluated the effect of bimodal hearing in the speech perception test and the increasing level of bimodal hearing over cochlear implantation in speech perception score regarding residual hearing. Subjects and Method：Nineteen prelingually deaf patients, who had used bimodal hearing over a period of 8 months, were divided in two groups in accordance to their low frequency residual hearing. The children were tested in open-set speech perception under +10 SNR and +20 SNR. The scores of bimodal hearing and unilateral cochlear implantation were compared by the Wilcoxon signed-rank test. Also, the increased level of speech perception scores of bimodal hearing over cochlear implantation alone were calculated using the formula, [bimodal score (%) - unilateral cochlear implantation (%)], and was compared between groups. Results：In pure tone audiometry under silent condition, the result of bimodal hearing was similar to unilateral cochlear implantation in magnitude in both groups. Under the noisy condition, both groups had better result in bimodal hearing in terms of speech perception. Also, the increased level was higher in bimodal hearing over unilateral cochlear implantation in the better residual hearing group. However, since the sample size was small enough, it is considered that there was no statistical significance. Conclusion：Cochlear implanted patients with residual hearing are expected to get better speech perception in noisy environment with bimodal hearing regardless of the level of residual hearing. (Korean J Otorhinolaryngol-Head Neck Surg 2009;52:29-35)