A CAUSAL DEEP LEARNING FRAMEWORK FOR CLASSIFYING PHONEMES IN COCHLEAR IMPLANTS.

Current Problems in Automatic Speech Recognition

Research in automatic speech production, like research in automatic speech recognition, seems to be regressing at an ever-increasing rate when compared with the over-all progress of technology. Both speech production and speech recognition involve human beings and perceptual phenomena about which very little information is available. A negative attitude towards either automatic speech recognition or automatic speech production logically must lead to a negative attitude towards the other. Hence, speech research aimed at automating either the recognition or the production of speech will require many years of effort before anything approaching practical results is obtained. The speech process should ultimately be investigated for its own intrinsic merit—not for satisfying man-machine communication needs.

A cochlear implant (CI) is a hearing device introduced in the 1980s for profoundly deaf subjects who gained little or no benefit from powerful hearing aids. This device comprises an electrode array inserted in the cochlea, connected to an internal receiver, and an externally worn speech processor. The CI transforms acoustic signals into electrical currents which directly stimulate the auditory nerve. Since the early 1990s, cochlear implantation in children has been developing rapidly. Although it is still difficult to predict how a child will perform with a cochlear implant, the success of cochlear implantation can no longer be denied. In this paper, some recent papers and reports, and the results of the various Nijmegen cochlear implant studies, are reviewed. Issues about selection, examinations, surgery and the outcome are discussed. Overall, our results were comparable with those of other authors. It can be concluded that cochlear implantation is an effective treatment for postlingually deaf as well as prelingually (congenital or acquired) deaf children with profound bilateral sensorineural deafness.

Current research in automatic speech recognition is primarily concerned with the correct evaluation of linguistic information transmitted in the speech signal and with the identification of variations, naturally present in speech. These differences in speech may be due to the individual’s age; gender; or speaking style influenced by his dialect. Undoubtedly, the focus of research in this field is to strengthen further the techniques developed thus far, regarding their reliability and accuracy. The endeavour of this research paper is to primarily concentrate on analysis and modelling of linguistic and paralinguistic information embedded in the speech signal for discovering the similarities and dissimilarities among acoustic characteristics arising out of different dialects. This paper investigates the influence of dialectal variations, by measuring and analysing certain acoustic features such as formant frequencies, pitch, pitch slope, duration and intensity of vowel sounds. For automatic identification of native dialect, these differences are further exploited, given a sample of native speaker’s speech. For the classification of dialect in the spoken utterances support vector machines along with dialect-specific Gaussian mixture models were used. The system performance is compared with human perception of dialects. The proposed study focuses on various dialects of one of the world’s major language; Hindi.

Recently with the wide development of computers, various forms of information exchange between human and computer are discovered. At present, interacting with the computer using speech is one of the active scientific research fields especially for people with disabilities who face variety of barriers to computer use. Such research in Automatic Speech Recognition (ASR) is investigated for different languages because each language has its specific features. This paper presents a speech recognition system for individually spoken word in Tamil language using multilayer feed forward network which falls under the category, "Networks for Classification and Prediction" and has widespread interesting applications and functions related to speech processing. To implement the above system, initially preprocessing is done with the input signal and the speech features being the main part of speech recognition system, are analyzed and extracted via Mel Frequency Cepstral Coefficients (MFCC). These feature vectors are given as the input to the Feed-Forward Neural Network for classifying and recognizing Tamil spoken word. Experiments are done with sample Tamil speech signals and its performance are measured based on Mean square error (MSE). The results indicate that the adopted network with specified parameters have produced the best MSE.KeywordsElliptic FilterMel Frequency Cepstral CoefficientsFeed-forward neural networksTamil Speech RecognitionMean square errorPeak signal to Noise Ratio

Intervention for a Child with Auditory Neuropathy/Dys-synchrony

Objective: The purpose of this study is to assess the individual effect of reverberation and noise, as well as their combined effect, on speech intelligibility by cochlear implant (CI) users. Design: Sentence stimuli corrupted by reverberation, noise, and reverberation + noise are presented to 11 CI listeners for word identification. They are tested in two reverberation conditions (T60 = 0.6 s, 0.8 s), two noise conditions (SNR = 5 dB, 10 dB), and four reverberation + noise conditions. Study sample: Eleven CI users participated. Results: Results indicated that reverberation degrades speech intelligibility to a greater extent than additive noise (speech-shaped noise), at least for the SNR levels tested. The combined effects were greater than those introduced by either reverberation or noise alone. Conclusions: The effect of reverberation on speech intelligibility by CI users was found to be larger than that by noise. The results from the present study highlight the importance of testing CI users in reverberant conditions, since testing in noise-alone conditions might underestimate the difficulties they experience in their daily lives where reverberation and noise often coexist.

Previous research in cochlear implant (CI) recipients with bilateral severe-to-profound sensorineural hearing loss showed improvements in speech recognition in noise using remote wireless microphone systems. However, to our knowledge, no previous studies have addressed the benefit of these systems in CI recipients with single-sided deafness. The objective of this study was to evaluate the potential improvement in speech recognition in noise for distant speakers in single-sided deaf (SSD) CI recipients obtained using the digital remote wireless microphone system, Roger. In addition, we evaluated the potential benefit in normal hearing (NH) participants gained by applying this system. Speech recognition in noise for a distant speaker in different conditions with and without Roger was evaluated with a two-way repeated-measures design in each group, SSD CI recipients, and NH participants. Post hoc analyses were conducted using pairwise comparison t-tests with Bonferroni correction. Eleven adult SSD participants aided with CIs and eleven adult NH participants were included in this study. All participants were assessed in 15 test conditions (5 listening conditions × 3 noise levels) each. The listening conditions for SSD CI recipients included the following: (I) only NH ear and CI turned off, (II) NH ear and CI (turned on), (III) NH ear and CI with Roger 14, (IV) NH ear with Roger Focus and CI, and (V) NH ear with Roger Focus and CI with Roger 14. For the NH participants, five corresponding listening conditions were chosen: (I) only better ear and weaker ear masked, (II) both ears, (III) better ear and weaker ear with Roger Focus, (IV) better ear with Roger Focus and weaker ear, and (V) both ears with Roger Focus. The speech level was fixed at 65 dB(A) at 1 meter from the speech-presenting loudspeaker, yielding a speech level of 56.5 dB(A) at the recipient's head. Noise levels were 55, 65, and 75 dB(A). Digitally altered noise recorded in school classrooms was used as competing noise. Speech recognition was measured in percent correct using the Oldenburg sentence test. In SSD CI recipients, a significant improvement in speech recognition was found for all listening conditions with Roger (III, IV, and V) versus all no-Roger conditions (I and II) at the higher noise levels (65 and 75 dB[A]). NH participants significantly benefited from the application of Roger in noise for higher levels, too. In both groups, no significant difference was detected between any of the different listening conditions at 55 dB(A) competing noise. There was also no significant difference between any of the Roger conditions III, IV, and V across all noise levels. The application of the advanced remote wireless microphone system, Roger, in SSD CI recipients provided significant benefits in speech recognition for distant speakers at higher noise levels. In NH participants, the application of Roger also produced a significant benefit in speech recognition in noise.

People with single-sided deafness (SSD) or asymmetric hearing loss (AHL) have particular difficulty understanding speech in noisy listening situations and in sound localization. The objective of this multicenter study is to evaluate the effect of a cochlear implant (CI) in adults with single-sided deafness (SSD) or asymmetric hearing loss (AHL), particularly regarding sound localization and speech intelligibility with additional interest in electric-acoustic pitch matching. A prospective longitudinal study at 7 European tertiary referral centers was conducted including 19 SSD and 16 AHL subjects undergoing cochlear implantation. Sound localization accuracy was investigated in terms of root mean square error and signed bias before and after implantation. Speech recognition in quiet and speech reception thresholds in noise for several spatial configurations were assessed preoperatively and at several post-activation time points. Pitch perception with CI was tracked using pitch matching. Data up to 12 months post activation were collected. In both SSD and AHL subjects, CI significantly improved sound localization for sound sources on the implant side, and thus overall sound localization. Speech recognition in quiet with the implant ear improved significantly. In noise, a significant head shadow effect was found for SSD subjects only. However, the evaluation of AHL subjects was limited by the small sample size. No uniform development of pitch perception with the implant ear was observed. The benefits shown in this study confirm and expand the existing body of evidence for the effectiveness of CI in SSD and AHL. Particularly, improved localization was shown to result from increased localization accuracy on the implant side.

Determine the impact of cochlear implantation on quality of life (QOL) and determine the correlation between QOL and speech recognition ability. Two authors independently searched PubMed, Medline, Scopus, and the Cumulative Index to Nursing and Allied Health Literature to identify studies reporting hearing-specific or cochlear implant (CI)-specific QOL outcomes before and after cochlear implantation, and studies reporting correlations between QOL and speech recognition after cochlear implantation. Data from the included articles were obtained independently by two authors. Standardized mean difference (SMD) for each measure and pooled effects were determined to assess improvement in QOL before and after cochlear implantation. From 14 articles with 679 CI patients who met the inclusion criteria, pooled analyses of all hearing-specific QOL measures revealed a very strong improvement in QOL after cochlear implantation (SMD = 1.77). Subset analysis of CI-specific QOL measures also showed very strong improvement (SMD = 1.69). Thirteen articles with 715 patients met the criteria to evaluate associations between QOL and speech recognition. Pooled analyses showed a low positive correlation between hearing-specific QOL and word recognition in quiet (r = 0.213), sentence recognition in quiet (r = 0.241), and sentence recognition in noise (r = 0.238). Subset analysis of CI-specific QOL showed similarly low positive correlations with word recognition in quiet (r = 0.213), word recognition in noise (r = 0.241), and sentence recognition in noise (r = 0.255). Using hearing-specific and CI-specific measures of QOL, patients report significantly improved QOL after cochlear implantation. However, widely used clinical measures of speech recognition are poor predictors of patient-reported QOL with CIs. Laryngoscope, 128:982-990, 2018.

Speech intelligibility and prosody production in children with cochlear implants

Saying that late 20th century automatic speech recognition (ASR) is pattern recognition, is something of a truism, but perhaps one of which the fundamental implications are not always fully appreciated. Essentially, a pattern recognition task boils down to measuring the distance between a physical representation of a new, as yet unknown token, and all elements of a set of preexisting patterns, of course in the same physical representation. On the one hand, the ‘patterns’ that can be recognized are, implicitly or explicitly, separate and invariable entities. For example, the command open in a Windows control application always has the same invariable and unique meaning. On the other hand, the unknown input tokens are continuous signals that typically show a high degree of variability. ASR research has centered around the problem of how to map continuous, variable acoustic representations onto discrete, invariable patterns. In ASR the physical representation of the speech tokens is some kind of dynamic power spectrum, for reasons which date back to the days of Ohm and von Helmholtz, who have shown that the power spectrum explains most of the perceptual phenomena in human speech processing. Since the inception of digital signal processing dynamic spectra are approximated by a sequence of short-time spectra (Rabiner & Schafer 1978). Consequently, the pattern match in ASR is invariably implemented as the accumulation of some distance measure between the acoustic features derived from a sequence of short-time spectra of the input token and the corresponding representation of the active patterns (see Fig. 2.1). Therefore, anything which adds to the variability of the short time spectrum of a speech signal will, as it were by definition, complicate pattern matching, and consequently complicate ASR.

<b>Purpose:</b> The aim of the study was to assess whether sequential cochlear implantation (CI) with a prolonged interimplant interval (<i>M </i>= 15.2 years) between the first and second CIs benefited speech recognition and health-related quality of life.<b>Method: </b>This prospective study included 14 prelingually deafened participants who received their second CI after a prolonged interimplant interval (<i>M</i> = 15.2 years). Additionally, speech recognition ability over a 12-month period of bilateral implant use was investigated. The results of the speech recognition test in both quiet and noisy conditions were statistically analyzed for each CI alone and both CIs together. Nijmegen Cochlear Implant Questionnaire scores were also collected at activation and at 12 months after activation.<b>Results:</b> Improvements in speech recognition ability were observed following the use of the first implant alone and with the use of both implants together; however, progress was much slower with the use of the second implant alone, following its introduction. Furthermore, a significant difference in the trajectory of speech recognition ability was observed between the first and the second implanted ear. According to Nijmegen Cochlear Implant Questionnaire scores, all participants benefitted from bilateral CI after 12 months.<b>Conclusions: </b>Prolonged interimplant intervals resulted in asymmetrical speech recognition abilities. A significant improvement in the speech recognition scores was observed with the first implanted ear, and much slower progress was observed with the second implanted ear. However, the “poorer” second implanted ear could provide a considerable beneficial effect on the improved speech recognition and health-related quality of life with the bilateral CI.<br><b>Supplemental Material S1. </b>Arcsine transform; comparison of recognition scores with three modalities over time; predictors of diverge time point.<br>Zhang, Z., Wei, C., Zhang, Y., Zeng, Z., Cao, K., &amp; Liu, Y. (2020). Sequential bilateral cochlear implantation with prolonged time intervals. <i>Journal of Speech, Language, and Hearing Research</i>. https://doi.org/10.1044/2020_JSLHR-20-00140

To evaluate the reliability of the Speech Intelligibility Rating scale to monitor the speech intelligibility of deaf children who have received cochlear implants. A prospective study assessing the speech intelligibility of deaf children with cochlear implants by local and cochlear implant program speech and language therapists. Pediatric tertiary referral center for cochlear implantation. Fifty-four children were each rated by two speech and language therapists, one working with the child locally and the other working with the child at the cochlear implant program. All children were between 1.2 and 10.9 years of age at the time of implantation (median, 4.0 years). The follow-up intervals ranged from before implantation to 9 years after implantation. Correlation coefficient, intraclass correlation coefficient, and kappa statistics were used to assess the interobserver reliability of the Speech Intelligibility Rating scale. Spearman rank correlation coefficient and intraclass correlation coefficient were 0.82 with high statistical significance (p < 0.00001). Kappa statistical analysis revealed a moderate to substantial agreement between the ratings. This agreement also reached a high statistical significance (overall kappa = 0.53, p < 0.000001). The Speech Intelligibility Rating scale was found to be able to discriminate speech intelligibility among subjects, and the ratings covaried with high consistency. The study found a high rate of agreement between observers when they used the Speech Intelligibility Rating scale to assess the speech intelligibility of deaf children after cochlear implantation. Because the scale presents information in a format that is understood by parents, local professionals, and health care purchasers, it will be useful to provide them with accessible information on speech intelligibility outcomes of deaf children who have received cochlear implants.

Psychoacoustic tests assessed shortly after cochlear implantation are useful predictors of the rehabilitative speech outcome. While largely independent, both spectral and temporal resolution tests are important to provide an accurate prediction of speech recognition. However, rapid tests of temporal sensitivity are currently lacking. Here, we propose a simple amplitude modulation rate discrimination (AMRD) paradigm that is validated by predicting future speech recognition in adult cochlear implant (CI) patients. In 34 newly implanted patients, we used an adaptive AMRD paradigm, where broadband noise was modulated at the speech-relevant rate of ~4 Hz. In a longitudinal study, speech recognition in quiet was assessed using the closed-set Freiburger number test shortly after cochlear implantation (t0) as well as the open-set Freiburger monosyllabic word test 6 months later (t6). Both AMRD thresholds at t0 (r = -0.51) and speech recognition scores at t0 (r = 0.56) predicted speech recognition scores at t6. However, AMRD and speech recognition at t0 were uncorrelated, suggesting that those measures capture partially distinct perceptual abilities. A multiple regression model predicting 6-month speech recognition outcome with deafness duration and speech recognition at t0 improved from adjusted R = 0.30 to adjusted R = 0.44 when AMRD threshold was added as a predictor. These findings identify AMRD thresholds as a reliable, nonredundant predictor above and beyond established speech tests for CI outcome. This AMRD test could potentially be developed into a rapid clinical temporal-resolution test to be integrated into the postoperative test battery to improve the reliability of speech outcome prognosis.