Robust automatic speech recognition with decoder oriented ideal binary mask estimation

Monaural speech separation is the process of separating the target speech from a noisy speech mixture recorded using single microphone. It can be used in wide range of applications including mobile telephony, hearing aid design and robust automatic speech and speaker recognition (ASR). Recently, researchers use computational auditory scene analysis (CASA) technique to successfully separate the target speech from the monaural noisy speech mixture. In CASA based monaural speech separation techniques, Ideal binary mask (IBM) and Ideal ratio mask (IRM) has been proposed as a computational goal to improve the speech intelligibility and speech quality. This paper reviews and reports various research works carried out using CASA techniques with IBM and IRM to improve speech intelligibility and quality. The experimental results show that CASA systems using IBM improves the speech intelligibility and using IRM improves the speech quality.

Combined speech enhancement and auditory modelling for robust distributed speech recognition

Robust automatic speech recognition with missing and unreliable acoustic data

In this work, we propose a novel feature extraction algorithm that improves the robustness of automatic speech recognition (ASR) systems in the presence of various types of noise. The proposed algorithm uses a new cepstral technique based on the differential power spectrum (DPS) instead of the power spectrum (PS), the algorithm replaces the logarithmic non linearity by the power function. In order to reduce cepstral coefficients mismatches between training and testing conditions, we used the mean and variance normalization, then we apply auto-regression movingaverage filtering (MVA) in the cepstral domain. The ASR experiments were conducted using two databases, the first is LASA digit database designed for recognition the isolated Arabic digits in the presence of different types of noise. The second is Aurora 2 noisy speech database designed to recognize connected English digits in various operating environments. The experimental results show a substantial improvement from the proposed algorithm over the baseline Mel Frequency Cepstral Coefficients (MFCC), the relative improvement is the 28.92% for LASA database and is the 44.43% for aurora 2 database. The performance of our proposed algorithm was tested and verified by extensive comparisons with the state-of-the-art noise-robust features in aurora 2.

Automatic Speech Recognition has been an active topic of research for the past four decades. The main objective of the automatic speech recognition task is to convert a speech segment into an interpretable text message without the need of human intervention. Many different algorithms and schemes based on different mathematical paradigms have been proposed in an attempt to improve recognition rates. Cepstral coefficients play an important part in speech theory and in automatic speech recognition in particular due to their ability to compactly represent relevant information that is contained in a short time sample of a continuous speech signal. The goal of this paper is to discuss comparison of speech parameterization methods: Mel-Frequency Cepstrum Coefficients (MFCC) and improved Mel-Frequency Cepstrum Coefficients (MFCC) using RASTA filters. Thus, in this study, we try to improve the MFCC algorithms to achieve much accuracy reducing the error rates in Automatic Speech Recognition. First, we remove signal correlation through normalization, then we use RASTA filter to filtering the cepstral coefficients. Finally, we reduce dimension of the cepstral coefficients by the variances of cepstral coefficients in different dimension and obtain our features. By using various classifiers, we try to simulate the speech feature extraction at much optimal and least error rate providing robust method for Automatic Speech Recognition (ASRs).KeywordsAutomatic Speech RecognitionMel frequency Cepstrum Coefficients (MFCC’s)ERB Gammatone FilteringHidden Markov Model

Code-switching is the frequent alternation between two or more languages within a single utterance and is a widespread phenomenon among bilingual and multilingual speakers. In India, more than 250 million people are estimated to engage in code-switched communication, especially blending English with Hindi (Hinglish), making it one of the largest bilingual populations globally, making challenging for developing accurate and robust Automatic Speech Recognition (ASR) systems. Existing ASR models, typically trained on monolingual corpus, struggle with code-switched input due to a lack of large, balanced, and representative datasets—particularly for diverse age groups. Recent evaluations have shown that ASR models experience a relative increase in Word Error Rate (WER) of 30–50 % when exposed to code-switched speech compared to monolingual input. To address this resource gap, we introduce a benchmark Hinglish speech corpus, HiACC, to improve ASR performance in resource-constrained settings. While several monolingual Hindi and English corpus exist, publicly available code-switched datasets remain scarce, and none till date include children's speech. Our corpus fills this gap by providing the first code-switched Hinglish speech dataset with recordings from both adults and children. It comprises 3,318 audio segments from adult participants and 1,858 segments from children, covering 5.24 hours of read and spontaneous speech. The transcriptions include detailed annotations and code-switching tags to assist in linguistic and computational analysis. The corpus is publicly available at [https://zenodo.org/records/15551669], offering segmented audio and aligned transcripts for open research. We also present baseline ASR experiments, which show that standard models trained on monolingual data underperform by approximately 42 % WER on our test set, highlighting the complexity of the task. To our knowledge, this is the first publicly available resource on code-switched Hinglish speech encompassing both adult and child speakers, designed to catalyse progress in this challenging yet important area of speech recognition.

We investigate how dominant-frequency information can be used in speech feature extraction to increase the robustness of automatic speech recognition against additive background noise. First, we review several earlier proposed auditory-based feature extraction methods and argue that the use of dominant-frequency information might be one of the major reasons for their improved noise robustness. Furthermore, we propose a new feature extraction method, which combines subband power information with dominant subband frequency information in a simple and computationally efficient way. The proposed features are shown to be considerably more robust against additive background noise than standard mel-frequency cepstrum coefficients on two different recognition tasks. The performance improvement increased as we moved from a small-vocabulary isolated-word task to a medium-vocabulary continuous-speech task, where the proposed features also outperformed a computationally expensive auditory-based method. The greatest improvement was obtained for noise types characterized by a relatively flat spectral density.

We propose a novel exemplar-based feature enhancement method for automatic speech recognition which uses coupled dictionaries: an input dictionary containing atoms sampled in the modulation (envelope) spectrogram domain and an output dictionary with atoms in the Mel or full-resolution frequency domain. The input modulation representation is chosen for its separation properties of speech and noise and for its relation with human auditory processing. The output representation is one which can be processed by the ASR back-end. The proposed method was investigated on the AURORA-2 and AURORA-4 databases and improved word error rates (WER) were obtained when compared to the system which uses Mel features in the input exemplars. The paper also proposes a hybrid system which combines the baseline and the proposed algorithm on the AURORA-2 database which in turn also yielded improvement over both the algorithms.

To address the limitations of traditional cepstrum or LPC based front-end processing methods for automatic speech recognition, more elaborate methods based on physiological models of the human auditory system may be used to achieve more robust speech recognition in adverse environments. For this purpose, a modified version of a model of the primary auditory cortex featuring a three dimensional mapping of auditory spectra [Wang and Shamma, IEEE Trans. Speech Audio Process. 3, 382–395 (1995)] is adopted and investigated for its use as an improved front-end processing method. The study is conducted in two ways: first, by relating the model’s redundant representation to traditional spectral representations and showing that the former not only encompasses information provided by the latter, but also reveals more relevant information that makes it superior in describing the identifying features of speech signals; and second, by observing the statistical features of the representation for various classes of sound to show how different identifying features manifest themselves as specific patterns on the cortical map, thereby becoming a place-coded data set on which detection theory could be applied to simulate auditory perception and cognition.

We consider the problem of robust automatic speech recognition (ASR) in the context of the CHiME-3 Challenge. The proposed system combines three contributions. First, we propose a deep neural network (DNN) based multichannel speech enhancement technique, where the speech and noise spectra are estimated using a DNN based regressor and the spatial parameters are derived in an expectation-maximization (EM) like fashion. Second, a conditional restricted Boltzmann machine (CRBM) model is trained using the obtained enhanced speech and used to generate simulated training and development datasets. The goal is to increase the similarity between simulated and real data, so as to increase the benefit of multicondition training. Finally, we make some changes to the ASR backend. Our system ranked 4th among 25 entries.

This paper proposes an ensemble neural network (ENN) framework for robust automatic speech recognition (ASR). The proposed ENN framework can be divided into offline and online phases. In the offline phase, the ENN framework first applies an environment clustering technique to partition the training data into several subsets, where each subset characterizes specific local information of the entire acoustic space. Next, each subset of training data is adopted to train an NN acoustic model. Finally, the entire set of training data is used to estimate a gating function, which can determine the most suitable NN acoustic model given an input utterance. In the online phase, given the testing utterance, the gating function specifies the optimal NN acoustic model to perform speech recognition. Because local environment information is incorporated, ENN can effectively determine the NN acoustic model that optimally matches the testing condition. The proposed framework was evaluated on the Aurora-2 task. Experimental results show that the proposed ENN framework can provide a notable word error rate reduction of 5.35% (from 5.05% to 4.78%) when compared to the baseline.

In this paper we investigate stereo-based stochastic mapping (SSM) with context for the noise robustness of automatic speech recognition, especially under unseen conditions. Probabilistic PCA (PPCA) is used in the SSM framework to reduce the high dimensionality of the noisy speech features with context and derive an eigen representation in the noisy feature space for the prediction of clean features. To reduce the computational cost in training, an approximation by single-pass re-training is considered for the estimation of joint GMM. We also show that the SSM estimate under the minimum mean square error (MMSE) in a space where low dimensional representation of clean speech and uncorrelated additive noise can be assumed is related to the subspace speech enhancement. Experiments on large vocabulary continuous speech recognition tasks observe gains from the proposed approach under the conditions with seen, unseen and real noise.

We investigate the effectiveness of generative adversarial networks (GANs) for speech enhancement, in the context of improving noise robustness of automatic speech recognition (ASR) systems. Prior work [1] demonstrates that GANs can effectively suppress additive noise in raw waveform speech signals, improving perceptual quality metrics; however this technique was not justified in the context of ASR. In this work, we conduct a detailed study to measure the effectiveness of GANs in enhancing speech contaminated by both additive and reverberant noise. Motivated by recent advances in image processing [2], we propose operating GANs on log-Mel filterbank spectra instead of waveforms, which requires less computation and is more robust to reverberant noise. While GAN enhancement improves the performance of a clean-trained ASR system on noisy speech, it falls short of the performance achieved by conventional multi-style training (MTR). By appending the GAN-enhanced features to the noisy inputs and retraining, we achieve a 7% WER improvement relative to the MTR system.

In this paper, the improvements in the recently implemented Kannada speech recognition system is demonstrated in detail. The Kannada automatic speech recognition (ASR) system consists of ASR models which are created by using Kaldi, IVRS call flow and weather and agricultural commodity prices information databases. The task specific speech data used in the recently developed spoken dialogue system had high level of different background noises. The different types of noises present in collected speech data had an adverse effect on the on line and off line speech recognition performances. Therefore, to improve the speech recognition accuracy in Kannada ASR system, a noise reduction algorithm is developed which is a fusion of spectral subtraction with voice activity detection (SS-VAD) and minimum mean square error spectrum power estimator based on zero crossing (MMSE-SPZC) estimator. The noise elimination algorithm is added in the system before the feature extraction part. An alternative ASR models are created using subspace Gaussian mixture models (SGMM) and deep neural network (DNN) modeling techniques. The experimental results show that, the fusion of noise elimination technique and SGMM/DNN based modeling gives a better relative improvement of 7.68% accuracy compared to the recently developed GMM-HMM based ASR system. The least word error rate (WER) acoustic models could be used in spoken dialogue system. The developed spoken query system is tested from Karnataka farmers under uncontrolled environment.

This paper introduces a robust human-robot interface (HRI) system using a speech recognition and a user localization. For a robust speech recognition indoors under unknown noises and acoustic reverberations, a blind source separation (BSS) algorithm is implemented by a block-wise processing and developed using digital signal processing board to guarantee real-time operation. And a reverberation-robust sound source localization algorithm using separated signals is proposed. Although the BSS method cannot completely preserve the room acoustic information, the proposed localization algorithm overcomes this problem using target channel selection and target-emphasized enhancement methods. The developed algorithms are integrated into the commercial robot system to provide overall voice-enabled HRI. A series of tests are conducted to evaluate the performance of the BSS-based speech recognition and user localization method, and the results show a remarkable performance even under severe non-stationary noise conditions.