Enhancing Pitch Robustness of Speech Recognition System through Spectral Smoothing

Combined speech enhancement and auditory modelling for robust distributed speech recognition

Performance evaluation of Hindi speech recognition system using optimized filterbanks

This paper describes the work done in improving the performance of Tamil speech recognition system by using Time Scale Modification (TSM) and Vocal Tract Length Normalization (VTLN) techniques. The speech recognition system for Tamil language was developed using a new approach of text independent speech segmentation, with a phoneme based language model for recognition. There is degradation in the performance of speech recognition due to variations in the speaking rate and vocal tract shape among different speakers. In order to improve the performance of speech recognition system, both TSM and VTLN normalization techniques were used in this work. The TSM was implemented using the Phase vocoder approach and the VTLN was implemented using speaker specific bark/mel scale in bark/mel domain. The performance of Tamil speech recognition system was improved by performing both TSM and VTLN normalization techniques.

Speech production knowledge has been used to enhance the phonetic representation and the performance of automatic speech recognition (ASR) systems successfully. Representations of speech production make simple explanations for many phenomena observed in speech. These phenomena can not be easily analyzed from either acoustic signal or phonetic transcription alone. One of the most important aspects of speech production knowledge is the use of articulatory knowledge, which describes the smooth and continuous movements in the vocal tract. In this paper, we present a new articulatory model to provide available information for rescoring the speech recognition lattice hypothesis. The articulatory model consists of a feature front-end, which computes a voicing feature based on a spectral harmonics correlation (SHC) function, and a back-end based on the combination of deep neural networks (DNNs) and hidden Markov models (HMMs). The voicing features are incorporated with standard Mel frequency cepstral coefficients (MFCCs) using heteroscedastic linear discriminant analysis (HLDA) to compensate the speech recognition accuracy rates. Moreover, the advantages of two different models are taken into account by the algorithm, which retains deep learning properties of DNNs, while modeling the articulatory context powerfully through HMMs. Mandarin speech recognition experiments show the proposed method achieves significant improvements in speech recognition performance over the system using MFCCs alone.

This paper describes a study of a set of features based on cepstral mean and variance normalization (CMVN) plus auto regressive moving average (ARMA) filtering technique which is called MVA and on histogram equalization (HEQ) for robust speech recognition. First, we use MVA then HEQ in combination with CMVN and ARMA filtering as a post-processing module to mel frequency cepstral coefficients (MFCC), Relative Spectral-Perceptual linear prediction (RASTA-PLP) and power normalized cepstral coefficients (PNCC) features to improve the performance of the automatic speech recognition (ASR) system. The results on the Arabic database task have shown that both methods MVA and HEQ+ARMA improves the success rate for all features compared to the baseline system however HEQ was not found to perform better than MVA. The results also provide that RASTA-PLP outperforms PNCC and MFCC features.

The automatic speech recognition (ASR) system based on deep neural network is easy to be attacked by an adversarial example due to the vulnerability of neural network, which is a hot topic in recent years. The adversarial example does harm to the ASR system, especially if the common-dependent ASR goes wrong, it will lead to serious consequences. To improve the robustness and security of the ASR system, the defense method against adversarial examples must be proposed. Based on this idea, we propose an algorithm of devastation and detection on adversarial examples which can attack the current advanced ASR system. We choose advanced text-dependent and command-dependent ASR system as our target system. Generating adversarial examples by the OPT on text-dependent ASR and the GA-based algorithm on command-dependent ASR. The main idea of our method is input transformation of the adversarial examples. Different random intensities and kinds of noise are added to the adversarial examples to devastate the perturbation previously added to the normal examples. From the experimental results, the method performs well. For the devastation of examples, the original speech similarity before and after adding noise can reach 99.68%, the similarity of the adversarial examples can reach 0%, and the detection rate of the adversarial examples can reach 94%.

A statistically designed Automatic Speech Recognition (ASR) system extracts features from speech signals using feature extraction methods, links the extracted features with the expected phonetics of the hypothesis using acoustic models, uses language model to add prior information about the structure of the target language. For many years, Mel-frequency Cepstral coefficients (MFCC), n-gram, and Hidden Markov Model (HMM) approaches have been used predominantly for feature extraction, language modeling and acoustic modeling, respectively. However, performance degradation of MFCC in noisy conditions and inaccuracy of HMMs while handling large vocabularies have made researchers to propose more efficient methods. The proposed work uses noise robust method Gammatone Frequency Cepstral Coefficients(GFCC) for feature extraction, trigram language modeling, and HMM-Gaussian mixture model (GMM) based acoustic modeling to implement a continuous Hindi language ASR system. Also, it applies Differential Evolution (DE) technique to refine the GFCC features and discriminative techniques to enhance performance of the acoustic model. The performance of the implemented system has been evaluated by using different feature extraction methods, variants of n-gram language modeling techniques and different discriminative techniques in clean as well as noisy conditions. Initially, the results reveal that DE optimized GFCC with HMM-Gaussian Mixture Model (GMM) acoustic modeling performs better than MFCC, PLP and MF-PLP feature extraction methods. Secondly, the experimental results show that the Minimum Phone Error (MPE) outperforms Maximum Mutual Information (MMI) and Maximum Likelihood Estimation (MLE) and trigram based language modeling gives more accurate results than unigram and bigram language modeling. Finally, it has been concluded that the continuous Hindi language ASR system implemented using DE refined GFFC feature extraction method with MPE discriminative training technique and trigram based language modeling gives better accuracy in clean as-well-as noisy environments.

A deep neural network (DNN)-based speech enhancement (SE) aiming to maximize the performance of an automatic speech recognition (ASR) system is proposed in this paper. In order to optimize the DNN-based SE model in terms of the character error rate (CER), which is one of the metric to evaluate the ASR system and generally non-differentiable, our method uses two DNNs: one for speech processing and one for mimicking the output CERs derived through an acoustic model (AM). Then both of DNNs are alternately optimized in the training phase. Even if the AM is a black-box, e.g., like one provided by a third-party, the proposed method enables the DNN-based SE model to be optimized in terms of the CER since the DNN mimicking the AM is differentiable. Consequently, it becomes feasible to build CER-centric SE model that has no negative effect, e.g., additional calculation cost and changing network architecture, on the inference phase since our method is merely a training scheme for the existing DNN-based methods. Experimental results show that our method improved CER by 8.8% relative derived through a black-box AM although certain noise levels are kept.

It is well known that the recognition performance of an automatic speech recognition (ASR) system is affected by intra-speaker as well inter-speaker variability. The differences in the geometry of vocal organs, pitch and speaking-rate among the speakers are some such inter-speaker variabilities affecting the recognition performance. A mismatch between the training and test data with respect to any of those aforementioned factors leads to increased error rates. An example of acoustically mismatched ASR is the task of transcribing children’s speech on adult data-trained system. A large number of studies have been reported earlier that present a myriad of techniques for addressing acoustic mismatch arising from differences in pitch and dimensions of vocal organs. At the same time, only a few works on speaking-rate adaptation employing timescale modification have been reported. Furthermore, those studies were performed on ASR systems developed using Gaussian mixture models. Motivated by these facts, speaking-rate adaptation is explored in this work in the context of children’s ASR system employing deep neural network-based acoustic modeling. Speaking-rate adaptation is performed by changing the frame-length and overlap during front-end feature extraction process. Significant reductions in errors are noted by speaking-rate adaptation. In addition to that, we have also studied the effect of combining speaking-rate adaptation with vocal-tract length normalization and explicit pitch modification. In both the cases, additive improvements are obtained. To summarize, relative improvements in 15–20% over the baselines are obtained by varying the frame-length and frame-overlap.

Recognition of broadcast data, such as TV and radio programs is a topic of great interest. One of the problems with such data is the frequent presence of background music that degrades the performance of speech recognition systems. In this paper we examine the effects of different kinds of music on automatic speech recognition systems by comparing the effects of music with the relatively well-known effects of white noise on these systems. We also examine the extent to which compensation algorithms that have been successfully applied to noisy speech are also helpful in improving recognition accuracy for speech that is corrupted by music. It is hoped that these experimental comparisons will lead to a better understanding of how to compensate for the effects of background music.

Automatic speech recognition (ASR) system is used to recognize the text transcript from the given speech signal. Such speech signal can contain either isolated words or large vocabulary continuous speech (LVCS). Isolated words can be recognized with high accuracy in clean environment, but recognizing continuous words involves various parameters like speech corpus, speaker, environment noise, etc., that affects the accuracy of automatic speech recognition system directly. In the proposed work, hybrid feature extraction technique combines the perceptual linear predictive (PLP) and mel frequency cepstral coefficient (MFCC) to improve the accuracy of ASR in noisy environment. Voice activity and detection (VAD)-based frame dropping is used for improving the phonemes modeling by removing the pauses and distorted elements from the given speech signal. The proposed hybrid model with VAD is implemented by using self-generated speech corpus and shows relatively 12% increase in recognition rate compared with the state-of-the-art methodology.KeywordsAutomatic speech recognition (ASR)Perceptual linear predictive (PLP)Mel frequency cepstral coefficient (MFCC)Voice activity and detection (VAC)

A mismatch between the training and testing in noisy circumstance often causes a drastic decrease in the performance of speech recognition system. The robust feature coefficients might suppress this sensitivity of mismatch during the recognition stage. In this paper, we investigate the noise robustness of LPC Cepstral Coefficients (LPCC) by using speech enhancement with feature post-processing. At front-end, speech enhancement in the wavelet domain is used to remove noise components from noisy signals. This enhanced processing adopts the combination of discrete wavelet transform (DWT), wavelet packet decomposition (WPD), multi-thresholds processing etc to obtain the estimated speech. The feature post-processing employs cepstral mean normalization (CMN) to compensate the signal distortion and residual noise of enhanced signals in the cepstral domain. The performance of digit speech recognition systems is evaluated under noisy environments based on NOISEX-92 database. The experimental results show that the presented method exhibits performance improvements in the adverse noise environment compared with the previous features.

The performance of the speech recognition system for English classroom teaching is largely affected by the surrounding environment. These interference signals will seriously reduce the quality and intelligibility of the speech signal, thereby greatly reducing the performance of the far-field speech recognition system. Aiming at word order detection in English classroom teaching, this paper proposes an analysis model based on block coding and improved genetic algorithm. Moreover, for DNN-based single-channel speech enhancement algorithms, this paper proposes PDNNs and PLSTMs to solve the problem of serious performance degradation of prototype DNN speech enhancement under low signal-to-noise ratio. This method decomposes the entire enhancement task into multiple subtasks to complete, and the previously completed subtasks will provide prior knowledge for the subsequent subtasks, so that the subsequent subtasks can learn its goals better. In general, the experimental results prove the reliability of the model constructed in this paper.

—The research on speech recognition systems currently focuses on the analysis of robust speech recognition systems. When the speech signals are combined with noise, the recognition system becomes distracted, struggling to identify the speech sounds. Therefore, the development of a robust speech recognition system continues to be carried out. The principle of a robust speech recognition system is to eliminate noise from the speech signals and restore the original information signals. In this paper, researchers conducted a frequency domain analysis on one stage of the Mel Frequency Cepstral Coefficients (MFCC) process, the Fast Fourier Transform (FFT), in children's speech recognition system. The FTT analysis in the feature extraction process determined the effect of frequency value characteristics utilized in the FFT output on the noise disruption. The analysis method was designed into three scenarios based on the value of the employed FFT points. The differences between scenarios were based on the number of shared FFT points. All FFT points were divided into four, three, and two parts in the first, second, and third scenarios, respectively. This study utilized children's speech data from the isolated TIDIGIT English digit corpus. As comparative data, the noise was added manually to simulate real-world conditions. The results showed that using a particular frequency portion following the scenario designed on MFCC affected the recognition system performance, which was relatively significant on the noisy speech data. The designed method in the scenario 3 (C1) version generated the highest accuracy, exceeded the accuracy of the conventional MFCC method. The average accuracy in the scenario 3 (C1) method increased by 1% more than all the tested noise types. Using various noise intensity values (SNR), the testing process indicates that scenario 3 (C1) generates a higher accuracy than conventional MFCC in all tested SNR values. It proves that the selection of specific frequency utilized in MFCC feature extraction significantly affects the recognition accuracy in a noisy speech.

Code-switching is the frequent alternation between two or more languages within a single utterance and is a widespread phenomenon among bilingual and multilingual speakers. In India, more than 250 million people are estimated to engage in code-switched communication, especially blending English with Hindi (Hinglish), making it one of the largest bilingual populations globally, making challenging for developing accurate and robust Automatic Speech Recognition (ASR) systems. Existing ASR models, typically trained on monolingual corpus, struggle with code-switched input due to a lack of large, balanced, and representative datasets—particularly for diverse age groups. Recent evaluations have shown that ASR models experience a relative increase in Word Error Rate (WER) of 30–50 % when exposed to code-switched speech compared to monolingual input. To address this resource gap, we introduce a benchmark Hinglish speech corpus, HiACC, to improve ASR performance in resource-constrained settings. While several monolingual Hindi and English corpus exist, publicly available code-switched datasets remain scarce, and none till date include children's speech. Our corpus fills this gap by providing the first code-switched Hinglish speech dataset with recordings from both adults and children. It comprises 3,318 audio segments from adult participants and 1,858 segments from children, covering 5.24 hours of read and spontaneous speech. The transcriptions include detailed annotations and code-switching tags to assist in linguistic and computational analysis. The corpus is publicly available at [https://zenodo.org/records/15551669], offering segmented audio and aligned transcripts for open research. We also present baseline ASR experiments, which show that standard models trained on monolingual data underperform by approximately 42 % WER on our test set, highlighting the complexity of the task. To our knowledge, this is the first publicly available resource on code-switched Hinglish speech encompassing both adult and child speakers, designed to catalyse progress in this challenging yet important area of speech recognition.