Estimating speech recognition error rate without acoustic test data

Mel-frequency Cesptral Coefficients (MFCC) and Predictive Linear Prediction (PLP) coefficients are two popular representations of continuous speech in existing Hidden Markov Model (HMM) based Automatic Speech Recognition (ASR) systems. Cepstral Mean Normalization (CMN) is often used as a post-processing step in the extraction of MFCC and PLP features to further enhance noise robustness at almost negligible computational cost. In this paper we build a closed dictionary, large vocabulary HMM-based Indonesian language ASR system using the CMU Sphinx in speech recognition toolkit implementing MFCC and PLP feature extraction, and CMN. We test the effect of various types and levels of noise on the word error rate (WER) of speech recognition. Utilizing CMN, an average improvement of 2% recognition over standard MFCC and PLP extraction methods is obtained at signal-to-noise ratios (SNR) below 24 decibels. A significant drop in recognition is observed between 12 and 6 dB SNR.

In this paper, the improvements in the recently implemented Kannada speech recognition system is demonstrated in detail. The Kannada automatic speech recognition (ASR) system consists of ASR models which are created by using Kaldi, IVRS call flow and weather and agricultural commodity prices information databases. The task specific speech data used in the recently developed spoken dialogue system had high level of different background noises. The different types of noises present in collected speech data had an adverse effect on the on line and off line speech recognition performances. Therefore, to improve the speech recognition accuracy in Kannada ASR system, a noise reduction algorithm is developed which is a fusion of spectral subtraction with voice activity detection (SS-VAD) and minimum mean square error spectrum power estimator based on zero crossing (MMSE-SPZC) estimator. The noise elimination algorithm is added in the system before the feature extraction part. An alternative ASR models are created using subspace Gaussian mixture models (SGMM) and deep neural network (DNN) modeling techniques. The experimental results show that, the fusion of noise elimination technique and SGMM/DNN based modeling gives a better relative improvement of 7.68% accuracy compared to the recently developed GMM-HMM based ASR system. The least word error rate (WER) acoustic models could be used in spoken dialogue system. The developed spoken query system is tested from Karnataka farmers under uncontrolled environment.

Combined speech enhancement and auditory modelling for robust distributed speech recognition

Challenges of Automatic Speech Recognition for medical interviews - research for Polish language

We report insights from translating Spanish conversational telephone speech into English text by cascading an automatic speech recognition (ASR) system with a statistical machine translation (SMT) system. The key new insight is that the informal register of conversational speech is a greater challenge for ASR than for SMT: the BLEU score for translating the reference transcript is 64%, but drops to 32% for translating automatic transcripts, whose word error rate (WER) is 40%. Several strategies are examined to mitigate the impact of ASR errors on the SMT output: (i) providing the ASR lattice, instead of the 1-best output, as input to the SMT system, (ii) training the SMT system on Spanish ASR output paired with English text, instead of Spanish reference transcripts, and (iii) improving the core ASR system. Each leads to consistent and complementary improvements in the SMT output. Compared to translating the 1-best output of an ASR system with 40% WER using an SMT system trained on Spanish reference transcripts, translating the output lattice of a better ASR system with 35% WER using an SMT system trained on ASR output improves BLEU from 32% to 38%.

This study investigates the effect of vowel-reduction dubbing practice on the Word Error Rate (WER) of automatic speech recognition (ASR)-generated captions in Zoom-based English presentations. Employing a convergent mixed-methods design, the research examines how misarticulation related to vowel reduction—particularly the realization of schwa—shapes ASR performance and caption accuracy. Twentyseven English teacher candidates completed structured dubbing tasks followed by ASR transcription analysis using Zoom’s auto-captioning system. WER was calculated using standard computational formulae, and both quantitative data (percentages, accuracy ratios, and error rates) and qualitative data (error types, substitution patterns, and prosodic deviations) were systematically analyzed. Results indicate that substitution errors remained the most dominant ASR error type, especially when vowel reduction occurred without adequate prosodic support. A substantial improvement was observed following the intervention: the average WER decreased by approximately 52% (from 27.4% to 13.2%), demonstrating a strong effect of targeted dubbing practice on caption intelligibility. Participants who maintained clearer vowel quality and rhythmic timing achieved consistently lower WER scores, while those applying excessive or inconsistent vowel reduction produced higher rates of substitution and deletion errors. The study highlights the pedagogical value of integrating ASR-based feedback with dubbing activities to strengthen learners’ pronunciation intelligibility and schwa awareness. It further underscores the importance of TPACK-informed instruction that cultivates learners’ sensitivity to both human comprehensibility and machine recognition. Overall, these findings contribute to ongoing developments in pronunciation pedagogy by bridging technology-enhanced learning with EFL teacher education.

The automatic speech recognition (ASR) system based on deep neural network is easy to be attacked by an adversarial example due to the vulnerability of neural network, which is a hot topic in recent years. The adversarial example does harm to the ASR system, especially if the common-dependent ASR goes wrong, it will lead to serious consequences. To improve the robustness and security of the ASR system, the defense method against adversarial examples must be proposed. Based on this idea, we propose an algorithm of devastation and detection on adversarial examples which can attack the current advanced ASR system. We choose advanced text-dependent and command-dependent ASR system as our target system. Generating adversarial examples by the OPT on text-dependent ASR and the GA-based algorithm on command-dependent ASR. The main idea of our method is input transformation of the adversarial examples. Different random intensities and kinds of noise are added to the adversarial examples to devastate the perturbation previously added to the normal examples. From the experimental results, the method performs well. For the devastation of examples, the original speech similarity before and after adding noise can reach 99.68%, the similarity of the adversarial examples can reach 0%, and the detection rate of the adversarial examples can reach 94%.

This study explores the feasibility of constructing a small-scale speech recognition system capable of competing with larger, modern automated speech recognition (ASR) systems in both performance and word error rate (WER). Our central hypothesis posits that a compact transformer-based ASR model can yield comparable results, specifically in terms of WER, to traditional ASR models while challenging contemporary ASR systems that boast significantly larger computational sizes. The aim is to extend ASR capabilities to under-resourced languages with limited corpora, catering to scenarios where practitioners face constraints in both data availability and computational resources. The model, comprising a compact convolutional neural network (CNN) and transformer architecture with 2.214 million parameters, challenges the conventional wisdom that large-scale transformer-based ASR systems are essential for achieving high accuracy. In comparison, contemporary ASR systems often deploy over 300 million parameters. Trained on a modest dataset of approximately 3000 h – significantly less than the 50,000 h used in larger systems – the proposed model leverages the Common Voice and LibriSpeech datasets. Evaluation on the LibriSpeech test-clean and test-other datasets produced character error rates (CERs) of 6.40% and 16.73% and WERs of 16.03% and 35.51% respectively. Comparisons with existing architectures showcase the efficiency of our model. A gated recurrent unit (GRU) architecture, albeit achieving lower error rates, incurred a computational cost 24 times larger than our proposed model. Large-scale transformer architectures, while achieving marginally lower WERs (2%–4% on LibriSpeech test-clean), require 200 times more parameters and 53,000 additional hours of training data. Modern large language models are used to improve the WERs, but require large computational resources. To further enhance performance, a small 4-g language model was integrated into our end-to-end ASR model, resulting in improved WERs. The overarching goal of this work is to provide a practical solution for practitioners dealing with limited datasets and computational resources, particularly in the context of under-resourced languages.

This paper describes open source software (available at https://github.com/robotology/natural-speech) to build automatic speech recognition (ASR) systems and run them within the YARP platform. The toolkit is designed (i) to allow non-ASR experts to easily create their own ASR system and run it on iCub and (ii) to build deep learning-based models specifically addressing the main challenges an ASR system faces in the context of verbal human–iCub interactions. The toolkit mostly consists of Python, C++ code and shell scripts integrated in YARP. As additional contribution, a second codebase (written in Matlab) is provided for more expert ASR users who want to experiment with bio-inspired and developmental learning-inspired ASR systems. Specifically, we provide code for two distinct kinds of speech recognition: “articulatory” and “unsupervised” speech recognition. The first is largely inspired by influential neurobiological theories of speech perception which assume speech perception to be mediated by brain motor cortex activities. Our articulatory systems have been shown to outperform strong deep learning-based baselines. The second type of recognition systems, the “unsupervised” systems, do not use any supervised information (contrary to most ASR systems, including our articulatory systems). To some extent, they mimic an infant who has to discover the basic speech units of a language by herself. In addition, we provide resources consisting of pre-trained deep learning models for ASR, and a 2.5-h speech dataset of spoken commands, the VoCub dataset, which can be used to adapt an ASR system to the typical acoustic environments in which iCub operates.

In this study, a real-time automatic speech recognition (ASR) system based on the Kaldi ASR toolkit, with low-latency and customizable models, without any internet connection, was developed. The proposed ASR system includes a voice activity detection (VAD) module and an audio transmitter as a front-end speech processing and a decoder for the received audio signals. The ASR system was evaluated in terms of ASR accuracy and speech processing speed. Consequently, the ASR system achieved high ASR accuracy on the CSJ (Corpus of Spontaneous Japanese) test set with super low-latency.

Recent developments in automatic speech recognition (ASR) systems have resulted in increased use of automated transcription services to analyze verbal communications. We aimed to analyze the accuracy of AWS General Transcribe (AWS-GT) on audio recordings of home healthcare patients. These audio recordings included in-person clinician visits and phone calls with research assistants (RAs). Study staff rated the clarity of the audio using a scale of low, medium and high. The audio quality was similar among the two types of recordings. Our results are a part of a larger study aiming to use automated speech processing to identify risk factors for hospitalization and emergency department visits. Overall, 4002 utterances, defined as uninterrupted blocks of speech with four or more words, were analyzed— 3,472 (87%) from clinician recordings and 520 (13%) from RA recordings. Word error rate (WER; range 0(best)-1(worst)) was used to assess the performance of AWS-GT, compared to a gold-standard manual transcription. We found a statistically significant difference in WER for clinician recordings (.26) and RA recordings (.19) (p < 0.05). We also observed that the WER was poorest for short, four-to-eight-words, utterances (.39), and that this improved as utterances grew longer (p < 0.05). Clinician recordings (35%) had a greater number of short utterances than RA recordings (25%) (p < 0.05). This shows how differences in conversation dynamics can impact the accuracy of ASR systems. It is incredibly important that we determine whether existing software is accurate enough for use in clinical documentation and prediction models dependent on ASR systems.

We present a novel approach for improving the performance of an End-to-End speech recognition system for the Gujarati language. We follow a deep learning-based approach that includes Convolutional Neural Network, Bi-directional Long Short Term Memory layers, Dense layers, and Connectionist Temporal Classification as a loss function. To improve the performance of the system with the limited size of the dataset, we present a combined language model (Word-level language Model and Character-level language model)-based prefix decoding technique and Bidirectional Encoder Representations from Transformers-based post-processing technique. To gain key insights from our Automatic Speech Recognition (ASR) system, we used the inferences from the system and proposed different analysis methods. These insights help us in understanding and improving the ASR system as well as provide intuition into the language used for the ASR system. We have trained the model on the Microsoft Speech Corpus, and we observe a 5.87% decrease in Word Error Rate (WER) with respect to base-model WER.

At present, most multi-dialect speech recognition models are based on a hard-parameter-sharing multi-task structure, which makes it difficult to reveal how one task contributes to others. In addition, in order to balance multi-task learning, the weights of the multi-task objective function need to be manually adjusted. This makes multi-task learning very difficult and costly because it requires constantly trying various combinations of weights to determine the optimal task weights. In this paper, we propose a multi-dialect acoustic model that combines soft-parameter-sharing multi-task learning with Transformer, and introduce several auxiliary cross-attentions to enable the auxiliary task (dialect ID recognition) to provide dialect information for the multi-dialect speech recognition task. Furthermore, we use the adaptive cross-entropy loss function as the multi-task objective function, which automatically balances the learning of the multi-task model according to the loss proportion of each task during the training process. Therefore, the optimal weight combination can be found without any manual intervention. Finally, for the two tasks of multi-dialect (including low-resource dialect) speech recognition and dialect ID recognition, the experimental results show that, compared with single-dialect Transformer, single-task multi-dialect Transformer, and multi-task Transformer with hard parameter sharing, our method significantly reduces the average syllable error rate of Tibetan multi-dialect speech recognition and the character error rate of Chinese multi-dialect speech recognition.

Analyzing teachers’ discourse plays a fundamental role in educational research and is a key component of Teaching Analytics. This usually involves transcribing lessons from audio recordings. As the number of recordings grows, Automatic Speech Recognition (ASR) systems gain popularity as a means for transcribing these recordings. However, most ASR systems are trained over very specific domains which usually involve read text and low environmental noise. This suggests common ASR systems available on the market may underperform over classroom recordings, as they present a unique type of environmental sound and spontaneous discourse, as opposed to the usual training domains. To address this challenge we present a system that automatically transcribes classroom discourse in a robust way with regard to classroom noise, which was trained over few annotated data. In particular, we used a state-of-the-art ASR model based on wav2vec 2.0 and fine-tuned it over a 6-h dataset of 4th to 8th grade Chilean lessons. We found that by leveraging its transformer-based architecture and changing the fine-tuning domain to classroom recordings, we can obtain a more accurate and robust transcriber for this source of audio which outperforms other popular cloud-based systems up to 35% and 59% in terms of Word and Character Error Rates, respectively. This work contributes by using state-of-the-art ASR techniques to develop a tool which is particularly adapted to classroom environments, making it robust and more reliable with regard to their environmental sound and the way teaching discourse is carried out.KeywordsAutomatic speech recognitionClassroom discourseTeaching analyticsTransfer learning

The aim of this study is to quantify the gap between the recognition performance of human listeners and an automatic speech recognition (ASR) system with special focus on intrinsic variations of speech, such as speaking rate and effort, altered pitch, and the presence of dialect and accent. Second, it is investigated if the most common ASR features contain all information required to recognize speech in noisy environments by using resynthesized ASR features in listening experiments. For the phoneme recognition task, the ASR system achieved the human performance level only when the signal-to-noise ratio (SNR) was increased by 15 dB, which is an estimate for the human-machine gap in terms of the SNR. The major part of this gap is attributed to the feature extraction stage, since human listeners achieve comparable recognition scores when the SNR difference between unaltered and resynthesized utterances is 10 dB. Intrinsic variabilities result in strong increases of error rates, both in human speech recognition (HSR) and ASR (with a relative increase of up to 120%). An analysis of phoneme duration and recognition rates indicates that human listeners are better able to identify temporal cues than the machine at low SNRs, which suggests incorporating information about the temporal dynamics of speech into ASR systems.