Lightly Supervised Acoustic Model Training for Mandarin Continuous Speech Recognition

Discriminative training techniques have been shown to consistently outperform the maximum likelihood (ML) paradigm for acoustic model training in automatic speech recognition (ASR). Consequently, today's discriminative training methods are fundamental components of state-of-the-art systems and are a major line of research in speech recognition. This article gives a comprehensive overview of discriminative training methods for acoustic model training in the context of ASR. The article covers all related aspects of discriminative training for speech recognition, i.e., specific training criteria and their relation, statistical modeling, different parameter optimization approaches, efficient implementation of discriminative training, and a performance overview.

For effective training of acoustic and language models for spontaneous speech such as meetings, it is significant to exploit the texts available in a large scale, which may not be faithful transcripts of the utterances. We have proposed a language model transformation scheme to cope with the differences between verbatim transcripts of spontaneous utterances and human-made transcripts such as those in proceedings. In this paper, we investigate its application to lightly supervised training of the acoustic model. By transforming the corresponding text in the proceedings, we can generate a very constrained model to predict the actual utterances. The experimental evaluation with the transcription system for the Japanese Congress meetings demonstrated that the proposed scheme can generate accurate labels for acoustic model training and thus realizes the comparable ASR (Automatic Speech Recognition) performance to the case using manual transcripts.

An efficient, scalable speech recognition architecture is proposed for multidomain dialog systems by combining topic detection and topic-dependent language modeling. The inferred domain is automatically detected from the user's utterance, and speech recognition is then performed with an appropriate domain-dependent language model. The architecture improves accuracy and efficiency over current approaches and is scaleable to a large number of domains. In this paper, a novel framework using a multilayer hierarchy of language models is introduced in order to improve robustness against topic detection errors. The proposed system provides a relative reduction in WER of 10.5% over a single language model system. Furthermore it achieves an accuracy that is comparable to using multiple language models in parallel while using only a fraction of the computational cost.

Semi-supervised and unsupervised discriminative language model training for automatic speech recognition

Automatic speech recognition (ASR) aims at understanding naturally spoken human speech to be used as text inputs to machines. In multi-speaker environments, where multiple speakers are talking simultaneously with a large amount of overlap, a significant performance degradation may occur with conventional ASR systems if they are trained by recordings of single talkers. This paper proposes a multi-speaker ASR method that incorporates speaker embedding information as an additional input. The embedding information for each of the speakers in the training set was extracted as numeric vectors, and all of the embedding vectors were stacked to construct a total speaker profile matrix. The speaker profile matrix from the training dataset enables finding embedding vectors that are close to the speakers of the input recordings in the test conditions, and it helps to recognize the individual speakers’ voices mixed in the input. Furthermore, the proposed method efficiently reuses the decoder from the existing speaker-independent ASR model, eliminating the need for retraining the entire system. Various speaker embedding methods such as i-vector, d-vector, and x-vector were adopted, and the experimental results show 0.33% and 0.95% absolute (3.9% and 11.5% relative) improvements without and with the speaker profile in the word error rate (WER).

This study analyzes the effect of stress in human and automatic stressed speech processing tasks for speech collected from non-professional speakers. The database of 33 keywords is collected under five stress conditions, namely, neutral, angry, happy, sad and Lombard from fifteen speakers. The first study is to understand the ability to identify stress by human and automatic speech processing. The average performance of human stress classification is 59.44%. The average performance of automatic stress classifier using Vector Quantization (VQ) and Hidden Markov model (HMM) is 54.65% and 56.02%, respectively. The second study has been done to understand the effect of stress in human and automatic speech recognition. The average performance of human stressed speech recognition is 99.60%. The automatic stressed speech recognition performance using VQ and HMM is 82.42% and 76.79%, respectively. Even in the non-professional speakers scenario, human performance is better than automatic processing. Also, automatic processing seem to show considerable degradation in performance that warrant development of new methods to handle stress information.

Introduction: Ambient artificial intelligence (AAI) is increasingly being integrated into clinical workflows to address the documentation burden contributing to clinician burnout. AAI systems use automatic speech recognition and natural language processing to capture real-time clinician–patient dialogue and generate structured documentation within the electronic health record. Methods: This review examines AAI’s technical foundations, clinical applications, and implementation considerations in oncology practice. We describe how current AAI tools extract, classify, and populate relevant clinical content, such as the history of present illness, physical examination findings, and assessment and plan, while adhering to established US coding standards (e.g., International Classification of Diseases [ICD-10]). Key commercially available systems, including Nuance DAX Copilot, Ambience Healthcare, Tali AI, Athelas Scribe, and Nabla, are compared in terms of functionality, integration capabilities, and reported outcomes. Practical guidance is provided on clinician onboarding, device setup, customization, and patient consent. Results: Early data suggest AAI implementation can reduce after-hours documentation, improve note quality, and enhance coding accuracy. However, barriers such as transcription errors, hallucinations, and workflow disruptions must be proactively addressed. To ensure safe and effective integration, oncology practices adopting AAI should prioritize targeted training, documentation oversight, and user feedback. Conclusion: Although preliminary findings are promising, further prospective, multi-institutional studies are warranted to evaluate the impact of AAI on clinical efficiency, data quality, and patient outcomes in oncology. This review provides a foundational framework for researchers, clinicians, and health systems seeking to implement AAI in precision oncology documentation.

The minimum classification error (MCE) framework for discriminative training is a simple and general formalism for directly optimizing recognition accuracy in pattern recognition problems. The framework applies directly to the optimization of hidden Markov models (HMMs) used for speech recognition problems. However, few if any studies have reported results for the application of MCE training to large-vocabulary, continuous-speech recognition tasks. This article reports significant gains in recognition performance and model compactness as a result of discriminative training based on MCE training applied to HMMs, in the context of three challenging large-vocabulary (up to 100 k word) speech recognition tasks: the Corpus of Spontaneous Japanese lecture speech transcription task, a telephone-based name recognition task, and the MIT Jupiter telephone-based conversational weather information task. On these tasks, starting from maximum likelihood (ML) baselines, MCE training yielded relative reductions in word error ranging from 7% to 20%. Furthermore, this paper evaluates the use of different methods for optimizing the MCE criterion function, as well as the use of precomputed recognition lattices to speed up training. An overview of the MCE framework is given, with an emphasis on practical implementation issues

Pretrained multilingual End-to-End (E2E) Automatic Speech Recognition (ASR) models demonstrate remarkable capabilities but struggle with domain-specific terminologies and underrepresented dialects. Fine-tuning requires expensive paired audio-transcript data, creating barriers to practical adaptation. This paper investigates language model (LM) integration via shallow fusion as an efficient, text-only adaptation method. We introduce Confidence Gated Fusion (CGF), a novel approach that dynamically determines the LM weight during decoding based on ASR model uncertainty, eliminating the expensive validation-set-dependent grid search required by traditional shallow fusion. We validate our approach using OpenAI’s Whisper across multiple model sizes on Arabic ASR, evaluating Modern Standard Arabic (MSA), Egyptian dialect (EGY), and a specialized judiciary domain. Integrating a domain-specific LM achieved up to 40.92% relative WER reduction on dialectal speech and 32.96% on the judiciary domain. Our CGF method achieved comparable performance to tuned baselines while requiring no hyperparameter optimization, with particular advantages in specialized domains (28.92% relative WER reduction on judiciary) and on smaller models where static weights often lack robustness.

Code-switching is a commonly occurring phenomenon in many multilingual communities, wherein a speaker switches between languages within a single utterance. Conventional Word Error Rate (WER) is not sufficient for measuring the performance of code-mixed languages due to ambiguities in transcription, misspellings and borrowing of words from two different writing systems. These rendering errors artificially inflate the WER of an Automated Speech Recognition (ASR) system and complicate its evaluation. Furthermore, these errors make it harder to accurately evaluate modeling errors originating from code-switched language and acoustic models. In this work, we propose the use of a new metric, transliteration-optimized Word Error Rate (toWER) that smoothes out many of these irregularities by mapping all text to one writing system and demonstrate a correlation with the amount of code-switching present in a language. We also present a novel approach to acoustic and language modeling for bilingual code-switched Indic languages using the same transliteration approach to normalize the data for three types of language models, namely, a conventional n-gram language model, a maximum entropy based language model and a Long Short Term Memory (LSTM) language model, and a state-of-the-art Connectionist Temporal Classification (CTC) acoustic model. We demonstrate the robustness of the proposed approach on several Indic languages from Google Voice Search traffic with significant gains in ASR performance up to 10% relative over the state-of-the-art baseline.

Topic identification techniques applied to dynamic language model adaptation for automatic speech recognition

This article describes a new automatic transcription system in the Japanese Parliament which deploys our automatic speech recognition (ASR) technology. To achieve high recognition performance in spontaneous meeting speech, we have investigated an efficient training scheme with minimal supervision which can exploit a huge amount of real data. Specifically, we have proposed a lightly-supervised training scheme based on statistical language model transformation, which fills the gap between faithful transcripts of spoken utterances and final texts for documentation. Once this mapping is trained, we no longer need faithful transcripts for training both acoustic and language models. Instead, we can fully exploit the speech and text data available in Parliament as they are. This scheme also realizes a sustainable ASR system which evolves, i.e. update/re-train the models, only with speech and text generated during the system operation. The ASR system has been deployed in the Japanese Parliament since 2010, and consistently achieved character accuracy of nearly 90%, which is useful for streamlining the transcription process.

Open Access Journal of Neurology & Neurosurgery has been developed as an opportunity for the free and open international exchange of information regarding the progress in the clinical and neurosciences. OAJNN publishes original research, reviews, mini reviews, case reports

Sequence-Level Unsupervised Training in Speech Recognition: A Theoretical Study

We present speech experiments that were carried out to evaluate a topically focusing language model in large vocabulary speech recognition. An ordered topical clustering is first computed as a self-organized mapping of a large document collection. Language models are then trained for each text cluster or for several neighboring clusters. The obtained organized collection of language models is efficiently utilized in continuous speech recognition to concentrate on the model that corresponds closest to the current topic of discussion. The speech recognition experiments are carried out on a novel Finnish speech database. A property of Finnish that is particularly challenging for speech recognition is the extremely fast vocabulary growth that makes many of the standard word-based language modeling methods impractical for large vocabulary tasks.