TED Talk Teaser Generation with Pre-Trained Models

Although over 300M around the world speak Bangla, scant work has been done in improving Bangla voice-to-text transcription due to Bangla being a low-resource language. However, with the introduction of the Bengali Common Voice 9.0 speech dataset, Automatic Speech Recognition (ASR) models can now be significantly improved. With 399hrs of speech recordings, Bengali Common Voice is the largest and most diversified open-source Bengali speech corpus in the world. In this paper, we outperform the State-of-the-Art (SOTA) pretrained Bengali ASR models by finetuning a pretrained wav2vec2 model on the common voice dataset. We also demonstrate how to significantly improve the performance of an ASR model by adding an n-gram language model as a post-processor. Finally, we do some experiments and hyperparameter tuning to generate a robust Bangla ASR model that is better than the existing ASR models.

This research introduces a comprehensive strategy to enhance the performance of an existing automatic speech recognition (ASR) model, which has been previously documented in published articles. The study sets out to achieve several objectives. Firstly, it concentrates on updating the ASR model by retraining it with new datasets. This involves integrating samples from the latest Common Voice corpus release and data collected independently via the armspeech.com web application. Another key focus lies in optimizing the ASR model for near-real-time processing, intending to improve its speed and efficiency. The proposed adjustments to the model’s architecture aim to balance accuracy and processing speed, which is essential for applications requiring prompt speech recognition. Furthermore, the research explores the integration of Transformer models into the post-processing pipeline to introduce punctuation and capitalization into the ASR output. This step not only enhances the linguistic quality of transcriptions but also improves their readability and usability. In tandem with these advancements, the research presents a systematic approach to gathering, annotating, and storing datasets specifically tailored for punctuation and capitalization tasks. The methodology outlines the acquisition and organization of a dataset conducive to training Transformer models for these linguistic tasks. This comprehensive approach, which encompasses dataset enrichment, architectural modifications, and post-processing enhancements, aims to elevate the ASR model’s accuracy, speed, and linguistic refinement, with a particular focus on addressing the intricacies of the Armenian language. The research contributes valuable insights into the optimization of ASR systems, tackling both language-specific challenges and broader issues related to linguistic post-processing.

Geriatricians are not adequately represented in national panels, health department guidelines, and the lay press.1 People, from numerous non-geriatric medicine fields, produce podcasts, promote products, and provide advice on aging issues that may be uninformed by expertise. We sought to better understand the aging-related content and backgrounds of experts portrayed by a leading national online lecture, TED Talks. TED talks are freely distributed under the slogan “ideas worth spreading” and are available on multiple platforms: Ted.com, YouTube, and podcasts.2 They are immensely popular, with the videos having been viewed more than a billion times online. Speaker backgrounds vary widely and include Nobel and Pulitzer Prize, Oscar, Grammy, and Emmy winners. It is important to distinguish between TED and TEDx Talks. TED is developed on a national platform, while TEDx is set up by local volunteer organizers without national supervision. Quality control has been a major issue as speakers are not always fully vetted (e.g., non-medical speakers providing medical advice). Unvetted speakers have reached both the TED and TEDx stages, but it is believed to be more frequent with TEDx as quality control lies solely in the hands of the local event organizers.3 We performed a search of all aging-related content produced by the TED Talk media and communication platform, from October 2006 to December 2021. Search terms included: aging, aged, seniors, older adults, geriatrics, geriatric medicine, geriatrician. We selected TED Talks (national) but excluded TEDx (local) Talks. The rationale was two-fold: (1) We anticipated better vetting of speakers from national groups. (2) A national cohort would reflect more generalizable findings instead of local subject-promoting enthusiasts. Data was collected examining: (1) content of lecture, (2) number of views, (3) date of activity, (4) speaker, and (5) presented background. When the background was unclear, we utilized LinkedIn profiles for further clarity. Descriptive frequencies were utilized for viewership. Qualitative analysis of lecture content (pertinence to aging-related or geriatrics) and presenter background was conducted using constant comparative thematic analysis between two coders, with a third coder available for tiebreak.4 Using the search terms “aging,” “seniors,” and “geriatrics,” 60 TED talks were determined to provide relevant geriatrics related content. Thematic analysis revealed major themes of (1) Delaying aging/living longer (e.g., roadmap to end aging, living to 100 years); (2) Wisdom from aging (e.g., how to live passionately no matter your age); (3) Testing for age-related diseases (e.g., Parkinson's); (4) Cures for common age-related diseases (Alzheimer's disease, vision loss, arthritis); (5) Older adult finance; and (6) Dying and Hospice. Among speakers, there were no board-certified geriatricians and only one palliative care certified physician. Backgrounds varied widely and included writer, actors, activist, filmmaker, molecular biologist, psychiatrist, psychologist, lawyer, philosopher (Table 1), healthcare professionals included neurologist, neuroscientist, developmental psychologist, and palliative care physician. Among the aging-related content in popular TED talks, no board-certified geriatricians have been tapped to provide expertise. Although capable, nationally renown geriatricians are available, most speakers are not from medical backgrounds and have unclear expertise in aging. This lack of geriatrics expertise may negatively impact age-friendly care and adversely affect older adults if misinformation is followed. It is unclear if geriatricians are applying to speak at TED talks, but through word of mouth, several have applied.5 There are also fewer geriatricians than other occupations (e.g., entertainment, journalism), which might account for the higher representation in other fields. From a content perspective, many topics were focused on avoiding aging altogether or finding cures instead of living with it or its treatments. While this might increase viewership, it is not realistic to the tenets of geriatric medicine.6 In addition, multiple topics could have potentially been presented by a geriatrician, displaying overlap in abilities (e.g., neurologist vs. geriatrician). Limitations existed in that we examined national TED Talks and not TEDx Talks or TEDMed; therefore, geriatricians may have presented locally but not been captured. TEDMed was an annual health-related speaking venue, active from 2009 to 2020, operating separately under license from TED Conferences. The majority of TEDMed talks have a lower viewership, with under 10,000 views, compared to TED talks, which often have a million or more views. On our review of TEDMed, there were only three geriatricians who presented, with only one speaking on aging-related issues (Dr. Louise Aronson, Embracing elderhood as a stage of life) and the other two discussing obesity and population health.7 Another limitation is that our results may not be representative of other media outlets, including podcasts (e.g., GeriPal.org), which may have a higher rate of geriatricians and geriatrics-related content. In conclusion, multiple online platforms, such as TED Talks, have emerged to disseminate geriatrics-related content. Unfortunately, geriatricians have not been utilized fully, and the resulting misinformation from non-geriatrician sources may be harmful to broad audiences. Platforms like TED Talks, YouTube, and TikTok are important for geriatricians to delve into as well as continue to have a strong presence on Twitter, Instagram, and Facebook.8-10 It is important that geriatricians are encouraged to have a strong presence in popular media to offset ill-informed messages regarding the health of older adults. All authors met criteria for authorship by Conception and design of the study: Liggett, Lindquist. Data acquisition: Liggett, Lindquist. Analysis and interpretation of data: All authors. Manuscript drafting: All authors. Revising the manuscript critically for important intellectual content: All authors. Approval of the version of the manuscript to be published: All authors. This research is supported through grants from the NIH/NIA P30AG059988. The authors declare that there is no conflict of interest. The sponsor was not involved in the design, methods, analysis and interpretation of the data, and preparation of the manuscript.

Error correction techniques have been used to refine the output sentences from automatic speech recognition (ASR) models and achieve a lower word error rate (WER). Previous works usually adopt end-to-end models and has strong dependency on Pseudo Paired Data and Original Paired Data. But when only pre-training on Pseudo Paired Data, previous models have negative effect on correction. While fine-tuning on Original Paired Data, the source side data must be transcribed by a well-trained ASR model, which takes a lot of time and not universal. In this paper, we propose UCorrect, an unsupervised Detector-Generator-Selector framework for ASR Error Correction. UCorrect has no dependency on the training data mentioned before. The whole procedure is first to detect whether the character is erroneous, then to generate some candidate characters and finally to select the most confident one to replace the error character. Experiments on the public AISHELL-1 dataset and WenetSpeech dataset show the effectiveness of UCorrect for ASR error correction: 1) it achieves significant WER reduction, achieves 6.83% even without fine-tuning and 14.29% after fine-tuning; 2) it outperforms the popular NAR correction models by a large margin with a competitive low latency; and 3) it is an universal method, as it reduces all WERs of the ASR model with different decoding strategies and reduces all WERs of ASR models trained on different scale datasets.

Pretrained multilingual End-to-End (E2E) Automatic Speech Recognition (ASR) models demonstrate remarkable capabilities but struggle with domain-specific terminologies and underrepresented dialects. Fine-tuning requires expensive paired audio-transcript data, creating barriers to practical adaptation. This paper investigates language model (LM) integration via shallow fusion as an efficient, text-only adaptation method. We introduce Confidence Gated Fusion (CGF), a novel approach that dynamically determines the LM weight during decoding based on ASR model uncertainty, eliminating the expensive validation-set-dependent grid search required by traditional shallow fusion. We validate our approach using OpenAI’s Whisper across multiple model sizes on Arabic ASR, evaluating Modern Standard Arabic (MSA), Egyptian dialect (EGY), and a specialized judiciary domain. Integrating a domain-specific LM achieved up to 40.92% relative WER reduction on dialectal speech and 32.96% on the judiciary domain. Our CGF method achieved comparable performance to tuned baselines while requiring no hyperparameter optimization, with particular advantages in specialized domains (28.92% relative WER reduction on judiciary) and on smaller models where static weights often lack robustness.

In the context of huge volume of data generated in the last couple of years and the need of processing audio stream in real time data processing flows, this paper aims to explore, identify and experiment Automatic Speech Recognition (ASR) models in the context of low latency applications depending on different contexts using specific data sets. Our work aims to explore whether if such models can be deployed in the real time stream audio workflow and for that we compare multiple size of the Whisper model for multiple data sets and analyze the results. Additionally, we tested the models for English and Romanian language. In the future, for the multi-language context it is highly important to analyze using model dedicated fine-tuned for specific language and compare with the results of multi-language models. This paper contributes to the field by providing evidence, comparison, statistics, level of accuracy in order to use ASR models in the context of low latency applications, exploring multiple versions of Whispers models and help researcher in dimensioning the environments for real time infrastructure and extend streaming capabilities with Artificial Intelligence models. Equally, the paper can used for IT professional in running critical systems in different industries to emphasize the techniques to be used in running ASR model in production.

This article concerns comparative studies on the Automatic Speech Recognition (ASR) model incorporated with the Large Language Model (LLM) used for medical interviews. The proposed solution is tested on polish language benchmarks and dataset with medical interviews. The latest ASR technologies are based on convolutional neural networks (CNNs), recurrent neural networks (RNNs) and Transformers. Most of them work as end-to-end solutions. The presented approach in the case of the Whisper model shows a two-stage solution with End-To-End ASR and LLM working together in a pipeline. The ASR output is an input for LLM. The LLM is a component by which the output from ASR is corrected and improved. Comparative studies for automatic recognition of the Polish language between modern End-To-End deep learning architectures and the ASR hybrid model were performed. The medical interview tests were performed with two state-of-the-art ASR models: OpenAI Whisper incorporated with LLM and Scribe ElevenLabs. Additionally, the results were compared with five more end-to-end models (QuartzNet, FastConformer, Wav2Vec 2.0 XLSR and ESPnet Model Zoo) on Mozilla Common Voice and VoxPopuli databases. Tests were conducted for clean audio signal, signal with bandwidth limitation, and degraded. The tested models were evaluated on the basis of Word Error Rate (WER) and Character Error Rate (CER). The results show that the Whisper model performs by far the best among the open-source models. ElevenLabs Scribe model, on the other hand, performs best for Polish on both general benchmark and medical data.

This paper investigates the issues that are associated with applying speech production models to automatic speech recognition (ASR). Here the applicability of articulatory representations to ASR is considered independently of the role of articulatory representations in speech perception. While the question of whether it is necessary or even possible for human listeners to recover the state of the articulators during the process of perceiving speech is an important one, it is not considered here. Hence, the authors refrain from posing completely new paradigms for ASR which more closely parallel the relationship between speech production and human speech understanding. Instead, work aimed at integrating speech production models into existing ASR formalisms is described.

This article describes a density ratio approach to integrating external Language Models (LMs) into end-to-end models for Automatic Speech Recognition (ASR). Applied to a Recurrent Neural Network Transducer (RNN-T) ASR model trained on a given domain, a matched in-domain RNN-LM, and a target domain RNN-LM, the proposed method uses Bayes' Rule to define RNN-T posteriors for the target domain, in a manner directly analogous to the classic hybrid model for ASR based on Deep Neural Networks (DNNs) or LSTMs in the Hidden Markov Model (HMM) framework (Bourlard & Morgan, 1994). The proposed approach is evaluated in cross-domain and limited-data scenarios, for which a significant amount of target domain text data is used for LM training, but only limited (or no) {audio, transcript} training data pairs are used to train the RNN-T. Specifically, an RNN-T model trained on paired audio & transcript data from YouTube is evaluated for its ability to generalize to Voice Search data. The Density Ratio method was found to consistently outperform the dominant approach to LM and end-to-end ASR integration, Shallow Fusion.

Subspace Gaussian mixture based language modeling for large vocabulary continuous speech recognition

With the surge of online meetings, it has become more critical than ever to provide high-quality speech audio and live captioning under various noise conditions.However, most monaural speech enhancement (SE) models introduce processing artifacts and thus degrade the performance of downstream tasks, including automatic speech recognition (ASR).This paper proposes a multi-task training framework to make the SE models unharmful to ASR.Because most ASR training samples do not have corresponding clean signal references, we alternately perform two model update steps called SE-step and ASR-step.The SEstep uses clean and noisy signal pairs and a signal-based loss function.The ASR-step applies a pre-trained ASR model to training signals enhanced with the SE model.A cross-entropy loss between the ASR output and reference transcriptions is calculated to update the SE model parameters.Experimental results with realistic large-scale settings using ASR models trained on 75,000-hour data show that the proposed framework improves the word error rate for the SE output by 11.82% with little compromise in the SE quality.Performance analysis is also carried out by changing the ASR model, the data used for the ASR-step, and the schedule of the two update steps.

We have proposed an automatic speech summarization approach that extracts words from transcription results obtained by automatic speech recognition (ASR) systems. To numerically evaluate this approach, the automatic summarization results are compared with manual summarization generated by humans through word extraction. We have proposed three metrics, weighted word precision, word strings precision and summarization accuracy (SumACCY), based on a word network created by merging manual summarization results. In this paper, we propose a new metric for automatic summarization results, weighted summarization accuracy (WSumACCY). This accuracy is weighted by the posterior probability of the manual summaries in the network to give the reliability of each answer extracted from the network. We clarify the goal of each metric and use these metrics to provide automatic evaluation results of the summarized speech. To compare the performance of each evaluation metric, correlations between the evaluation results using these metrics and subjective evaluation by hand are measured. It is confirmed that WSumACCY is an effective and robust measure for automatic summarization.

This research concentrates on the progression and present status of automatic speech recognition systems powered by deep neural networks. It discusses model architectures, training approaches, evaluating model efficacy, and recent advancements specific to deep neural networks applied in automatic speech recognition models. It considers the challenges faced in crafting these speech recognition models, such as data scarcity and the necessity for adaptability. Our exploration traces the evolution of automatic speech recognition through deep neural networks, presenting valuable insights aimed at propelling the domain of speech recognition for diverse applications, spanning from smart devices to healthcare. KEYWORDS - Automatic Speech Recognition, Deep Neural Networks, Language Modeling, Robustness to Noise, Speech Modelling

In this letter, a spoken dialogue system is demonstrated to disseminate the agricultural commodity price information in Kannada language/dialects. The spoken dialogue system mainly includes interactive voice response system (IVRS), automatic speech recognition (ASR) models and price information database. The IVRS call flow is used for speech data collection from the farmers across various dialect regions of Karnataka. The collected speech data is transcribed and validated using Indic transliteration tool (IT3: UTF-8). The dictionary and corresponding phoneme set is created for Kannada language. The ASR acoustic models are created using the speech recognition toolkit called Kaldi. The 85% and 15% of transcribed and validated speech corpus is used for Kaldi system training and decoding respectively. The agricultural marketing network (AGMARKNET) website is used as the price information database for different agricultural commodities. The postgresql database is used for managing the crawled data. The word error rates (WERs) of ASR models for districts, mandis, commodities and mixed data are discussed and the spoken dialogue system is developed and tested from the real farmers to access the real time agricultural commodity price information in language/dialects of Kannada.

In this paper, we explore the benefits of incorporating context into a Recurrent Neural Network (RNN-T) based Automatic Speech Recognition (ASR) model to improve the speech recognition for virtual assistants. Specifically, we use meta information extracted from the time at which the utterance is spoken and the approximate location information to make ASR context aware. We show that these contextual information, when used individually, improves overall performance by as much as 3.48% relative to the baseline and when the contexts are combined, the model learns complementary features and the recognition improves by 4.62%. On specific domains, these contextual signals show improvements as high as 11.5%, without any significant degradation on others. We ran experiments with models trained on data of sizes 30K hours and 10K hours. We show that the scale of improvement with the 10K hours dataset is much higher than the one obtained with 30K hours dataset. Our results indicate that with limited data to train the ASR model, contextual signals can improve the performance significantly.