An Interactive System leveraging Automatic Speech Recognition and Machine Translation for learning Hindi as a Second Language

This paper attempts to show how healthcare professionals navigate in relation to Automatic Speech Recognition (ASR) by examining workarounds. The use of ASR in entering information into electronic health records (EHRs) is an important topic to study as hospitals increasingly adapt to the technology. This paper is based on ethnographic multi-sited fieldwork. It gathers information about the use of ASR in workflows, its limitations, how physicians perceive it, and what workarounds they employ when it does not work as expected. The study analyses a Danish hospital that underwent an ASR implementation almost two decades ago, yet they still experience frequent system modifications. Once the physicians experience these changes, they promptly criticize the technology for failing to live up to proclaimed expectations. The paper demonstrates how a simplistic understanding and a following of a predetermined plan of complex technology, such as ASR, contributes to workflow-related challenges and workarounds. As a result, the research proclaims a concern towards information recording, saving, and retrieval related to differences in how information is entered into an EHR by using ASR. Further, studies of workarounds proclaim these to be a way for physicians to achieve the desired goal of patient safety, where this study finds some workarounds as contradictory actions based on revolt practices towards managerial decisions.KeywordsElectronic Health RecordsAutomatic Speech RecognitionHuman-Computer InteractionWorkarounds

Purpose :This investigation explored the speech-production behaviors associated with the use of automatic speech recognition (ASR) software for dictation of spontaneous and scripted material by individuals with and without spinal cord injury (SCI). The variables of interest in determining speaking workload and efficiency included syllables per breath group, frequency of breath groups, frequency of apnea, time needed for dictation and number of words spoken during a dictation task. Method : Twelve individuals participated, six with SCI and six able-bodied cohorts matched for age, sex and height. Subjects dictated with continuous-speech ASR, discrete-word ASR and no ASR in a spontaneous-speaking situation, as well as in a scripted speaking situation. Results : For all variables, differences amongst dictation conditions were significant. No significant differences were found between speaker groups. Dictation with both discrete-word and continuous-speech ASR resulted in a decrease in the number of syllables produced per breath group, increases in the frequency of breath groups and apnea, with differences from normal being greater for dictation with discrete-word ASR. In addition, when participants dictated with either type of ASR, the amount of time and number of words produced were significantly greater than that associated with production of the same message without ASR, requiring `more work' on the part of the speaker and ultimately reducing the efficiency with which a message was produced. Conclusions : From a human factors perspective, these results suggest that ASR software, especially discrete-word ASR, has the potential to increase energy expenditure during dictation over a prolonged period of time, thereby increasing speech workloads and the potential for overuse of the laryngeal system.

In this article I will discuss the possible use of automatic speech recognition (ASR) for training students to improve their accents in a foreign language.Principles of good language training as well as the limits of the use of Automatic Speech Recognition (ASR) and how to deal with them will be discussed first.I will then use the example of the Carnegie Mellon FLUENCY system to show concretely how such a system may function.Prosody training as well as phonetics will be emphasized.Finally, I will underscore the importance of having a system that adapts to the user, again using the FLUENCY system as an example.

The use of digital technologies in the daily routine of second language (L2) classes became unavoidable when teachers needed to overcome mobility restrictions imposed by the COVID-19 pandemic. In countries where access to technological resources in regular classrooms is still limited, the use of free digital resources for pedagogical purposes is a valuable alternative. This pedagogical paper explores the use of Automatic Speech Recognition (ASR) for L2 pronunciation teaching and learning. Three ASR-based activities drawn from Gottardi (2023) are presented and discussed. The assessment is based on the criteria provided by two frameworks. From the pedagogical perspective, we adopted the communicative pronunciation teaching framework (Celce-Murcia; Brinton; Goodwin, 2010), which proposes five steps to teach pronunciation and design materials, including explicit instruction, perception, and production tasks, with different degrees of teacher guidance and feedback. Considering the use of technology and the design of pedagogical activities, we draw on Chapelle’s (2001) six criteria to evaluate educational software and activities. We analyzed each ASRbased pronunciation activity in terms of language learning potential, learner fit, meaning focus, authenticity, positive impact, and practicality. Thus, the objective of this pedagogical paper is to 1) discuss how ASR technology can assist L2 pronunciation practice; 2) provide practical examples of ASR-based pronunciation activities designed to be used by L2 teachers; and 3) examine the pedagogical appropriateness of the proposed activities. Finally, pedagogical implications of using ASR for pronunciation practice are discussed.

The passage discusses a proposed method for proofreading English translations of long sentences, particularly in the context of machine translation (MT). The traditional manual proofreading process is deemed inefficient, and existing intelligent proofreading systems are considered inadequate when dealing with a large volume of English long sentence translations. The proposed method involves a part-of-speech tagging approach based on an improved Markov model and an antagonistic learning framework. It utilizes a distribution model of text feature concept sets to identify and correct English translation errors. The results indicate that this method outperforms the traditional ID3 algorithm, with a reduction in Mean Absolute Error (MAE) by about 15% and an increase in recall rate by approximately 10%. These improvements demonstrate the effectiveness of the proposed intelligent proofreading algorithm for enhancing the quality of translations, particularly for complex English long sentences. Overall, the system described in the paper is shown to have a high level of accuracy in detecting grammatical errors in English sentences, automating the detection of translation errors, and delivering superior performance.

Resisting Linguistic Imperialism in English Teaching: A. Suresh Canagarajah; Oxford University Press, Oxford, 1999, 224 pp., £18.10 pb, ISBN 0-19-442154-6

With the further improvement of basic equipment, more and more data can be discovered and used. With the continuous improvement of computer computing power and the perfection of machine learning and other theories, speech recognition technology gradually tends to be practical. Research on the application of data mining in speech recognition and constructing an English translator speech recognition system has practical significance in our country. Therefore, this article mainly studies the application of data mining in the design of English translator speech recognition system. This paper analyzes the speech data of data mining technology, speech recognition technology and machine translation technology, and proposes a speech error correction algorithm based on the Jaro-winkler algorithm, which improves the accuracy of speech recognition. This paper designs a platform development plan for an English translator speech recognition system, and tests the translation performance of the system. The results show that the overall recognition rate of the system is 89.45%, and the translation accuracy rate is 91.64%. This shows that the accuracy of the system is high, and the input sentence can be translated more accurately, which meets the requirements of the English speech translation system in specific situations.

Clinician progress notes are an important record for care and communication, but there is a perception that electronic notes take too long to write and may not accurately reflect the patient encounter, threatening quality of care. Automatic speech recognition (ASR) has the potential to improve clinical documentation process; however, ASR inaccuracy and editing time are barriers to wider use. We hypothesized that automatic text processing technologies could decrease editing time and improve note quality. To inform the development of these technologies, we studied how physicians create clinical notes using ASR and analyzed note content that is revised or added during asynchronous editing. We analyzed a corpus of 649 dictated clinical notes from 9 physicians. Notes were dictated during rounds to portable devices, automatically transcribed, and edited later at the physician's convenience. Comparing ASR transcripts and the final edited notes, we identified the word sequences edited by physicians and categorized the edits by length and content. We found that 40% of the words in the final notes were added by physicians while editing: 6% corresponded to short edits associated with error correction and format changes, and 34% were associated with longer edits. Short error correction edits that affect note accuracy are estimated to be less than 3% of the words in the dictated notes. Longer edits primarily involved insertion of material associated with clinical data or assessment and plans. The longer edits improve note completeness; some could be handled with verbalized commands in dictation. Process interventions to reduce ASR documentation burden, whether related to technology or the dictation/editing workflow, should apply a portfolio of solutions to address all categories of required edits. Improved processes could reduce an important barrier to broader use of ASR by clinicians and improve note quality.

This study addresses the issue of automatic pronunciation assessment (APA) and its contribution to the teaching of second language (L2) pronunciation. Several attempts have been made at designing such systems, and some have proven operationally successful. However, the automatic assessment of the pronunciation of short words in segmental approaches has still remained a significant challenge. Free and off-the-shelf automatic speech recognition (ASR) systems have been used in integration with other tools with the hopes of facilitating improvement in the domain of computer-assisted pronunciation training (CAPT). The use of ASR in APA stands on the premise that a word that is recognized is intelligible and well-pronounced. Our goal was to explore and test the functionality of Google ASR as the core component within a possible automatic British English pronunciation assessment system. After testing the system against standard and non-standard (foreign) pronunciations provided by participating pronunciation experts as well as non-expert native and non-native speakers of English, we found that Google ASR does not and cannot simultaneously meet two necessary conditions (here defined as intrinsic and derived) for performing as an APA system. Our study concludes with a synthetic view on the requirements of a reliable APA system.

Altered nasality influences speech intelligibility. Automatic speech recognition (ASR) has proved suitable for quantifying speech intelligibility in patients with different degrees of nasal emissions. We investigated the influence of hyponasality on the results of speech recognition before and after nasal surgery using ASR. Speech recordings, nasal peak inspiratory flow and self-perception measurements were carried out in 20 German-speaking patients (8 women, 12 men; aged 38 ± 22 years) who underwent surgery for various nasal and sinus pathologies. The degree of speech intelligibility was quantified as the percentage of correctly recognized words of a standardized word chain by ASR (word recognition rate; WR). WR was measured 1 day before (t1), 1 day after with nasal packings (t2), and 3 months after (t3) surgery; nasal peak flow on t1 and t3. WR was calculated with program for the automatic evaluation of all kinds of speech disorders (PEAKS). WR as a parameter of speech intelligibility was significantly decreased immediately after surgery (t1 vs. t2 p < 0.01) but increased 3 months after surgery (t2 vs. t3 p < 0.01). WR showed no association with age or gender. There was no significant difference between WR at t1 and t3, despite a post-operative increase in nasal peak inspiratory flow measurements. The results show that ASR is capable of quantifying the influence of hyponasality on speech; nasal obstruction leads to significantly reduced WR and nasal peak flow cannot replace evaluation of nasality.

A common metric for evaluating Automatic Speech Recognition (ASR) is Word Error Rate (WER) which solely takes into account discrepancies at the word-level. Although WER is useful, it is not guaranteed to correlate well with intelligibility or performance on downstream tasks that make use of ASR. Meaningful assess- ment of ASR mistakes becomes even more important in high-stake scenarios such as health-care. I propose 2 general measures to evaluate the quality or severity of mistakes made by ASR systems, one based on sentiment analysis and another based on text embeddings. Both have the potential to overcome the limitations of WER. I evaluate these measures on simulated patient-doctor conversations. Measures of severity based on sentiment ratings and text embeddings correlate with human ratings of severity. Measures based on text embeddings have the capability to predict human severity ratings better than WER. These measures are used in metrics in the overall evaluation of 5 ASR engines alongside WER. Results show that these metrics capture characteristics of ASR errors that WER does not. Furthermore, I train an ASR system using severity as a penalty in the loss function and demonstrate the potential for using severity not only in the evaluation, but in the development of ASR. Advantages and limitations of this methodology are analyzed and discussed.

Speech recognition can be defined as the process of converting voice signals into the ranks of the word, by applying a specific algorithm that is implemented in a computer program. The research of speech recognition in Indonesia is relatively limited. This paper has studied methods of feature extraction which is the best among the Linear Predictive Coding (LPC) and Mel Frequency Cepstral Coefficients (MFCC) for speech recognition in Indonesian language. This is important because the method can produce a high accuracy for a particular language does not necessarily produce the same accuracy for other languages, considering every language has different characteristics. Thus this research hopefully can help further accelerate the use of automatic speech recognition for Indonesian language. There are two main processes in speech recognition, feature extraction and recognition. The method used for comparison feature extraction in this study is the LPC and MFCC, while the method of recognition using Hidden Markov Model (HMM). The test results showed that the LPC method is better than MFCC in Indonesian language speech recognition.

The paper investigated whether Korean middle school were able to resolve ambiguous sentences in spoken English by relying on prosodic cues. Fifty eight junior high school in Seoul and 6 peer native English listeners (Korean-Americans) in the US participated in a comprehension test, which consisted of 20 questions of main-subordinate and parenthetical syntactic structures. The results indicated that, unlike the native English listeners, a vast majority of the Korean had much difficulty resolving ambiguous sentences although the sentences were disambiguated by prosodic cues such as phrase final lengthening, phrasing, and pauses. The Korean students listening comprehension was not correlated with their general English ability. However, an analysis of major textbooks being used in middle schools in Korea suggests that the Korean students poor performance in the comprehension test may be ascribable partly to their lack of exposure to the relationship between syntactic structure and prosodic structure. Based on the findings of the study, implications for the teaching of intonational cues for the resolution of ambiguous sentences in English were drawn.

SLIM prosodic automatic tools for self-learning instruction

Artificial intelligence is getting shrewd in real time with massive amounts of computational power and high in demand in last quarter more than billion dollars. The Intensification of Neural Networks in Machine learning playing major revolution in Machine Translation (MT). Looking same aspect, we have prepared a paper on Machine Translation for Hindi using Neural Machine Translation techniques. The Basis of this paper is to translate the source language into target language with the help of sentence structure for source language (English sentence) and corresponding reordering rules for target language (Hindi sentence) using a deep neural network (DNN).When the source input that is English sentences are exactly matched with the target language database then the suitable translation for related input is directly fetched from the database. The fuzzy matching between the input sentence and the sample sentence in the database is done with cosine similarity between the sentence structure of input sentence and the sample sentence and then the reordering rule of the most alike sentence to the input sentence is used for translation.Extracting the structure of the source language sentence and extracting the reordering rule for target sentence is the core part of the translator. Reordering of words in spite of that remains one of the complex problems.It is exciting to implement Deep Neural Network and fuzzy logic to build intelligent reordering rules machine learning system.