Evolution of the performance of automatic speech recognition algorithms in transcribing conversational telephone speech

We report insights from translating Spanish conversational telephone speech into English text by cascading an automatic speech recognition (ASR) system with a statistical machine translation (SMT) system. The key new insight is that the informal register of conversational speech is a greater challenge for ASR than for SMT: the BLEU score for translating the reference transcript is 64%, but drops to 32% for translating automatic transcripts, whose word error rate (WER) is 40%. Several strategies are examined to mitigate the impact of ASR errors on the SMT output: (i) providing the ASR lattice, instead of the 1-best output, as input to the SMT system, (ii) training the SMT system on Spanish ASR output paired with English text, instead of Spanish reference transcripts, and (iii) improving the core ASR system. Each leads to consistent and complementary improvements in the SMT output. Compared to translating the 1-best output of an ASR system with 40% WER using an SMT system trained on Spanish reference transcripts, translating the output lattice of a better ASR system with 35% WER using an SMT system trained on ASR output improves BLEU from 32% to 38%.

This paper presents an approach to tone recognition in mandarin conversational telephone speech (CTS) based on a real context model. The real context model is proposed as a new concept designed with special consideration on the fact that mandarin CTS is characterized by complicated tone behaviors due to physiological articulation. As pitch is a supra-segmental feature, current tone's pitch value is influenced by its context especially in CTS for its fast speaking rate. A real context model covers not only the current tone but also the relative pitch level of pre-tone. Then we cluster the real context annotated training data into a few subsets to generate a more refined tone model. Gaussian Mixture Model (GMM) is used for the tone modeling. In addition, a kind of similarity measurements to compute the distance between two tones is employed, which should reveal the similarity of their pitch contour shapes and also include their different pitch height. All experiments are based on the mandarin CTS database, Train04. Our methods can improve tone recognition accuracy 4.7%.

This paper presents two different approaches utilizing sta-tisticallanguagemodel(SLM)andsupportvectormachines(SVM) for sentence boundary detection of spontaneousJapanese. In the SLM-based approach, linguistic likeli-hoods and occurrence of pause are used to determine sen-tence boundaries. To suppress false alarms, heuristic pat-terns of end-of-sentence expressions are also incorporated.On the other hand, SVM is adopted to realize robust classi-fication against a wide variety of expressions and speechrecognition errors. Detection is performed by an SVM-based text chunker using lexical and pause information asfeatures. We evaluated these approaches on manual and au-tomatic transcription of spontaneous lectures and speeches,and achieved F-measures of 0.85 and 0.78, respectively. Index Terms : sentence boundary detection, spontaneousspeech, statistical language model, support vector ma-chines. 1. Introduction Recent advance of automatic speech recognition (ASR)technology, especially for spontaneous speech, enables var-ious applications such as spoken document archiving andretrieval, speech summarization and speech translation. Toorganizeaspokendocumentinastructuredformandtogiveuseful indices, transcriptions should be segmented into ap-propriate units like sentences. Moreover, these applicationsare usually built by combining an ASR system with naturallanguage processing (NLP) systems such as a parser and amachine translator, which often assume that input text is asentence. However,sentencesinspontaneousspeechareill-formed, and sentence boundaries are indistinct. Output textby ASR systems is just a sequence of words and has no ex-plicitsentenceboundaries, sothefurtherstepofsegmentingthe ASR output is required for these applications.Automatic boundary detection of spoken sentences hasbeen explored mainly on broadcast news (BN)tasks[1,2,3]and conversational telephone speech (CTS) tasks[3, 4, 5] inEnglish. As features for detection, pause, prosodic and lin-guistic information is often used. Most popular approach isa combination of prosodic and linguistic information[2, 3],which realizes high performance on BN and CTS tasks.Prosody-based approaches[1, 5] have also been investi-gated. Meanwhile, linguistic information is not used by it-self, since most of these works were performed on Englishdata, where cue words or expressions of sentence bound-aries are not easily defined.In Japanese, cue expressions are typically observed atthe end of sentences and expected to be useful for bound-ary detection. However, variety of such expressions is solarge in spoken Japanese, that it is hard to collect sufficientamountofdatafortrainingstatisticalmodelssuchasamax-imum entropy (ME) model. Moreover, many of cue expres-sions consist of particles, which are apparently difficult tobedetectedinASR.Thus,robustnessforASRerrorsshouldalso be investigated.In this paper, we address two approaches of sen-tence boundary detection for spontaneous Japanese. Asframeworks of detection, we adopt and compare statisti-cal language model (SLM) and support vector machines(SVM). The proposed approaches are evaluated with reallectures and speeches included in the Corpus of Sponta-neous Japanese (CSJ).

Front-end feature extraction techniques have long been a critical component in Automatic Speech Recognition (ASR). Nonlinear filtering techniques are becoming increasingly important in this application, and are often better than linear filters at removing noise without distorting speech features. However, design and analysis of nonlinear filters are more difficult than for linear filters. Mathematical morphology, which creates filters based on shape and size characteristics, is a design structure for nonlinear filters. These filters are limited to minimum and maximum operations that introduce a deterministic bias into filtered signals. This work develops filtering structures based on a mathematical morphology that utilizes the bias while emphasizing spectral peaks. The combination of peak emphasis via LP analysis with morphological filtering results in more noise robust speech recognition rates. To help understand the behavior of these pre-processing techniques the deterministic and statistical properties of the morphological filters are compared to the properties of feature extraction techniques that do not employ such algorithms. The robust behavior of these algorithms for automatic speech recognition in the presence of rapidly fluctuating speech signals with additive and convolutional noise is illustrated. Examples of these nonlinear feature extraction techniques are given using the Aurora 2.0 and Aurora 3.0 databases. Features are computed using LP analysis alone to emphasize peaks, morphological filtering alone, or a combination of the two approaches. Although absolute best results are normally obtained using a combination of the two methods, morphological filtering alone is nearly as effective and much more computationally efficient.

RETRACTED: Real time speech recognition algorithm on embedded system based on continuous Markov model

Recognition of Hungarian conversational telephone speech is challenging due to the informal style and morphological richness of the language. Recurrent Neural Network Language Model (RNNLM) can provide remedy for the high perplexity of the task; however, two-pass decoding introduces a considerable processing delay. In order to eliminate this delay we investigate approaches aiming at the complexity reduction of RNNLM, while preserving its accuracy. We compare the performance of conventional back-off n-gram language models (BNLM), BNLM approximation of RNNLMs (RNN-BNLM) and RNN n-grams in terms of perplexity and word error rate (WER). Morphological richness is often addressed by using statistically derived subwords - morphs - in the language models, hence our investigations are extended to morph-based models, as well. We found that using RNN-BNLMs 40% of the RNNLM perplexity reduction can be recovered, which is roughly equal to the performance of a RNN 4-gram model. Combining morph-based modeling and approximation of RNNLM, we were able to achieve 8% relative WER reduction and preserve real-time operation of our conversational telephone speech recognition system.

This paper explores the issues involved in using symbolic metric algorithms for automatic speech recognition (ASR), via a structural representation of speech. This representation is based on a set of phonological distinctive features which is a linguistically well-motivated alternative to the "beads-on-a-string" view of speech that is standard in current ASR systems. We report the promising results of phoneme classification experiments conducted on a standard continuous speech task.

This paper describes the development of a speech recognition system for the processing of telephone conversations, starting with a state-of-the-art broadcast news transcription system. We identify major changes and improvements in acoustic and language modeling, as well as decoding, which are required to achieve state-of-the-art performance on conversational speech. Some major changes on the acoustic side include the use of speaker normalization (VTLN), the need to cope with channel variability, and the need for efficient speaker adaptation and better pronunciation modeling. On the linguistic side the primary challenge is to cope with the limited amount of language model training data. To address this issue we make use of a data selection technique, and a smoothing technique based on a neural network language model. At the decoding level lattice rescoring and minimum word error decoding are applied. On the development data, the improvements yield an overall word error rate of 24.9% whereas the original BN transcription system had a word error rate of about 50% on the same data.

We propose a method for zero-resource domain adaptation of DNN acoustic models, for use in low-resource situations where the only in-language training data available may be poorly matched to the intended target domain. Our method uses a multi-lingual model in which several DNN layers are shared between languages. This architecture enables domain adaptation transforms learned for one well-resourced language to be applied to an entirely different low- resource language. First, to develop the technique we use English as a well-resourced language and take Spanish to mimic a low-resource language. Experiments in domain adaptation between the conversational telephone speech (CTS) domain and broadcast news (BN) domain demonstrate a 29% relative WER improvement on Spanish BN test data by using only English adaptation data. Second, we demonstrate the effectiveness of the method for low-resource languages with a poor match to the well-resourced language. Even in this scenario, the proposed method achieves relative WER improvements of 18-27% by using solely English data for domain adaptation. Compared to other related approaches based on multi-task and multi-condition training, the proposed method is able to better exploit well-resource language data for improved acoustic modelling of the low-resource target domain.

Speech recognition is one of the important areas in digital speech processing. The study of speech recognition is a part of a quest for artificially intelligent machines that can hear and understand spoken information. The conventional methods for speech recognition are very complicated and time consuming. To apply fuzzy logic to speech recognition is a new attempt in digital speech processing. The approach proposed in the paper simplifies the algorithm in speech recognition and makes the real-time processing time shorter. The situation considered in this paper is the simplest, i.e., the situation of speaker dependence, small vocabulary and isolated words.

The accurate recognition of speech is beneficial to the fields of machine translation and intelligent human–computer interaction. After briefly introducing speech recognition algorithms, this study proposed to recognize speech with a recurrent neural network (RNN) and adopted the connectionist temporal classification (CTC) algorithm to align input speech sequences and output text sequences forcibly. Simulation experiments compared the RNN-CTC algorithm with the Gaussian mixture model–hidden Markov model and convolutional neural network-CTC algorithms. The results demonstrated that the more training samples the speech recognition algorithm had, the higher the recognition accuracy of the trained algorithm was, but the training time consumption increased gradually; the more samples a trained speech recognition algorithm had to test, the lower the recognition accuracy and the longer the testing time. The proposed RNN-CTC speech recognition algorithm always had the highest accuracy and the lowest training and testing time among the three algorithms when the number of training and testing samples was the same.

The paper describes the use of several advanced acoustic modeling techniques for the 2004 CU-HTK large vocabulary speech recognition systems. These techniques include Gaussianization for speaker normalization, discriminative cluster adaptive training (CAT), subspace for precision and mean (SPAM) modeling of inverse covariances, and discriminative complexity control. Acoustic models featuring these techniques were integrated into a state-of-the-art 10 real-time multi-pass system with sophisticated adaptation for performance evaluation. Experimental results are presented on both broadcast news (BN) and conversational telephone speech (CTS) transcription tasks.

Speech recognition can be defined as the process of converting voice signals into the ranks of the word, by applying a specific algorithm that is implemented in a computer program. The research of speech recognition in Indonesia is relatively limited. This paper has studied methods of feature extraction which is the best among the Linear Predictive Coding (LPC) and Mel Frequency Cepstral Coefficients (MFCC) for speech recognition in Indonesian language. This is important because the method can produce a high accuracy for a particular language does not necessarily produce the same accuracy for other languages, considering every language has different characteristics. Thus this research hopefully can help further accelerate the use of automatic speech recognition for Indonesian language. There are two main processes in speech recognition, feature extraction and recognition. The method used for comparison feature extraction in this study is the LPC and MFCC, while the method of recognition using Hidden Markov Model (HMM). The test results showed that the LPC method is better than MFCC in Indonesian language speech recognition.

Proficiency in oral English plays a significant role in different professional and academic settings. However, evaluating oral English proficiency precisely is challenging because of the subjective nature of oral communication. In this study, we developed an innovative Oral English Evaluation System (OEES) by combining the efficiencies of speech recognition technology and scoring algorithms. Initially, an intensive oral English database was collected, containing numerous persons' voice recordings. The collected voice recordings are pre-processed to remove the background noise, ensuring clarity in the voice signals. Then, an automatic speech recognition algorithm was developed using the recurrent neural network (RNN) to transcribe voice signals into text. This module trains the system to map the voice signals to their corresponding textual characteristics. Finally, an adaptive scoring algorithm was incorporated into the designed OEES to evaluate the oral English proficiency of the individuals. The adaptive scoring algorithm considers different factors like fluency, grammar, pronunciation, and vocabulary for evaluating the individual's oral English proficiency as "excellent," "good," and "poor." The presented framework was modeled in Python and validated across diverse natural English-speaking databases. The experimental results are assessed in terms of accuracy, precision, recall, and error rate. The implementation outcomes suggest that the proposed OEES framework accurately evaluates oral English proficiency.

Keyword spotting becomes a very important branch of speech recognition. But the acoustic mismatch between training and testing environments often causes a severe degradation in the recognition performance. This paper presents an improved keyword spotting strategy. A fuzzy search algorithm is proposed to extract keyword hypotheses from a syllable confusion network (SCN). SCN is linear and naturally suitable for indexing. To accelerate search process, SCN are pruned to feasible sizes. As a post-processing method, minimum classification error (MCE) optimized confidence measure is adopted to reject false accepts. On Mandarin conversational telephone speech (CTS), the proposed algorithms reduce the equal error rate (EER) by 7.2% relative.