Modeling word‐boundary phonological processes in continuous speech recognition

The authors present an NN-IIR (neural network/infinite impulse response filter) system for phoneme recognition in continuous speech based on the idea of modeling the recognition process by state evolution and interpretation equations. This work gives a solution to temporal information representation in phoneme recognition using neural networks and recursive filters, yielding better recognition results for continuous speech. This recognition system has two promising properties, i.e., capabilities for dealing with sequential properties and for interpreting speech signals by means of a training process. It was shown experimentally that the NN-IIR network obtained good performance for continuous speech recognition. Preliminary experiments with limited training data indicate that the NN-IIR provides good discrimination power for plosives, which are highly context-dependent.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

Detection of word boundaries in continuous speech is a tedious process due to the absence of a definite pause or silence in the word boundary position. Thus, continuous speech recognition is a very challenging task. However, the prosodic word boundaries, unlike the written word boundaries, can be predicted using the prosodic parameters of continuous speech. This paper proposes a method for detecting such prosodic word boundaries from Bengali continuous speech. Bengali is a bound-stress language, where stress is observed on the first syllable of a prosodic word. Empirical Mode Decomposition is applied to the logarithm of fundamental frequency (F0) contour of continuous speech to detect prosodic word boundaries. 200 Bengali readout sentences, read by ten speakers, are analyzed for the present work. An overall prosodic boundary detection accuracy of 88.05% is achieved, whereas precision and recall values are 90.73% and 88.31%, respectively, with f-score as 89.5. A prosodic word dictionary comprising 5031 prosodic words has been developed by analyzing 1526 Bengali sentences with the proposed prosodic word boundary detection method.

Since 1972, the Speech Recognition Group at IBM's Thomas J. Watson Research Center has worked on a system for the automatic recognition of natural continuous speech. The goal of our research is to produce a voice-activated text-processing system. We have recently worked on a 5,000-word isolated-speech system for recognition of business correspondence and interoffice memoranda.Although speech recognition is performed with beguiling ease by human beings, the task of transferring this ability to computers has proven quite difficult. Our approach to the problem has been to develop general statistical techniques that allow the recognition system to organize itself when presented with a large sample of speech called training data. In essence, our system refines itself from some crudely specified initial configuration so as to take advantage of regularities present in a speech sample.Suppose that a talker wishes to convey to a listener the string of words W1, W2, ..., Wn. Each of the words has a pronunciation, in accordance with which the talker will guide his/her speech apparatus through an intricate cascade of gestures, with the result that a series of sound pressure waves will emanate. A microphone placed within the influence of these waves will respond to them in a fashion that is characteristic of the word sequence W1, W2, ..., Wn.Let the response of the microphone be denoted by the sequence A1, ..., Am. We can regard A1, ..., Am as a garbled version of the intended message W1, ..., Wn. The problem of recovering a message from a garbled transmission is the province of communication theory. The problem may be stated as follows: for a garbled transmission A (=A1, ...Am), determine the message W (=W1, ..., Wn) that is most probable given A. That is, find W such that Pr(W|A) is as large as possible.From Bayes's rule, we have Pr(W|A) = {Pr(W|A)}Pr(W)}Pr(A). Since we are interested in maximizing this for a particular fixed A, we can ignore the denominator on the right. The first term in the numerator constitutes a probabilistic characterization of the transmission channel. It tells us what A's to expect when the talker intends a particular word string W. The second term in the numerator is a probabilistic characterization of the word strings that the talker desires to convey. Pr(W) embodies all rules of grammar (ungrammatical word strings are less probable than grammatical ones) as well as all semantic constraints (meaningless word strings are less probable than meaningful ones).It is, of course, impossible to compute these probabilities exactly. We have, however, developed methods for obtaining useful approximations to them. We refer to the approximation of Pr(A|W) as a statistical model for the speech production process and to the approximation of Pr(W) as a statistical model for the language. A discussion of these models constitutes the body of the talk.

This paper aims to design and implement English digits speech recognition system using Matlab (GUI). This work was based on the Hidden Markov Model (HMM), which provides a highly reliable way for recognizing speech. The system is able to recognize the speech waveform by translating the speech waveform into a set of feature vectors using Mel Frequency Cepstral Coefficients (MFCC) technique This paper focuses on all English digits from (Zero through Nine), which is based on isolated words structure. Two modules were developed, namely the isolated words speech recognition and the continuous speech recognition. Both modules were tested in both clean and noisy environments and showed a successful recognition rates. In clean environment and isolated words speech recognition module, the multi-speaker mode achieved 99.5% whereas the speaker-independent mode achieved 79.5%. In clean environment and continuous speech recognition module, the multi-speaker mode achieved 72.5% whereas the speaker-independent mode achieved 56.25%. However in noisy environment and isolated words speech recognition module, the multi-speaker mode achieved 88% whereas the speaker-independent mode achieved 67%. In noisy environment and continuous speech recognition module, the multi-speaker mode achieved 82.5% whereas the speaker-independent mode achieved 76.67%. These recognition rates are relatively successful if compared to similar systems.

Studies the problem of phonetic modeling for continuous Mandarin speech recognition by providing a systematic performance comparison for systems based on following primitive speech units: syllables, demi-syllables (initial and final), context-independent phones, left-or-right context-dependent phones (diphones) and left-and-right context-dependent phones (triphones). In our speaker-dependent continuous speech recognition experiments, a generalized triphone system has achieved the best performance of all. Our best system is in contrast to most other Mandarin speech recognition systems, which have been based on demi-syllable units.

Use of fuzzy mathematical concepts in character spotting for automatic recognition of continuous speech in Hindi

Continuous speech is far more natural and efficient than isolated speech for communication. However, for current state-of-the-art automatic speech recognition systems, isolated speech recognition (ISR) is far more accurate than continuous speech recognition (CSR). It is common practice in the speech research community to build CSR systems using only CS data. However, slowing of the speaking rate is a natural reaction for a user faced with the high error rates of current CSR systems. Ironically, CSR systems typically have a much higher word error rate when speakers slow down since the acoustic models are usually derived exclusively from continuous speech corpora. In this paper, we summarize our efforts to improve the robustness of our speaker-independent CSR system against speaking styles, without suffering a recognition accuracy penalty. In particular the multi-style trained system described in this paper attains a 7.0% word error rate for a test set consisting of both isolated and continuous speech, in contrast to the 10.9% word error rate achieved by the same system trained only on continuous speech.

Automatic disorders recognition in speech can be very helpful for a therapist while monitoring therapy progress of patients with disordered speech. This article is focused on sound repetitions. The signal is analyzed using Continuous Wavelet Transform with 16 bark scales. Using the silence finding algorithm, only speech fragments are automatically found and cut. Each cut fragment is converted into a fixed-length vector and passed into the Kohonen network. Finally, the Kohonen winning neuron result is put on the 3-layer perceptron. Most of the analysis was performer and the results were obtained using the authors’ program WaveBlaster. We use the STATISTICA package for finding the best perceptron which was then imported back into WaveBlaster and used for automatic blockades finding. The problem presented in this article is a part of our research work aimed at creating an automatic disordered speech recognition system.

For extra-large vocabulary continuous speech recognition a language model that describes permissible phrases is needed. In the paper, the results of experiments on extra-large vocabulary (above 100 K words) continuous speech recognition, with usage of n-gram models, are presented. A quantitative comparison of recognition accuracy of words, symbols, and phonemes depending on n-gram model, with n value varying from 0 till 3, was made.

A connectionist structure for phoneme recognition in continuous speech is described. This has two main parts. The first is a sound subunit classifier in the form of a three-layer back propagation network which classifies speech subunits from frames of spectral speech data. This is followed by a sequence classifier in the form of a network of neural like-units which classifies phonemes from input sequences of subunits by their occurrence and duration. Results are given for a 15-phoneme subset of British English, for a single speaker. These include the difficult syllable initial and final stop consonants, fricatives, vowels, and diphthongs. The overall recognition accuracy achieved is 87%. >

The choice of basic modeling unit in building acoustic model for a continuous Mandarin speech recognition task is a very important issue [1]. Unlike traditional phoneme or Initial/Finals (IFs) units based acoustic modeling methods, which usually suffer from the limitations of less accuracy in modeling intrasyllable variations and long scale temporal dependencies, in this paper, a practicable syllable based approach is presented. In contrast with IFs, syllable can implicitly model the intrasyllable variations in good accuracy. Also, by carefully choosing context modeling schemes and parameter tying methods, syllable based acoustic model can capture longer temporal variations while keeping the complexity of model well controlled. Meanwhile, considering the data unbalanced problem, multiple sized unit model based approaches are also implemented in this research. The experiment result shows the acoustic model based on the presented syllable based approach is effective in improving the performance of the Chinese continuous speech recognition.

The major difficulty of prosody modeling and automatic tone recognition of continuous Mandarin speech is the complex interaction of tones and prosody/intonation on FO contours. In this study, we propose a latent prosody model (LPM) aiming to jointly model the affections of tone and prosody state on FO. The main purposes are twofold including (1) automatic prosody state labeling and (2) improving tone recognition accuracy. The basic idea is to introduce latent prosody state variables into an additive statistic model of FO which already considers the affecting factors of tone and speaker. Experiments on the Tree-Bank corpus showed that LPM not only gave meaningful prosody state labeling results but also improved the average tone recognition rate from 80.86% of a multi-layer perceptron (MLP) baseline to 82.55%.

Inherent features of the Bangla (widely used as Bengali) language like long and short vowels and many instances of allophones make it difficult to build a continuous speech recognizer for the language. Stress and accent vary in spoken Bangla language from region to region. But in formal read Bangla speech, stress and accents are ignored. There are three approaches to continuous speech recognition (CSR) based on the sub-word unit viz. word, phoneme and syllable. Pronunciation of words and sentences are strictly governed by set of linguistic rules. Many attempts have been made to build continuous speech recognizers for Bangla for small and restricted tasks. However, medium and large vocabulary CSR for Bangla is relatively new and not explored. In this paper, the authors have attempted for extracting local features (LFs) from a Bangla input speech for tri-phone based automatic speech recognition (ASR) method. The method comprises two stages, where the first stage extracts LFs from input speech and the final stage generates word strings based on trip hone hidden Markov models (HMMs). The objective of this research is to build a medium vocabulary trip hone based continuous speech recognizer for Bangla language by extracting LFs over mel frequency cepstral coefficients (MFCCs). In this experimentation using Bangla speech corpus prepared by us, the recognizer provides higher word accuracy, word correct rate as well as sentence correct rate for trained and tested sentences with fewer mixture components in HMMs.

Automatic lip-reading (ALR) is a challenging task and a significant amount of research has been devoted to this topic in recent years. However, continuous Russian speech recognition still remains a not well-investigated area. In this paper, we present the results of Russian visual speech recognition (VSR) system using pixel-based and advanced geometry-based features. A HAVRUS video database, comprising of 4000 utterances of continuous Russian speech, collected from 20 speakers, is used in this study. Pixel-based features (principal component analysis-based or PCA) and geometry-based features (active appearance model-based or AAM) were used for the feature representation, and a Gaussian mixture hidden Markov models (HMM) were used for classification. Our evaluation experiments show a significant improvement (up to 9%) in recognition accuracy by using proposed geometry-based features when compared to pixel-based PCA features. The combined VSR is planned for future studies to augment the performance of audio-based automatic speech recognition systems in human–robot interfaces.

Hybrid speech recognition systems incorporating deep neural networks (DNNs) with Hidden Markov Models/Gaussian Mixture Models have achieved good results. We propose applying various DNNs in automatic recognition of Russian continuous speech. We used different neural network models such as Convolutional Neural Networks (CNNs), modifications of Long short-term memory (LSTM), Residual Networks and Recurrent Convolutional Networks (RCNNs). The presented model achieved \(7.5\%\) reducing of word error rate (WER) compared with Kaldi baseline. Experiments are performed with extra-large vocabulary (more than 30 h) of Russian speech.