A new hybrid system based on MMI-neural networks for the RM speech recognition task

The current state-of-the-art in large-vocabulary, continuous speech recognition is based on the use of hidden Markov models (HMM). In an attempt to improve over HMM performance, the authors developed a hybrid system that combines the advantages of neural networks and HMM using a multiple hypothesis (or N-best) paradigm. The connectionist component of the system, the segmental neural net (SNN), models all the frames of a phonetic segment simultaneously, thus overcoming the well-known conditional-independence limitation of the HMM. They describe the hybrid system and discuss various aspects of SNN modeling, including network architectures, training algorithms and context modeling. Finally, they evaluate the hybrid system by performing several speaker-independent experiments with the DARPA Resource Management (RM) corpus, and demonstrate that the hybrid system shows a consistent improvement in performance over the baseline HMM system. >

A speech recognition method which integrates a VQ (vector quantization)-distortion measure and a discrete HMM (hidden Markov model) is proposed. This VQ-distortion-based HMM uses a VQ-distortion measure at each state instead of the discrete output probability used by a discrete HMM. Although this method is regarded as a refined version of the VQ-distribution based recognition method proposed by D.K. Burton et al (IEEE Trans. vol. ASSP-33, no.4, p.837-49 of 1985), it is also considered as a special case of a mixtured distribution density HMM. The authors describe the relationship between the VQ-distortion-based HMM and conventional HMMs, and compare their speech recognition performance through experiments on speaker-dependent digit recognition. A recognition accuracy of 100% using the new method was obtained. >

Hybrids that use a neural network to estimate the output probability for a hidden Markov model (HMM) word recognizer have been competitive with traditional HMM recognizers when both use monophone context. While traditional HMM recognizers can easily utilize more context (e.g., triphones) to achieve better results, the size of the task has made it impractical to use phonetic context directly in the neural network front end of a hybrid. In this paper, we suggest a simple method to incorporate more context by modeling the phone distributions obtained from the neural network. This allows the HMM to easily handle stochastic pronunciations as well as errors from the neural network phone recognizer. The re-estimation equations are derived for the new model. Results for the Resource Management task illustrate that SOHMM increases recognition accuracy for the cases of no grammar, unigram grammar, and word pair grammar.

In this paper, we present time-sequence binary classifiers (TSBC) for phonetic feature extraction. A large neural network is divided into an array of TSBCs, and each TSBC is a multilayer neural network which is dedicatedly trained to extract low level acoustic features of only one phoneme category, resulting in lower neural network complexity. TSBC has a feature that its output units are sequentially arranged and trained to reflect the temporal information of phonetic features, which is very important in speech recognition. In our speaker-independent all-Chinese-Syllable continuous speech recognition system, TSBCs are efficiently combined with HMM techniques, where TSBCs are used to extract low level phonetic features and HMMs are used to recognize high level speech units. The evaluation experiments obtain 97.0% word accuracy for speaker-independent large-vocabulary and continuous speech recognition.

The Relative Distance Vector Neural Network (RDVNN) Model: A Hybrid Approach to Speech Recognition

Accurate and computationally efficient means of recognizing continuous speech has been a subject of research in recent years. This paper reports the development of a small vocabulary, speaker independent continuous Malayalam speech recognition system based on Hidden Markov Models (HMMs). Continuous density HMM, which is used in this work to model phonemes, represents the general case where the observation probability density functions (pdfs) are continuous. The observation pdf is approximated using a Gaussian mixture density. Mel-frequency Cepstral Coefficients (MFCC) method is used to extract acoustic features from the input signal. To represent temporal variations in the speech signal, the first and second order derivatives of MFCC are added to the set of static parameters. The training and decoding are performed by the Baum-Welch and Viterbi algorithms respectively. The corpus contains naturally and continuously spoken sentences with multiple pronunciations and speaker variations. On evaluation the proposed system has produced promising results with 94.67% word accuracy and 93.33% sentence correct.

Up till recently, state-of-the-art, large vocabulary, continuous speech recognition (CSR) had employed hidden Markov modeling (HMM) to model speech sounds. In an attempt to improve over HMM we developed a hybrid system that integrates HMM technology with neural networks. We present the concept of a Segmental Neural Net (SNN) for phonetic modeling in CSR. By taking into account all the frames of a phonetic segment simultaneously, the SNN overcomes the well-known conditional-independence limitation of HMMs. We have developed a novel hybrid SNN/HMM system that combines the advantages of SNNs and HMMs using a multiple hypothesis (or N-best) paradigm. In this system, we generate likely phonetic segmentations from the HMM N-best list of word sequences, which are scored by the SNN. The HMM and SNN scores are then combined to optimize performance. In several speaker-independent, 1000-word CSR tests, the error rate for the hybrid system dropped 20% from that of a state-of-the-art HMM system alone.

Multilayer perceptrons for state-dependent weightings of HMM likelihoods

Arabic language can be used by native and non- native speakers; due to Arabic is the language of the holy book of Muslims. In this paper, Arabic phoneme recognition system is proposed based on Malay speakers. This system consists of three main stages. The first stage is noise reduction and it aims to enhance the phoneme signals by excluding the unvoiced signals and keep only the voiced signal. Wiener filter is adapted to accomplish this task. The second stage is based on Mel-Frequency Cepstral Coefficients method to extract a vector of features to represent each phoneme signal. Eventually, pattern recognition neural network is designed as recognizer. The proposed system produces sufficient outcomes with 20 hidden neurons. Keywords—Arabic; Malay; pattern recognition; Wiener; Mel- Frequency Cepstral Coefficients I. INTRODUCTION It is well known the wide range of Artificial Neural Networks (ANN) application in speech recognition, financial, telecommunications, electronics and medical. As for speech recognition system for Arabic language, (1) & (2) have addressed the implementation of recognition system based on Arabic native speakers. In general, neural networks (NNs) for phonemes recognition are divided into several categories for instance Very Large Scale Integration (VLSI) NN that can be divided into digital and analogue with digital NNs being more compatible with feed-forward neural networks while analogue NNs are found to be more successful with recurrent NNs (3). From Automatic Speech System (ASR) point of view, NNs for recognition purpose can be divided into two main category namely conventional neural networks such as Multilayer Perceptron (MLP) and Radial Basis Function (RBF) and secondly Recurrent Neural Networks (RNN). The first category of NNs was implemented as pattern classifiers and proven capable for recognition purpose but not at par as compared to Hidden Markov Model (HMM) performance (2). At present, HMM has been proven to be the most successful approach in ASR research area until recently the possibility to hybrid both HMM and ANN (4) & (5). In general, speech recognition systems consist of three stages specifically noise reduction, feature extraction and classification. The aim of noise reduction is to reduce the level of noise in the speech signal and make it more compatible for the ASR. As for feature extraction, this process attempted to extract the most relevant features related to the phoneme signal. The extracted features acted as input features of the NN during training and testing. Finally, the purpose of classification process is to arrange the most similar and related features in categories based on the training phase. Basically, networks training database are being observed and updated during training session of the networks, involving the weights and biases arguments. Networks with lower errors can be considered as better networks (6). Hence, this paper deems to explore further the ability of NN for Arabic phonemes recognition spoken by Malay individuals. This paper is arranged as following; related work is discussed in section II followed by section III that discussed method to be implemented based on Wiener filter, Mel-Frequency Cepstral Coefficients and pattern recognition neural network. Section IV elaborated experimental analysis and results and followed by Conclusion section.

This paper describes an automatic text‐ and speaker‐independent language identification method based on hidden Markov models (HMMs) for acoustic features. The hidden Markov modeling is used to represent the phonotactics for each language. Each language has its proper phonotactics.The HMM topology here is a fully structured (ergodic) model wherein any state could transit to all states. Two kinds of HMMs are used: the discrete HMM (DHMM) with the codebook and the continuous density HMM (CHMM). The HMM was trained using both the Baum‐Welch (forward‐backward) algorithm and the Viterbi algorithm. The latter was used for emphasizing the state transition probability.For comparison, experiments also were conducted on the identification using a mixtured Gaussian distribution model with one state. This single‐state Gaussian distribution model gave the same performance as the HMM trained with the Baum‐Welch algorithm. This is because the transition between states which reflects the characteristics of each language does not affect the likelihood scores very much. This problem was addressed by emphasizing the transition probabilities and using the Viterbi algorithm, which resulted in an improvement in the recognition rates. The trigram for optimal state sequence is introduced. Combining it with the HMM produced the best results.

A model-based framework of classification error rate estimation is proposed for speech and speaker recognition. It aims at predicting the run-time performance of a hidden Markov model (HMM) based recognition system for a given task vocabulary and grammar without the need of running recognition experiments using a separate set of testing samples. This is highly desirable both in theory and in practice. However, the error rate expression in HMM-based speech recognition systems has no closed form solution due to the complexity of the multi-class comparison process and the need for dynamic time warping to handle various speech patterns. To alleviate the difficulty, we propose a one-dimensional model-based misclassification measure to evaluate the distance between a particular model of interest and a combination of many of its competing models. The error rate for a class characterized by the HMM is then the value of a smoothed zero-one error function given the misclassification measure. The overall error rate of the task vocabulary could then be computed as a function of all the available class error rates. The key here is to evaluate the misclassification measure in terms of the parameters of environmental-matched models without running recognition experiments, where the models are adapted by very limited data that could be just the testing utterance itself. In this paper, we show how the misclassification measure could be approximated by first computing the distance between two mixture Gaussian densities, then between two HMMs with mixture Gaussian state observation densities and finally between two sequences of HMMs. The misclassification measure is then converted into classification error rate. When comparing the error rate obtained in running actual experiments and that of the new framework, the proposed algorithm accurately estimates the classification error rate for many types of speech and speaker recognition problems. Based on the same framework, it is also demonstrated that the error rate of a recognition system in a noisy environment could also be predicted.

The authors describe the latest development by the speech research group at CRIM (Centre de Recherche Informatique de Montreal) in speaker-independent connected digit recognition, using hidden Markov Models (HMMs) trained with maximum mutual information estimation, in conjunction with connectionist models. The experiments described were all done on the complete adult portion of the 10 kHz speaker-independent TI/NIST connected digit database. The baseline system, using discrete HMMs and maximum likelihood estimation, has a 98.6% word recognition rate and a 96.1% string recognition rate. The authors describe techniques that made it possible to improve greatly the baseline system recognition rate. The 99.3% recognition rate and 98.0% string recognition rate were obtained with a single model per unit using discrete HMMs and recurrent neural networks. Using semi-continuous HMMs with two models per digit (one for male and one for female speakers), a 99.5% word recognition rate and a 98.4% string recognition rate were achieved.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

Evolutionary approach for integration of multiple pronunciation patterns for enhancement of dysarthric speech recognition

In recent years, with the rapid development of Deep Learning (DL) and the widespread uses of Deep Neural Networks (DNN), speech recognition technology has attracted great attention. This technology enables machines to convert human speech into text and plays an important role in some areas such as virtual assistants, conference transcription and smart home devices. Traditional models such as Hidden Markov Models (HMM) and Gaussian Mixture Models (GMM) used to play a key role in modeling and recognizing speech signals. Even though these methods still have challenges in dealing with high‐dimensional feature vectors as they struggle to efficiently learn large amounts of model parameters while maintaining good generalization performance. The emergence of DNNs has revolutionized the field of speech recognition, making it possible to automatically extract features from raw speech signals. Thereby, modern speech recognition systems can exhibit higher adaptability and generalization capabilities, achieving higher accuracy and stability under various conditions. This review explores the application of DL techniques to speech recognition, including convolutional neural networks (CNNs), recurrent neural networks (RNNs), and Transformer‐based architectures, etc. Advances in these techniques have dramatically improved recognition accuracy, noise immunity, and adaptability to multiple accents, thereby improving the total efficiency of automatic speech systems. By leveraging advanced feature extraction and modeling capabilities, deep learning‐based systems support real‐ time processing and minimize the dependency on traditional manually designed features. However, there are still limitations in terms of interpretability, computational efficiency, and fairness to different accents and languages, which impede the widespread use of these techniques. Future research should focus on solving these issues in order to achieve fairer, efficient, and scalable deep learning‐driven speech recognition schemes.

In this paper a novel algorithm for Gaussian Selection (GS) of mixtures used in a continuous speech recognition system is presented. The system is based on hidden Markov models (HMM), using Gaussian mixtures with full covariance matrices as output distributions. The purpose of Gaussian selection is to increase the speed of a speech recognition system, without degrading the recognition accuracy. The basic idea is to form hyper-mixtures by clustering close mixtures into a single group by means of Vector Quantization (VQ) and assigning it unique Gaussian parameters for estimation. In the decoding process only those hyper-mixtures which are above a designated threshold are selected, and only mixtures belonging to them are evaluated, improving computational efficiency. There is no problem with the clustering and evaluation if overlaps between the mixtures are small, and their variances are of the same range. However, in real case, there are numerous models which do not fit this profile. A Gaussian selection scheme proposed in this paper addresses this problem. For that purpose, beside the clustering algorithm, it also incorporates an algorithm for mixture grouping. The particular mixture is assigned to a group from the predefined set of groups, based on a value aggregated from eigenvalues of the covariance matrix of that mixture using Ordered Weighted Averaging operators (OWA). After the grouping of mixtures is carried out, Gaussian mixture clustering is performed on each group separately.