Noisy hidden Markov models for speech recognition

This paper presents a Bayesian learning approach to large margin classifier for hidden Markov model (HMM) based speech recognition. We build the Bayesian large margin HMMs (BLM-HMMs) and improve the model generalization for handling unknown test environments. Using BLM-HMMs, the variational Bayesian HMM parameters are estimated by maximizing lower bound of a marginal likelihood over the uncertainties of HMM parameters. The Bayesian large margin estimation is performed with frame selection mechanism, and is illustrated to meet the objective of support vector machines, i.e. maximal class margin and minimal training errors. The new objective function is not only interpreted as a discriminative criterion, but also feasible to deal with model selection and adaptive training. Experiments on phone recognition show that BLM-HMMs perform better than other generative and discriminative models.

In this dissertation, a novel approach of integrating template matching with statistical modeling is proposed to improve continuous speech recognition. Hidden Markov Modeling (HMMs) has been the dominant approach in statistical speech recognition since it provides a principled way of jointly modeling speech spectral variations and time dynamics. However, HMMs have the shortcoming of assuming the observations being independent within each state, which makes it ineffective in modeling the details of speech temporal evolutions that are important for characterizing nonstationary speech sounds. Template-based methods make comparisons between a test pattern and the templates derived from training data, and therefore they are able to capture speech dynamics and time correlation of speech frames better than HMM based methods. However, template matching requires large memory space and computational time since feature vectors of training data need to be stored in computer memory for access at the recognition stage, which is difficult in large vocabulary continuous speech recognition (LVCSR). Our proposed approach takes advantages of both statistical modeling and template matching, which overcomes the weakness of conventional template-based method and is feasible for LVCSR. We use multiple Gaussian Mixture Model (GMM) indices to represent each frame of speech templates, and define the template unit to be context-dependent phone segments (triphone context). We also use phonetic decision trees borrowed from those commonly used in HMMs to tie triphone templates and predict triphones unseen in training data. Two local distances, log likelihood ratio (LLR) and Kullback-Leibler (KL) divergence, are proposed for dynamic time warping (DTW) based template matching. In order to reduce computational complexity and storage space, we propose methods of minimum distance template selection (MDTS) and maximum log-likelihood template selection (MLTS), and investigate a template compression method on top of template selection to further improve recognition performance. The template based methods were used to rescore lattices generated by baseline HMMs on the tasks of TIMIT continuous phone recognition and teleheath LVCSR and experimental results demonstrated that the proposed approach of integrating template matching with statistical modeling significantly improved recognition performances over the HMM baselines. The template selection methods also provided significant recognition accuracy improvements over the HMM baseline while largely reducing the computation and storage complexities. When all templates or MDTS were used, using the LLR local distance obtained better recognition performance than the KL divergence local distance. For MLTS and template compression, KL divergence local distance provided better performance than the LLR local distance, and the template compression method made further improvements over KL based MLTS. Since the templates were constructed based on the GMM indices extracted from HMM baselines, we also validated the effectiveness of the proposed template methods based on enhanced HMM baselines. Experimental results showed that LLR based all template method was able to consistently improve TIMIT phone recognition accuracies based on four enhanced HMM baselines. Prosodic features such as duration, energy, and pitch can reflect longer span information of speech than conventional single frame vectors but they have commonly been ignored by HMMs. Template based methods provide possibilities to conveniently integrate prosodic features into speech recognition, which has not been well studied in the past. In this dissertation, we investigate combining template based methods with the speech prosodic features of duration, energy and pitch to further improve speech recognition accuracy. The scores of prosodic information were computed by a GMM based method and a non-parametric method, and the prosodic scores were combined with the acoustic scores in triphone template matching. Experimental results obtained on the telehealth task showed that prosodic information had positive effects on vowel sound recognition.

A hybrid model comprised of Gaussian Mixtures Models (GMMs) and Hidden Markov Models (HMMs) is used to model generic sounds with large intra class perceptual variations. Each class has variable number of mixture components in the GMM. The number of mixture components is derived using the Minimum Description Length (MDL) criterion. The overall performance of the hybrid model was compared against models based on HMMs and GMMs with a fixed number of mixture components across all classes. We show that a hybrid model outperforms both class-based GMMs, HMMs, and GMMs based on fixed number of components. Further, our experiments revealed that the contribution of transitions between states in HMMs has no significant effect on the overall classification performance of generic sounds when large intra class perceptual variations are present among sounds in the training and test datasets. Sounds that show multi-event structure with events that tend to be similar (repetitive) indicated improved performance when modeled with HMMs that can be attributed to HMM’s state transition property. Conversely, GMMs indicate better performance when the sound samples show subtle or no repetitive behavior. These results were validated using the MuscleFish sound database.

Discriminative training has been a leading factor for improving automatic speech recognition (ASR) performance over the last decade. The traditional discriminative training, however, has been aimed to minimize empirical error rates on training sets, which may not be well generalized to test sets. Many attempts have been made recently to incorporate the principle of large margin (PLM) into the training of hidden Markov models (HMMs) in ASR to improve the generalization abilities. Significant error rate reduction on the test sets has been observed on both small vocabulary and large vocabulary continuous ASR tasks using large-margin discriminative training (LMDT) techniques. In this paper, we introduce the PLM, define the concept of margin in the HMMs, and survey a number of popular LMDT algorithms proposed and developed recently. Specifically, we review and compare the large-margin minimum classification error (LM-MCE) estimation, soft-margin estimation (SME), large margin estimation (LME), large relative margin estimation (LRME), and large margin training (LMT) with a focus on the insights, the training criteria, the optimization techniques used, and the strengths and weaknesses of these different approaches. We suggest future research directions in our conclusion of this paper.

A stochastic version of Expectation Maximization algorithm for better estimation of Hidden Markov Model

The Hidden Markov models are generalized by defining a new emission probability which takes the correlation between successive feature vectors into account. Estimation formulas for the iterative learning both along Viterbi and Maximum likelihood criteria are presented.

A new model has been constructed to generalise the force and torque information during a manual peg-in-a-hole (PiH) assembly process. The paper uses Hidden Markov Model analysis to interpret the state topology (transition probability) and observations (force/torque signal) in the manipulation task. The task can be recognised as several discrete states that reflect the intrinsic nature of the process. Since the whole manipulation process happens so fast, even the operator themselves cannot articulate the exact states. Those are tacit skills which are difficult to extract using human factors methodologies. In order to programme a robot to complete tasks at skill level, numerical representation of the sub-goals are necessary. Therefore, those recognised ‘hidden’ states become valuable when a detail explanation of the task is needed and when a robot controller needs to change its behaviour in different states. The Gaussian Mixture model (GMM) is used as the initial guess of observations distribution. Then a Hidden Markov Model is used to encode the state (sub-goal) topology and observation density associated with those sub-goals. The Viterbi algorithm is then applied for the model-based analysis of the force and torque signal and the classification into sub-goals. The Baum-Welch algorithm is used for training and to estimate the most likely model parameters. In addition to generic states recognition, the proposed method also enhances our understanding of the skill based performances in manual tasks.

This paper presents a novel diagnosis method for the classification of cardiac arrhythmia based on electrocardiogram (ECG). The recognition of abnormal heart beat was performed using the context-independent hidden Markov model (HMM) and context-dependent HMM. Gaussian mixture model (GMM) was used as the state emission probability density function. The input feature of the recognition system was the raw heartbeat waveform. There were no other feature transform steps. Six classes of heartbeats, including normal beat, paced beat, left bundle branch block beat, right bundle branch block beat, premature ventricular contraction, and ventricular escape beat, were categorized by the system. The performance was evaluated on the MIT-BIH Arrhythmia Database and the European ST-T Database. The experimental results showed that the context-dependent HMM was more precise than the context-independent HMM. The context-dependent HMM system achieved a mean level of 95.26%, 90.67% and 95.60% in heart beats recognition accuracy, sensibility and specificity respectively. The results confirmed the importance of the generative model and the raw waveform feature strategy in ECG study.

BACKGROUND A hidden Markov model (HMM) is used for exon prediction on DNA of genes Plasmodium falciparum that has a model structure based on exon region structure in coding sequence (CDS). The objective research was to develop a new structure model to predict exon on DNA-genes of Plasmodium falciparum based on CDS structure using the HMM system. METHODS Model design in CDS, between two exon regions can be found one intron region and the model state number is used for its region. Its state number is used by separating start codon from first exon region and stop codon from the last exon region up to 9. The Viterbi algorithm and the backward-forward method for transition as well as emission states are used for training process. Furthermore, Viterbi and Baum-Welch algorithms are used for the testing process. The correlation coefficient (CC) was used as performance indicator, as the ratio of the estimated state in the output and the original state in the input of the model. RESULTS The simulation results has shown that the CC values depend on the given of the backward-forward transition state values randomly. The model with state number 9 showed the highest average of CC values of 0.7289 for Viterbi algorithm, and is 0.7166 for Baum-Welch algorithm. However, the lowest average of CC values has been found for the model with state number five. Its values are 0.6735 by using Viterbi algorithm and 0.6661 by using Baum-Welch algorithm. CONCLUSION The new structure model based on HMM system was valid to predict exon on DNA-genes of Plasmodium falciparum.

This paper models stochastic volatility using Hidden Markov Models in Kenya. The univariate Stochastic volatility Model is calibrated to the Nairobi Securities Exchange 20 share index daily data from January 2012 to February 2021. The Hidden Markov model (HMM) is employed to establish volatility regimes while the Expected Maximization (EM) algorithm is applied in parameter estimation. Markov Chain Monte Carlo (MCMC) and Sequential Monte Carlo (SMC) techniques are employed in filtering out noisy observations in parameter estimation. The 4-state model, which divides the economy into periods of very high, high, low, and very low volatility, is established to be optimal.

One effective iterative method for estimating parameters in latent variable probabilistic models is the Expectation-Maximization (EM) algorithm, which finds extensive use in classification issues. A versatile probabilistic framework for modeling data from multiple Gaussian distributions, Gaussian Mixture Models (GMMs) employ EM for parameter estimation, which is explored in this article. Problems with class membership estimation (including means, covariances, and mixing coefficients) are the primary focus of this study. Starting with initial parameter estimations, estimating posterior probabilities in the E-step, and repeatedly updating model parameters in the M-step until convergence, the technique entails applying the EM algorithm on both synthetic and real-world datasets. In order to measure performance, we compare it to K-means and other baseline classifiers using measures like classification accuracy, Adjusted Rand Index (ARI), and log-likelihood progression. When data is distributed according to a multimodal Gaussian distribution, the findings show that EM-GMM provides better parameter estimation and better classification accuracy. Initialization procedures have a substantial effect on performance, as convergence study further shows. In addition to reiterating the EM algorithm's usefulness for probabilistic classification, this paper also describes its advantages and disadvantages and offers suggestions for improving its performance in settings with high-dimensional data and noise.

Speech Emotion Recognition (SER) plays a vital role in various real-world applications, from mental health diagnostics to human-computer interaction. This research examines the integration of Stacked Long Short-Term Memory (LSTM) networks and Hidden Markov Models (HMM) for SER, addressing the limitations of traditional models like Gaussian Mixture Models (GMM) and Support Vector Machines (SVM). While GMMs and SVMs offer decent performance, their inability to model temporal dynamics limits their accuracy in detecting complex emotions. The proposed hybrid model leverages LSTM’s strength in handling sequential data and HMM’s ability to model emotional transitions, making it highly suitable for real-world noisy environments. Preprocessing techniques such as MFCC and LPCC are applied to enhance feature extraction, and benchmark datasets like IEMOCAP and RAVDESS are used for evaluation. Finally, The Systematic review highlights the superior role of the hybrid model's performance on SER and sets the stage for a significant shift in future research in addressing bias and fairness in SER systems by combining LSTM and HMM as hybrid model. In recent studies, context-aware emotion recognition models are commonly evaluated using existing datasets such as IEMOCAP, MELD, DailyDialog, and SEMAINE, which provide conversational and multimodal emotional data. Features used often include acoustic features as MFCCs, spectrograms, prosodic features, linguistic features as BERT or transformer embeddings, and visual cues such as facial landmarks or video frames. Many models also incorporate contextual embeddings from transformers, graph attention networks, or sequential memory modules to capture speaker history and emotional shifts.

A network intrusion can be any unauthorized activity on a network and network intrusion detection is a significant topic in cybersecurity. Data analytics is conducted on the database ‘spambase’ as an example of analysis for network intrusion detection based on the Naïve Bayesian classification, deep learning with the algorithm of Rprop+ and the hidden Markov model (HMM), respectively. All the analysis is fulfilled using R language and its functions. An HMM based on the Baum–Welch algorithm has been created on the database ‘spambase’ through training and parameter estimation. An HMM-based spam-email prediction has been performed through the probability evaluation based on the forward algorithm. The analytics results obtained from the above three methods are compared. It is shown that HMM-based analytics can achieve the best accuracy in the spam-email classification although only a few features are used in the HMM while all features are used in the Naïve Bayesian classification and deep learning.

The expectation maximisation (EM) algorithm is an iterative maximum likelihood procedure often used for estimating the parameters of a mixture model. Theoretically, increases in the likelihood function are guaranteed as the algorithm iteratively improves upon previously derived parameter estimates. The algorithm is considered to converge when all parameter estimates become stable and no further improvements can be made to the likelihood value. However, to reduce computational time, it is often common practice for the algorithm to be stopped before complete convergence using heuristic approaches. In this paper, we consider various stopping criteria and evaluate their effect on fitting Gaussian mixture models (GMMs) to patient length of stay (LOS) data. Although the GMM can be successfully fitted to positively skewed data such as LOS, the fitting procedure often requires many iterations of the EM algorithm. To our knowledge, no previous study has evaluated the effect of different stopping criteria on fitting GMMs to skewed distributions. Hence, the aim of this paper is to evaluate the effect of various stopping criteria in order to select and justify their use within a patient spell classification methodology. Results illustrate that criteria based on the difference in the likelihood value and on the GMM parameters may not always be a good indicator for stopping the algorithm. In fact we show that the values of the difference in the variance parameters should be used instead, as these parameters are the last to stabilise. In addition, we also specify threshold values for the other stopping criteria.

EM (expectation-maximization) algorithm is a classical method for parameter estimation of HMM (Hidden Markov model ). Concerning that EM algorithm is easily affected by initial parameter values, we proposed a mixture splitting algorithm based on decision boundary confusion (DBC) to describe more about boundary distribution. The algorithm mainly includes three aspects: firstly the number of incremented mixtures for every decision boundary could be determined according to decision boundary confusion; secondly the mixtures which are the closest to the decision boundary are chosen to split, thirdly the split mean of mixture is in the direction of decision boundary. Our experiments show that our proposed algorithm is more effective for classification using HMM.