Hybrid support vector machine/hidden Markov model approach for continuous speech recognition

In this dissertation, a novel approach of integrating template matching with statistical modeling is proposed to improve continuous speech recognition. Hidden Markov Modeling (HMMs) has been the dominant approach in statistical speech recognition since it provides a principled way of jointly modeling speech spectral variations and time dynamics. However, HMMs have the shortcoming of assuming the observations being independent within each state, which makes it ineffective in modeling the details of speech temporal evolutions that are important for characterizing nonstationary speech sounds. Template-based methods make comparisons between a test pattern and the templates derived from training data, and therefore they are able to capture speech dynamics and time correlation of speech frames better than HMM based methods. However, template matching requires large memory space and computational time since feature vectors of training data need to be stored in computer memory for access at the recognition stage, which is difficult in large vocabulary continuous speech recognition (LVCSR). Our proposed approach takes advantages of both statistical modeling and template matching, which overcomes the weakness of conventional template-based method and is feasible for LVCSR. We use multiple Gaussian Mixture Model (GMM) indices to represent each frame of speech templates, and define the template unit to be context-dependent phone segments (triphone context). We also use phonetic decision trees borrowed from those commonly used in HMMs to tie triphone templates and predict triphones unseen in training data. Two local distances, log likelihood ratio (LLR) and Kullback-Leibler (KL) divergence, are proposed for dynamic time warping (DTW) based template matching. In order to reduce computational complexity and storage space, we propose methods of minimum distance template selection (MDTS) and maximum log-likelihood template selection (MLTS), and investigate a template compression method on top of template selection to further improve recognition performance. The template based methods were used to rescore lattices generated by baseline HMMs on the tasks of TIMIT continuous phone recognition and teleheath LVCSR and experimental results demonstrated that the proposed approach of integrating template matching with statistical modeling significantly improved recognition performances over the HMM baselines. The template selection methods also provided significant recognition accuracy improvements over the HMM baseline while largely reducing the computation and storage complexities. When all templates or MDTS were used, using the LLR local distance obtained better recognition performance than the KL divergence local distance. For MLTS and template compression, KL divergence local distance provided better performance than the LLR local distance, and the template compression method made further improvements over KL based MLTS. Since the templates were constructed based on the GMM indices extracted from HMM baselines, we also validated the effectiveness of the proposed template methods based on enhanced HMM baselines. Experimental results showed that LLR based all template method was able to consistently improve TIMIT phone recognition accuracies based on four enhanced HMM baselines. Prosodic features such as duration, energy, and pitch can reflect longer span information of speech than conventional single frame vectors but they have commonly been ignored by HMMs. Template based methods provide possibilities to conveniently integrate prosodic features into speech recognition, which has not been well studied in the past. In this dissertation, we investigate combining template based methods with the speech prosodic features of duration, energy and pitch to further improve speech recognition accuracy. The scores of prosodic information were computed by a GMM based method and a non-parametric method, and the prosodic scores were combined with the acoustic scores in triphone template matching. Experimental results obtained on the telehealth task showed that prosodic information had positive effects on vowel sound recognition.

This paper presents the basis-based speaker adaptation method that includes approaches using principal component analysis (PCA) and two-dimensional PCA (2DPCA). The proposed method partitions the hidden Markov model (HMM) mean vectors of training models into subvectors of smaller dimension. Consequently, the sample covariance matrix computed using the partitioned HMM mean vectors has various dimensions according to the dimension of the subvectors. From the eigen-decomposition of the sample covariance matrix, basis vectors are constructed. Thus, the dimension of basis vectors varies according to the dimension of the sample covariance matrix, and the proposed method includes PCA and 2DPCA-based approaches. We present the adaptation equation in both the maximum likelihood (ML) and maximum a posteriori (MAP) frameworks. We perform continuous speech recognition experiments using the Wall Street Journal (WSJ) corpus. The results show that the model with basis vectors whose dimensions are between those of PCA and 2DPCA-based approaches shows good overall performance. The proposed approach in the MAP framework shows additional performance improvement over the ML counterpart when the number of adaptation parameters is large but the amount of available adaptation data is small. Furthermore, the performance of the approach in the MAP framework approach is less sensitive to the choice of model order than the ML counterpart.

Up till recently, state-of-the-art, large vocabulary, continuous speech recognition (CSR) had employed hidden Markov modeling (HMM) to model speech sounds. In an attempt to improve over HMM we developed a hybrid system that integrates HMM technology with neural networks. We present the concept of a Segmental Neural Net (SNN) for phonetic modeling in CSR. By taking into account all the frames of a phonetic segment simultaneously, the SNN overcomes the well-known conditional-independence limitation of HMMs. We have developed a novel hybrid SNN/HMM system that combines the advantages of SNNs and HMMs using a multiple hypothesis (or N-best) paradigm. In this system, we generate likely phonetic segmentations from the HMM N-best list of word sequences, which are scored by the SNN. The HMM and SNN scores are then combined to optimize performance. In several speaker-independent, 1000-word CSR tests, the error rate for the hybrid system dropped 20% from that of a state-of-the-art HMM system alone.

Hidden Markov models (HMM) with Gaussian mixture observation densities are the dominant approach in speech recognition. These systems typically use a representational model for acoustic modeling which can often be prone to overfitting and does not translate to improved discrimination. We propose a new paradigm centered on principles of structural risk minimization using a discriminative framework for speech recognition based on support vector machines (SVMs). SVMs have the ability to simultaneously optimize the representational and discriminative ability of the acoustic classifiers. We have developed the first SVM-based large vocabulary speech recognition system that improves performance over traditional HMM-based systems. This hybrid system achieves a state-of-the-art word error rate of 10.6% on a continuous alphadigit task—a 10% improvement relative to an HMM system. On SWITCHBOARD, a large vocabulary task, the system improves performance over a traditional HMM system from 41.6% word error rate to 40.6%. This dissertation discusses several practical issues that arise when SVMs are incorporated into the hybrid system.

Speech emotion recognition as a significant part has become a challenge to artificial emotion. It is particularly difficult to recognize emotion independent of the person concentrating on the speech channel. In the paper, an integrated system of hidden Markov model (HMM) and support vector machine (SVM), which combining advantages on capability to dynamic time warping of HMM and pattern recognition of SVM, has been proposed to implement speaker independent emotion classification. Firstly, all emotions are divided into two groups by SVM. Then, HMMs are used to discriminate emotions from each group. For a more robust estimation, we also combine four HMMs classifiers into a system. The recognition result of the fusion system has been compared with the isolated HMMs using Mandarin database. Experimental results demonstrate that comparing with the method based on only HMMs, the proposed system is more effective and the average recognition rate reaches 76.1% when speaker is independent.

The paper explores the hybrid of SVM and system of Artificial Neural Network as the finest binary classification system for calculating the diabetic nature of people in comparison to Support Vector Machine (SVM). In this research, the sets of all the parameters describing the diabetic nature of a person are taken from the laboratories. This approach is chosen so that a better learning method can be used for various problems. The testing results were found to be in accordance with the accepted results that resemble with the direct diagnosis of a physician. The results of this research shows that this hybrid SVM and Artificial Neural Network (ANN) model is more precise than the SVM model. These results of the hybrid SVM and ANN model suggest that it is very effective for the classification of Diabetic and Non Diabetic nature of a person. This paper highlights the concept of Support Vector Machines and its integration with Artificial Neural Network, the two key characteristics with one being the generalization theory of the SVM model that best describes how to select a hypothesis and functions given by Kernel that introduces the idea of non-linearity without the inclusion of the actual algorithm.

Hypertension is a chronic disease that often goes undetected in its early stages, increasing the risk of complications such as stroke and heart failure. Accurate classification of hypertensive patients is essential to support early intervention and reduce morbidity and mortality. This study aims to evaluate the performance of the Support Vector Machine (SVM) algorithm and to develop a hybrid classification model by integrating Particle Swarm Optimization (PSO) to improve the predictive performance of SVM. The research was conducted using 400 patient records from Medika Palopo General Hospital, equally divided into hypertensive and non-hypertensive groups, with 12 clinical features as input variables. The SVM model was built using a sigmoid kernel with default parameters (C = 1.0, gamma = auto), while the hybrid model utilized PSO to optimize the values of C and gamma. Evaluation results show that the conventional SVM model achieved an accuracy of 61.25%, precision of 63.22%, recall of 56.50%, F1-score of 59.31%, and AUC of 0.6400. After optimization using PSO, the hybrid model significantly improved with an accuracy of 96.75%, precision of 97.14%, recall of 96.31%, F1-score of 96.73%, and AUC of 0.9725. The findings indicate that the hybrid SVM-PSO approach effectively enhances the classification performance of hypertension prediction models and offers promising potential to be developed into an AI-based medical decision support system.

In order to solve the problems of current machine learning in fault diagnosing system of the chemical plants, a better and effective multilayer architecture model is used in this paper. Hidden Markov model (HMM) is good at dealing with dynamic continuous data and support vector machine (SVM) shows superior performance for classification, especially for limited samples. Combining their respective virtues, we propose a new multilayer architecture model to improve classification accuracy for a fault diagnosis example. The simulation result shows that this two level architecture framework combining HMM and SVM is better than the single HMM method in high classification accuracy with small training samples.

Classifiers in Automatic Speech Recognition (ASR) aims to improve the generalization ability of the machine learning and improve the recognition accuracy in noisy environments. This paper discusses the classification performance of Hidden Markov Models (HMM) and Support Vector Machines (SVM) applied to a wavelet front end based ASR. The experiments are performed on speaker independent TIMIT database which are trained in a clean environment and later tested in the presence of Additive White Gaussian Noise (AWGN) for various SNR levels using the HTK toolkit and SVM Light software tool. Experiments indicate that for large vocabulary the wavelet front end and the Multiclass SVM classifier with RBF kernel performs better than the conventional HMM classifier.

Internet of Things (IoT) based healthcare monitoring system is becoming the present and the future of the medical field around the world. Here the monitoring system acquires the regular health details of hospital discharged patients like elderly patients, patients out of critical operations, and patients from remote areas, etc., and transmits it to the doctors. But the system is highly susceptible to sensor faults. Hence a data-driven hybrid approach of Hidden Markov Model (HMM) based on baum-welch algorithm with Support Vector Machine (SVM) is proposed to predict the abnormality caused by the medical sensors. The proposed work first perform the abnormality detection on the sensor data using the HMM based on baum-welch algorithm in which the normal data is separated from abnormal data followed by classifying the abnormal data as critical patient data or sensor fault data using the SVM. Here the proposed work efficiently performs fault diagnosis with an overall accuracy of 99.94% which is 0.59% better than the existing SVM model. And also a comparison is made between the hybrid approach and the existing ML algorithms in terms of recall and F1-score where the proposed approach outperforms the other algorithms with a recall value of 100% and F1-score of 99.7%.

A major aspect in the development of advanced driving assistance systems (ADASs) is the research in developing human driving behavior prediction and recognition models. Recent contributions focus on developing these models for estimating different driving behaviors like lane or speed change. Thus, the models are incorporated into the ADAS to generate warnings and hints for safe maneuvers. Driving behavior recognition and prediction models are generally developed based on machine learning (ML) algorithms and are proven to generate accurate estimations. Previous review research contributions tend to focus on ML-based models for the prediction and recognition of speed change, trajectory change, and even driving styles. Due to high number of driving errors occurring during a lane change, a state-of-art review of different ML-based models for lane-changing behavior prediction and recognition is helpful to present a comparison between different models in terms of structure, influencing input variables, and performance. This enables the integration of the most efficient model for the development of ADASs to avoid accidents during a lane change. First, definitions and terms related to the model’s task and evaluation metrics used to evaluate the model’s performance are described to improve the readability. Then, the different input variables of the models affecting the lane-changing behaviors are presented. Next, a review of the models developed based on well-known approaches, such as artificial neural network (ANN), hidden Markov model (HMM), and support vector machine (SVM), using different input variables is given. Three lane-changing behaviors are focused on here: left/right lane change and lane keeping. The advantages and disadvantages of the different ML models with a comparison are summarized as well. Finally, the improvements required in the future are discussed.

Promoters are regions in DNA that play important role in the regulation of gene expression. The ability to locate promoters within a section of DNA is known to be a very difficult and important task in DNA analysis. Since experimental techniques to identify promoters are costly and time consuming, in silico methods offer an alternative. In this study, we have developed a tool for identification of s70 promoters in the –10 and –35 regions of sequences from Pseudomonas spp. Promoters were predicted using both Support Vector Machine (SVM) and Hidden Markov Model (HMM) based approaches. SVM performed better when trained using RBF kernel with a cross-validation of 5 and a value of 0.03 for the gamma parameter. The module developed using SVM showed a sensitivity of 78% and a specificity of 80%. The programmes required to process the user input were written using Perl and HTML codes were used to create a user interface. The user interface accepts a query sequence and the processed result will be displayed in a new window. The tool named ‘PROMIT’ (PROMoter Identification Tool), was developed in the Windows platform, has a user friendly interface and works well for sequences from Pseudomonas spp.

In metabonomics, metabolic profiles of high complexity bring out tremendous challenges to existing chemometric methods. Variable selection (ie, biomarker discovery) and pattern recognition (ie, classification) are two important tasks of chemometrics in metabonomics, especially biomarker discovery that can be potentially used for disease diagnosis and pathology discovery. Typically, the informative variables are elicited from a single classifier; however, it is often unreliable in practice. To rectify this, in the current study, bagging and classification tree (CT) were combined to form a general framework (ie, BAGCT) for robustly selecting the informative variables, based on the advantages of CT in automatically carrying out variable selection as well as measuring variable importance and the properties of bagging in improving the reliability and robustness of a single model. In BAGCT, a set of parallel CT models were established based on the idea of bagging, each CT providing some endowed information such as the splitting variables and their corresponding importance values. The informative variables can be successfully spied via inspecting the variable importance values over all CTs in BAGCT. Taking the promising properties of support vector machine (SVM) into account, we used the informative variables identified by BAGCT as the inputs of SVM, forming a new classification tool abbreviated as BAGCT‐SVM. A metabonomic dataset by hydrogen‐1 nuclear magnetic resonance from the patients with lung cancer and the healthy controls was used to validate BAGCT‐SVM with CT and SVM as comparisons. Results showed that BAGCT‐SVM with less number of variables can give better predictive ability than CT and SVM.

Hybrid methods which combine hidden Markov models (HMMs) and connectionist techniques take advantage of what are believed to be the strong points of each of the two approaches: the powerful discrimination-based learning of connectionist networks and the time-alignment capability of HMMs. Connectionist Viterbi Training (CVT) is a simple variation of Viterbi training which uses a back-propagation network to represent the output distributions associated with the transitions in the HMM. The work reported here represents the culmination of three years of investigation of various means by which HMMs and neural networks (NNs) can be combined for continuous speech recognition. This paper describes the CVT procedure, discusses the factors most important to its design and reports its recognition performance. Several changes made to the system over the past year are reported here, including: (1) the change from recurrent to non-recurrent NNs, (2) the change from Sphinx-style phone-based HMMs to word-based HMMS, (3) the addition of a corrective training procedure, and (3) the addition of an alternate model for every word. The CVT system, incorporating these changes, achieves 99.1% word accuracy and 98.0% string accuracy on the TI/NBS Connected Digits task (“TI Digits”).

Accent classification technologies directly influence the performance of speech recognition. Currently, two models are used for accent detection namely: Hidden Markov Model (HMM) and Artificial Neural Networks (ANN). However, both models have some drawbacks of their own. In this paper, we use Support Vector Machine (SVM) to detect different speakers’ accents. To examine the performance of SVM, Hidden Markov Model is used to classify the same problem set. Simulation results show that SVM can effectively classify different accents. Its performance is found to be very similar to that of HMM.