Speech recognition methods applied to biomedical signals processing

Dynamic time warping in phoneme modeling for fast pronunciation error detection

In this dissertation, a novel approach of integrating template matching with statistical modeling is proposed to improve continuous speech recognition. Hidden Markov Modeling (HMMs) has been the dominant approach in statistical speech recognition since it provides a principled way of jointly modeling speech spectral variations and time dynamics. However, HMMs have the shortcoming of assuming the observations being independent within each state, which makes it ineffective in modeling the details of speech temporal evolutions that are important for characterizing nonstationary speech sounds. Template-based methods make comparisons between a test pattern and the templates derived from training data, and therefore they are able to capture speech dynamics and time correlation of speech frames better than HMM based methods. However, template matching requires large memory space and computational time since feature vectors of training data need to be stored in computer memory for access at the recognition stage, which is difficult in large vocabulary continuous speech recognition (LVCSR). Our proposed approach takes advantages of both statistical modeling and template matching, which overcomes the weakness of conventional template-based method and is feasible for LVCSR. We use multiple Gaussian Mixture Model (GMM) indices to represent each frame of speech templates, and define the template unit to be context-dependent phone segments (triphone context). We also use phonetic decision trees borrowed from those commonly used in HMMs to tie triphone templates and predict triphones unseen in training data. Two local distances, log likelihood ratio (LLR) and Kullback-Leibler (KL) divergence, are proposed for dynamic time warping (DTW) based template matching. In order to reduce computational complexity and storage space, we propose methods of minimum distance template selection (MDTS) and maximum log-likelihood template selection (MLTS), and investigate a template compression method on top of template selection to further improve recognition performance. The template based methods were used to rescore lattices generated by baseline HMMs on the tasks of TIMIT continuous phone recognition and teleheath LVCSR and experimental results demonstrated that the proposed approach of integrating template matching with statistical modeling significantly improved recognition performances over the HMM baselines. The template selection methods also provided significant recognition accuracy improvements over the HMM baseline while largely reducing the computation and storage complexities. When all templates or MDTS were used, using the LLR local distance obtained better recognition performance than the KL divergence local distance. For MLTS and template compression, KL divergence local distance provided better performance than the LLR local distance, and the template compression method made further improvements over KL based MLTS. Since the templates were constructed based on the GMM indices extracted from HMM baselines, we also validated the effectiveness of the proposed template methods based on enhanced HMM baselines. Experimental results showed that LLR based all template method was able to consistently improve TIMIT phone recognition accuracies based on four enhanced HMM baselines. Prosodic features such as duration, energy, and pitch can reflect longer span information of speech than conventional single frame vectors but they have commonly been ignored by HMMs. Template based methods provide possibilities to conveniently integrate prosodic features into speech recognition, which has not been well studied in the past. In this dissertation, we investigate combining template based methods with the speech prosodic features of duration, energy and pitch to further improve speech recognition accuracy. The scores of prosodic information were computed by a GMM based method and a non-parametric method, and the prosodic scores were combined with the acoustic scores in triphone template matching. Experimental results obtained on the telehealth task showed that prosodic information had positive effects on vowel sound recognition.

It's difficult to build precise mathematical models for complex engineering systems because of the complexity of the structure and dynamics characteristics. Intelligent fault diagnosis introduces artificial intelligence and works in a different way without building the analytical mathematical model of a diagnostic object, so it's a practical approach to solve diagnostic problems of complex systems. This paper presents an intelligent fault diagnosis method, an integrated fault-pattern classifier based on Hidden Markov Model (HMM). This classifier consists of dynamic time warping (DTW) algorithm, self-organizing feature mapping (SOFM) network and Hidden Markov Model. First, after dynamic observation vector in measuring space is processed by DTW, the error vector including the fault feature of being tested system is obtained. Then a SOFM network is used as a feature extractor and vector quantization processor. Finally, fault diagnosis is realized by fault patterns classifying with the Hidden Markov Model classifier. The importing of dynamic time warping solves the problem of feature extracting from dynamic process vectors of complex system such as aeroengine, and makes it come true to diagnose complex system by utilizing dynamic process information. Simulating experiments show that the diagnosis model is easy to extend, and the fault pattern classifier is efficient and is convenient to the detecting and diagnosing of new faults.

In this paper, we present a novel occlusion detection scheme for hidden Markov modeled shapes. First, hidden Markov model (HMM) is built using multiple examples of the shape. A reference path for the shape is built from the HMM, which is nothing but optimal path followed by the most likely example. The reference path stores temporal information about the entire shape, while the HMM only retains relationship between temporal information. For the shape of interest, its optimal path through HMM is calculated and warped to match the reference path using dynamic time warping (DTW). Occluded part of the shape is detected by identifying imbalance among various components of the matching cost. Detection results obtained for two shape data sets are presented for varying degrees of occlusion.

This paper proposes a novel similarity measure between vector sequences. We work in the framework of model-based approaches, where each sequence is first mapped to a Hidden Markov Model (HMM) and then a measure of similarity is computed between the HMMs. We propose to model sequences with semicontinuous HMMs (SC-HMMs). This is a particular type of HMM whose emission probabilities in each state are mixtures of shared Gaussians. This crucial constraint provides two major benefits. First, the a priori information contained in the common set of Gaussians leads to a more accurate estimate of the HMM parameters. Second, the computation of a similarity between two SC-HMMs can be simplified to a Dynamic Time Warping (DTW) between their mixture weight vectors, which significantly reduces the computational cost. Experiments are carried out on a handwritten word retrieval task in three different datasets-an in-house dataset of real handwritten letters, the George Washington dataset, and the IFN/ENIT dataset of Arabic handwritten words. These experiments show that the proposed similarity outperforms the traditional DTW between the original sequences, and the model-based approach which uses ordinary continuous HMMs. We also show that this increase in accuracy can be traded against a significant reduction of the computational cost.

Keyword spotting on embedded platforms must balance accuracy and strict resource limits while remaining independent of network connectivity. The aim of the study was to develop and experimentally validate a classical, frugal recognition method that increases feature informativeness without increasing model complexity and is suitable for autonomous use on edge devices that rely only on a central processing unit. A weighted acoustic f ingerprinting mechanism was proposed. Mel-frequency cepstral coefficients, together with their derivatives, were reweighted, aggregated, and serialised into compact discrete “fingerprints”, which were then classified using the Levenshtein edit distance. Experiments were carried out on a Ukrainian-language command corpus from six native speakers (three male, three female), recorded with both headsets and far-field microphones; lexicons of 10, 100, and 200 words were evaluated under speaker-independent splits of 70%/15%/15%. The methodology comprised f ixed parametrisation of mel-frequency cepstral coefficients, construction of a static weighting vector, voice activity detection with spectral subtraction, uniform quantisation and serialisation, and deterministic edit-distance classification; for comparison, equal-weight baselines, hidden Markov models with Gaussian mixture emissions, Dynamic Time Warping, a lightweight convolutional neural network, and a reference depthwise-separable convolutional neural network were considered. The proposed method achieved macro-averaged harmonic means of precision and recall of 0.96/0.92/0.89 for 10/100/200-word lexicons in clean audio, and 0.78 at a signal-to-noise ratio of 5 decibels (100-word lexicon). The implementation required approximately 250 kilobytes of memory and operated with a real-time factor of 0.005 on Raspberry Pi 4 with 4 gigabytes, i.e., faster than real time. Superiority over equal-weight baselines, hidden Markov models with Gaussian mixture emissions, and Dynamic Time Warping was demonstrated, with performance approaching that of a compact convolutional neural network. It is concluded that weighted acoustic fingerprinting provides a robust, efficient, and autonomous keyword-spotting solution for deployments that use only a central processing unit

In this paper, based on the study of two speech recognition algorithms, two designs of speech recognition system are given to realize this isolated speech recognition mobile robot control system based on ARM9 processor. The speech recognition process includes pretreatment of speech signal, characteristic extrication, pattern matching and post-processing. Mel-Frequency cepstrum coefficients (MFCC) and linear prediction cepstrum coefficients (LPCC) are the two most common parameters. Through analysis and comparison the parameters, MFCC shows more noise immunity than LPCC, so MFCC is selected as the characteristic parameters. Both dynamic time warping (DTW) and hidden markov model (HMM) are commonly used algorithm. For the different characteristics of DTW and HMM recognition algorithm, two different programs were designed for mobile robot control system. The effect and speed of the two speech recognition system were analyzed and compared.

Gesture recognition has become a very active research area with the advent of the Kinect sensor. The most common approaches for gesture recognition use temporal information and are based on methods such as Hidden Markov Models (HMM) and Dynamic Time Warping (DTW). In this paper, we present a novel non-temporal alternative for gesture recognition using the Microsoft Kinect device. The proposed approach, Recognition by Characteristic Window (RCW), identifies, using clustering techniques and a sliding window, distinctive portions of individual gestures which have low overlapping information with other gestures. Once a distinctive portion has been identified for each gesture, all these sub-sequences are used to recognize a new instance. The proposed method was compared against HMM and DTW on a benchmark gesture’s dataset showing very competitive performance.

Continuous hidden Markov models integrating transitional and instantaneous features for Mandarin syllable recognition

Rehabilitation systems are becoming more impor-tant now because patients can access motor skills recovery treatment from home, reducing the limitations of time, space and cost of treatment in a medical facility. Traditional rehabilitation systems served as movement guides, later as movement mirrors, and in recent years research has sought to generate feedback messages to the patient based on the evaluation of his or her movements. Currently the most commonly used algorithms for exercise evaluation are Dynamic time warping (DTW), Hidden Markov model (HMM), Support vector machine (SVM). However, the larger the set of exercises to be evaluated, the less accurate the recognition becomes, generating confusion between exercises that have similar posture descriptors. This research paper compares two HMM classifiers and Hidden Conditional Random Fields (HCRF) plus two types of posture descriptors, based on points and based on angles. Point representation proves to be superior to angle representation, although the latter is still acceptable. Similar results are found in HCRF and HMM.

In this paper, we proposed a dynamic time warping (DTW) method with a training part. DTW is a popular automatic speech recognition (ASR) method based on template matching. Conventional DTW is fast and of low complexity, however its recognition accuracy is limited. Recently, a DTW with multireferences (mDTW) algorithm has also been developed to improve the recognition accuracy to be comparable to that of the hidden Markov model (HMM) algorithm under noisy conditions. However the mDTW algorithm increases the calculation cost. Therefore, in order to reduce the calculation cost, in this paper, a training part will be added to the DTW-based ASR system, unlike the mDTW, which tries to find appropriate reference utterances to replace the increasing utterances. The results show that the average recognition accuracy of the proposed method is similar to that of the mDTW, and the calculation cost was reduced by 41.6%.

Modeling the driver behavior is expected to play a fundamental role in designing systems of driver monitoring, warning, assist control and training. In this paper, we present an identification method of automobile driver operations based on a hierarchical clustering approach, which leads to a stochastic piecewise affine (PWA) model. The driver behavior can be viewed as an outcome of the hybrid system that consists of (continuous) primitive driving operations and their (discrete) switchings. We describe the driving primitives by PWA models and the switchings by hidden Markov models (HMMs). One significant issue of this hybrid modeling is to extract the distinct states of driving operation from the driver behavior and determine the number of the states. To this problem, we propose a method to estimate the number of states using an idea of hierarchical clustering. We apply our identification method to the accelerator operations of driver, and demonstrate its efficacy through numerical experiments using the real data of four drivers.

Growth in Digital signal processing technology gives positives influences in human life. One of the branch of science that gives significant influence is digital speech processing. It can be expand into some applications that make human life easier. Digital speech processing is appropriate to speech recognition. Speech recognition is used to arrange the movement of remote control car. The remote control car will move according to our speech. This final project is closely related with speech recognition. The LPC (Linear Predictive Coding) method will extract the speech signal features and HMM (Hidden Markov Model) to modelling the speech signal are used. It is done by comparing model from extracting feature that is available in HMM modelling. Models will be used in the speech recognition process, if the models have highest level of conformity. The experiment has been done in two conditions, i.e ideal condition in room wih low noise level and unideal condition in room with noise. Result of the experiment from the whole sistem performace at ideal condition is 97,71% for people that have been inputed in the database, and 95,42% for people which have not been inputed in the database. Result of the experiment of database say it regulary is 97,14%. Result of the experiment of database at unideal condition is 54,28% for inputing word at noisy area, result of the experiment of database for inputing word at high frequency from siren voice is 98,57% and result of the experiment of inputing similar word with database is 97,6% Keywords: speech recognition, LPC, HMM

Chapter 10 - Robot Trajectory Generation in Joint Space

A non-probabilistic recognizer of stochastic signals based on KLT