A hidden Markov model for multimodal biometrics score fusion

A face recognition system usually consists of one recognition algorithm by processing single spectral images. For example, face pattern byte (FPB) algorithm was initially created using thermal (LWIR) images, while Elastic Bunch Graphic Matching (EBGM) algorithm was originated with visible (RGB) images. When there are two or more recognition algorithms and/or spectral images available, system performance can be enhanced using information fusion. In this paper, a score fusion with multispectral images is proposed to improve system performance, which is termed as an integrated multispectral face recognition system. Score fusion actually combines several scores from multiple matchers (algorithms) and/or multiple modalities (multispectra). The system performance is measured by the recognition accuracy (AC; the higher the better) and false accept rate (FAR; the lower the better). Specifically, a fusion method will combine the face scores from three matchers (Circular Gaussian Filter, FPB, EBGM) and from two-spectral bands (visible and thermal). We present and compare the system performance using seven fusion methods: linear discriminant analysis (LDA), k-nearest neighbor (KNN), support vector machine (SVM), binomial logistic regression (BLR), Gaussian mixture model (GMM), artificial neural network (ANN), and hidden Markov model (HMM). Our experiments are conducted with the Alcon State University multispectral face dataset that currently consists of two spectral images from 105 subjects. The experimental results show that the KNN score fusion produces the best performance (AC = 98.98%; FAR = 0.35%); and the SVM yields the second best. Compared with the performance of the single best matcher (AC = 91.67%, FAR = 8.33%), the integrated system with score fusion highly improves the accuracy, meanwhile dramatically reduces the FAR.

A hybrid model comprised of Gaussian Mixtures Models (GMMs) and Hidden Markov Models (HMMs) is used to model generic sounds with large intra class perceptual variations. Each class has variable number of mixture components in the GMM. The number of mixture components is derived using the Minimum Description Length (MDL) criterion. The overall performance of the hybrid model was compared against models based on HMMs and GMMs with a fixed number of mixture components across all classes. We show that a hybrid model outperforms both class-based GMMs, HMMs, and GMMs based on fixed number of components. Further, our experiments revealed that the contribution of transitions between states in HMMs has no significant effect on the overall classification performance of generic sounds when large intra class perceptual variations are present among sounds in the training and test datasets. Sounds that show multi-event structure with events that tend to be similar (repetitive) indicated improved performance when modeled with HMMs that can be attributed to HMM’s state transition property. Conversely, GMMs indicate better performance when the sound samples show subtle or no repetitive behavior. These results were validated using the MuscleFish sound database.

Computational cancer phylogenetics can play an important role in delineating possible tumor progression pathways and identifying molecular subtypes and mechanisms of action. We previously developed a pipeline for constructing tumor phylogenies from recurring cell types computationally inferred from whole genome copy number data. The accuracy and detail of these tumor phylogenies, however, depends on the identification of accurate and high-resolution molecular markers of progression, i.e., reproducible regions of copy number variation that can be used to robustly differentiate different subtypes and stages of progression. Here we present a new method for the problem using hidden Markov models (HMMs) to derive robust, high resolution progression markers from sets of tumor samples. We demonstrate our method on a publicly available array comparative genome hybridization (aCGH) dataset (NCBI GEO GSE16672, Navin et al., 2010) from sectioned primary ductal breast tumors. Our method uses an HMM, a class of probabilistic models, to classify sets of aCGH data into a partitioning of samples into normal (diploid) or amplified at each copy number probe. It differs from other similar HMM methods primarily in seeking a parsimonious set of combinations of amplification states able to explain all aCGH profiles simultaneously in order to identify robust markers of progression across samples. The model learns frequencies with which different combinations of amplifications are observed across the samples by modeling individual probes as Gaussian random variables with either normal or tetraploid means, with data more consistent with tetraploid being classified as amplified and those more consistent with diploid classified as normal. To handle a combinatorial explosion in combinations of amplification states with increasing numbers of samples, the method introduces a Gibbs sampling algorithm to learn a parsimonious model of the most frequently occurring combinations of amplification states. We applied our methods to a previously constructed set of inferred cell types derived from the Navin et al. data (Tolliver et al., 2010) and to a comparison set of 9 random samples from the raw aCGH data. We validated our model relative to manual labeling of amplicons on the same data. In both experiments, the HMM method was able to pick up significantly larger numbers of robustly amplified segments per chromosome than did prior methods or manual analysis. The resulting segments can be directly fed into downstream analysis routines for phylogeny inference or other predictions. In future work, the HMM method may be improved by fine-tuning the underlying model for copy number variation. Citation Format: {Authors}. {Abstract title} [abstract]. In: Proceedings of the 103rd Annual Meeting of the American Association for Cancer Research; 2012 Mar 31-Apr 4; Chicago, IL. Philadelphia (PA): AACR; Cancer Res 2012;72(8 Suppl):Abstract nr 3964. doi:1538-7445.AM2012-3964

In this paper, we propose a new objective evaluation method for hidden Markov model (HMM)-based speech synthesis using Kullback-Leibler divergence (KLD). The KLD is used to measure the difference between the probability density functions (PDFs) of the acoustic feature vectors extracted from natural training and synthetic speech data. For the evaluation, Gaussian mixture model (GMM) is used to model the distribution of acoustic feature vectors, including the fundamental frequency (F0). Continuous F0, obtained with linear interpolation, is used in the evaluation. In essence, the KLD is the expectation of the logarithmic difference between the likelihoods calculated on training and synthetic speech. This likelihood difference is appropriate to characterize the quality of a HMM-based speech synthesis system in generating synthetic speech using a maximum likelihood criterion. The objective evaluation is tested with 3 different HMM-based speech synthesis systems which use multi-space distribution (MSD) to model discontinuous F0. These systems are trained on a common speech corpus in French. We propose an index to evaluate HMM-based speech synthesis system which takes into account the relative variation of the KLDs on test sets of synthetic and natural speech. This index correlates inversely with the result of the MOS (mean opinion score) perceptual test.

Speech to Text was one of speech recognition applications which speech signal was processed, recognized and converted into a textual representation. Hidden Markov Model (HMM) was the widely used method in speech recognition. However, the level of accuracy using HMM was strongly influenced by the optimalization of extraction process and modellling methods. Hence in this research, the use of genetic algorithm (GA) method to optimize the Ergodic HMM was tested. In Hybrid HMM-GA, GA was used to optimize the Baum-welch method in the training process. It was useful to improve the accuracy of the recognition result which is produced by the HMM parameters that generate the low accuracy when the HMM are tested. Based on the research, the percentage increases the level of accuracy of 20% to 41%. Proved that the combination of GA in HMM method can gives more optimal results when compared with the HMM system that not combine with any method.

Multi-modal fusion model for Time-Varying medical Data: Addressing Long-Term dependencies and memory challenges in sequence fusion.

This paper introduces a non-temporal multiple silhouettes in Hidden Markov Model (HMM) for offering view independent human posture recognition. The multiple silhouettes are used to reduce the ambiguity problem of posture recognition. A simple feature extraction of the 2D shape contour based histogram is used for image encoding and K-Means algorithm is applied for clustering and code-wording of eight simple postures from multiple views. Therefore, 3D volume reconstruction is not required, in return helps to reduce the complexity of modeling and computational power of feature extraction. HMM is trained to obtain view independent recognition model using multiple silhouettes. A combination of non-temporal multiple silhouettes, code-wording and HMM methods in this proposed approach make it possible to recognize human posture in view independent. The experimental results demonstrate the effectiveness of non-temporal multiple silhouettes in HMM for recognizing posture.

A codebook design method for Hidden Markov Model (HMM) by using a Centroid Neural Network (CNN) is applied to a Korean monophone recognition problem in this paper. In order to alleviate the accuracy degradation problem in tied mixture HMM (TMHMM), this paper utilizes a clustering algorithm, called Centroid Neural Network with State Dependence measure (CNN(SD)), for TMHMMs. The CNN(SD) uses a novel distance measure that discriminates between two observation densities in the HMM from the same state and those from different states. When compared with other conventional unsupervised neural networks, the CNN(SD) successfully allocates more code vectors to the regions where Gaussian Probability Density Function (GPDF) data of different states overlap each other while it allocates fewer code vectors to the regions where GPDF data are from the same states. Experiments and results on a Korean monophone data, the CNN(SD) shows improvements on the recognition accuracy over CNN and the traditional k-means algorithm.

The finger vein recognition system uses blood vessels inside the finger of an individual for identity verification. The public is in favor of a finger vein recognition system over conventional passwords or ID cards as the biometric technology is harder to forge, misplace, and share. In this study, the histogram of oriented gradients (HOG) features, which are robust against changes in illumination and position, are extracted from the finger vein for personal recognition. To further increase the amount of information that can be used for recognition, different instances of the finger vein, ranging from the index, middle, and ring finger are combined to form a multi-instance finger vein representation. This fusion approach is preferred since it can be performed without requiring additional sensors or feature extractors. To combine different instances of finger vein effectively, score level fusion is adopted to allow greater compatibility among the wide range of matches. Towards this end, two methods are proposed: Bayesian optimized support vector machine (SVM) score fusion (BSSF) and Bayesian optimized SVM based fusion (BSBF). The fusion results are incrementally improved by optimizing the hyperparameters of the HOG feature, SVM matcher, and the weighted sum of score level fusion using the Bayesian optimization approach. This is considered a kind of knowledge-based approach that takes into account the previous optimization attempts or trials to determine the next optimization trial, making it an efficient optimizer. By using stratified cross-validation in the training process, the proposed method is able to achieve the lowest EER of 0.48% and 0.22% for the SDUMLA-HMT dataset and UTFVP dataset, respectively.

In this dissertation, a novel approach of integrating template matching with statistical modeling is proposed to improve continuous speech recognition. Hidden Markov Modeling (HMMs) has been the dominant approach in statistical speech recognition since it provides a principled way of jointly modeling speech spectral variations and time dynamics. However, HMMs have the shortcoming of assuming the observations being independent within each state, which makes it ineffective in modeling the details of speech temporal evolutions that are important for characterizing nonstationary speech sounds. Template-based methods make comparisons between a test pattern and the templates derived from training data, and therefore they are able to capture speech dynamics and time correlation of speech frames better than HMM based methods. However, template matching requires large memory space and computational time since feature vectors of training data need to be stored in computer memory for access at the recognition stage, which is difficult in large vocabulary continuous speech recognition (LVCSR). Our proposed approach takes advantages of both statistical modeling and template matching, which overcomes the weakness of conventional template-based method and is feasible for LVCSR. We use multiple Gaussian Mixture Model (GMM) indices to represent each frame of speech templates, and define the template unit to be context-dependent phone segments (triphone context). We also use phonetic decision trees borrowed from those commonly used in HMMs to tie triphone templates and predict triphones unseen in training data. Two local distances, log likelihood ratio (LLR) and Kullback-Leibler (KL) divergence, are proposed for dynamic time warping (DTW) based template matching. In order to reduce computational complexity and storage space, we propose methods of minimum distance template selection (MDTS) and maximum log-likelihood template selection (MLTS), and investigate a template compression method on top of template selection to further improve recognition performance. The template based methods were used to rescore lattices generated by baseline HMMs on the tasks of TIMIT continuous phone recognition and teleheath LVCSR and experimental results demonstrated that the proposed approach of integrating template matching with statistical modeling significantly improved recognition performances over the HMM baselines. The template selection methods also provided significant recognition accuracy improvements over the HMM baseline while largely reducing the computation and storage complexities. When all templates or MDTS were used, using the LLR local distance obtained better recognition performance than the KL divergence local distance. For MLTS and template compression, KL divergence local distance provided better performance than the LLR local distance, and the template compression method made further improvements over KL based MLTS. Since the templates were constructed based on the GMM indices extracted from HMM baselines, we also validated the effectiveness of the proposed template methods based on enhanced HMM baselines. Experimental results showed that LLR based all template method was able to consistently improve TIMIT phone recognition accuracies based on four enhanced HMM baselines. Prosodic features such as duration, energy, and pitch can reflect longer span information of speech than conventional single frame vectors but they have commonly been ignored by HMMs. Template based methods provide possibilities to conveniently integrate prosodic features into speech recognition, which has not been well studied in the past. In this dissertation, we investigate combining template based methods with the speech prosodic features of duration, energy and pitch to further improve speech recognition accuracy. The scores of prosodic information were computed by a GMM based method and a non-parametric method, and the prosodic scores were combined with the acoustic scores in triphone template matching. Experimental results obtained on the telehealth task showed that prosodic information had positive effects on vowel sound recognition.

The central nervous system (CNS) plays an important role in regulation of human gait. Parkinson's disease (PD) is a common neurodegenerative disease that may cause neurophysiologic change in the CNS and as a result change the gait cycle duration (stride interval). This article used the Hidden Markov Model (HMM) with Gaussian Mixtures to separate the patients with PD from healthy subjects. The results showed that the performance of the HMM classifier in classifying the gait data corresponding to 16 healthy and 15 PD subjects is comparable to the results obtained from the least squares support vector machine (LS-SVM) classifier. In this study, the leave-one-out cross-validation method was used to evaluate the performance of each classifier. The HMM method could effectively separate the gait data in terms of stride interval obtained from healthy subjects and PD patients with an accuracy rate of 90.3 % . All in all, the results showed that the proposed method can be used for distinguishing PD patients from healthy subjects based on the gait data classification.

A method is presented for using hidden Markov models (HMMs) for the analysis of force, torque, and position signals from sensors in manufacturing machines. The HMM can detect the transitions between contact states and compute a measure of the task quality using a model of the task developed by the manufacturing engineer and optimized on training data. The HMM method has been evaluated in extensive experimentation with teleoperation and the results suggest even higher effectiveness in automation and manufacturing applications. >

Face recognition based on multi-class mapping of Fisher scores

Hidden Markov models (HMMs) are probabilistic functions of finite Markov chains, or, put in other words, state space models with finite state space. In this paper, we examine subspace estimation methods for HMMs whose output lies a finite set as well. In particular, we study the geometric structure arising from the nonminimality of the linear state space representation of HMMs, and consistency of a subspace algorithm arising from a certain factorization of the singular value decomposition of the estimated linear prediction matrix. For this algorithm, we show that the estimates of the transition and emission probability matrices are consistent up to a similarity transformation, and that the m-step linear predictor computed from the estimated system matrices is consistent, i.e., converges to the true optimal linear m-step predictor.

This paper presents a minimum generation error (MGE) training method for hidden Markov model (HMM) based prediction of articulatory movements when both text and audio inputs are given. In this method, MGE criterion is adopted to replace the maximum likelihood (ML) criterion to estimate model parameters for the unified acoustic-articulatory HMMs. Different from the MGE training for HMM-based acoustic speech synthesis, the generation error used here is defined as the distance between the generated and natural articulatory features. Experimental results show that our proposed method can improve the accuracy of articulatory movement prediction significantly. The average root mean square (RMS) error reduces from 1.002 mm to 0.913 mm on the test set.