Hidden Markov Model Based Weighted Likelihood Discriminant for Minimum Error Shape Classification

A Poisson ridge regression estimator

This paper presents the design and implementation of text-independent speaker verification. We apply the maximum model distance (MMD) algorithm to the Gaussian mixture model (GMM) training. The traditional maximum likelihood (ML) method only utilizes the labeled utterances for each speaker model, which probably leads to a local optimization solution. By maximizing the model distance between the target and competing speakers, MMD could add the discriminative capability into the training procedure and then improve the verification performance. Based on the TIMIT corpus, we designed the verification experiments and the results show that the equal error rate (EER) could be reduced greatly compared with the traditional ML method.

Deconvolution techniques provide efficient implementations for super-resolution imaging for forward-looking scanning radar. However, deconvolution is normally an ill-posed problem, and the solution is extremely sensitive to noise. From a statistical perspective, maximum likelihood (ML) methods are able to condition the ill-posed problem into a well-posed one. Nevertheless, traditional ML methods only consider the amplitude of the echo and by ignoring the phase that do not adequately model the radar imaging system. In this letter, an I/Q-channel modeling ML method is proposed for forward-looking scanning radar. First, the probability model of the echo is deduced by jointly considering noise in the I and Q channels. Then, a probability density function of the received data is deduced and used to formulate the likelihood function. Finally, the targets can be precisely estimated by maximizing this likelihood function. The results of simulations and experiments are provided to illustrate the effectiveness of the proposed method.

The beta regression model is commonly used when analyzing data that come in the form of rates or percentages. However, a problem that may encounter when analyzing these kinds of data that has not been investigated for this model is the multicollinearity problem. It is well known that the maximum likelihood (ML) method is very sensitive to high inter-correlation among the explanatory variables. Therefore, this paper proposes some ridge estimators for the beta regression model to remedy the problem of instability of the traditional ML method and increase the efficiency of estimation. The performance of ridge estimators is compared to the ML estimator through the mean squared error (MSE) and the mean absolute error (MAE) criteria by conducting a Monte-Carlo simulation study and through an empirical application. According to the simulation and application results, the proposed estimators outperform the ML estimator in terms of MSE and MAE.

We propose a super-resolution resolution algorithm on the basis of maximum likelihood (ML) method and edge-orient diffusion. By using Hammerseley-Clifford theorem, an image field assumed to be a Markov random field is Gibbs distributed. An edge-orient diffusion function is introduced and employed in the Gibbs prior. According to Bayesian theorem, the solution to the maximum likelihood function is equal to that to maximum a posterior function. Therefore we incorporate ML with a prior distributed function. Experimental results illustrate that our method has a powerful super-resolution restoration performance. Compared with traditional ML method, our approach can not only obtain super-resolution images, but also eliminate noise artifacts effectively without smoothing edges.

The minimum classification error (MCE) framework for discriminative training is a simple and general formalism for directly optimizing recognition accuracy in pattern recognition problems. The framework applies directly to the optimization of hidden Markov models (HMMs) used for speech recognition problems. However, few if any studies have reported results for the application of MCE training to large-vocabulary, continuous-speech recognition tasks. This article reports significant gains in recognition performance and model compactness as a result of discriminative training based on MCE training applied to HMMs, in the context of three challenging large-vocabulary (up to 100 k word) speech recognition tasks: the Corpus of Spontaneous Japanese lecture speech transcription task, a telephone-based name recognition task, and the MIT Jupiter telephone-based conversational weather information task. On these tasks, starting from maximum likelihood (ML) baselines, MCE training yielded relative reductions in word error ranging from 7% to 20%. Furthermore, this paper evaluates the use of different methods for optimizing the MCE criterion function, as well as the use of precomputed recognition lattices to speed up training. An overview of the MCE framework is given, with an emphasis on practical implementation issues

The authors developed a generalized probabilistic descent (GPD) method by extending the classical theory on adaptive training by Amari (1967). Their generalization makes it possible to treat dynamic patterns (of a variable duration or dimension) such as speech as well as static patterns (of a fixed duration or dimension), for pattern classification problems. The key ideas of GPD formulations include the embedding of time normalization and the incorporation of smooth classification error functions into the gradient search optimization objectives. As a result, a family of new discriminative training algorithms can be rigorously formulated for various kinds of classifier frameworks, including the popular dynamic time warping (DTW) and hidden Markov model (HMM). Experimental results are also provided to show the superiority of this new family of GPD-based, adaptive training algorithms for speech recognition. >

Spectral information of ground objects target in remote sensing image is complex, more noise, and highly nonlinear. It makes traditional data processing method no longer significant, effective, and efficient. The BP neural network classification-recognition method provides a more ideal solution. Using the TM remote sensing images as the example, this paper experimented the application of the BP neural network to the remote sensing image classification and recognition. Results showed that the classification precision of cultivated land was very low for both the BP neural network and traditional maximum likelihood methods because the spectrum difference between the new cultivated land and the bare land having low plant covered in this area was not significant. Maximum likelihood method wrongly regarded the bare land which had higher soil moisture content by lakeshore as water body. Except the grassland, the classification effect of the BP neural network was superior to maximum likelihood method. The overall classification accuracy by the BP neural network reached 81.79 %; however, the one by the maximum likelihood method was 79.08 %, indicating that the BP neural network classification and recognition was superior to the traditional maximum likelihood method.KeywordsHide LayerMaximum Likelihood MethodBare LandHide Layer NeuronGround ObjectThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

This paper proposes a method of facial expression recognition based on Zernike moments and the minimum classification error (MCE) based hidden Markov model (HMM). In the feature extraction of face, the method of Zernike moments feature extraction based on local feature regions is adopted. First, eyes and mouths are segmented from the facial expression image and Zernike moments feature vectors of eyes and mouth are extracted. In the classification of expression, using MCE criteria is adopted to improve the performance of HMM. The results show that the performance of expression recognition system can be improved to some extent by using Zernike moments and MCE criterion.

This paper considers both the garbage modelling and the discriminative training of a speech recognition system consisting of a multilayer perceptron (MLP) network and hidden Markov models (HMMs). The string level training of the system is based on Minimum Classification Error (MCE) algorithm using especially the ideas recently proposed by Reichl et al. In order to improve the MCE training scheme, they defined a derivative for MCE cost function putting heavier weight to the misclassified utterances than the traditional translated sigmoidal. In this paper, we give a definition of a new cost function, whose derivative has similar properties as the one proposed by Reichl et al. The new and the sigmoidal cost function are compared with the string level (MCE) algorithm. The string recognition rates show that both methods achieve equal performances, but the convergence of the MCE algorithm is a bit faster with the new cost function. Moreover, the performance of two garbage models, the nth best and an interpolative garbage model, are also compared in this paper. The latter one is an approximation of the former one requiring less operations, but achieving comparable performance. The recognition system achieved 93.32% accuracy for test set at best. The test set contained 29188 Finnish digit strings from two environments.

A new look at discriminative training for hidden Markov models

In this paper we compare three frameworks for discriminative training of continuous-density hidden Markov models (CD-HMMs). Specifically, we compare two popular frameworks, based on conditional maximum likelihood (CML) and minimum classification error (MCE), to a new framework based on margin maximization. Unlike CML and MCE, our formulation of large margin training explicitly penalizes incorrect decodings by an amount proportional to the number of mislabeled hidden states. It also leads to a convex optimization over the parameter space of CD-HMMs, thus avoiding the problem of spurious local minima. We used discriminatively trained CD-HMMs from all three frameworks to build phonetic recognizers on the TIMIT speech corpus. The different recognizers employed exactly the same acoustic front end and hidden state space, thus enabling us to isolate the effect of different cost functions, parameterizations, and numerical optimizations. Experimentally, we find that our framework for large margin training yields significantly lower error rates than both CML and MCE training.

In this paper, we present the theoretical framework of minimum classification error (MCE) training of generalized linear classifiers for text classification. We show that many important text classifiers, either probabilistic or non-probabilistic, can be unified under this framework, and the proposed MCE classifier training approach can be applied to improve the classifier performance. In addition, we describe an effective MCE classifier training algorithm that uses AdaBoost to generate alternative initial classifiers, as opposed to combining multiple classifiers as it is typically used. This method is applied to MCE classifier training to overcome local minimums in optimal classifier parameter search, utilizing the fact that the family of generalized linear classifiers is closed under AdaBoost. Moreover, we extend the loss function in MCE training to incorporate training sample prior distributions to compensate the imbalanced training data distribution in each category. Experimental studies are performed on the text classification tasks, and the significant classification error reductions of 25% - 55% are observed.

We investigate incremental word learning with few training examples in a Hidden Markov Model (HMM) framework suitable for an interactive learning scenario with little prior knowledge. When using only a few training examples the initialization of the models is a crucial step. In the bootstrapping approach proposed, an unsupervised initialization of the parameters is performed, followed by the retraining and construction of a new HMM using multiple sequence alignment (MSA). Finally we analyze discriminative training techniques to increase the separability of the classes using minimum classification error (MCE). Recognition results are reported on isolated digits taken from the TIDIGITS database.

A hidden Markov model (HMM) – based methodology for simultaneous design of extraction and classification stages is presented. Such a methodology is based on the minimum classification error (MCE) algorithm. The feature extraction is model state – dependent and is optimized using the same criterion of parameter estimation of the HMM. Validation is carried out over an automatic detection of pathological voices problem. The result shows that the MCE training improves the accuracy against the classical maximum likelihood training. The proposed methodology diminished the similarity between models of different classes and improves the performance system.