Comparison of optimization methods for discriminative training criteria

We propose a method for sound monitoring, which enables one to selectively detect unexpected irregular sounds and ignore the other sounds, i.e., regular sounds. The proposed method is based on the generalized probabilistic descent (GPD) method, which was originally developed as a general concept for the discriminative design of pattern recognizers, and is referred to as minimum detection error (MDE) training. The formulation and implementation of MDE training are described in detail, and its utility is demonstrated in a task of detecting irregular events; more specifically, sounds due to the mis-operation of a tool in a noisy environment.

The authors developed a generalized probabilistic descent (GPD) method by extending the classical theory on adaptive training by Amari (1967). Their generalization makes it possible to treat dynamic patterns (of a variable duration or dimension) such as speech as well as static patterns (of a fixed duration or dimension), for pattern classification problems. The key ideas of GPD formulations include the embedding of time normalization and the incorporation of smooth classification error functions into the gradient search optimization objectives. As a result, a family of new discriminative training algorithms can be rigorously formulated for various kinds of classifier frameworks, including the popular dynamic time warping (DTW) and hidden Markov model (HMM). Experimental results are also provided to show the superiority of this new family of GPD-based, adaptive training algorithms for speech recognition. >

The goal of this communication is to present a weighted likelihood discriminant for minimum error shape classification. Different from traditional maximum likelihood (ML) methods in which classification is carried out based on probabilities from independent individual class models as is the case for general hidden Markov model (HMM) methods, our proposed method utilizes information from all classes to minimize classification error. Proposed approach uses a hidden Markov model as a curvature feature based 2D shape descriptor. In this contribution we present a generalized probabilistic descent (GPD) method to weight the curvature likelihoods to achieve a discriminant function with minimum classification error. In contrast with other approaches, a weighted likelihood discriminant function is introduced. We believe that our sound theory based implementation reduces classification error by combining hidden Markov model with generalized probabilistic descent theory. We show comparative results obtained with our approach and classic maximum-likelihood calculation for fighter planes in terms of classification accuracies

We propose to use data sampling in the extended Baum-Welch (EBW) algorithm for maximum mutual information (MMI) based estimation of speech acoustic models, and to randomize the configurations of the sampled training sets and aggregate the numerator and denominator sufficient statistics for improving model robustness. We further combine data sampling based ensemble acoustic modeling with the data sampling based EBW, forming a two-level data sampling mechanism for acoustic model training. We conducted experiments on a telehealth conversational speech recognition task, where the two-level data sampling mechanism gave a statistically significant, absolute word accuracy gain of 3.56% over the conventional MMI baseline, corresponding to a 19.44% relative word error rate reduction.

The authors suggest a linear discriminant function to complete the distance score instead of a conventional average distance. Several discriminative algorithms are proposed to learn the discriminant function. These include one heuristic method, two methods based on the error propagation algorithm, and one method based on the generalized probabilistic descent (GPD) algorithm. The authors study these methods in a speaker-independent speech recognition task involving utterances of the highly confusable English E-set. The results show that the best performance is obtained by using the GPD method, which achieved a 78.1% accuracy, compared to 67.6% with the traditional average method.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

This paper presents a minimum unit selection error (MUSE) training method for HMM-based unit selection speech synthesis system, which selects the optimal phone-sized unit sequence from the speech database by maximizing the combined likelihood of a group of trained HMMs. Under MUSE criterion, the weights and distribution parameters of these HMMs are estimated to minimize the number of different units between the selected phone sequences and the natural phone sequences for the training sentences. The optimization is realized by discriminative training using generalized probabilistic descent (GPD) algorithm. Results of our experiment show that this proposed method is able to improve the performance of the baseline system where model weights are set manually and distribution parameters are trained under maximum likelihood criterion.

This paper introduces an improved HMM (hidden Markov model) for low altitude acoustic target recognition. To overcome the limitation of the classical CDHMM (continuous density hidden Markov model) training algorithm and the generalization ability deficiency of existing discriminative learning methods, a new discriminative training method for estimating the CDHMM in acoustic target recognition is proposed based on the principle of maximizing the minimum relative separation margin. According to the definition of the relative margin, the new training criterion can be equation as a standard constrained minimax optimization problem. Then, the optimization problem can be solved by a GPD (generalized probabilistic descent) algorithm. The experimental results show that the performance of the algorithm is significantly improved compared with the former training method, which can effectively improve the recognition ability of the acoustic target recognition system.

This paper proposes a new approach, named discriminative metric design (DMD), to pattern recognition. DMD optimizes discriminant functions with the minimum classification error/generalized probabilistic descent method (MCE/GPD) such that intrinsic features of each pattern class can be represented efficiently. Resulting metrics accordingly lead to robust recognizers. The DMD is quite general. Several existing methods, such as learning vector quantization and the continuous hidden Markov model, are defined as its special cases. The paper specially elaborates the DMD formulation for the quadratic discriminant function, and clearly demonstrates its utility in a speaker-independent Japanese vowel recognition task.

The current trend toward increasing database applications and end-user computing brings about the need for effective training and support tools that enable end users to perform logical database designs. This paper presents a unique comparison of a network-based data model (the extended entity relationship (EER)) and a hierarchical-based data model (the structured object model (SOM)) and their associated logical database design methods, LRDM (EER-based) and the SOM method, in end-user logical database design. The SOM method, based on an object-oriented knowledge representation scheme, is a new structured method designed to be less susceptible to designer errors and yet capable of maintaining ease of understanding and use. The Logical Relational Design Methodology (LRDM), based on the EER model, shares some common characteristics with the SOM method. In the analytic comparison the basic components and constructs of EER and SOM, as well as their use in the logical design process, are compared to derive a qualitative assessment of the two methods. Two experiments using novice database designers were conducted to examine the effectiveness of each method in leading to accurate logical designs, the time needed for task completion and the improvement in performance over a short training period. To confirm the findings from other studies on logical database design using non-graphic methods versus using graphic methods, the normalization method was included in one of the experiments. The results showed that end users performed better with graphic methods and also preferred them to the non-graphic (normalization) method. Furthermore, users learned and improved in a short period using LRDM (EER-based). However, users of the SOM method generated more accurate designs, were able to identify more complex relationships, and had more significant improvements during the training period than did users of LRDM (EER-based).

We present a novel HMM based approach to simultaneous segmentation of vessel walls in Lymphatic confocal images. The vessel borders are parameterized using RBFs to minimize the number of tracking points. The proposed method tracks the hidden states that indicate border locations for both the inner and outer walls. The observation for both borders is obtained using edge-based features from steerable filters. Two separate Gaussian probability distributions for the vessel borders and background are used to infer the emission probability, and the transmission probability is learned using a Baum-Welch algorithm. We transform the segmentation problem into a minimization of an s-excess graph cost, with each node in the graph corresponding to a hidden state and the weight for each node being defined by its emission probability. We define the inter-relations between neighboring nodes based on the transmission probability. We present both qualitative and quantitative analysis in comparison to the popular Viterbi algorithm.

We present a novel method to segment the lymph vessel wall in confocal microscopy images using Optimal Surface Segmentation (OSS) and hidden Markov Models (HMM). OSS is used to preform a pre-segmentation on the images, to act as the initial state for the HMM. We utilize a steerable filter to determine edge based filters for both of these segmentations, and use these features to build Gaussian probability distributions for both the vessel walls and the background. From this we infer the emission probability for the HMM, and the transmission probability is learned using a Baum-Welch algorithm. We transform the segmentation problem into one of cost minimization, with each node in the graph corresponding to one state, and the weight for each node being defined using its emission probability. We define the inter-relations between neighboring nodes using the transmission probability. Having constructed the problem, it is solved using the Viterbi algorithm, allowing the vessel to be reconstructed. The optimal solution can be found in polynomial time. We present qualitative and quantitative analysis to show the performance of the proposed method.

Accurate and computationally efficient means of recognizing continuous speech has been a subject of research in recent years. This paper reports the development of a small vocabulary, speaker independent continuous Malayalam speech recognition system based on Hidden Markov Models (HMMs). Continuous density HMM, which is used in this work to model phonemes, represents the general case where the observation probability density functions (pdfs) are continuous. The observation pdf is approximated using a Gaussian mixture density. Mel-frequency Cepstral Coefficients (MFCC) method is used to extract acoustic features from the input signal. To represent temporal variations in the speech signal, the first and second order derivatives of MFCC are added to the set of static parameters. The training and decoding are performed by the Baum-Welch and Viterbi algorithms respectively. The corpus contains naturally and continuously spoken sentences with multiple pronunciations and speaker variations. On evaluation the proposed system has produced promising results with 94.67% word accuracy and 93.33% sentence correct.

Coverage interval estimation of the measurement of Gross Heat of Combustion of fuel by bomb calorimetry: Comparison of ISO GUM and adaptive Monte Carlo method

In solving classification task of data mining the traditional polynomial neural network (PNN) algorithm takes longer time while generating complex mathematical models. PNN algorithm takes the combinations two or three inputs to generates one partial description (PD) for the next layer. The output of the PDs becomes the input to the next layer. The number of PDs in each layer increases very fast, which consume lot of time for evaluation of the coefficients of the PDs, consume huge memory and increase complexity of the model. We propose simplified polynomial neural network (SPNN) for the task of classification. PDs for a single layer of the PNN model are developed. The outputs of these PDs along with the original inputs from the dataset are fed to a single perception model of artificial neural network (ANN) without any hidden layers. The ANN is trained with gradient descent method as well as with particle swarm optimization (PSO) technique. The results of both techniques for training are considered for the comparison of the performance. Simulation and result shows that the performance of SPNN is better than PNN model.

This paper describes how we can combine our previously proposed fast extended Baum-Welch algorithm and generalized discriminative feature transformation to achieve single pass discriminative training, which we only process the data once. Compared to the state of the art training procedure, which uses feature space maximum mutual information (fMMI) and boosted maximum mutual information (BMMI), our proposed training procedure can achieve around 80% of the improvement available from discriminative training. We also show that if we are allowed to process the data twice, it is possible to achieve almost all of the improvement. We evaluate different training procedures on various large scale tasks using Iraqi and modern standard Arabic speech recognition systems.