Discriminative analysis of distortion sequences in speech recognition

The authors developed a generalized probabilistic descent (GPD) method by extending the classical theory on adaptive training by Amari (1967). Their generalization makes it possible to treat dynamic patterns (of a variable duration or dimension) such as speech as well as static patterns (of a fixed duration or dimension), for pattern classification problems. The key ideas of GPD formulations include the embedding of time normalization and the incorporation of smooth classification error functions into the gradient search optimization objectives. As a result, a family of new discriminative training algorithms can be rigorously formulated for various kinds of classifier frameworks, including the popular dynamic time warping (DTW) and hidden Markov model (HMM). Experimental results are also provided to show the superiority of this new family of GPD-based, adaptive training algorithms for speech recognition. >

In this work we compare two parameter optimization techniques for discriminative training using the MMI criterion: the extended Baum-Welch (EBW) algorithm and the generalized probabilistic descent (GPD) method. Using Gaussian emission densities we found special expressions for the step sizes in GPD, leading to reestimation formula very similar to those derived for the EBW algorithm. Results were produced for both the TI digitstring and the SieTill corpus for continuously spoken American English and German digitstrings. The results for both techniques do not show significant differences. This experimental results support the strong link between EBW and GPD as expected from the analytic comparison.

The goal of this communication is to present a weighted likelihood discriminant for minimum error shape classification. Different from traditional maximum likelihood (ML) methods in which classification is carried out based on probabilities from independent individual class models as is the case for general hidden Markov model (HMM) methods, our proposed method utilizes information from all classes to minimize classification error. Proposed approach uses a hidden Markov model as a curvature feature based 2D shape descriptor. In this contribution we present a generalized probabilistic descent (GPD) method to weight the curvature likelihoods to achieve a discriminant function with minimum classification error. In contrast with other approaches, a weighted likelihood discriminant function is introduced. We believe that our sound theory based implementation reduces classification error by combining hidden Markov model with generalized probabilistic descent theory. We show comparative results obtained with our approach and classic maximum-likelihood calculation for fighter planes in terms of classification accuracies

This paper proposes a new mathematical programming approach to represent the dimensions of the discriminant space for the multiple-group classification problem. Few papers have investigated generalizations of two-group mathematical programming approaches for the classification of multiple groups. While several papers have proposed mathematical programming models for separating groups of observations, the issue of considering the classification problem by finding discriminant linear functions to describe the groups in fewer dimensions has not been addressed. The new mathematical programming approach proposed in this paper first solves the multiple-group problem using a single discriminant function, which essentially represents the separation of the groups in one dimension. Then the multiple-group problem is successively solved using single discriminant functions with the requirement that successive linear discriminant functions have a sample covariance equal to zero. An algorithm is proposed to classify observations from multiple groups using the linear discriminant functions from the mathematical programming approach in a reduced number of dimensions.

This paper presents a minimum unit selection error (MUSE) training method for HMM-based unit selection speech synthesis system, which selects the optimal phone-sized unit sequence from the speech database by maximizing the combined likelihood of a group of trained HMMs. Under MUSE criterion, the weights and distribution parameters of these HMMs are estimated to minimize the number of different units between the selected phone sequences and the natural phone sequences for the training sentences. The optimization is realized by discriminative training using generalized probabilistic descent (GPD) algorithm. Results of our experiment show that this proposed method is able to improve the performance of the baseline system where model weights are set manually and distribution parameters are trained under maximum likelihood criterion.

Fisher proposes Fisher’s linear discriminant function (LDF) based on maximization of variance ratio. If data satisfies Fisher’s assumption, the same LDF is easily derived by variance-covariance matrix. If variance-covariance matrices of two classes are not same, quadratic discriminant function (QDF) is proposed. These discriminant functions have three problems. First, all discriminant functions except for Revised IP-OLDF can’t discriminate the cases xi on the discriminant hyper plane correctly (the unresolved problem of discriminant analysis). Only Revised IP-OLDF can solve this problem in theoretically. Second, LDF and QDF can’t recognize linear separable data in many cases, and the numbers of misclassifications (NMs) of these functions are usually higher than logistic regression that is used in many users. Third, these are not obtained if the values of some independent variable are constant, because inverse matrix can’t be calculated. JMP prepare the regularized discriminant function if QDF can’t analyze the difficult data. These facts mean that LDF and QDF shouldn’t be used for an important discrimination. On the contrary, Revised IP-OLDF based on Minimum Number of Misclassifications (MNM) criterion resolves these problems. In addition to this, means of error rates of Revised IP-OLDF are less than those of LDF and logistic regression and S-SVM by 100-fold cross validation in many cases.

The decision tree-based classification is a popular approach for pattern recognition and data mining. Most decision tree induction methods assume training data being present at one central location. Given the growth in distributed databases at geographically dispersed locations, the methods for decision tree induction in distributed settings are gaining importance. This paper extends two well-known decision tree methods for centralized data to distributed data settings. The first method is an extension of CHAID algorithm and generates single feature based multi-way split decision trees. The second method is based on Fisher's linear discriminant (FLO) function and generates multifeature binary trees. Both methods aim to generate compact trees and are able to handle multiple classes. The suggested extensions for distributed environment are compared to their centralized counterparts and also to each other. Theoretical analysis and experimental tests demonstrate the effectiveness of the extensions. In addition, the side-by-side comparison highlights the advantages and deficiencies of these methods under different settings of the distribution environments.

We propose a new algorithm for the allocation of an individual to one of several possible groups or populations. The algorithm enables us to define a finite partition over the sample space, based on distance function. This partition is used, jointly with the application of a standard Bayesian decision rule, to allocate individuals to the populations. The algorithm also provides a measure of the allocation confidence for each individual, in a similar manner to that of logistic regression. The error rates for classification are also computed using the leave-one-out method. Results are compared with those obtained with other discriminant analysis techniques previously reported: Fisher's linear discriminant function, the quadratic discriminant function, logistic discrimination, and others.

We propose a method for sound monitoring, which enables one to selectively detect unexpected irregular sounds and ignore the other sounds, i.e., regular sounds. The proposed method is based on the generalized probabilistic descent (GPD) method, which was originally developed as a general concept for the discriminative design of pattern recognizers, and is referred to as minimum detection error (MDE) training. The formulation and implementation of MDE training are described in detail, and its utility is demonstrated in a task of detecting irregular events; more specifically, sounds due to the mis-operation of a tool in a noisy environment.

This paper introduces an improved HMM (hidden Markov model) for low altitude acoustic target recognition. To overcome the limitation of the classical CDHMM (continuous density hidden Markov model) training algorithm and the generalization ability deficiency of existing discriminative learning methods, a new discriminative training method for estimating the CDHMM in acoustic target recognition is proposed based on the principle of maximizing the minimum relative separation margin. According to the definition of the relative margin, the new training criterion can be equation as a standard constrained minimax optimization problem. Then, the optimization problem can be solved by a GPD (generalized probabilistic descent) algorithm. The experimental results show that the performance of the algorithm is significantly improved compared with the former training method, which can effectively improve the recognition ability of the acoustic target recognition system.

Mexico is adopting the competence-based model for education improvement. One of the major problems is to quantify the results of assessment to provide a grade considering subjective data. It commonly results in the assignation of an arbitrary grade which is estimated by the experience of the teacher using the evidence from the students. This paper presents a fuzzy system for calculating the grade assignment at undergraduate courses considering the student competences. The obtained results are compared against the grades that are calculated with the traditional average method and those obtained with the pass-fail checklists method.

Chapter 4 - Comparing samples from two populations: The discriminant function

In recent years, speech processing resides a major application in the domain of signal processing. Due to the audibility loss of some speech signals, people with hearing impairment have difficulty in understanding speech, which reintroduces a crucial role in speech recognition. Automatic Speech Recognition (ASR) development is a major challenge in research in the case of noise, domain, vocabulary size, and language and speaker variability. Speech recognition system design needs careful attention to challenges or issues like performance and database evaluation, feature extraction methods, speech representations and speech classes. In this paper, HDF-DNN model has been proposed with the hybridization of discriminant fuzzy function and deep neural network for speech recognition. Initially, the speech signals are pre-processed to eliminate the unwanted noise and the features are extracted using Mel Frequency Cepstral Coefficient (MFCC). A hybrid Deep Neural Network and Discriminant Fuzzy Logic is used for assisting hearing-impaired listeners with enhanced speech intelligibility. Both DNN and DF have some problems with parameters to address this problem, Enhanced Modularity function-based Bat Algorithm (EMBA) is used as a powerful optimization tool. The experimental results show that the proposed automatic speech recognition-based hybrid deep learning model is effectively-identifies speech recognition more than the MFCC-CNN, CSVM and Deep auto encoder techniques. The proposed method improves the overall accuracy of 8.31%, 9.71% and 10.25% better than, MFCC-CNN, CSVM and Deep auto encoder respectively.

Revised IP-OLDF (optimal linear discriminant function by integer programming) is a linear discriminant function to minimize the number of misclassifications (NM) of training samples by integer programming (IP). However, IP requires large computation (CPU) time. In this paper, it is proposed how to reduce CPU time by using linear programming (LP). In the first phase, Revised LP-OLDF is applied to all cases, and all cases are categorized into two groups: those that are classified correctly or those that are not classified by support vectors (SVs). In the second phase, Revised IP-OLDF is applied to the misclassified cases by SVs. This method is called Revised IPLP-OLDF.In this research, it is evaluated whether NM of Revised IPLP-OLDF is good estimate of the minimum number of misclassifications (MNM) by Revised IP-OLDF. Four kinds of the real data—Iris data, Swiss bank note data, student data, and CPD data—are used as training samples. Four kinds of 20,000 re-sampling cases generated from these data are used as the evaluation samples. There are a total of 149 models of all combinations of independent variables by these data. NMs and CPU times of the 149 models are compared with Revised IPLP-OLDF and Revised IP-OLDF. The following results are obtained: 1) Revised IPLP-OLDF significantly improves CPU time. 2) In the case of training samples, all 149 NMs of Revised IPLP-OLDF are equal to the MNM of Revised IP-OLDF. 3) In the case of evaluation samples, most NMs of Revised IPLP-OLDF are equal to NM of Revised IP-OLDF. 4) Generalization abilities of both discriminant functions are concluded to be high, because the difference between the error rates of training and evaluation samples are almost within 2%. Therefore, Revised IPLP-OLDF is recommended for the analysis of big data instead of Revised IP-OLDF. Next, Revised IPLP-OLDF is compared with LDF and logistic regression by 100-fold cross validation using 100 re-sampling samples. Means of error rates of Revised IPLP-OLDF are remarkable fewer than those of LDF and logistic regression.

Estimation of the multivariate logistic risk function: A comparison of the discriminant function and maximum likelihood approaches