Offline handwritten numeral recognition using orthogonal Gaussian mixture model

In the field of speaker recognition, the Gaussian mixture model with diagonal covariance matrices is a popular technique, in this way, it simplified model and reduced the amount of computation, but lost the correlation information between feature vectors, and then influenced the classification performance. In this paper, in order to compensate the correlation between feature elements, we proposed a novel method based on clustering transformation algorithm, we calculate the similarity between Gaussian components, and the cluster of same components will share one transformation matrix, thus multi-transformation matrices, together with weights and means vectors are obtained simultaneously by maximum likelihood estimation. Theory analysis and experimental results demonstrated that this proposed method can get a better balance between training speed and recognition rate, improve the performance of classifier and reduce the complexity and memory burden relatively.

Exploring similarity-based classification of larynx disorders from human voice

This paper introduced an approach to design and implement an embedded SoPC (System on a Programmable Chip) technique with Altera Nios II processor for real-time speech recognition system by developing hardware/software with minimum usage of resources (hardware components) and relatively small size software to reduce memory utilization. This is achieved by using Mel Frequency Cepstral Coefficients (MFCCs) technique as speech signal feature extraction (observation vector). Using Gaussian Mixture Model (GMM) to model the observation vector of voice information. Finally, this model passed to the Hidden Markov Model (HMM) as probabilistic model to process the GMM statistically to make decision on utterance words recognition, whether a single or composite, one or more syllable words (i.e. one, six, target). The total framework was implemented on Altera Cyclone II EP2C70F896C6N FPGA chip sitting on ALTERA DE2-70 Development Board. The utility software which are used as tools for design and development hardware/software are Quartus II 11.0sp1 (32-Bit) and Nios II 11.0spl IDE/C++ respectively. Each word model (template) stored as Transition Matrix, Diagonal Covariance Matrices, and Mean Vectors in the system memory. Each word model utilizes only 4.45Kbytes regardless of the spoken word length. Accuracy of the recognition words (digit/0 to digit/10) given 100% for the individual speaker. Training and recognition software has size (code + initialized data) equal to 312Kbytes.

Non-stationary feature fusion of face and palmprint multimodal biometrics

Summary The primary purpose of this paper is to propose empirical measures of the structural differences between two communities of plants or animals composed of the same species. Structure is defined to consist of; 1) the species in the community, 2) the pattern of interactions as represented by the covariance or correlation matrix of successive observations on each species, and 3) the mean abundances of each species in each of the two communities. Statistical tests are proposed for testing whether the covariance matrices and the vectors of mean densities for each community are equal and empirical measures of the differences between the covariance matrices and mean vectors are proposed. Given unequal covariance or correlation matrices the factor analysis model is used to derive empirical measures of the degree to which each variable of the ecosystem is responsible for the observed defferences in the pattern of interactions in each community. These tests and measures were applied to data gathered by Hunter (1966) on the abundances of six species of Drosophila censused monthly over a period of approximately two and a half years in two adjacent, but different habitats near Bogota, colombia. The two covariance matrices were significantly different indicating different patterns of interactions in the two Drosophila communities.

Gaussian mixture model (GMM) techniques are popular for speaker identification. Theoretically, each Gaussian function should have a full covariance matrix. However, the diagonal covariance matrix is usually used because the inverse of diagonal covariance matrix can be easily calculated via expectation maximization (EM) algorithm. This paper proposes a new probabilistic principal component analysis (PPCA) model for speaker identification. The full covariance of speaker’s data is considered. This model is originated from factor analysis theory. The probability distributions using PPCA are well defined. In particular, GMM and PPCA are found to be equivalent when using diagonal covariance matrix. In this study, we derive a novel PPCA model selection and establish models for different speakers. Applying PPCA model selection, we can dynamically determine the numbers of speech features and mixture components. Experiments show that PPCA achieves desirable speaker recognition performance with proper model regularization.

Sentence alignment using P-NNT and GMM

We present a discriminant measure that can be used to determine the model complexity in a speech recognition system. In the speech recognition process, given a test feature vector the conditional probability of the feature vector has to be obtained for several allophone (sub-phonetic units) classes using a Gaussian-mixture density model for each class. The Gaussian-mixture models are constructed from the training data belonging to the allophone classes, and the number of mixture components that are required to adequately model the PDF of each class is determined by using some simple rule of thumb-for instance the number of components has to be sufficient to model the data reasonably well but not so many as to overmodel the data. A typical example of the choice of the number is to make it proportional to the number of data samples. However, such methods may result in models that are sub-optimal as far as classification accuracy is concerned. We present a new discriminant measure that can be used to determine in an objective fashion, the number of Gaussians required to best model the PDF of an allophone class. We also present the results of experiments showing the improvement in recognition performance when the number of mixture components is chosen based on the discriminant measure as opposed to the rule of thumb. These results are presented both for the speaker-independent and speaker-adapted case.

We consider Gaussian mixture models in high dimensions, focusing on the twin tasks of detection and feature selection. Under sparsity assumptions on the difference in means, we derive minimax rates for the problems of testing and of variable selection. We find these rates to depend crucially on the knowledge of the covariance matrices and on whether the mixture is symmetric or not. We establish the performance of various procedures, including the top sparse eigenvalue of the sample covariance matrix (popular in the context of Sparse PCA), as well as new tests inspired by the normality tests of Malkovich and Afifi [J. Amer. Statist. Assoc. 68 (1973) 176–179].

Fuzzy C-Means (FCM) clustering and Gaussian Mixture Model (GMM) are two popular tools for data processing. In this study the unsupervised algorithm combining FCM clustering in the Kernel Space (KFCM) and GMM is presented. First, a trick is applied to the FCM algorithm. Then, the number of clusters is chosen automatically in the kernel space. On the basis of obtained starting parameters, i.e. number of mixture components, mean vector, covariance matrices and mixing proportion coefficients, the final GMM parameters are estimated. For this estimation the Expectation Maximization (EM) algorithm is used. The presented methodology - combination of KFCM and GMM methods named unsupervised cascade clustering - constitutes the basic step in Ewing's sarcoma segmentation. On this basis the voxels intensity values describing segmented tumour and surrounded healthy tissue are defined and fuzzy connectedness analysis is performed. The obtained mixture parameters estimation results are compared with the results obtained using two different methods described in literature.

Let Y = Xβ + ϵ be a general linear statistical model with co-variance matrix of the form , where δi, is a known diagonal matrix and σi is an unknown scalar. This form includes the I common linear regression model with uncorrelated observations, with or without replications; the random coefficients model of Hildreth and Houck (1968); and numerous mixed effects models after a suit-able diagonalizing transformation. The MINQUE of is derived by a simple, intuitive approach in which a weighted least squares criterion is applied to the vector of squared residuals. In addition to providing a different perspective, this approach has the advantage that estimation can be restricted to the known parameter space. The restricted estimates are always nonnegative, but still equal to the MINQUE values when the latter are also nonnegative. A hypothetical example is simulated in which the restricted and unrestricted estimates are compared. The practical significance for an iterative feedback procedure is discussed, and som...

On the accuracy of covariance matrix: Hessian versus Gauss–Newton methods in atmospheric remote sensing with infrared spectroscopy

Covariance matrices play an important role in many multivariate techniques and hence a good covariance estimation is crucial in this kind of analysis. In many applications a sparse covariance matrix is expected due to the nature of the data or for simple interpretation. Hard thresholding, soft thresholding, and generalized thresholding were therefore developed to this end. However, these estimators do not always yield well-conditioned covariance estimates. To have sparse and well-conditioned estimates, we propose doubly shrinkage estimators: shrinking small covariances towards zero and then shrinking covariance matrix towards a diagonal matrix. Additionally, a richness index is defined to evaluate how rich a covariance matrix is. According to our simulations, the richness index serves as a good indicator to choose relevant covariance estimator.

Estimating large covariance and precision (inverse covariance) matrices has become increasingly important in high dimensional statistics because of its wide applications. The estimation problem is challenging not only theoretically due to the constraint of its positive definiteness, but also computationally because of the curse of dimensionality. Many types of estimators have been proposed such as thresholding under the sparsity assumption of the target matrix, banding and tapering the sample covariance matrix. However, these estimators are not always guaranteed to be positive-definite, especially, for finite samples, and the sparsity assumption is rather restrictive. We propose a novel two-stage adaptive method based on the Cholesky decomposition of a general covariance matrix. By banding the precision matrix in the first stage and adapting the estimates to the second stage estimation, we develop a computationally efficient and statistically accurate method for estimating high dimensional precision matrices. We demonstrate the finite-sample performance of the proposed method by simulations from autoregressive, moving average, and long-range dependent processes. We illustrate its wide applicability by analyzing financial data such S&P 500 index and IBM stock returns, and electric power consumption of individual households. The theoretical properties of the proposed method are also investigated within a large class of covariance matrices.

In this paper, a set of feature vector normalization methods based on the minimum mean square error (MMSE) criterion and stereo data is presented. They include multi-environment model-based linear normalization (MEMLIN), polynomial MEMLIN (P-MEMLIN), multi-environment model-based histogram normalization (MEMHIN), and phoneme-dependent MEMLIN (PD-MEMLIN). Those methods model clean and noisy feature vector spaces using Gaussian mixture models (GMMs). The objective of the methods is to learn a transformation between clean and noisy feature vectors associated with each pair of clean and noisy model Gaussians. The direct approach to learn the transformation is by using stereo data; that is, noisy feature vectors and the corresponding clean feature vectors. In this paper, however, a nonstereo data based training procedure, is presented. The transformations can be modeled just like a bias vector (MEMLIN), or by using a first-order polynomial (P-MEMLIN) or a nonlinear function based on histogram equalization (MEMHIN). Further improvements are obtained by using phoneme-dependent bias vector transformation (PD-MEMLIN). In PD-MEMLIN, the clean and noisy feature vector spaces are split into several phonemes, and each of them is modeled as a GMM. Those methods achieve significant word error rate improvements over others that are based on similar targets. The experimental results using the SpeechDat Car database show an average improvement in word error rate greater than 68% in all cases compared to the baseline when using the original clean acoustic models, and up to 83% when training acoustic models on the new normalized feature space