A combination between VQ and covariance matrices for speaker recognition

In this study, different vector quantization methods are used to compress EMG signals. EMG is a versatile biosignal used in many fields of medical research and its compression is necessary for long-term recordings and narrow-band wireless transmission. The quality of lossy compression-decompression process of a biosignal has to be evaluated with the fidelity of essential medical parameters in mind. In this study, the analysis is focused on the preservation of primarily used spectral variables, namely mean frequency and median frequency. Modified versions of the basic vector quantization, classified vector quantization with two codebooks, classified vector quantization with three codebooks, classified vector quantization with four codebooks, gain-shape vector quantization with unquantized gain value, gain-shape vector quantization with eight gain values, and gain-shape vector quantization with four gain values have been implemented and tested with the segment length of 8 samples and the codebook sizes of 32 codevectors and 64 codevectors. The gain values, used both for criteria of codebook selection in classified vector quantization, and as product code in gain-shape vector quantization, was calculated as a sum of absolute segment values. The gain-shape method with eight quantized gain values shows best results in terms of the compression ratio. The general results reveal systematic bias up to 17.34 Hz in the mean frequency value, but a linear model for correcting the error is presented

In this paper a novel algorithm for Gaussian Selection (GS) of mixtures used in a continuous speech recognition system is presented. The system is based on hidden Markov models (HMM), using Gaussian mixtures with full covariance matrices as output distributions. The purpose of Gaussian selection is to increase the speed of a speech recognition system, without degrading the recognition accuracy. The basic idea is to form hyper-mixtures by clustering close mixtures into a single group by means of Vector Quantization (VQ) and assigning it unique Gaussian parameters for estimation. In the decoding process only those hyper-mixtures which are above a designated threshold are selected, and only mixtures belonging to them are evaluated, improving computational efficiency. There is no problem with the clustering and evaluation if overlaps between the mixtures are small, and their variances are of the same range. However, in real case, there are numerous models which do not fit this profile. A Gaussian selection scheme proposed in this paper addresses this problem. For that purpose, beside the clustering algorithm, it also incorporates an algorithm for mixture grouping. The particular mixture is assigned to a group from the predefined set of groups, based on a value aggregated from eigenvalues of the covariance matrix of that mixture using Ordered Weighted Averaging operators (OWA). After the grouping of mixtures is carried out, Gaussian mixture clustering is performed on each group separately.

In this paper we present different approaches to structuring covariance matrices within Statistical classifiers. This is motivated by the fact that the use of full covariance matrices is infeasible in many applications. On the one hand, this is due to the high number of model Parameters that have to be estimated, on the other hand the computational complexity of a classifier based on full covariance matrices is very high. We propose the use of diagonal and band-matrices to replace full covariance matrices and we also show that computation of tangent distance is equivalent to using a structured covariance matrix within a Statistical classifier.

Summary The primary purpose of this paper is to propose empirical measures of the structural differences between two communities of plants or animals composed of the same species. Structure is defined to consist of; 1) the species in the community, 2) the pattern of interactions as represented by the covariance or correlation matrix of successive observations on each species, and 3) the mean abundances of each species in each of the two communities. Statistical tests are proposed for testing whether the covariance matrices and the vectors of mean densities for each community are equal and empirical measures of the differences between the covariance matrices and mean vectors are proposed. Given unequal covariance or correlation matrices the factor analysis model is used to derive empirical measures of the degree to which each variable of the ecosystem is responsible for the observed defferences in the pattern of interactions in each community. These tests and measures were applied to data gathered by Hunter (1966) on the abundances of six species of Drosophila censused monthly over a period of approximately two and a half years in two adjacent, but different habitats near Bogota, colombia. The two covariance matrices were significantly different indicating different patterns of interactions in the two Drosophila communities.

In this paper, we apply orthogonally equivariant spatial sign covariance matrices as well as their affine equivariant counterparts in principal component analysis. The influence functions and asymptotic covariance matrices of eigenvectors based on robust covariance estimators are derived in order to compare the robustness and efficiency properties. We show in particular that the estimators that use pairwise differences of the observed data have very good efficiency properties, providing practical robust alternatives to classical sample covariance matrix based methods.

We examine the performance of several neural network vector quantization (VQ) methods on image coding. The VQ methods we look at are Kohonen's Self-Organizing Feature Map (KSOFM), Frequency-Sensitive Competitive Learning (FSCL), and Self-Creating and Organizing Neural Network (SCONN). We also look at variations of these algorithms by combining different methods. Our simulation results show that the best performance is achieved by the SCONN and the combination KSOFM and FSCL.

Estimating large covariance and precision (inverse covariance) matrices has become increasingly important in high dimensional statistics because of its wide applications. The estimation problem is challenging not only theoretically due to the constraint of its positive definiteness, but also computationally because of the curse of dimensionality. Many types of estimators have been proposed such as thresholding under the sparsity assumption of the target matrix, banding and tapering the sample covariance matrix. However, these estimators are not always guaranteed to be positive-definite, especially, for finite samples, and the sparsity assumption is rather restrictive. We propose a novel two-stage adaptive method based on the Cholesky decomposition of a general covariance matrix. By banding the precision matrix in the first stage and adapting the estimates to the second stage estimation, we develop a computationally efficient and statistically accurate method for estimating high dimensional precision matrices. We demonstrate the finite-sample performance of the proposed method by simulations from autoregressive, moving average, and long-range dependent processes. We illustrate its wide applicability by analyzing financial data such S&P 500 index and IBM stock returns, and electric power consumption of individual households. The theoretical properties of the proposed method are also investigated within a large class of covariance matrices.

Vector quantization is an essential tool for tasks involving large scale data, for example, large scale similarity search, which is crucial for content-based information retrieval and analysis. In this paper, we propose a novel vector quantization framework that iteratively minimizes quantization error. First, we provide a detailed review on a relevant vector quantization method named residual vector quantization (RVQ). Next, we propose generalized residual vector quantization (GRVQ) to further improve over RVQ. Many vector quantization methods can be viewed as special cases of our proposed method. To enable GRVQ on billion scale data, we introduce a nonexhaustive search scheme named aggregating tree (A-Tree) for high dimensional data that uses GRVQ encodings to build a radix tree and perform the nearest neighbor search by beam search. To search accurately and efficiently, VQ-encodings should satisfy locally aggregating encoding criterion: For any node of the corresponding A-Tree, neighboring vectors should aggregate in fewer subtrees to make beam search efficient. We show that the proposed GRVQ encodings best satisfy the suggested criterion, and the joint use of GRVQ and A-Tree shows significantly better performances on billion scale datasets. Our methods are validated on several standard benchmark datasets. Experimental results and empirical analysis show the superior efficiency and effectiveness of our proposed methods compared to the state-of-the-art for large scale search.

The generalized Lloyd algorithm (GLA), better known as the Linde-Buzo-Gray (LBG) algorithm, is the most widely used technique in classical vector quantization (VQ) for speech or image signal compression. However, the encoding complexity of the algorithm grows exponentially with the product of coding rate and vector dimension, which prohibits applying the technique to tasks with moderate to large encoding rates or vector dimensions. The authors present a new VQ scheme which overcomes the successive search coding computation of traditional techniques by using a quasi-parallel mapping technique and makes VQ practical for higher encoding rates and/or vector dimensions. Neural computing techniques are used to implement parallel encoding and decoding mappings in VQ and the developed algorithm was applied to quantizing Gauss-Markov processes and artificial data sources. Comparisons of performance with the LBG algorithm are given. >

Upcoming and existing large-scale surveys of galaxies require accurate theoretical predictions of the dark matter clustering statistics for thousands of mock galaxy catalogs. We demonstrate that this goal can be achieve with our new Parallel Particle-Mesh (PM) Nbody code (PPM-GLAM) at a very low computational cost. We run about 15,000 simulations with ~2 billion particles that provide ~1% accuracy of the dark matter power spectra P(k) for wave-numbers up to k~ 1h/Mpc. Using this large data-set we study the power spectrum covariance matrix, the stepping stone for producing mock catalogs. In contrast to many previous analytical and numerical results, we find that the covariance matrix normalised to the power spectrum C(k,k')/P(k)P(k') has a complex structure of non-diagonal components. It has an upturn at small k, followed by a minimum at k=0.1-0.2h/Mpc. It also has a maximum at k=0.5-0.6h/Mpc. The normalised covariance matrix strongly evolves with redshift: C(k,k')~delta(t)^alpha P(k)P(k'), where delta is the linear growth factor and alpha ~ 1-1.25, which indicates that the covariance matrix depends on cosmological parameters. We also show that waves longer than 1Gpc have very little impact on the power spectrum and covariance matrix. This significantly reduces the computational costs and complexity of theoretical predictions: relatively small volume ~ (1Gpc)^3 simulations capture the necessary properties of dark matter clustering statistics. All the power spectra obtained from many thousands of our simulations are publicly available.

Estimation of the high-dimensional banded covariance matrix is widely used in multivariate statistical analysis. To ensure the validity of estimation, we aim to test the hypothesis that the covariance matrix is banded with a certain bandwidth under the high-dimensional framework. Though several testing methods have been proposed in the literature, the existing tests are only powerful for some alternatives with certain sparsity levels, whereas they may not be powerful for alternatives with other sparsity structures. The goal of this paper is to propose a new test for the bandedness of high-dimensional covariance matrix, which is powerful for alternatives with various sparsity levels. The proposed new test also be used for testing the banded structure of covariance matrices of error vectors in high-dimensional factor models. Based on these statistics, a consistent bandwidth estimator is also introduced for a banded high dimensional covariance matrix. Extensive simulation studies and an application to a prostate cancer dataset from protein mass spectroscopy are conducted for evaluating the effectiveness of the proposed adaptive tests blue and bandwidth estimator for the banded covariance matrix.

High dimensional covariance matrices have attracted much attention of statisticians and econometricians during the past decades. Vast literature is devoted to the research in high dimensional covariance matrices. However, most of them are for constant covariance matrices. In many applications, constant covariance matrices are not appropriate, e.g. in portfolio allocation, dynamic covariance matrices would make much more sense. Simply assuming each entry of a covariance matrix is a function of time to introduce a dynamic structure would not work. In this paper, we are going to introduce a class of high dimensional dynamic covariance matrices in which a kind of additive structure is embedded. We will show the proposed high dimensional dynamic covariance matrices have many advantages in applications. An estimation procedure is also proposed to estimate the proposed high dimensional dynamic covariance matrices. Asymptotic properties are built to justify the proposed estimation procedure. Intensive simulation studies show the proposed estimation procedure works very well when sample size is finite. Finally, we apply the proposed high dimensional dynamic covariance matrices, together with the proposed estimation procedure, to portfolio allocation. The results look very interesting.

In this paper we present a coding scheme for color images aiming at high compression ratios. It is based on perceptually classified vector quantization (VQ), where the different classes are chosen to achieve a better quality of the decoded image. Spatial correlation is reduced by a tree structured wavelet decomposition, then prediction of the insignificant coefficients is performed across subbands. Afterwards, the reduced set of data is organized in vectors in such a way that residual intra and inter band correlation are exploited. Finally, such vectors are coded by a classified VQ. Reconstructed images with good quality and a compression ratio higher that 100:1 have been produced by the proposed scheme, which has been shown to outperform the JPEG standard.

Compared to scalar quantization (SQ), vector quantization (VQ) has memory, space-filling, and shape advantages. If the signal statistics are known, direct vector quantization (DVQ) according to these statistics provides the highest coding efficiency, but requires unmanageable storage requirements if the statistics are time varying. In code-excited linear predictive (CELP) coding, a single compromise codebook is trained in the excitation-domain and the space-filling and shape advantages of VQ are utilized in a nonoptimal, average sense. In this paper, we propose Karhunen-Loe/spl grave/ve transform (KLT)-based adaptive classified VQ (CVQ), where the space-filling advantage can be utilized since the Voronoi-region shape is not affected by the KLT. The memory and shape advantages can be also used, since each codebook is designed based on a narrow class of KLT-domain statistics. We further improve basic KLT-CVQ with companding. The companding utilizes the shape advantage of VQ more efficiently. Our experiments show that KLT-CVQ provides a higher SNR than basic CELP coding, and has a computational complexity similar to DVQ and much lower than CELP. With companding, even single-class KLT-CVQ outperforms CELP, both in terms of SNR and codebook search complexity.

We consider the estimation of a Gaussian random vector x observed through a linear transformation H and corrupted by additive Gaussian noise with a known covariance matrix, where the covariance matrix of x is known to lie in a given region of uncertainty that is described using bounds on the eigenvalues and on the elements of the covariance matrix. Recently, two criteria for minimax estimation called difference regret (DR) and ratio regret (RR) were proposed and their closed form solutions were presented assuming that the eigenvalues of the covariance matrix of x are known to lie in a given region of uncertainty, and assuming that the matrices H T Cw -1 H and C x are jointly diagonalizable, where C w and C x denote the covariance matrices of the additive noise and of x respectively. In this work we present a new criterion for the minimax estimation problem which we call the generalized difference regret (GDR), and derive a new minimax estimator which is based on the GDR criterion where the region of uncertainty is defined not only using upper and lower bounds on the eigenvalues of the parameter's covariance matrix, but also using upper and lower bounds on the individual elements of the covariance matrix itself. Furthermore, the new estimator does not require the assumption of joint diagonalizability and it can be obtained efficiently using semidefinite programming. We also show that when the joint diagonalizability assumption holds and when there are only eigenvalue uncertainties, then the new estimator is identical to the difference regret estimator. The experimental results show that we can obtain improved mean squared error (MSE) results compared to the MMSE, DR, and RR estimators.