Training sequence size and vector quantizer performance

Suppose we are given n real valued samples Z/sub 1/, Z/sub 2/, ..., Z/sub n/ from a stationary source P. We consider the following question. For a compression scheme that uses blocks of length k, what is the minimal distortion (for encoding the true source P) induced by a vector quantizer of fixed rate R, designed from the training sequence. For a certain class of dependent sources, we derive conditions ensuring that the empirically designed quantizer performs as well (on the average) as the optimal quantizer, for almost every training sequence emitted by the source. In particular, we observe that for a code rate R, the optimal way to choose the dimension of the quantizer is k/sub n/=[(1-/spl delta/)R/sup -1/ log n]. The problem of empirical design of a vector quantizer of fixed dimension k based on a vector valued training sequence X/sub 1/, X/sub 2/, ..., X/sub n/ is also considered. For a class of dependent sources, it is shown that the mean squared error (MSE) of the empirically designed quantizer w.r.t the true source distribution converges to the minimum possible MSE at a rate of O(/spl radic/(log n/n)), for almost every training sequence emitted by the source. In addition, the expected value of the distortion redundancy-the difference between the MSEs of the quantizers-converges to zero for a sequence of increasing block lengths k, if we have at our disposal corresponding training sequences whose length grows as n=2/sup (R+/spl delta/)k/. Some of the derivations extend results in empirical quantizer design using an i.i.d. Training sequence, obtained by Linder et al. (see IEEE Trans. on Info. Theory, vol.40, p.1728-40, 1994) and Merhav and Ziv (see IEEE Trans. on Info. Theory, vol.43, p.1112-23, 1997). Proof of the techniques rely on the results in the theory of empirical processes, indexed by VC function classes.

The article deals with the problem of constructing a classifier for a set of n physical objects encoded by vectors of the space . In this well-known problem of pattern recognition, we are interested in the case when the number of factors N is much larger than the number of training vectors. As a classifier we use a linear homogeneous function . It is assumed that for each object with the number i there is a cluster set consisting of Ki training vectors corresponding to the observations of this object, for example, a set of its digitized photographs. In the case when a linear homogeneous classifier, as a rule, exists. We describe the algorithm for constructing the classifier and discuss the properties of this algorithm.

Network anomaly detection is a critical task to ensure network security. With increasing network traffic, detecting network anomaly would require solving a large-scale pattern classification problem that often contains millions of training vectors. Each training vector may represent a particular signature of network traffic pattern and some of them may be linked to security breaching activities that need to be detected and eradicated. In this paper, a popular statistical learning algorithm known as the support vector machine (SVM) was consider to solve the network anomaly detection problem. However, it is well known that SVM would require excessively long computing time and exceedingly large amount of memory when number of training vectors becomes huge. Hence, direct application of the standard SVM algorithm to solve large-scale network anomaly detection problems is impractical. In this paper, based on computational geometry theory, a new algorithm called convex-hull SVM (CH-SVM) was proposed, which can yield the same solution as original SVM while using significantly less training data, and hence less computing time. Then experiments were done on KDD'99 intrusion detection dataset to compare the performance of the proposed algorithm to a standard SVM and observed reduced training time and improved classification accuracy.

Multimedia data is a popular communication medium, but requires substantial storage space and network bandwidth. Vector quantization (VQ) is suitable for multimedia data applications because of its simple architecture, fast decoding ability, and high compression rate. Full‐search VQ can typically be used to determine optimal codewords, but requires considerable computational time and resources. In this study, a hybrid VQ combining a tree structure and a Voronoi diagram is proposed to improve VQ efficiency. To efficiently reduce the search space, a tree structure integrated with principal component analysis is proposed, to rapidly determine an initial codeword in low‐dimensional space. To increase accuracy, a Voronoi diagram is applied to precisely enlarge the search space by modeling relations between each codeword. This enables an optimal codeword to be efficiently identified by rippling an optimal neighbor from parts of neighboring Voronoi regions. The experimental results demonstrated that the proposed approach improved VQ performance, outperforming other approaches. The proposed approach also satisfies the requirements of handheld device application, namely, the use of limited memory and network bandwidth, when a suitable number of dimensions in principal component analysis is selected.

The authors point out that the LBG algorithm (see Y. Linde et al., (1980)) requires a lot of computation as the training vectors increase, and proposes a fast VQ (vector quantization) algorithm for a large amount of training data. This algorithm consists of three steps: first, divide training vectors into small groups; second, quantize each group into a few codewords by the LBG algorithm; finally, construct a codebook by clustering these codewords using the LBG algorithm again. The authors also report they can reduce the distortion error of the algorithm by adapting an effective data-dividing method. In experiments of quantizing 17500 training vectors into 512 codewords, this algorithm requires only 1/6 computation time compared with the conventional algorithm, while the increase of distortion is only 0.5 dB.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

This paper presents an iterative fuzzy vector quantization approach used in codebook design and its neural net algorithm, the fuzzy self-organizing feature map (FSOFM) algorithm, which is the development of the self-organizing feature map (SOFM) algorithm and the fuzzy vector quantization (FVQ) algorithms. The FVQ algorithm allows that each training vector is assigned to multiple codewords in the early stages of the codebook design. ALthough, the FVQ algorithm reduces the dependence of the resulting codebook on the initial codebook, the codewords are calculated in batch mode. The iterative fuzzy vector quantization approach is based on a gradient decent approach, and the concept of fuzzy is introduced into it. The FSOFM algorithm considers the winning output node and its neighborhood as a fuzzy set of the input node. As a result, the feature vector of the output node in the fuzzy set of the input sample can be updated by the membership function and the training vector just completing once iteration. In this paper, the LBG, FVQ, SOFM and FSOFM algorithms are used in image compression based on vector quantization. This paper evaluates the computing efficiency of these algorithms and compares the quality of the resulting codebooks.© (1997) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.

We propose the distortion sensitive competitive learning (DSCL) algorithm for codebook design in image vector quantization. The algorithm is based on the equidistortion principle for an asymptotically optimal vector quantizer after Gersho (1979) and from Ueda and Nakano (1994). The DSCL is simple and efficient in that a single weight vector update is performed per training vector, and the processing speed of the DSCL in a sequential or multiprocessor environment can further be improved by applying a modified partial distance elimination (MPDE) method. Simulations indicate that the DSCL outperforms some previously proposed neural algorithms, including the neural-gas from Martinetz et al. (1993) and the DEFCL from Butler and Jiang (1996). In combining with the MPDE, the DSCL is faster than the neural-gas up to a factor of 45 times on a sequential machine, and yet arrives at better codebooks with the same number of iterations.

The authors study codeword index assignment to allow for progressive image transmission of fixed rate full-search vector quantization (VQ). They develop three new methods of assigning indices to a vector quantization codebook and formulate these assignments as labels of nodes of a full-search progressive transmission tree. The tree is used to design intermediate codewords for the decoder so that full-search VQ has a successive approximation character. The binary representation for the path through the tree represents the progressive transmission code. The methods of designing the tree that they apply are the generalized Lloyd algorithm, minimum cost perfect matching from optimization theory, and a method of principal component partitioning. Their empirical results show that the final method gives intermediate signal-to-noise ratios (SNRs) that are close to those obtained with tree-structured vector quantization, yet they have higher final SNRs.

This correspondence discusses a progressive vector quantization (VQ) compression approach, which decomposes image data into a number of levels using full-search VQ. The final level is losslessly compressed, enabling lossless reconstruction. The computational difficulties are addressed by implementation on a massively parallel SIMD machine. We demonstrate progressive VQ on multispectral imagery obtained from the advanced very high resolution radiometer (AVHRR) and other earth-observation image data, and investigate the tradeoffs in selecting the number of decomposition levels and codebook training method.

We describe new 2-channel multiple-description speech coders based on the ITU-T recommendation G.711 PCM speech coder. The new coders operate in the PCM code domain in order to exploit the companding gain of PCM. They apply pairs of complementary asymmetric 2D vector quantizers to each pair of PCM codes, thus exploiting the correlation between adjacent speech samples. If both quantizer outputs are received (two channels working), they are combined to generate an approximation to the original pair of PCM codes. If only one quantizer output is received (one channel failed, one channel working), a coarser approximation is still possible. The vector quantizers use rectangular cells, and the aspect ratio of the cells controls the speech-quality trade-off between the two-channel and one-channel cases.

The authors present some preliminary results of their ongoing study on subjective VQ distortion measure in the time/spatial domain. They first propose a context based distortion measure between two vectors. The new measure is intuitively appealing, and they include some empirical evidence for its subjective significance. Although the measure is formulated as a matrix norm, it is computationally no more difficult than the mean-squares error. This measure quantifies the quantization distortion in the context (shape) of the signal waveform, but it is amplitude-invariant. So they combine the context distortion measure with a weighted mean distortion measure to obtain a unified subjective distortion measure D. They show that D is a distance measure and can be easily computed. Moreover, the process of computing the centroid of a set of training vectors and designing the VQ codebook under the new subjective distortion measure D is as simple as the conventional VQ. Specifically, the LBG algorithm can be applied to design the subjective VQ codebook after a simple linear transformation of the vector space in which signal samples are originally taken. They also analytically relate their subjective distortion measure to the ubiquitous mean-squares measure, and demonstrate that the latter is only a special case of the former. They also observe that the mean-removed VQ in a sense clusters training vectors under the proposed context distortion measure.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

Digital image coding using vector quantization [VQ] based techniques provides low-bit rates and high quality coded images, at the expenses of intensive computational demands. The computational requirement which is due to the encoding search process had hindered application of VQ to real-time and high quality coding of color images. Reduction of the encoding search complexity through partitioning of a large code book by tree structured vector quantization [TSVQ] in the spatial domain is proposed in this paper. A. real-time TSVQ algorithm for encoding color is developed on Apollo workstation HP 9000/433. This algorithm contributes to the use of a free distortion measure and reduction of code vector dimension. The mean/quantized residual vector quantizer has been used for reducing computational complexity of the encoding search. Running at clock 33MHz, the proposed algorithm is capable of real-time processing of 512x492 pixels per frame with rate 30 frames/second. Application of this algorithm on a training image(lady face) shows the convergence with low distortion rate as shown in Fig.1(a–e).

An image that is mapped into a bit stream suitable for communication over or storage in a digital medium is said to have been compressed. Using tree-structured vector quantizers (TSVQs) is an approach to image compression in which clustering algorithms are combined with ideas from tree-structured classification to provide code books that can be searched quickly and simply. The overall goal is to optimize the quality of the compressed image subject to a constraint on the communication or storage capacity, i.e. on the allowed bit rate. General goals of image compression and vector quantization are summarized in this paper. There is discussion of methods for code book design, particularly the generalized Lloyd algorithm for clustering, and methods for splitting and pruning that have been extended from the design of classification trees to TSVQs. The resulting codes, called pruned TSVQs, are of variable rate, and yield lower distortion than fixed-rate, full-search vector quantizers for a given average bit rate. They have simple encoders and a natural successive approximation (progressive) property. Applications of pruned TSVQs are discussed, particularly compressing computerized tomography images. In this work, the key issue is not merely the subjective attractiveness of the compressed image but rather whether the diagnostic accuracy is adversely aflected by compression. In recent work, TSVQs have been combined with other types of image processing, including segmentation and enhancement. The relationship between vector quantizer performance and the size of the training sequence used to design the code and other asymptotic properties of the codes are discussed.

Image compression is a basic need for the progressive transmission or storage of image data via any media. For the effective processing of image data the coding standards were developed. The most commonly used standard in current scenario is the JPEG and JPEG-2000 coding. These standards have become the industry standards and are progressively been in use for almost all image processing applications. In the process of image coding various sub-processing operations occur which process the image blocks by transformation, quantization and encoding before transmitting or storing. In the process, quantization is observed to be a time consuming and is needed to be processed in high precision so as to achieve less processing error and also in reasonable time. The process of quantization has shown greater efficiency in vector quantization than earlier methods. Vector quantization is observed to be an efficient block based lossy compression technique. Due to simpler coding process and high compression ratio VQ are widely used. Predictive vector Quantization (PVQ) is commonly used memory based vector Quantization method. The main drawback with PVQ is the high computational complexity which limits PVQ to use in real time applications. Shorter coding time can be archived by reducing the number of code words or instructions, but the reconstructed image quality will degraded in the case of fewer code words. Hence, it is necessary to propose fast coding algorithms for PVQ. In this paper a modified version of the PVQ approach has been suggested in order to process faster, which ensue to be a compatible mechanism for real time image coding in JPEG based coding system. Keywords: Compression, quantization, PVQ encoder, mean-square-error, Discrete Cosine Transform.

In this paper, we create a framework for training-based channel estimation under different channel and interference statistics. The minimum mean square error (MMSE) estimator for channel matrix estimation in Rician fading multi-antenna systems is analyzed, and especially the design of mean square error (MSE) minimizing training sequences. By considering Kronecker-structured systems with a combination of noise and interference and arbitrary training sequence length, we collect and generalize several previous results in the framework. We clarify the conditions for achieving the optimal training sequence structure and show when the spatial training power allocation can be solved explicitly. We also prove that spatial correlation improves the estimation performance and establish how it determines the optimal training sequence length. The analytic results for Kronecker-structured systems are used to derive a heuristic training sequence under general unstructured statistics. The MMSE estimator of the squared Frobenius norm of the channel matrix is also derived and shown to provide far better gain estimates than other approaches. It is shown under which conditions training sequences that minimize the non-convex MSE can be derived explicitly or with low complexity. Numerical examples are used to evaluate the performance of the two estimators for different training sequences and system statistics. We also illustrate how the optimal length of the training sequence often can be shorter than the number of transmit antennas.