Vector quantizer design by constrained global optimization

Novel vector quantiser design using reinforced learning as a pre-process

Codebook design plays an important role in the performance of signal processing based on vector quantization (VQ) just as speech coding, data compression and pattern recognition. LBG is one of the most effective algorithms and is widely used in codebook generation. Some problems still exist while its performance is remarkable. The LBG algorithm is easy to fall into local optimum. Usually there is strong correlation between the best solution and the initial selection for codebook design. It means that the quantization performance of codebooks from the same training data may varies in a certain range. There is also a certain probability for the algorithm to generate empty voronoi cell. In order to solve these problems, a novel algorithm based on ant colony clustering algorithm and genetic algorithm is proposed in this paper. The new algorithm takes advantage of the excellent global optimal searching ability of genetic algorithm. At the same time, the ant colony clustering algorithm is combined into the process. The dynamic change of the searching direction is adopted during crossover stage. The simulation results of line spectrum frequency parameters in mixed linear excitation prediction (MELP) show that the proposed algorithm is more efficient in its quantization performance compared to that of the LBG and genetic algorithms. Meanwhile, it has good stability in quantization performance. Index Terms - Codebook design, Vector quantization, Genetic algorithm, Ant colony clustering algorithm.

A geometric formulation is presented for source coding and vector quantizer design. Motivated by the asymptotic equipartition principle, the authors consider two broad classes of source codes and vector quantizers: elliptical codes and quantizers based on the Gaussian density function, and pyramid codes and quantizers based on the Laplacian density function. Elliptical and weighted pyramid vector quantizers are developed by selecting codewords as points in a lattice that lie on (or near) a specified ellipse or pyramid. The combination of geometric structure and lattice basis allows simple encoding and decoding algorithms. >

Vector quantization (VQ) has long been recognized for its potential in image compression due to its ability to represent high-dimensional signal vectors compactly. However, the adoption of VQ in modern codecs has been hindered by practical limitations such as fixed-rate codebooks, lack of scalability, and inefficiency in entropy coding. In this work, we revisit VQ from a practical perspective and explore effective techniques to integrate it into modern image coding frameworks. We propose a rate-distortion-oriented (RD-oriented) design of tree-structured vector quantization (TSVQ) that enables flexible codebook construction at multiple bit rates through a growing method. Moreover, we introduce a cascade VQ strategy to promote the VQ ability by successively quantizing residual vectors, and propose a switching point decision method based on RD slope analysis. Our proposed methods enable scalable and RD-optimized vector quantization, demonstrating the potential of being applied to practical image coding frameworks.

This paper presents a novel hardware architecture for memetic vector quantizer (VQ) design. The architecture uses steady-state genetic algorithm (GA) for global search, and C-Means algorithm for local refinement. It adopts a shift register based circuit for accelerating mutation and crossover operations for steady state CA operations. It also uses a pipeline architecture for the hardware implementation of C-Means algorithm. The proposed architecture is embedded in a softcore CPU, and implemented on a field programmable logic array (FPGA) device for physical performance measurement. Experimental results show that the proposed architecture is an effective method for VQ optimization attaining both high performance and low computational time.

The paper presents applications of neural network algorithms to the design of an adaptive vector quantizer. Vector quantization has been applied to the problem of displaying natural images with a reduced set of colors (colormap) and to the interframe coding of image sequences. The first step was to test classical Linde Buzo Gray (LGB), self organizing feature maps (SOFM) and Competitive Learning (CL) algorithms for the codebook design. The best results for the reconstructed quality image and the computational time are obtained using a CL algorithm with a new initialization strategy that solves the problem of underutilized nodes. An adaptive vector quantization algorithm is proposed and tested in a motion compensated image coder. The results of the simulations are very promising. In fact the coder performance, compared with that using a fixed VQ, is considerably improved and the subjective quality of the coded images is much better than that obtained using standard vector quantization, especially when rapid motion is present in the scene.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

The design of vector quantizers (VQs) that yield minimal distortion is one of the most challenging problems in source coding. The problem of VQ design is to find a codebook that gives the least overall distortion (or equivalently, the largest signal-to-noise ratio (SNR)) for a given set of input vectors. This problem is known to be difficult as there are no known closed-form solutions. The generalized Lloyd algorithm (GLA) uses a finite set of training sequences as the data source and employs an iterative refinement. Given an initial codebook, the algorithm computes the nearest focally optimum codebook only. Genetic algorithms (GAs) are emerging as widely accepted optimization and search methods. These search methods are rooted in the mechanisms of evolution and natural genetics. They have a high probability of locating the globally optimal solution in a multimodal search space. A genetic algorithmic (GA) approach to vector quantizer design that combines the GLA is presented. We refer to this hybrid as the genetic generalized Lloyd algorithm (GGLA). It works briefly as follows. Initially, a finite number of codebooks, called chromosomes, are selected. In contrast to the GLA which refines only one codebook at a time, those codebooks undergo iterative cycles of reproduction in parallel. During an iteration, each codebook is updated by GLA or GA operations (i.e., mutation, crossover, and chromosome replacement). Three versions of the GGLAs are investigated depending on how the GLA or GA is selected.

The authors provide a convergence analysis for the Kohonen learning algorithm (KLA) with respect to vector quantizer (VQ) optimality criteria and introduce a stochastic relaxation technique which produces the global minimum but is computationally expensive. By incorporating the principles of the stochastic approach into the KLA, a deterministic VQ design algorithm, the soft competition scheme (SCS), is introduced. Experimental results are presented where the SCS consistently provided better codebooks than the generalized Lloyd algorithm (GLA), even when the same computation time was used for both algorithms. The SCS may therefore prove to be a valuable alternative to the GLA for VQ design.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

Some applications of neural network algorithms for the design of vector quantizer codebooks are presented. Vector quantization has been applied to the problem of displaying natural images with a reduced set of colors (color palette) and to the interframe coding of image sequences. Self-organizing feature maps and competitive learning (CL) algorithms have been used for the codebook design and their results are compared with those obtained using a classical LBG (Linde, Buzo, Gray) algorithm. The best results are obtained using a CL algorithm with a new initialization strategy. With this technique the codebook design is very fast, and some experiments have been carried out in order to introduce adaptive vector quantization in the interframe coder. Frame by frame the codebooks used to quantize the most important information are updated and then the more active parts of the motion-compensated luminance differences are quantized with these new codebooks. A considerable rise of the coder performance was obtained. >

Vector quantization (VQ) based lossy compression techniques are ideally suited for image archival and distribution applications due to their asymmetrical computational properties. The practical use of VQ is however limited due to complications arising from design and maintenance of codebooks. Model-based VQ (MVQ) is a variant of VQ with the computational properties of VQ but without complications from explicit handling of codebooks. In MVQ, error models combined with a human visual system (HVS) model are used to generate an implicit model codebook. The decoder generates same codebook using the model parameters thus eliminating the need for training algorithm, transmitting and maintaining the codebooks. This algorithm was tested on several NASA images and results were published, M. Manchar et al. (1995). The present authors describe progressive MVQ along the lines of progressive VQ and evaluate its performance on NASA images.

Vector quantization is the process of encoding vector data as an index to a dictionary-or codebook-of representative vectors. Entropy-constrained vector quantizers (ECVQ) explicitly control the entropy of the output, and are superior to simple nearest-neighbor vector quantizers in terms of rate-distortion performances. ECVQ codebook design based on empirical data involves an expensive training phase in which a Lagrangian cost measure has to be minimized over the set of codebook vectors. In this paper, we describe two new general elimination rules allowing significant acceleration in the codebook design process. These rules use features of the codebook vectors in order to discard most of them as fast as possible during the search. Experimental results are presented on image block data. They show that those new rules perform slightly better than the previously known methods.

The authors present results of a comparative study of the efficiency of neural-net based approaches (Kohonen and NNVQ) and conventional (LBG and K-means) approaches for vector quantization. They focus on the accuracy and speed of the four methods for the VQ (vector quantization) design problem using two different input sources: a Gaussian Markov source and a speech signal (digit strings). It is shown that the LBG (Y. Linde, A. Buzo, and R. M. Gray 1980) and NNVQ methods achieve more accurate vector quantization than the K-means and Kohonen methods. The NNVQ method offers computational advantages, since the neural-net-based algorithm can be implemented with the use of parallel processors due to its inherent parallelism. >

In codebook based precoding MIMO system, the precoding codebook affects the system performance remarkably. Consequently, it is a crucial technology to design the optimum codebook for precoding MIMO system. Codebook design is related to the channel fading, antenna number, spatial correlation etc. So specific distributed channel corresponds to respective optimum codebook. In this paper, in order to design the optimum codebooks for the precoding MIMO system, a universal unitary space vector quantization (USVQ) codebook design criterion is provided, which can design the optimum codebooks for various spatial correlated channels with arbitrary antenna configurations. The unitary space K-mean(USK) algorithm is also provided to generate the USVQ codebook, which is iterative and convergent. Simulations show that the capacities of the precoding MIMO schemes using the USVQ codebooks are very close to those of the ideal precoding cases and outperform those of the schemes using the traditional Grassmannian codebooks.

Summary form only given. Vector quantization (VQ),is a well-known compression technique which has been widely used in many speech and image coding systems. Techniques for codebook design attempt to produce a codebook that is optimum for a given source. To date, the most widely used technique for VQ codebook design is the LBG (Linde-Buzo-Gray) algorithm. Madeim et al. (1999) show that an unsupervised neural network algorithm, referred to as SOA (self-organizing algorithm), provides good VQ codebooks, leading to reconstructed signals with better quality when compared to the ones obtained by using LBG codebooks. In this paper, an investigation is carried out to evaluate the inherent quality of SOA and LBG codebooks regarding the computational complexity of medium distortion encoding. The present work shows that the SOA codebooks overperforms the LBG codebooks in the sense that they yield a smaller average number of multiplications per sample for image VQ.

An embedded image coder based on wavelet transform coding and multistage vector quantization (VQ) is proposed in this research. We have examined several critical issues to make the proposed algorithm practically applicable. They include the complexity of embedded VQ, design of the successive vector quantizer, significance evaluation of a vector symbol, and integration of wavelet transform coding and vector quantization. It is shown in experiments that the new method achieves a superior rate-distortion performance. © 1997 John Wiley & Sons, Inc. Int J Imaging Syst Technol, 8, 444–449, 1997