Geometric source coding and vector quantization

We propose an iterative design algorithm for jointly optimizing source and channel codes. The joint design combines channel-optimized vector quantization (COVQ) for the source code with rate-compatible punctured convolutional (RCPC) coding for the channel code. Our objective is to minimize the average end-to-end distortion. For a given channel SNR and transmission rate, our joint source and channel code design achieves an optimal allocation of bits between the source and channel coders. This optimal allocation can reduce distortion by up to 6 dB over suboptimal allocations for the source data set considered. We also compare the distortion of our joint iterative design with that of two suboptimal design techniques: COVQ optimized for a given channel bit-error-probability, and RCPC channel coding optimized for a given vector quantizer. We conclude by relaxing the fixed transmission rate constraint and jointly optimizing the transmission rate, source code, and channel code.

We study the performance of joint source and channel codes designed to minimize end-to-end distortion over a Rayleigh fading channel. We consider two joint code designs. The first joint code uses a sequential design: a standard vector quantizer (VQ) source code is designed for a perfect channel (noiseless and distortionless) and then an RCPC channel code is optimized relative to the VQ and the channel statistics. The second design jointly optimizes a channel optimized VQ (COVQ) and an RCPC channel code through an iterative design process. We consider both hard-decision and soft-decision decoding for the channel codes. In both designs the bit allocation between the source and channel codes is optimized. At this optimal bit allocation, the performance of the iterative joint design and the simpler sequential design are nearly the same over the range of SNR values that we considered. Both code designs outperform standard COVQ and by up to 6 dB, and this performance improvement is most pronounced at low SNRs.

A criticism of using the Gaussian density function to model the classes in multispectral remote sensing data is that the classes may be multimodal, and therefore, not well represented by the Gaussian density function. The normal mixture density function, which is the sum of one or more weighted Gaussian components, is a compromise between Gaussian and non-parametric density functions. It can model multimodal density functions, yet requires fewer parameters to be estimated than non-parametric density functions, which is an advantage in remote sensing applications where training samples are difficult and expensive to obtain. Usually in practice, the number of components is not known, and must be estimated from the training samples. A new approach for estimating the appropriate number of components in a normal mixture density is described. The approach is to divide the data from each class into various numbers of clusters using the nearest means algorithm, compute a measure of fit which measures how well the training data are represented by the clusters, and to select the number of clusters that best fits the data. The measure of fit is computed by leaving one sample out and estimating the mean vectors and covariance matrices of the mixture density with the rest of the samples. Then the likelihood of the left-out sample is computed, and the process is repeated, each time leaving out a different sample. In this way, the samples used to test the estimates are independent of the samples used to compute estimates, and so the measure of fit, which is the average log likelihood of the left-out samples, does not increase monotonically like the joint likelihood, but will reach a maximum. The number of clusters that results in the maximum value of the measure of fit is selected as the estimate of the number of components in the mixture density.

Vector quantization (VQ) has long been recognized for its potential in image compression due to its ability to represent high-dimensional signal vectors compactly. However, the adoption of VQ in modern codecs has been hindered by practical limitations such as fixed-rate codebooks, lack of scalability, and inefficiency in entropy coding. In this work, we revisit VQ from a practical perspective and explore effective techniques to integrate it into modern image coding frameworks. We propose a rate-distortion-oriented (RD-oriented) design of tree-structured vector quantization (TSVQ) that enables flexible codebook construction at multiple bit rates through a growing method. Moreover, we introduce a cascade VQ strategy to promote the VQ ability by successively quantizing residual vectors, and propose a switching point decision method based on RD slope analysis. Our proposed methods enable scalable and RD-optimized vector quantization, demonstrating the potential of being applied to practical image coding frameworks.

An image that is mapped into a bit stream suitable for communication over or storage in a digital medium is said to have been compressed. Using tree-structured vector quantizers (TSVQs) is an approach to image compression in which clustering algorithms are combined with ideas from tree-structured classification to provide code books that can be searched quickly and simply. The overall goal is to optimize the quality of the compressed image subject to a constraint on the communication or storage capacity, i.e. on the allowed bit rate. General goals of image compression and vector quantization are summarized in this paper. There is discussion of methods for code book design, particularly the generalized Lloyd algorithm for clustering, and methods for splitting and pruning that have been extended from the design of classification trees to TSVQs. The resulting codes, called pruned TSVQs, are of variable rate, and yield lower distortion than fixed-rate, full-search vector quantizers for a given average bit rate. They have simple encoders and a natural successive approximation (progressive) property. Applications of pruned TSVQs are discussed, particularly compressing computerized tomography images. In this work, the key issue is not merely the subjective attractiveness of the compressed image but rather whether the diagnostic accuracy is adversely aflected by compression. In recent work, TSVQs have been combined with other types of image processing, including segmentation and enhancement. The relationship between vector quantizer performance and the size of the training sequence used to design the code and other asymptotic properties of the codes are discussed.

Central to vector quantization is the design of optimal code book. The construction of a globally optimal code book has been shown to be NP-complete. However, if the partition halfplanes are restricted to be orthogonal to the principal direction of the training vectors, then the globally optimal K-partition of a set of N D-dimensional data points can be computed in O((N+KM/sup 2/)D) time by dynamic programming, where M is the intensity resolution. This constrained optimization strategy improves the performance of vector quantizer over the classic LBG algorithm and the popular methods of tree-structured recursive greedy bipartition of the training data set. >

The authors present results of a comparative study of the efficiency of neural-net based approaches (Kohonen and NNVQ) and conventional (LBG and K-means) approaches for vector quantization. They focus on the accuracy and speed of the four methods for the VQ (vector quantization) design problem using two different input sources: a Gaussian Markov source and a speech signal (digit strings). It is shown that the LBG (Y. Linde, A. Buzo, and R. M. Gray 1980) and NNVQ methods achieve more accurate vector quantization than the K-means and Kohonen methods. The NNVQ method offers computational advantages, since the neural-net-based algorithm can be implemented with the use of parallel processors due to its inherent parallelism. >

This paper presents a novel hardware architecture for memetic vector quantizer (VQ) design. The architecture uses steady-state genetic algorithm (GA) for global search, and C-Means algorithm for local refinement. It adopts a shift register based circuit for accelerating mutation and crossover operations for steady state CA operations. It also uses a pipeline architecture for the hardware implementation of C-Means algorithm. The proposed architecture is embedded in a softcore CPU, and implemented on a field programmable logic array (FPGA) device for physical performance measurement. Experimental results show that the proposed architecture is an effective method for VQ optimization attaining both high performance and low computational time.

The design of vector quantizers (VQs) that yield minimal distortion is one of the most challenging problems in source coding. The problem of VQ design is to find a codebook that gives the least overall distortion (or equivalently, the largest signal-to-noise ratio (SNR)) for a given set of input vectors. This problem is known to be difficult as there are no known closed-form solutions. The generalized Lloyd algorithm (GLA) uses a finite set of training sequences as the data source and employs an iterative refinement. Given an initial codebook, the algorithm computes the nearest focally optimum codebook only. Genetic algorithms (GAs) are emerging as widely accepted optimization and search methods. These search methods are rooted in the mechanisms of evolution and natural genetics. They have a high probability of locating the globally optimal solution in a multimodal search space. A genetic algorithmic (GA) approach to vector quantizer design that combines the GLA is presented. We refer to this hybrid as the genetic generalized Lloyd algorithm (GGLA). It works briefly as follows. Initially, a finite number of codebooks, called chromosomes, are selected. In contrast to the GLA which refines only one codebook at a time, those codebooks undergo iterative cycles of reproduction in parallel. During an iteration, each codebook is updated by GLA or GA operations (i.e., mutation, crossover, and chromosome replacement). Three versions of the GGLAs are investigated depending on how the GLA or GA is selected.

Concurrent genetic optimization for joint design of source and channel codes

A vector quantizer maps a <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">k</tex> -dimensional vector into one of a finite set of output vectors or "points". Although certain lattices have been shown to have desirable properties for vector quantization applications, there are as yet no algorithms available in the quantization literature for building quantizers based on these lattices. An algorithm for designing vector quantizers based on the root lattices <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">A_{n}, D_{n}</tex> , and <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">E_{n}</tex> and their duals is presented. Also, a coding scheme that has general applicability to all vector quantizers is presented. A four-dimensional uniform vector quantizer is used to encode Laplacian and gamma-distributed sources at entropy rates of one and two bits/sample and is demonstrated to achieve performance that compares favorably with the rate distortion bound and other scalar and vector quantizers. Finally, an application using uniform four- and eight-dimensional vector quantizers for encoding the discrete cosine transform coefficients of an image at <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">0.5</tex> bit/pel is presented, which visibly illustrates the performance advantage of vector quantization over scalar quantization.

An embedded image coder based on wavelet transform coding and multistage vector quantization (VQ) is proposed in this research. We have examined several critical issues to make the proposed algorithm practically applicable. They include the complexity of embedded VQ, design of the successive vector quantizer, significance evaluation of a vector symbol, and integration of wavelet transform coding and vector quantization. It is shown in experiments that the new method achieves a superior rate-distortion performance. © 1997 John Wiley & Sons, Inc. Int J Imaging Syst Technol, 8, 444–449, 1997

The paper presents applications of neural network algorithms to the design of an adaptive vector quantizer. Vector quantization has been applied to the problem of displaying natural images with a reduced set of colors (colormap) and to the interframe coding of image sequences. The first step was to test classical Linde Buzo Gray (LGB), self organizing feature maps (SOFM) and Competitive Learning (CL) algorithms for the codebook design. The best results for the reconstructed quality image and the computational time are obtained using a CL algorithm with a new initialization strategy that solves the problem of underutilized nodes. An adaptive vector quantization algorithm is proposed and tested in a motion compensated image coder. The results of the simulations are very promising. In fact the coder performance, compared with that using a fixed VQ, is considerably improved and the subjective quality of the coded images is much better than that obtained using standard vector quantization, especially when rapid motion is present in the scene.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

Novel vector quantiser design using reinforced learning as a pre-process

The authors provide a convergence analysis for the Kohonen learning algorithm (KLA) with respect to vector quantizer (VQ) optimality criteria and introduce a stochastic relaxation technique which produces the global minimum but is computationally expensive. By incorporating the principles of the stochastic approach into the KLA, a deterministic VQ design algorithm, the soft competition scheme (SCS), is introduced. Experimental results are presented where the SCS consistently provided better codebooks than the generalized Lloyd algorithm (GLA), even when the same computation time was used for both algorithms. The SCS may therefore prove to be a valuable alternative to the GLA for VQ design.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>