Optimal maximal encoding different from Huffman encoding

Novel synchronous coding schemes are introduced and relationships between optimal synchronous codes and Huffman codes are discussed. Although the problem of existence of the optimal synchronous codes has not yet resolved, we show that any synchronous code can be considered as an optimal synchronous code for some information source alphabet. In other words, synchronous codes are almost optimal and, therefore, are regarded as near optimal with respect to average code word length. It is shown that there always exist optimal synchronous codes for the information source alphabets with a dyadic probability distribution. Comparing with the Huffman coding, the synchronous coding is used not only for statistical modeling but also for dictionary methods. It is also good at using in a large information retrieval system like the Huffman coding. Moreover, from the viewpoint of computational difficulty, it is proven that breaking a synchronous or an optimal synchronous code is NP-complete. E-mail: itjia@cityu.edu.hk This work was partially sponsored by City U Grants 7001355, 863 Program (Project No. 2002AA144060) and the National Natural Science Foundation of China (Project No. 60273062). E-mail: mli@ee.ecnu.edu.cn

In this paper we investigate the semigroup structure of the syntactic monoid $\mathrm {Syn}(C^+)$ of $C^+$, the semigroup generated by a maximal prefix code $C$ for which $C^+$ is a single class of the syntactic congruence. In particular we prove that for such a prefix code $C$, either $\mathrm {Syn}(C^+)$ is a group or it is isomorphic to a special type of submonoid of $G\times \mathcal {T}(R)$ where $G$ is a group and $\mathcal {T}(R)$ is the full transformation semigroup on a set $R$ with more than one element. From this description we conclude that $\mathrm {Syn}(C^+)$ has a kernel $J$ which is a right group. We further investigate separately the case when $J$ is a right zero semigroup and the case when $J$ is a group.

SummaryHuffman algorithm allows for constructing optimal prefix‐codes withO(n·logn) complexity. As the number of symbolsngrows, so does the complexity of building the code‐words. In this paper, a new algorithm and implementation are proposed that achieve nearly optimal coding without sorting the probabilities or building a tree of codes. The complexity is proportional to the maximum code length, making the algorithm especially attractive for large alphabets. The focus is put on achieving almost optimal coding with a fast implementation, suitable for real‐time compression of large volumes of data. A practical case example about checkpoint files compression is presented, providing encouraging results. Copyright © 2015 John Wiley & Sons, Ltd.

Optimal prefix codes with fewer distinct codeword lengths are faster to construct

We study asymptotically the redundancy of Huffman (and other) codes. It has been known from the inception of the Huffman (1952) code that in the worst case its redundancy-defined as the excess of the code length over the optimal (ideal) code length-is not more than one. However, to the best of our knowledge no precise asymptotic results have been reported in literature thus far. We consider here a memoryless binary source generating a sequence of length n distributed as binomial (n, p) with p being the probability of emitting 0. Based on the results of Stubley (1994), we prove that for p<1/2 the average redundancy R~/sub n//sup H/ of the Huffman code becomes as n/spl rarr//spl infin/: R~/sub n//sup H/={(3/2-(1/ln2+o(1))=0.057304..., /spl alpha/ irrational); (3/2-(1/M)(- 1/2 )); (-(1/M(1-2/sup -1/M/))2/sup - <n M="" beta="" spl="" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">/M/); (+O(/spl rho//sup n/), /spl alpha/=N/M rational); where /spl alpha/=log/sub 2/(1-p)/p and /spl beta/=-log/sub 2/(1-p), /spl rho/&lt;1, M, N are integers such that gcd (N, M)=1, and <x>=x-[x] is the fractional part of x. The appearance of the fractal-like function explains the erratic behavior of the Huffman redundancy, and its "resistance" to succumb to a precise analysis. As a side result, we prove that the average redundancy of the Shannon block code is as n/spl rarr//spl infin/: R~/sub n//sup S/{( 1/2 +o(1), /spl alpha/ irrational); ( 1/2 -1/M (<Mn beta="" spl=""/>- 1/2 )); (+O(/spl rho//sup n/), /spl alpha/=N/M rational); where /spl rho/&lt;1. Finally, we derive the redundancy of the Golomb (1966) code (for the geometric distribution) which can be viewed as a special case of the Huffman and Shannon codes, Golomb's code redundancy has only oscillating behavior (i.e., there is not convergent mode). These findings are obtained by analytic methods such as theory of distribution of sequences modulo 1 and Fourier series.</x></n>

Prefix codes such as Huffman codes are commonly used for loseless data compression. The class of synchronizing codes is often chosen to improve error resilience or to enable parallel decoding of data. Such codes have a special sequence whose occurrence realigns decoding process leading to recovery from errors in a data stream. In the present paper we identify a class of codes whose synchronizability depends only on the lengths of codewords. Namely, we show that every maximal finite prefix code with only two codeword lengths is synchronizing if and only if these lengths are coprime.

On the lattice of prefix codes

Iwata and Yamamoto proposed an iterative algorithm to obtain the optimal AIFV-m code with m code trees for a given source probability distribution, which can attain better compression rate than Huffman codes generally. In this paper, we generalize the optimization problem of AIFV-m code trees to the optimization problem of the average performance of finite Markov systems with m states, which have a unique stationary distribution. Then, we prove that the generalized iterative algorithm can derive the optimal system with m states, and hence, the original iterative algorithm can derive the optimal AIFV-m code.

Huffman codes, combined with word-based models, are considered efficient compression schemes for full-text retrieval systems. The decoding rate for these schemes can be substantially improved if the maximum length of the codewords is not greater then the machine word size L. However, if the vocabulary is large, simple methods for generating optimal length-restricted codes are either too slow or require a significantly large amount of memory. We present an in-place, simple and fast implementation for the BRCI (Build, Remove, Condense and Insert) algorithm, an approximative method for length-restricted coding. It overwrites a sorted input list of n weights with the corresponding codeword lengths in O(n) time. In addition, the worst-case compression loss introduced by BRCI codes with respect to unrestricted Huffman codes is proved to be negligible for all practical values of both L and n.

In order to transmit images and audio securely, the authors present the Quantum Image Teleportation Protocol (QITP) and Quantum Audio Teleportation Protocol (QATP), which utilizes the Quantum Teleportation (QT) technique combined with Huffman Coding. The QITP secures the teleportation of quantum states of an image while simultaneously encrypting and decrypting them using Huffman Coding since it is only possible to recover or decode data if the prefix codes are known. To test their approach, the authors transformed pixels or RGB values from digital images into text, which was then fed into the Huffman Coding Technique. It has the advantage of compressing the entire text, which makes it faster to transmit vast amounts of information. This work also demonstrates the Quantum Audio Teleportation Protocol (QATP) with and without Huffman coding. For proof of concept, experimental evaluations were performed for both suggested QITPs and QATPs (Standard QITP, QITP with Huffman Coding, Standard QATP, QATP with Huffman Coding), using IBM Quantum Assembly Language (IBM QASM) Simulator and real quantum hardware using the Quantum Information Science Kit (Qiskit), a quantum computing platform.

In this study, the authors consider the class of anti-uniform Huffman (AUH) codes. The authors derived tight lower and upper bounds on the average codeword length, entropy and redundancy of finite and infinite AUH codes in terms of the alphabet size of the source. These bounds are tighter than similar bounds. Also a tight upper bound on the entropy of AUH codes is presented in terms of the average cost of the code. The Fibonacci distribution is introduced, which plays a fundamental role in AUH codes. It is shown that such distributions maximise the average length and the entropy of the code for a given alphabet size. The authors also show that the minimum average cost of a code is achieved by an AUH codes in a highly unbalanced cost regime.

In coding theory, we study various properties of codes for application in data compression, cryptography, error correction and network coding. The study of codes is introduced in information theory, electrical engineering, mathematics and computer sciences for the transmission of data through reliable and efficient methods. We have to consider how coding of messages can be done efficiently so that maximum number of messages can be sent over a noiseless channel in a given time. Thus, minimum value of mean codeword length subject to a given constraint on codeword lengths has to be founded. In this paper, average code word length is proposed. Corresponding to newly defined average code word length a relationship with a result of generalized fuzzy information measures is introduced. Some Noiseless coding theorems connected with fuzzy information measure are also analyzed.

The aim of source coding is to remove as much redundancy as possible to achieve only the information in a file. This chapter shows that with the requirement of unique decompression, it is possible to achieve the average codeword length arbitrarily close to the entropy, but not less. A simple algorithm is given to construct a code, the Huffman code, that is optimal in terms of codeword length for a source with independent and identically distributed symbols. With Kraft inequality, a mathematical foundation needed to consider an optimization function for the codeword lengths is obtained. One standard method for minimization of a function with some side criterion is the Lagrange multiplication method. The idea of arithmetic coding is to consider the probabilities for vectors and to build a code with codeword length that grows as the logarithm of the probability for the vector, but without the exponential growth in complexity.

A new algorithm is presented for constructing nearly optimal prefix codes in the case of unequal letter costs and unequal probabilities. A bound on the maximal deviation from the optimum is derived and numerical examples are given. The algorithm has running time <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">O(t \cdot n)</tex> , where <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">t</tex> is the number of letters and <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">n</tex> is the number of probabilities.

Joint source-channel decoding has recently received extensive attention due to the rise in the applications of multimedia wireless communication. Based on a code trellis rather than on a code tree, this work presents a maximum a posteriori (MAP) soft-decision priority-first decoding algorithm and its approximations for variable-length error-correcting codes. Simulation results indicate that for the code with average codeword length 6.269 bits and free distance 3, under moderate signal-to-noise ratio, one of the proposed algorithms almost reaches the lowest decoding complexity, and has nearly the same performance on symbol error probability as the MAP decoding.