Reexamination of quantum data compression and relative entropy

In this paper, we provide a simple framework for deriving one-shot achievable bounds for some problems in quantum information theory. Our framework is based on the joint convexity of the exponential of the collision relative entropy and is a (partial) quantum generalization of the technique of Yassaee et al. from classical information theory. Based on this framework, we derive one-shot achievable bounds for the problems of communication over classical-quantum channels, quantum hypothesis testing, and classical data compression with quantum side information. We argue that our one-shot achievable bounds are strong enough to give the asymptotic achievable rates of these problems even up to the second order.

: Calculating the amount of information about a random function contained in another random function has important uses in communication theory. An expression for the mutual information for continuous time random processes has been given by Gelfand and Yaglom, Chiang, and Perez by generalizing Shannon's result in a natural way. Under a condition of absolute continuity of measures the continuous time expression has the same form as Shannon's result. For two Gaussian processes Gelfand and Yaglom express the mutual information in terms of a mean square estimation error. We generalize this result to diffusion processes and express the solution in a different form which is more naturally related to a corresponding filtering problem. We also use these results to calculate some information rates.

A source coding paradigm called interactive encoding and decoding (IED) is considered for a source network where a finite alphabet source <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">X</i> is to be encoded, and another finite alphabet source <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Y</i> correlated with <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">X</i> is available only to the decoder as a helper. The optimal performance achievable asymptotically (OPAA) by IED is investigated, where the performance is measured as the average number of bits per symbol exchanged by the encoder and decoder until the decoder learns <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">X</i> with high probability. First, it is shown that for any stationary <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">(X</i> , <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Y</i> ), the OPAA by IED is given by the conditional entropy rate <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">H</i> ( <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">X</i> | <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Y</i> ) of <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">X</i> given <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Y</i> . This is in contrast with noninteractive Slepian-Wolf (SW) coding, where the OPAA is shown in general to be strictly greater than <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">H</i> ( <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">X</i> | <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Y</i> ) when <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">(X</i> , <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Y</i> ) is not ergodic. Second, for a memoryless source pair ( <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">X, Y</i> ), it is shown that IED approaches <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">H</i> ( <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">X</i> | <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Y</i> ) faster than SW coding does. Finally, it is demonstrated that one can convert any classical universal data compression algorithm with side information to a universal IED algorithm for the class <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">¿</i> of all stationary ergodic source pairs. In contrast, universal SW coding algorithms for the class <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">¿</i> do not exist.

The quantum relative entropy is frequently used as a distance measure between two quantum states, and inequalities relating it to other distance measures are important mathematical tools in many areas of quantum information theory. We have derived many such inequalities in previous work. The present paper is a follow-up on this, and provides a sharp upper bound on the relative entropy in terms of the trace norm distance and of the smallest eigenvalues of both states concerned. The result obtained here is more general than the corresponding one from our previous work. As a corollary, we obtain a sharp upper bound on the regularised relative entropy introduced by Lendi, Farhadmotamed, and van Wonderen.

Assume ${P_{\theta}: \theta \epsilon \Theta}$ is a set of probability distributions with a common dominating measure on a complete separable metric space Y. A state $\theta^* \epsilon \Theta$ is chosen by Nature. A statistician obtains n independent observations $Y_1, \dots, Y_n$ from Y distributed according to $P_{\theta^*}$. For each time t between 1 and n, based on the observations $Y_1, \dots, Y_{t-1}$, the statistician produces an estimated distribution $\hat{P}_t$ for $P_{\theta^*}$ and suffers a loss $L(P_{\theta^*}, \hat{P}_t)$. The cumulative risk for the statistician is the average total loss up to time n. Of special interest in information theory, data compression, mathematical finance, computational learning theory and statistical mechanics is the special case when the loss $L(P_{\theta^*}, \hat{P}_t)$ is the relative entropy between the true distribution $P_{\theta^*}$ and the estimated distribution $\hat{P}_t$. Here the cumulative Bayes risk from time 1 to n is the mutual information between the random parameter $\Theta^*$ and the observations $Y_1, \dots, Y_n$. New bounds on this mutual information are given in terms of the Laplace transform of the Hellinger distance between pairs of distributions indexed by parameters in $\Theta$. From these, bounds on the cumulative minimax risk are given in terms of the metric entropy of $\Theta$ with respect to the Hellinger distance. The assumptions required for these bounds are very general and do not depend on the choice of the dominating measure. They apply to both finite- and infinite-dimensional $\Theta$. They apply in some cases where Y is infinite dimensional, in some cases where Y is not compact, in some cases where the distributions are not smooth and in some parametric cases where asymptotic normality of the posterior distribution fails.

Multimedia data may be transmitted or stored either according to the classical Shannon information theory or according to the newer Autosophy information theory. Autosophy algorithms combine very high "lossless" data and image compression with virtually unbreakable "codebook" encryption. Shannon's theory treats all data items as "quantities", which are converted into binary digits (bit), for transmission in meaningless bit streams. Only "lossy" data compression is possible. A new "Autosophy" theory was developed by Klaus Holtz in 1974 to explain the functioning of natural self-assembling structures, such as chemical crystals or living trees. The same processes can also be used for growing self-assembling data structures, which grow like data crystals or data trees in electronic memories. This provides true mathematical learning algorithms, according to a new Autosophy information theory. Information in essence is only that which can be perceived and which is not already known by the receiver. The transmission bit rates are dependent on the data content only. Applications already include the V.42bis compression standard in modems, the gif and tif formats for lossless image compression, and Autosophy Internet television. A new 64bit data format could make all future communications compatible and solve the Internet's Quality of Service (QoS) problems.

The subject of this chapter is the use of classical/quantum information theory and coding in genetics and evolution. The chapter starts with the description of using the concepts from both classical and quantum information theories to describe the evolution of biological channel capacity through generations. In order to do so, several classical and quantum biological channel models are employed including the Markovian classical and Markovian-like quantum model, hybrid quantum-classical model, multilevel symmetric channel model, and Kimura model-based Markovian process. In order to describe the reliable long-time storage of genetic information in DNA, the use of unequal error protection (UEP) coding is studied. Several classes of error-correction codes suitable for UEP on a cellular level are described including nested coding, multilevel coding (MLC), rate-adaptive coding, and generalized LDPC coding. The use of concepts of constrained coding to describe the genetic information flow from DNA to proteins is also described as well as joint-constrained and error-correction coding. After that, the use of quantum error-correction concepts to deal with environmental errors including canonical quantum error-correction and stabilizer codes is briefly described. One particular class of stabilizer codes, known as topological codes, is then described that might be relevant to biological processes as they only involve the local qubits in encoding process. Another relevant class of codes, the subsystem codes, is then described. The key idea behind subsystem codes is to decompose the quantum code as the tensor product of two subsystems, exon subsystem A and intron subsystem B, and we are concerned with correcting errors only on the exon subsystem. Finally, we describe the use of nonbinary quantum stabilizer codes to deal with nucleobase substitution errors, both random and burst errors. We also briefly discuss the possible use of both classical and quantum error-correction concepts to improve tolerance to tumor and cancer introducing errors.

In coding theory, we study various properties of codes for application in data compression, cryptography, error correction and network coding. The study of codes is introduced in information theory, electrical engineering, mathematics and computer sciences for the transmission of data through reliable and efficient methods. We have to consider how coding of messages can be done efficiently so that maximum number of messages can be sent over a noiseless channel in a given time. Thus, minimum value of mean codeword length subject to a given constraint on codeword lengths has to be founded. In this paper, average code word length is proposed. Corresponding to newly defined average code word length a relationship with a result of generalized fuzzy information measures is introduced. Some Noiseless coding theorems connected with fuzzy information measure are also analyzed.

In coding theory, we study various properties of codes for application in data compression, cryptography, error correction, and network coding. The study of codes is introduced in Information Theory, electrical engineering, mathematics, and computer sciences for the transmission of data through reliable and efficient methods. We have to consider how coding of messages can be done efficiently so that the maximum number of messages can be sent over a noiseless channel in a given time. Thus, the minimum value of mean codeword length subject to a given constraint on codeword lengths has to be founded. In this paper, we have introduced mean codeword length of orderαand typeβfor 1:1 codes and analyzed the relationship between average codeword length and fuzzy information measures for binary 1:1 codes. Further, noiseless coding theorem associated with fuzzy information measure has been established.

The quantum relative entropy is frequently used as a distance, or distinguishability measure between two quantum states. In this article we study the relation between this measure and a number of other measures used for that purpose, including the trace norm distance. More specifically, we derive lower and upper bounds on the relative entropy in terms of various distance measures for the difference of the states based on unitarily invariant norms. The upper bounds can be considered as statements of continuity of the relative entropy distance in the sense of Fannes. We employ methods from optimisation theory to obtain bounds that are as sharp as possible.

Relative entropies play important roles in classical and quantum information theory. In this paper, we discuss the sandwiched Rényi relative entropy for [Formula: see text] on [Formula: see text] (the cone of positive trace-class operators acting on an infinite-dimensional complex Hilbert space [Formula: see text]) and characterize all surjective maps preserving the sandwiched Rényi relative entropy on [Formula: see text]. Such transformations have the form [Formula: see text] for each [Formula: see text], where [Formula: see text] and [Formula: see text] is either a unitary or an anti-unitary operator on [Formula: see text]. Particularly, all surjective maps preserving sandwiched Rényi relative entropy on [Formula: see text] (the set of all quantum states on [Formula: see text]) are necessarily implemented by either a unitary or an anti-unitary operator.

Chapter 6 - Information Theory and Classical Error Correcting Codes

Image processing has gone through many years of research and development. But none of these techniques can be applied to compressed images, i.e., the compressed codes. Modern telecommunication and multimedia networks may require image processing and data compression being integrated into one entity. A visual information system is one example in which it is desired that image databases are constructed by storing compressed codes rather than original image pixels in order to save the storage space. Hence new algorithms should be developed for image indexing directly operated on those compressed codes. This paper reports some pioneering work in this direction to show that compressed image processing could well become a new research area and challenge right through the next century.

We consider a two-parameter family of Rényi relative entropies Dα,z(ρ ∥ σ) that are quantum generalisations of the classical Rényi divergence Dα(p ∥ q). This family includes many known relative entropies (or divergences) such as the quantum relative entropy, the recently defined quantum Rényi divergences, as well as the quantum Rényi relative entropies. All its members satisfy the quantum generalizations of Rényi’s axioms for a divergence. We consider the range of the parameters α, z for which the data-processing inequality holds. We also investigate a variety of limiting cases for the two parameters, obtaining explicit formulas for each one of them.

The quantum relative entropy between two states satisfies a monotonicity\nproperty meaning that applying the same quantum channel to both states can\nnever increase their relative entropy. It is known that this inequality is only\ntight when there is a "recovery map" that exactly reverses the effects of the\nquantum channel on both states. In this paper we strengthen this inequality by\nshowing that the difference of relative entropies is bounded below by the\nmeasured relative entropy between the first state and a recovered state from\nits processed version. The recovery map is a convex combination of rotated Petz\nrecovery maps and perfectly reverses the quantum channel on the second state.\nAs a special case we reproduce recent lower bounds on the conditional mutual\ninformation such as the one proved in [Fawzi and Renner, Commun. Math. Phys.,\n2015]. Our proof only relies on elementary properties of pinching maps and the\noperator logarithm.\n