Generalization of fuzzy noiseless source coding with utilities

In coding theory, we study various properties of codes for application in data compression, cryptography, error correction, and network coding. The study of codes is introduced in Information Theory, electrical engineering, mathematics, and computer sciences for the transmission of data through reliable and efficient methods. We have to consider how coding of messages can be done efficiently so that the maximum number of messages can be sent over a noiseless channel in a given time. Thus, the minimum value of mean codeword length subject to a given constraint on codeword lengths has to be founded. In this paper, we have introduced mean codeword length of orderαand typeβfor 1:1 codes and analyzed the relationship between average codeword length and fuzzy information measures for binary 1:1 codes. Further, noiseless coding theorem associated with fuzzy information measure has been established.

The aim of source coding is to remove as much redundancy as possible to achieve only the information in a file. This chapter shows that with the requirement of unique decompression, it is possible to achieve the average codeword length arbitrarily close to the entropy, but not less. A simple algorithm is given to construct a code, the Huffman code, that is optimal in terms of codeword length for a source with independent and identically distributed symbols. With Kraft inequality, a mathematical foundation needed to consider an optimization function for the codeword lengths is obtained. One standard method for minimization of a function with some side criterion is the Lagrange multiplication method. The idea of arithmetic coding is to consider the probabilities for vectors and to build a code with codeword length that grows as the logarithm of the probability for the vector, but without the exponential growth in complexity.

: Calculating the amount of information about a random function contained in another random function has important uses in communication theory. An expression for the mutual information for continuous time random processes has been given by Gelfand and Yaglom, Chiang, and Perez by generalizing Shannon's result in a natural way. Under a condition of absolute continuity of measures the continuous time expression has the same form as Shannon's result. For two Gaussian processes Gelfand and Yaglom express the mutual information in terms of a mean square estimation error. We generalize this result to diffusion processes and express the solution in a different form which is more naturally related to a corresponding filtering problem. We also use these results to calculate some information rates.

In this study, the authors consider the class of anti-uniform Huffman (AUH) codes. The authors derived tight lower and upper bounds on the average codeword length, entropy and redundancy of finite and infinite AUH codes in terms of the alphabet size of the source. These bounds are tighter than similar bounds. Also a tight upper bound on the entropy of AUH codes is presented in terms of the average cost of the code. The Fibonacci distribution is introduced, which plays a fundamental role in AUH codes. It is shown that such distributions maximise the average length and the entropy of the code for a given alphabet size. The authors also show that the minimum average cost of a code is achieved by an AUH codes in a highly unbalanced cost regime.

With the rapid advancement in electronics technologies, software-based beamformers for ultrasound array imaging have become feasible, and thus allowing rapid development of a high performance and potentially low cost system. However, to realize a fully software based system, one challenge is data transfer from the analog front end to the software backend, whose data rate can be up to a few GB/sec. It is therefore the purpose of this study to investigate the use of data compression for reducing data transfer requirements and the associated tradeoff with beamforming quality. JPEG (Joint Photographic Experts Group) and JPEG2000 compression were adopted. The acoustic data were acquired with a 128-channel array transducer at a center frequency of 3.5MHz. On each transmit event, the receive channel data are separated into 8×8 blocks, followed by the application of data compression. In one scheme, the compression is done on raw RF data. In another scheme, baseband data are used and only amplitude of the data is compressed. The maximum compression ratio of RF data compression is 15 with JPEG and 20 with JPEG2000 when the average error is less than 5 dB. With baseband data compression, on the other hand, although the maximum compression ratio is less than 8, the image quality is better than RF data compression. The average error in this case is less than 1.5 dB when the compression ratio is less than 7.

We study joint network-coding and channel-coding design to optimize delay performance. In our systems, network codes (network layer) are on top of channel codes (physical layer). Network codes run in a rateless random method, i.e., source and intermediate nodes randomly produce and transmit codewords until transmission succeeds. Thus, the rateless network codes have block-erasure-correction capability. The physical layer codewords are disturbed by channel noise. For the constraint of finite transmission time (finite block length), transmission errors are inevitable in the physical layer. We show that the physical-layer erasure probability is affected by both layers. Then, there is an interaction between network codes and channel codes, concerning the delay measure of the networks. We model the delay of each generation as the independent, identically distributed random variables. We show tradeoffs between the network layer and the physical layer on the length of network codewords, and on the transmission time of physical-layer codewords. To measure delay performance, we use expected delay and network-layer decoding error probability of a given delay, which are measures for networks without and with strict maximum-delay constraints, respectively. We show how to evaluate these measures for the coded networks with erasure channels. Then, we formulate problems to optimize the delay performance, and solutions are suggested.

The problem of constructing statistically synchronizable codes over arbitrary alphabets and for any finite source is considered. It is shown how to efficiently construct a statistically synchronizable code whose average codeword length is within the least likely codeword probability from that of the Huffman code for the same source. Moreover, a method is given for constructing codes having a synchronizing codeword. The method yields synchronous codes that exhibit high synchronizing capability and low redundancy.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

Joint source-channel decoding has recently received extensive attention due to the rise in the applications of multimedia wireless communication. Based on a code trellis rather than on a code tree, this work presents a maximum a posteriori (MAP) soft-decision priority-first decoding algorithm and its approximations for variable-length error-correcting codes. Simulation results indicate that for the code with average codeword length 6.269 bits and free distance 3, under moderate signal-to-noise ratio, one of the proposed algorithms almost reaches the lowest decoding complexity, and has nearly the same performance on symbol error probability as the MAP decoding.

After K. Bostro/spl uml/m and T. Felbinger observed that lossless quantum data compression does not exist unless decoders know the lengths of codewords, they introduced a classical noiseless channel to inform the decoder of a quantum source about the lengths of codewords. In this paper we analyze their codes and present: 1) a sufficient and necessary condition for the existence of such codes for given lists of lengths of codes; 2) a characterization of the optimal compression rate for their codes. However our main contribution is a more efficient way to use the classical channel. We propose a more general coding scheme. It turned out that the optimal compression can always be achieved by a code obtained by this scheme. A von Neumann entropy lower bound to rates of our codes and a necessary and sufficient condition to achieve the bound are obtained. The gap between this lower bound and the compression rates is also well analyzed. For a special family of quantum sources we provide a sharper lower bound in terms of Shannon entropy. Finally, we propose some problems for further research.

We investigate the joint application of compressive sensing and network coding to the problem of energy efficient data gathering in wireless sensor networks. We consider the problem of optimally constructing forwarding trees to carry compressed data to projection nodes; each compressed data refers to a weighted aggregation of measurements from sensors collected at one projection node. Projection nodes then forward their received compressed data to the sink, which subsequently recovers the original measurements. This aggregation technique based on compressive sensing is shown to reduce significantly the number of transmissions. We observe that the presence of multiple forwarding trees gives rise to many-to-many communication patterns which in turn can be exploited to perform network coding on the compressed data being forwarded on these trees. Such technique will further reduce the number of transmissions required to gather the measurements, and consequently result in a better network-wide energy efficiency. This paper addresses the problem of network coding aware construction of forwarding/aggregation trees and we present a mathematical model to optimally construct such trees. We also develop a decentralized method for solving the problem and we show that our method is both very scalable and accurate. We also show that when both network coding and compressive data gathering are considered jointly, modest gains may be attained.

In this letter, we present an efficient cross-entropy (CE)-based algorithm for the design of variable-length error-correcting (VLEC) codes under the joint source and channel coding (JSCC) framework. The algorithm enables us to construct the near-optimal VLEC codes that have the minimum average codeword length (ACL) with low search complexity. The efficiency of the proposed CE-based algorithm makes it possible to construct the VLEC codes that have small ACL values under various free-distance constraints, especially for large-sized signal alphabets.

This article investigates the joint application of compressive sensing (CS) and network coding (NC) to the problem of energy-efficient data gathering in wireless sensor networks. We consider the problem of optimally constructing forwarding trees to carry compressed data to projection nodes. Each compressed dataset refers to a weighted aggregation (or sum) of sensed measurements from network sensors collected at one projection node. Projection nodes then forward their received compressed data to the sink, which subsequently recovers the original measurements. This aggregation technique, based on CS, is shown to reduce significantly the number of transmissions in the network. We observe that the presence of multiple forwarding trees gives rise to many-to-many communication patterns in sensor networks that, in turn, can be exploited to perform NC on the compressed data being forwarded on these trees. Such a technique will further reduce the number of transmissions required to gather the measurements, resulting in a better network-wide energy efficiency. This article addresses the problem of NC--aware construction of forwarding/aggregation trees. We present a mathematical model to optimally construct such forwarding trees, which encourage NC operations on the compressed data. Owing to its complexity, we further develop algorithmic methods (both centralized and distributed) for solving the problem and analyze their complexities. We show that our algorithmic methods are scalable and accurate, with worst-case optimality gap not exceeding 3.96% in the studied scenarios. We also show that, when bothNC and compressive data gathering are considered jointly, performance gains (reduction in number of transmissions) of up to 30% may be attained. Finally, we show that the proposed methods distribute the workload of data gathering throughout the network nodes uniformly, resulting in extended network life times.

There are numerous scenarios in source coding where not only the code length but the importance of each value should also be taken into account. Different from the traditional coding theorems, by adding the importance weights for the length of the codes, we define the average cost of the weighted codeword length as an importance-aware measure of the codes. This novel information theoretical measure generalizes the average codeword length by assigning importance weights for each symbol according to users' concerns through focusing on user's selections. With such definitions, coding theorems of the bounds are derived and the outcomes are shown to be extensions of traditional coding theorems.

Initial investigations of the re-synchronisation properties of fifth augmentation level, variable length, self-synchronising T-codes are described. It has been found from simulation studies that within the fifth augmentation level family there are considerable differences in the resynchronisation delay statistics of individual code sets, with some code sets consistently outperforming others with the same codeword length distribution. Furthermore, the code sets with the shortest average re-synchronisation delays are generally those with the shortest average code-word length (i.e. the most efficient). The coding efficiency of these code sets has been found to be within 1% of the efficiency of a Huffman code optimised for the particular information source.

A superstring of a set of words is a string that contains each input word as a substring. Given such a set, the Shortest Superstring Problem (SSP) asks for a superstring of minimum length. SSP is an important theoretical problem related to the Asymmetric Travelling Salesman Problem, and also has practical applications in data compression and in bioinformatics. Indeed, it models the question of assembling a genome from a set of sequencing reads. Unfortunately, SSP is known to be NP-hard even on a binary alphabet and also hard to approximate with respect to the superstring length or to the compression achieved by the superstring. Even the variant in which all words share the same length r, called r-SSP, is NP-hard whenever r>2. Numerous involved approximation algorithms achieve approximation ratio above 2 for the superstring, but remain difficult to implement in practice. In contrast the greedy conjecture asked in 1988 whether a simple greedy algorithm achieves ratio of 2 for SSP. Here, we present a novel approach to bound the superstring approximation ratio with the compression ratio, which, when applied to the greedy algorithm, shows a 2 approximation ratio for 3-SSP, and also that greedy achieves ratios smaller than 2. This leads to a new version of the greedy conjecture.