Reliability Function for Streaming Over a DMC With Feedback

We study the problem of strong coordination of the actions of two nodes $X$ and $Y$ that communicate over a discrete memoryless channel (DMC) such that the actions follow a prescribed joint probability distribution. We propose two novel random coding schemes and a polar coding scheme for this noisy strong coordination problem, and derive inner bounds for the respective strong coordination capacity region. The first scheme is a joint coordination-channel coding scheme that utilizes the randomness provided by the DMC to reduce the amount of local randomness required to generate the sequence of actions at Node $Y$. Based on this random coding scheme, we provide a characterization of the capacity region for two special cases of the noisy strong coordination setup, namely, when the actions at Node $Y$ are determined by Node $X$ and when the DMC is a deterministic channel. The second scheme exploits separate coordination and channel coding where local randomness is extracted from the channel after decoding. The third scheme is a joint coordination-channel polar coding scheme for strong coordination. We show that polar codes are able to achieve the established inner bound to the noisy strong coordination capacity region and thus provide a constructive alternative to a random coding proof. Our polar coding scheme also offers a constructive solution to a channel simulation problem where a DMC and shared randomness are employed together to simulate another DMC. Finally, by leveraging the random coding results for this problem, we present an example in which the proposed joint scheme is able to strictly outperform the separate scheme in terms of achievable communication rate for the same amount of injected randomness into both systems. Thus, we establish the sub-optimality of the separation of strong coordination and channel coding with respect to the communication rate over the DMC.

In this paper, we propose a novel code for transmitting a sequence of <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$n$</tex> message bits in real time over a discrete-memoryless channel (DMC) with noiseless feedback, where the message bits stream into the encoder one by one at random time instants. Similar to existing posterior matching schemes with block encoding, the encoder in our work takes advantage of the channel feedback to form channel inputs that contain the information the decoder does not yet have, and that are distributed close to the capacity-achieving input distribution, but dissimilar to the existing posterior matching schemes, the encoder performs instantaneous encoding - it immediately weaves the new message bits into a continuing transmission. A posterior matching scheme by Naghshvar et al. partitions the source messages into groups so that the group posteriors have a small-enough difference (SED) to the capacity-achieving distribution, and transmits the group index that contains the actual message. Our code adopts the SED rule to apply to the evolving message alphabet that contains all the possible variable-length strings that the source could have emitted up to that time. Our instantaneous SED code achieves better delay-reliability tradeoffs than existing feedback codes over 2-input DMCs: we establish this dominance both by simulations and via an analysis comparing the performance of the instantaneous SED code to Burnashev's reliability function. Due to the message alphabet that grows exponentially with time <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$t$</tex> , the complexity of the instantaneous SED code is double-exponential in <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$t$</tex> . To overcome this complexity barrier to practical implementation, we design a low-complexity code for binary symmetric channels that we name the instantaneous type set SED code. It groups the message strings into sets we call type sets and tracks their prior and posterior probabilities jointly, resulting in the reduction of complexity from double-exponential to <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$O(t^{4})$</tex> . Simulation results show that the gap in performance between the instantaneous SED code and the instantaneous type-set SED code is negligible.

Layered Wyner–Ziv video coding for transmission over unreliable channels

We study the transmission of two discrete memoryless correlated sources, consisting of a common and a private source, over a discrete memoryless multiterminal channel with two transmitters and two receivers. At the transmitter side, the common source is observed by both encoders but the private source can only be accessed by one encoder. At the receiver side, both decoders need to reconstruct the common source, but only one decoder needs to reconstruct the private source. We hence refer to this system by the asymmetric two-user source-channel coding system. We derive a universally achievable lossless joint source-channel coding (JSCC) error exponent pair for the two-user system by using a technique which generalizes Csiszar's type-packing lemma (1980) for the point-to-point (single-user) discrete memoryless source-channel system. We next investigate the largest convergence rate of asymptotic exponential decay of the system (overall) probability of erroneous transmission, i.e., the system JSCC error exponent. We obtain lower and upper bounds for the exponent. As a consequence, we establish a JSCC theorem with single-letter characterization and we show that the separation principle holds for the asymmetric two-user scenario. By introducing common randomization, we also provide a formula for the tandem (separate) source-channel coding error exponent. Numerical examples show that for a large class of systems consisting of two correlated sources and an asymmetric multiple-access channel with additive noise, the JSCC error exponent considerably outperforms the corresponding tandem coding error exponent.

This paper finds new tight finite-blocklength bounds for the best achievable lossy joint source-channel code rate, and demonstrates that joint source-channel code design brings considerable performance advantage over a separate one in the nonasymptotic regime. A joint source-channel code maps a block of <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">k</i> source symbols onto a length- <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">n</i> channel codeword, and the fidelity of reproduction at the receiver end is measured by the probability ε that the distortion exceeds a given threshold <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">d</i> . For memoryless sources and channels, it is demonstrated that the parameters of the best joint source-channel code must satisfy <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">nC</i> - <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">kR</i> ( <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">d</i> ) ≈ √( <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">nV</i> + <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">k V</i> ( <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">d</i> )) <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Q</i> <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">-1</sup> (ε), where <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">C</i> and <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">V</i> are the channel capacity and channel dispersion, respectively; <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">R</i> ( <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">d</i> ) and <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">V</i> ( <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">d</i> ) are the source rate-distortion and rate-dispersion functions; and <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Q</i> is the standard Gaussian complementary cumulative distribution function. Symbol-by-symbol (uncoded) transmission is known to achieve the Shannon limit when the source and channel satisfy a certain probabilistic matching condition. In this paper, we show that even when this condition is not satisfied, symbol-by-symbol transmission is, in some cases, the best known strategy in the nonasymptotic regime.

Noisy channels are a valuable resource from a cryptographic point of view. They can be used for exchanging secret-keys as well as realizing other cryptographic primitives such as commitment and oblivious transfer. To be really useful, noisy channels have to be considered in the scenario where a cheating party has some degree of control over the channel characteristics. Damgard et al. (EUROCRYPT 1999) proposed a more realistic model where such level of control is permitted to an adversary, the so called unfair noisy channels, and proved that they can be used to obtain commitment and oblivious transfer protocols. Given that noisy channels are a precious resource for cryptographic purposes, one important question is determining the optimal rate in which they can be used. The commitment capacity has already been determined for the cases of discrete memoryless channels and Gaussian channels. In this work we address the problem of determining the commitment capacity of unfair noisy channels. We compute a single-letter characterization of the commitment capacity of unfair noisy channels. In the case where an adversary has no control over the channel (the fair case) our capacity reduces to the well-known capacity of a discrete memoryless binary symmetric channel.

The objective of joint source channel coding (JSCC) is to optimally allocate the available channel bandwidth between source coding and channel coding in order to minimize the average distortion in the presence of packet losses. Unlike previous research, which focuses on determining distortion-optimized methods for JSCC using either assumptions about the rate-distortion (R-D) curve properties of source data (e.g., convexity) or experimental R-D data, we explicitly employ the analytical operational R-D model of a three-dimensional wavelet video coder to solve the JSCC problem. Using this model, we formulated the JSCC optimization as a nonlinear programming (NP) problem. By expressing the NP problem in terms of a dynamic programming equation, we derived the necessary conditions that facilitate the search for global optimal solutions. In this letter, we focus on finding the optimal JSCC problem for subband video coding and not on deriving computationally efficient methods for JSCC.

The bit commitment can be realized by using a noisy channel. Winter-Nascimento-Imai showed that the bit commitment capacity of a noisy channel is given by C <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">b</sub> = max <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">pN</sub> H(X\Y) where X and Y are the input and output of the channel. In this paper, the idea of a multiplex coding, which was first introduced into the coding problem of a wiretap channel by Kobayashi-Yamamoto-Ogawa, is applied to the bit commitment coding. The coding theorem is proved for the multiplex coding of bit commitment based on a noisy channel, and it is shown that several mutually independent secrets can be committed at the same time via a nosiy channel attaining the perfect secrecy of each secret with the total coding rate log \X\, where \X\ is the cardinality of channel input alphabet.

Chapter 12 - Joint Source-Channel Coding and Decoding

Recent research on joint source channel coding (JSCC) for wireless communications has achieved great success owing to the employment of deep learning (DL). However, the existing work on DL based JSCC usually trains the designed network to operate under a specific signal-to-noise ratio (SNR) regime, without taking into account that the SNR level during the deployment stage may differ from that during the training stage. A number of networks are required to cover the scenario with a broad range of SNRs, which is computational inefficiency (in the training stage) and requires large storage. To overcome these drawbacks our paper proposes a novel method called Attention DL based JSCC (ADJSCC) that can successfully operate with different SNR levels during transmission. This design is inspired by the resource assignment strategy in traditional JSCC, which dynamically adjusts the compression ratio in source coding and the channel coding rate according to the channel SNR. This is achieved by resorting to attention mechanisms because these are able to allocate computing resources to more critical tasks. Instead of applying the resource allocation strategy in traditional JSCC, the ADJSCC uses the channel-wise soft attention to scaling features according to SNR conditions. We compare the ADJSCC method with the state-of-the-art DL based JSCC method through extensive experiments to demonstrate its adaptability, robustness and versatility. Compared with the existing methods, the proposed method takes less storage and is more robust in the presence of channel mismatch.

Communication Systems and Information Theory. A Measure of Information. Coding for Discrete Sources. Discrete Memoryless Channels and Capacity. The Noisy-Channel Coding Theorem. Techniques for Coding and Decoding. Memoryless Channels with Discrete Time. Waveform Channels. Source Coding with a Fidelity Criterion. Index.

<para xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> In this paper, we present a cross-layer approach for video transmission in wireless LANs that employs joint source and application-layer channel coding, together with rate adaptation at the wireless physical layer (PHY). While the purpose of adopting PHY rate adaptation in modern wireless LANs like the IEEE 802.11a/b is to maximize the throughput, in this paper we exploit this feature to increase the robustness of wireless video. More specifically, we investigate the impact of adapting the PHY transmission rate, thus changing the throughput and packet loss channel characteristics, on the rate-distortion performance of a transmitted video sequence. To evaluate the video quality at the decoder, we develop a cross-layer modeling framework that considers jointly the effect of application-layer joint source-channel coding (JSCC), error concealment, and the PHY transmission rate. The resulting models are used by an optimization algorithm that calculates the optimal JSCC allocation for each video frame, and PHY transmission rate for each outgoing transport packet. The comprehensive simulation results obtained with the H.264/AVC codec demonstrate considerable increase in the PSNR of the decoded video when compared with a system that employs separately JSCC and PHY rate adaptation. Furthermore, our performance analysis indicates that the optimal PHY transmission rate calculated by the proposed algorithm, can be significantly different when compared with rate adaptation algorithms that target throughput improvement. </para>

To study the fundamental limits on the joint transmission of data of different levels of sensitivity, we establish the deterministic-code capacity region of a network with one transmitter and two receivers: an “ordinary receiver” and a “robust receiver.” The channel to the ordinary receiver is a given (known) discrete memoryless channel, whereas the channel to the robust receiver is an arbitrarily varying channel. Both receivers are required to decode the “common message” (the more sensitive data), whereas only the ordinary receiver is required to decode the “private message” (the less sensitive data).

In this paper, we consider a point-to-point communication network of discrete memoryless channels. In the network, there are a source node and possibly more than one sink node. Information is generated at the source node and is multicast to each sink node. We allow a node to encode its received information before loading it onto an outgoing channel, where the channels are independent of each other. We also allow the nodes to pass along messages asynchronously. In this paper, we characterize the admissibility of single-source multi-sink communication networks. Our result can be regarded as a network generalization of Shannon's result that feedback does not increase the capacity of a discrete memoryless channels (DMCs), and it implies a separation theorem for network coding and channel coding in such a communication network.

So far we have discussed source and channel coding problems separately. In this chapter we address the combination of the two, known as joint source-channel coding (JSCC). In a JSCC scheme, the encoder maps a source sequence into a channel input block, while the decoder produces a source estimate from the corresponding channel outputs. This mapping can be constructed using a concatenation of source and channel codes, in which case it is known as a separation-based scheme. While Shannon's separation principle states that this approach yields asymptotically the smallest possible average distortion, other schemes may have significant advantages as we will see in the following. For example, analog transmission (symbol-by-symbol mapping) has very desirable properties of low complexity and high robustness. However, except for some very special source-channel pairs, it is inferior to the digital (separation-based) approach in terms of average distortion. Therefore, finding schemes which are neither fully digital nor fully analog is of great interest.