Recurrent Multiple-Repetition Coding for Channels With Feedback

Suppose Q is a family of discrete memoryless channels. An unknown member of Q will be available, with perfect, causal output feedback for communication. Is there a coding scheme (possibly with variable transmission time) that can achieve the Burnashev error exponent uniformly over Q ? For two families of channels we show that the answer is yes. Furthermore, for each of these two classes, in addition to achieve the maximum error exponent, it is possible to uniformly attain any given fraction of the channel capacity. Therefore, in terms of achievable rates and delay, there are situations in which the knowledge of the channel becomes irrelevant. In the second part of the thesis, we show that for arbitrary sets of channels the Burnashev error exponent cannot in general be uniformly achieved. In particular we give a sufficient condition for a pair of channels so that no coding strategy reaches Burnashev's exponent simultaneously on both channels. As a third part we study a scenario where communication is carried by first testing the channel by means of a training sequence, then coding according to the channel estimate. We provide an upper bound on the maximum achievable error exponent of such coding schemes. This bound is typically much lower than the maximum achievable error exponent over a channel with feedback. For example in the case of binary symmetric channels this bound has a slope that vanishes at capacity. This result suggests that in terms of error exponent, a good universal feedback scheme combines channel estimation with information delivery, rather than separating them. In the final chapter, we address the question of communicating quickly and reliably. We consider a simple situation of two message communication over a known channel with feedback. We propose a simple decoding rule, and show that it minimizes a weighted combination of the probability of error and decoding delay for a certain range of crossover probabilities and combination weights.

Along with the channel capacity, the error exponent is one of the most important information-theoretic measures of reliability, as it sets ultimate bounds on the performance of communication systems employing codes of finite complexity. In this paper, we derive an exact analytical expression for the random coding error exponent, which provides significant insight regarding the ultimate limits to communications through Nakagami-m fading channels. An important fact about this error exponent is that it determines the behavior of error probability in terms of the transmission rate and the code length that reflects the coding complexity required to achieve a certain level of reliability. Moreover, from the derived analytical expression, we can easily compute the necessary codeword length without extensive Monte-Carlo simulation to achieve a predefined upper bound for error probability at a rate below the channel capacity. We also improve the random coding bound by expurgating bad codewords from the code ensemble, since random coding error exponent is determined by selecting codewords independently according to the input distribution where good and bad codewords contribute equally to the overall average error probability. Finally, we derive exact analytical expressions for the cutoff rate, critical rate, and expurgation rate and verify the analytical expressions via Monte-Carlo simulation.

We study the transmission of two discrete memoryless correlated sources, consisting of a common and a private source, over a discrete memoryless multiterminal channel with two transmitters and two receivers. At the transmitter side, the common source is observed by both encoders but the private source can only be accessed by one encoder. At the receiver side, both decoders need to reconstruct the common source, but only one decoder needs to reconstruct the private source. We hence refer to this system by the asymmetric two-user source-channel coding system. We derive a universally achievable lossless joint source-channel coding (JSCC) error exponent pair for the two-user system by using a technique which generalizes Csiszar's type-packing lemma (1980) for the point-to-point (single-user) discrete memoryless source-channel system. We next investigate the largest convergence rate of asymptotic exponential decay of the system (overall) probability of erroneous transmission, i.e., the system JSCC error exponent. We obtain lower and upper bounds for the exponent. As a consequence, we establish a JSCC theorem with single-letter characterization and we show that the separation principle holds for the asymmetric two-user scenario. By introducing common randomization, we also provide a formula for the tandem (separate) source-channel coding error exponent. Numerical examples show that for a large class of systems consisting of two correlated sources and an asymmetric multiple-access channel with additive noise, the JSCC error exponent considerably outperforms the corresponding tandem coding error exponent.

Capacity of a compound channel without feedback is defined in a pessimistic manner as the maximum rate determined before the start of communication such that communication is reliable. In the presence of feedback, the transmission rate can adapt to the channel chosen by nature. Thus, capacity can be defined in an opportunistic manner as the maximum rate determined at the end of communication such that communication is reliable. Under this definition, transmission rate and error exponents are regions rather than scalars. In this paper, variable length communication over a compound channel with feedback is formulated, its opportunistic capacity region is characterized, and lower bounds for its error exponent region are provided.

A secret key agreement framework involving three users is considered in which each of the users 1 and 2 intends to share a secret key with user 3 and users 1 and 2 are eavesdroppers with respect to each other. There is a generalized discrete memoryless multiple access channel (GDMMAC) from users 1 and 2 to user 3 where the three users receive outputs from the channel. Furthermore, there is a feedback channel from user 3 to users 1 and 2 through which user 3 sends information extracted from the received output from the GDMMAC to increase the key rates. We consider both noiseless and noisy feedback. In the case of noiseless feedback, a public channel of unlimited capacity from user 3 to users 1 and 2 is used only once. In the case of noisy feedback, a noisy broadcast channel (BC) from user 3 to users 1 and 2 can be repeatedly used, like GDMMAC. In both setups, inner bounds of the secret key capacity region are derived. The secret key capacity region is derived in some special cases where the channel inputs and outputs form Markov chains in certain orders. For illustration, the corresponding results are also derived and discussed for Gaussian channels. The cases with noiseless feedback, noisy feedback, and no feedback at all are compared with each other.

In a two-way relay network, two terminals exchange information over a shared wireless half-duplex channel with the help of a relay. Due to its fundamental and practical importance, there has been an increasing interest in this channel. However, surprisingly, there has been little work that characterizes the fundamental tradeoff between the communication reliability and transmission rate across all signal-to-noise ratio (SNR) ratios. In this paper, we consider amplify-and-forward (AF) two-way relaying due to its simplicity. We first derive the random coding error exponent for the link in each direction. From the exponent expression, the capacity and cutoff rate for each link are also deduced. We then put forth the notion of the bottleneck error exponent, which is the worst exponent decay between the two links, to give us insight into the fundamental tradeoff between the rate pair and information-exchange reliability of the two terminals.

The issue of whether feedback can significantly increase reliability in the fixed-length channel code setting is further investigated. This paper considers the problem of error exponents for block codes with noiseless, delayed feedback used over discrete memoryless channels (DMCs) - including asymmetric channels with and without zeros in their transition matrix. We show that when output feedback is given to the encoder with a delay of T symbols, the error exponent is upper bounded by E <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">sp</sub> (R - O((log T)/T )) + O((log T)/T ), where E <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">sp</sub> denotes the sphere-packing exponent.

Universally achievable error exponents pertaining to certain families of channels (most notably, discrete memoryless channels (DMCs) and various ensembles of random codes, are studied by combining the competitive minimax approach, proposed by Feder and Merhav, with Chernoff bound and Gallager's techniques for the analysis of error exponents. In particular, we derive a single-letter expression for the largest, universally achievable fraction xi of the optimum error exponent pertaining to the optimum maximum-likelihood (ML) decoding. Moreover, a simpler single-letter expression for a lower bound to xi is presented. To demonstrate the tightness of this lower bound, we use it to show that xi = 1, for the binary symmetric channel (BSC), when the random coding distribution is uniform over: i) all codes (of a given rate), and ii) all linear codes, in agreement with well-known results. We also show that xi = 1 for the uniform ensemble of systematic linear codes, and for that of time-varying convolutional codes in the bit-error-rate sense. For the latter case, we also derive the corresponding universal decoder explicitly and show how it can be efficiently implemented using a slightly modified version of the Viterbi algorithm which employs two trellises.

In light of recent results by Verdú and Han on channel capacity, we examine three problems: the strong converse condition to the channel coding.theorem, the capacity of arbitrary channels with feedback and the Neyman‐Pearson hypothesis testing type‐II error exponent. It is first remarked that the strong converse condition holds if and only if the sequence of normalized channel information densities converges in probability to a constant. Examples illustrating this condition are provided. A general formula for the capacity of arbitrary channels with output feedback is then obtained. Finally, a general expression for the Neyman‐Pearson type‐II error exponent based on arbitrary observations subject to a constant bound on the type‐I error probability is derived.

We define the error exponent of the typical random code as the long-block limit of the negative normalized expectation of the logarithm of the error probability of the random code, as opposed to the traditional random coding error exponent, which is the limit of the negative normalized logarithm of the expectation of the error probability. For the ensemble of uniformly randomly drawn fixed composition codes, we provide exact error exponents of typical random codes for a general discrete memoryless channel (DMC) and a wide class of (stochastic) decoders, collectively referred to as the generalized likelihood decoder (GLD). This ensemble of fixed composition codes is shown to be no worse than any other ensemble of independent codewords that are drawn under a permutation--invariant distribution (e.g., i.i.d. codewords). We also present relationships between the error exponent of the typical random code and the ordinary random coding error exponent, as well as the expurgated exponent for the GLD. Finally, we demonstrate that our analysis technique is applicable also to more general communication scenarios, such as list decoding (for fixed-size lists) as well as decoding with an erasure/list option in Forney's sense.

The problem of characterizing achievable rates and error exponents for discrete memoryless channels with mismatched decoding is addressed. A mismatched cognitive multiple-access channel is introduced, and an inner bound on its capacity region is derived using two alternative encoding methods: 1) superposition coding and 2) random binning. The inner bounds are derived by analyzing the average error probability of the code ensemble for both methods and by a tight characterization of the resulting error exponents. Random coding converse theorems are also derived. A comparison of the achievable regions shows that in the matched case, random binning performs as well as superposition coding, i.e., the region achievable by random binning is equal to the capacity region. The achievability results are further specialized to obtain a lower bound on the mismatch capacity of the single-user channel by investigating a cognitive multiple-access channel whose achievable sum-rate serves as a lower bound on the single-user channel's capacity. While the achievable rate presented here may not improve the rate achieved by Lapidoth's scheme in optimizing over the parameters of the random coding scheme, it can improve the achieved rate for given parameters, and thereby may reduce the computational complexity required to find a good code.

This paper studies channel coding for the discrete memoryless multiple-access channel with a given (possibly suboptimal) decoding rule. A multi-letter successive decoding rule depending on an arbitrary non-negative decoding metric is considered, and achievable rate regions and error exponents are derived both for the standard MAC (independent codebooks), and for the cognitive MAC (one user knows both messages) with superposition coding. In the cognitive case, the rate region and error exponent are shown to be tight with respect to the ensemble average. The rate regions are compared with those of the commonly considered decoder that chooses the message pair maximizing the decoding metric, and numerical examples are given for which successive decoding yields a strictly higher sum rate for a given pair of input distributions.

The <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">K</i> -user cyclic Z-interference channel models a situation in which the <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">k</i> th transmitter causes interference only to the ( <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">k</i> -1)th receiver in a cyclic manner, e.g., the first transmitter causes interference only to the <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">K</i> th receiver. The impact of noiseless feedback on the capacity of this channel is studied by focusing on the Gaussian cyclic Z-interference channel. To this end, the symmetric feedback capacity of the linear shift deterministic cyclic Z-interference channel is completely characterized for all interference regimes. Using insights from the linear deterministic channel model, the symmetric feedback capacity of the Gaussian cyclic Z-interference channel is characterized up to within a constant number of bits. As a byproduct of the constant gap result, the symmetric generalized degrees of freedom with feedback for the Gaussian cyclic Z-interference channel are also characterized. These results highlight that the symmetric feedback capacities for both linear and Gaussian channel models are in general functions of <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">K</i> , the number of users. Furthermore, the capacity gain obtained due to feedback decreases as <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">K</i> increases.

One of the most stunning results of information theory is the channel coding theorem addressing the maximum rate of reliable communication over a noisy channel, known as channel capacity. In this thesis, we consider two problems emerging from the classic channel coding theorem. First, we study the extremal problems of the channel reliability function, which is the exponent with which the probability of making a wrong decision vanishes. To this end, we introduce a set of fundamental channels which exhibit significant monotonicity properties and invoke the theory of Chebychev systems to utilize such properties. We show that the binary symmetric channel (BSC) and binary erasure channel (BEC), which happen to be among the fundamental channels, are the two extremes of the channel reliability function. Also, we show that given a rate and a probability of error as a performance measure, BSC (BEC) needs the longest (shortest) code length to achieve such performance. While the first problem is pure theoretical, the second problem addresses a challenging practical scenario. The most fundamental assumption in the classic channel coding theorem is that we receive as many symbols as we send. In reality, however, this is not always true, e.g., a miss-sampling at a conventional receiver might duplicate a symbol. The extra symbol confuses a receiver as it has no clue about the position of duplication. Such scenarios are collectively known as channels with synchronization errors. Unlike their classic counterparts, there is only little known about either the capacity or coding techniques for channels with synchronization errors, even in their simplest forms. In this part, we study the duplication channel by introducing a series expansion for its capacity.

Ultra-reliable low-latency communication (URLLC) is a key technology in future wireless communications, and finite blocklength (FBL) coding is the core of the URLLC. In this paper, FBL coding schemes for the wireless multi-antenna channels are proposed, which are based on the classical Schalkwijk-Kailath scheme for the point-to-point additive white Gaussian noise channel with noiseless feedback. Simulation examples show that the proposed feedback-based schemes almost approach the corresponding channel capacities.