Strong converse, feedback channel capacity and hypothesis testing

Nonlinear feedback communication schemes proposed up to now have been restricted to coherent channels. This paper describes a scheme which not only lacks this restriction but yields performance better than the coherent ones that have been analyzed. Block coded orthogonal signals together with incoherent receiver forms are used on both forward and feedback links. It is assumed that the transmitter has a high peak power capability. However, it is shown that by increasing the code length the duty cycle of this mode can be made sufficiently small so that the contribution to average power is negligible. It is found that the probability of error for every message is upper-bounded by exp[-E(R)T], where T represents the code length and E(R) is a function of the transmission rate, R , the capacities of the forward and feedback channels, and a parameter that determines the rate with which the duty cycle decreases with increasing code length. When the forward and feedback capacities are equal, say C , the E(R) versus R curve can be made to approach a curve that starts at 2C and decreases monotonically to the value C at R = C . This contrasts with the corresponding curve without feedback which starts at C/2 and decreases to zero at R = C . The main advantage of information feedback over no feedback, namely E(R = C) > 0 , where C is the forward channel capacity, can also be obtained even when the capacity of the feedback channel is less than that of the forward. To obtain such behavior, existing schemes require that the feedback channel capacity be at least as great as the forward.

Along with the channel capacity, the error exponent is one of the most important information-theoretic measures of reliability, as it sets ultimate bounds on the performance of communication systems employing codes of finite complexity. In this paper, we derive an exact analytical expression for the random coding error exponent, which provides significant insight regarding the ultimate limits to communications through Nakagami-m fading channels. An important fact about this error exponent is that it determines the behavior of error probability in terms of the transmission rate and the code length that reflects the coding complexity required to achieve a certain level of reliability. Moreover, from the derived analytical expression, we can easily compute the necessary codeword length without extensive Monte-Carlo simulation to achieve a predefined upper bound for error probability at a rate below the channel capacity. We also improve the random coding bound by expurgating bad codewords from the code ensemble, since random coding error exponent is determined by selecting codewords independently according to the input distribution where good and bad codewords contribute equally to the overall average error probability. Finally, we derive exact analytical expressions for the cutoff rate, critical rate, and expurgation rate and verify the analytical expressions via Monte-Carlo simulation.

Suppose Q is a family of discrete memoryless channels. An unknown member of Q will be available, with perfect, causal output feedback for communication. Is there a coding scheme (possibly with variable transmission time) that can achieve the Burnashev error exponent uniformly over Q ? For two families of channels we show that the answer is yes. Furthermore, for each of these two classes, in addition to achieve the maximum error exponent, it is possible to uniformly attain any given fraction of the channel capacity. Therefore, in terms of achievable rates and delay, there are situations in which the knowledge of the channel becomes irrelevant. In the second part of the thesis, we show that for arbitrary sets of channels the Burnashev error exponent cannot in general be uniformly achieved. In particular we give a sufficient condition for a pair of channels so that no coding strategy reaches Burnashev's exponent simultaneously on both channels. As a third part we study a scenario where communication is carried by first testing the channel by means of a training sequence, then coding according to the channel estimate. We provide an upper bound on the maximum achievable error exponent of such coding schemes. This bound is typically much lower than the maximum achievable error exponent over a channel with feedback. For example in the case of binary symmetric channels this bound has a slope that vanishes at capacity. This result suggests that in terms of error exponent, a good universal feedback scheme combines channel estimation with information delivery, rather than separating them. In the final chapter, we address the question of communicating quickly and reliably. We consider a simple situation of two message communication over a known channel with feedback. We propose a simple decoding rule, and show that it minimizes a weighted combination of the probability of error and decoding delay for a certain range of crossover probabilities and combination weights.

A central challenge in hypothesis testing (HT) lies in determining the optimal balance between Type I (false positive) and Type II (non-detection or false negative) error probabilities. Analyzing these errors' exponential rate of convergence, known as error exponents, provides crucial insights into system performance. Error exponents offer a lens through which we can understand how operational restrictions, such as resource constraints and impairments in communications, affect the accuracy of distributed inference in networked systems. This survey presents a comprehensive review of key results in HT, from the foundational Stein's Lemma to recent advancements in distributed HT, all unified through the framework of error exponents. We explore asymptotic and non-asymptotic results, highlighting their implications for designing robust and efficient networked systems, such as event detection through lossy wireless sensor monitoring networks, collective perception-based object detection in vehicular environments, and clock synchronization in distributed environments, among others. We show that understanding the role of error exponents provides a valuable tool for optimizing decision-making and improving the reliability of networked systems.

Information theory tells us that if the rate of sending information across a noisy channel were above the capacity of that channel, then the transmission would necessarily be unreliable. For classical information sent over classical or quantum channels, one could, under certain conditions, make a stronger statement that the reliability of the transmission shall decay exponentially to zero with the number of channel uses, and the proof of this statement typically relies on a certain fundamental bound on the reliability of the transmission. Such a statement or the bound has never been given for sending quantum information. We give this bound and then use it to give the first example where the reliability of sending quantum information at rates above the capacity decays exponentially to zero. We also show that our framework can be used for proving generalized bounds on the reliability.

The classical result of Blahut, which characterizes achievable error exponents in binary hypothesis testing, is generalized in two different directions. First, in M-ary hypothesis testing, the tradeoff of all M(M-1) types of error exponents and corresponding optimal decision schemes are explored. Then, motivated by a power-constrained distributed detection scenario, binary hypothesis testing is revisited, and the tradeoff of power consumption versus error exponents is fully characterized. In the latter scenario, sensors are allowed to make random decisions as to whether they should remain silent and save power, or transmit and improve detection quality. It is then shown by an example that optimal sensor decisions may indeed be random

A single-sensor two-detectors system is considered where the sensor communicates with both detectors and Detector 1 communicates with Detector 2, all over noise-free rate-limited links. The sensor and both detectors observe discrete memoryless source sequences whose joint probability mass function depends on a binary hypothesis. The goal at each detector is to guess the binary hypothesis in a way that, for increasing observation lengths, the probability of error under one of the hypotheses decays to zero with largest possible exponential decay, whereas the probability of error under the other hypothesis can decay to zero or to a small positive number arbitrarily slow. For the setting with positive communication rates from the sensor to the detectors and when both detectors are interested in maximizing the error exponent under the same hypothesis, we characterize the set of all possible exponents in a special case of testing against independence. In this case the cooperation link allows Detector 2 to increase its Type-II error exponent by an amount that is equal to the exponent attained at Detector 1. We also provide a general inner bound on the set of achievable error exponents that shows a tradeoff between the exponents at the two detectors in most cases. When the two detectors aim at maximizing the error exponent under different hypotheses and the distribution at the Sensor is different under the two hypotheses, then we show that such a tradeoff does not exist. We propose a general scheme that allows each detector to attain the same exponent as if it was the only detector in the system. For the setting with zero-rate communication on both links, we exactly characterize the set of possible exponents and the gain brought up by cooperation, in function of the number of bits that are sent over the two links. Notice that, for this setting, tradeoffs between the exponents achieved at the two detectors arise only in few particular cases. In all other cases, each detector achieves the same performance as if it were the only detector in the system.

This paper explores the relationship between two ideas in the network information theory: edge removal and strong converses. Edge removal properties state that if an edge of small capacity is removed from a network, the capacity region does not change too much. Strong converses state that, for rates outside the capacity region, the probability of error converges to 1 as the blocklength goes to infinity. Various notions of edge removal and strong converse are defined, depending on how edge capacity and error probability scale with blocklength, and relations between them are proved. Each class of strong converse implies a specific class of edge removal. The opposite directions are proved for deterministic networks. Furthermore, a technique based on a novel, causal version of the blowing-up lemma is used to prove that for discrete memoryless networks, the weak edge removal property—that the capacity region changes continuously as the capacity of an edge vanishes—is equivalent to the exponentially strong converse—that outside the capacity region, the probability of error goes to 1 exponentially fast. This result is used to prove exponentially strong converses for several examples, including the discrete two-user interference channel with strong interference, with only a small variation from traditional weak converse proofs.

In this paper, a new class of lower bounds on the probability of error for m-ary hypothesis tests is proposed. Computation of the minimum probability of error which is attained by the maximum a-posteriori probability (MAP) criterion, is usually not tractable. The new class is derived using Holder¿s inequality. The bounds in this class are continuous and differentiable function of the conditional probability of error and they provide good prediction of the minimum probability of error in multiple hypothesis testing. It is shown that for binary hypothesis testing problem this bound asymptotically coincides with the optimum probability of error provided by the MAP criterion. This bound is compared with other existing lower bounds in several typical detection and classification problems in terms of tightness and computational complexity.

This work studies error exponent limits in hypothesis testing (HT) in a distributed scenario with partial communication constraints. We derive general conditions on the Type I error restriction under which the error exponent of the optimal Type II error has a closed-form characterization for the task of testing against independence. We show that the error exponent is preserved for a family of decreasing Type I error restrictions. Complementing this analysis, new expressions are derived to bound the optimal Type II error probability for a finite number of observations. These bounds shed light about the velocity at which error exponent limits are attained with the number of samples.

The problem of multiple hypothesis testing (HT) for arbitrarily varying sources (AVS) is considered. The achievable error probability exponents (reliabilities) region is derived, optimal decision schemes are described. The result extends the known ones by Fu and Shen and by Tuncel. The Chernoff bounds for AVS binary and M-ary HT are specified via indication of a Sanov theorem for those sources.

We prove that several known upper bounds on the classical capacity of thermal and additive noise bosonic channels are actually strong converse rates. Our results strengthen the interpretation of these upper bounds, in the sense that we now know that the probability of correctly decoding a classical message rapidly converges to 0 in the limit of many channel uses if the communication rate exceeds these upper bounds. In order for these theorems to hold, we need to impose a maximum photon number constraint on the states input to the channel (the strong converse property need not hold if there is only a mean photon number constraint). Our first theorem demonstrates that Koenig and Smith's upper bound on the classical capacity of the thermal bosonic channel is a strong converse rate, and we prove this result by utilizing the structural decomposition of a thermal channel into a pure-loss channel followed by an amplifier channel. Our second theorem demonstrates that Giovannetti et al.'s upper bound on the classical capacity of a thermal bosonic channel corresponds to a strong converse rate, and we prove this result by relating the success probability to the rate, the effective dimension of the output space, and the purity of the channel as measured by the R\'enyi collision entropy. Finally, we use similar techniques to prove that similar previously known upper bounds on the classical capacity of an additive noise bosonic channel correspond to strong converse rates.

In this work we show how an improved lower bound to the error exponent of the memoryless multiple-access (MAC) channel is attained via the use of linear codes, thus demonstrating that structure can be beneficial even in cases where there is no capacity gain. We show that if the MAC channel is modulo-additive, then any error probability, and hence any error exponent, achievable by a linear code for the corresponding single-user channel, is also achievable for the MAC channel. Specifically, for an alphabet of prime cardinality, where linear codes achieve the best known exponents in the single-user setting and the optimal exponent above the critical rate, this performance carries over to the MAC setting. At least at low rates, where expurgation is needed, our approach strictly improves performance over previous results, where expurgation was used at most for one of the users. Even when the MAC channel is not additive, it may be transformed into such a channel. While the transformation is lossy, we show that the distributed structure gain in some "nearly additive" cases outweighs the loss, and thus the error exponent can improve upon the best known error exponent for these cases as well. Finally we apply a similar approach to the Gaussian MAC channel. We obtain an improvement over the best known achievable exponent, given by Gallager, for certain rate pairs, using lattice codes which satisfy a nesting condition.

We consider multiple repetition strategies with fixed delay decoding for discrete memoryless channels with noiseless feedback. Existing binary schemes by Schalkwijk and Zigangirov are analyzed and their results are extended. The general error exponents are computed and presented by elegant expressions in the strictly symmetric case. An important class of precoded sequences, so-called flip sequences, is found and their degrading effect on the error exponent is investigated. This effect is shown negligible when the repetition parameters are chosen such that the transmission rate is maximized. Even when signalling at channel capacity, the error exponent is shown to be strictly positive.

The <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">K</i> -user cyclic Z-interference channel models a situation in which the <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">k</i> th transmitter causes interference only to the ( <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">k</i> -1)th receiver in a cyclic manner, e.g., the first transmitter causes interference only to the <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">K</i> th receiver. The impact of noiseless feedback on the capacity of this channel is studied by focusing on the Gaussian cyclic Z-interference channel. To this end, the symmetric feedback capacity of the linear shift deterministic cyclic Z-interference channel is completely characterized for all interference regimes. Using insights from the linear deterministic channel model, the symmetric feedback capacity of the Gaussian cyclic Z-interference channel is characterized up to within a constant number of bits. As a byproduct of the constant gap result, the symmetric generalized degrees of freedom with feedback for the Gaussian cyclic Z-interference channel are also characterized. These results highlight that the symmetric feedback capacities for both linear and Gaussian channel models are in general functions of <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">K</i> , the number of users. Furthermore, the capacity gain obtained due to feedback decreases as <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">K</i> increases.