Distributed Hypothesis Testing over Noisy Broadcast Channels

A central challenge in hypothesis testing (HT) lies in determining the optimal balance between Type I (false positive) and Type II (non-detection or false negative) error probabilities. Analyzing these errors' exponential rate of convergence, known as error exponents, provides crucial insights into system performance. Error exponents offer a lens through which we can understand how operational restrictions, such as resource constraints and impairments in communications, affect the accuracy of distributed inference in networked systems. This survey presents a comprehensive review of key results in HT, from the foundational Stein's Lemma to recent advancements in distributed HT, all unified through the framework of error exponents. We explore asymptotic and non-asymptotic results, highlighting their implications for designing robust and efficient networked systems, such as event detection through lossy wireless sensor monitoring networks, collective perception-based object detection in vehicular environments, and clock synchronization in distributed environments, among others. We show that understanding the role of error exponents provides a valuable tool for optimizing decision-making and improving the reliability of networked systems.

A secret key agreement framework involving three users is considered in which each of the users 1 and 2 intends to share a secret key with user 3 and users 1 and 2 are eavesdroppers with respect to each other. There is a generalized discrete memoryless multiple access channel (GDMMAC) from users 1 and 2 to user 3 where the three users receive outputs from the channel. Furthermore, there is a feedback channel from user 3 to users 1 and 2 through which user 3 sends information extracted from the received output from the GDMMAC to increase the key rates. We consider both noiseless and noisy feedback. In the case of noiseless feedback, a public channel of unlimited capacity from user 3 to users 1 and 2 is used only once. In the case of noisy feedback, a noisy broadcast channel (BC) from user 3 to users 1 and 2 can be repeatedly used, like GDMMAC. In both setups, inner bounds of the secret key capacity region are derived. The secret key capacity region is derived in some special cases where the channel inputs and outputs form Markov chains in certain orders. For illustration, the corresponding results are also derived and discussed for Gaussian channels. The cases with noiseless feedback, noisy feedback, and no feedback at all are compared with each other.

A single-sensor two-detectors system is considered where the sensor communicates with both detectors and Detector 1 communicates with Detector 2, all over noise-free rate-limited links. The sensor and both detectors observe discrete memoryless source sequences whose joint probability mass function depends on a binary hypothesis. The goal at each detector is to guess the binary hypothesis in a way that, for increasing observation lengths, the probability of error under one of the hypotheses decays to zero with largest possible exponential decay, whereas the probability of error under the other hypothesis can decay to zero or to a small positive number arbitrarily slow. For the setting with positive communication rates from the sensor to the detectors and when both detectors are interested in maximizing the error exponent under the same hypothesis, we characterize the set of all possible exponents in a special case of testing against independence. In this case the cooperation link allows Detector 2 to increase its Type-II error exponent by an amount that is equal to the exponent attained at Detector 1. We also provide a general inner bound on the set of achievable error exponents that shows a tradeoff between the exponents at the two detectors in most cases. When the two detectors aim at maximizing the error exponent under different hypotheses and the distribution at the Sensor is different under the two hypotheses, then we show that such a tradeoff does not exist. We propose a general scheme that allows each detector to attain the same exponent as if it was the only detector in the system. For the setting with zero-rate communication on both links, we exactly characterize the set of possible exponents and the gain brought up by cooperation, in function of the number of bits that are sent over the two links. Notice that, for this setting, tradeoffs between the exponents achieved at the two detectors arise only in few particular cases. In all other cases, each detector achieves the same performance as if it were the only detector in the system.

This letter investigates the role of index coding in the capacity of AWGN broadcast channels with receiver message side information. We first show that index coding is unnecessary where there are two receivers; multiplexing coding and superposition coding are sufficient to achieve the capacity region. We next show that, for more than two receivers, multiplexing coding and superposition coding alone can be suboptimal. We give an example where these two coding schemes alone cannot achieve the capacity region, but adding index coding can. This demonstrates that, in contrast to the two-receiver case, multiplexing coding cannot fulfill the function of index coding where there are three or more receivers.

This paper investigates the joint source-channel coding problem of sending a memoryless source over a memoryless broadcast channel. An inner bound and several outer bounds on the admissible distortion region are derived, which respectively generalize and unify several existing bounds. As a consequence, we also obtain an inner bound and an outer bound for the degraded broadcast channel case. When specialized to the Gaussian or binary source broadcast, the inner bound and outer bound not only recover the best known inner bound and outer bound in the literature, but also generate some new results. Besides, we also extend the inner bound and outer bounds to the Wyner-Ziv source broadcast problem, i.e., source broadcast with side information available at decoders. Some new bounds are obtained when specialized to the Wyner-Ziv Gaussian and Wyner-Ziv binary cases.

We determine the capacity region of a 3-receiver less noisy broadcast channel. The difficulty in extending the two-receiver result to three-receivers involves extending the Csiszar-sum lemma to three or more sequences, a standard difficulty in this area. In this work we bypass the difficulty by using a new information inequality, for less noisy receivers, that is employed in the converse. We also generalize our result to obtain the capacity region for a class of less noisy receivers.

Capacity formulas and random-coding and sphere-packing exponents are derived for a generalized family of Gel'fand-Pinsker coding problems. Information is to be reliably transmitted through a noisy channel with random state sequence. Partial information about the state sequence is available to the encoder and decoder. Two families of channels are considered: (1) compound discrete memoryless channels (C-DMC); and (2) channels with arbitrary memory, subject to an additive cost constraint, or more generally to a constraint on the conditional type of the channel output given the input. Both problems are closely connected. For the C-DMC case, our random-coding and sphere-packing exponents coincide at high rates, thereby determining the reliability function of the channel family. The random-coding exponent is achieved using a 3D binning scheme and a maximum penalized mutual information decoder. In the case of arbitrary channels, with memory, a larger random-coding error exponent than in the C-DMC case is obtained. Applications of this study include watermarking, data hiding, communication in presence of partially known interferers, and problems such as broadcast channels, all of which involve the fundamental idea of binning.

A distributed binary hypothesis testing (HT) problem involving two parties, one referred to as the observer and the other as the detector is studied. The observer observes a discrete memoryless source (DMS) and communicates its observations to the detector over a discrete memoryless channel (DMC). The detector observes another DMS correlated with that at the observer, and performs a binary hypothesis test on the joint distribution of the two DMS's using its own observed data and the information received from the observer. The trade-off between the type I error probability and the type II error-exponent of the HT is explored. Single-letter lower bounds on the optimal type II error-exponent are obtained by using two different coding schemes, a separate HT and channel coding scheme and a joint HT and channel coding scheme based on hybrid coding for the matched bandwidth case. Exact single-letter characterization of the same is established for the special case of testing against conditional independence, and it is shown to be achieved by the separate HT and channel coding scheme. An example is provided where the joint scheme achieves a strictly better performance than the separation based scheme.

Hypothesis tests

Information theory was developed to determine the fundamental limits on the performance of communication systems. Detection theory involves an application of statistical decision theory to the problem of determining the presence or absence of signals in noise. Both of these theories deal with the communication problem. The relationship between information theory and conventional (centralized) detection theory has been discussed in the literature. Kullback [Ku159] discussed the use of discrimination to study hypothesis testing problems. Middleton [Mid60] employed cost functions based on information theory for optimizing signal detection systems. Csiszar et al. [CsL71] and Blahut [Bla74] have formulated the detection problem as a coding problem and have carried out an asymptotic analysis of detection systems based on the error exponent function. In this chapter, we briefly present some extensions of the above work for the distributed detection problem. In Section 7.2, we discuss the design of distributed detection systems based on an information theoretic cost function. In Section 7.3, we present a brief summary of some asymptototic results on the performance of distributed detection systems.

The classical result of Blahut, which characterizes achievable error exponents in binary hypothesis testing, is generalized in two different directions. First, in M-ary hypothesis testing, the tradeoff of all M(M-1) types of error exponents and corresponding optimal decision schemes are explored. Then, motivated by a power-constrained distributed detection scenario, binary hypothesis testing is revisited, and the tradeoff of power consumption versus error exponents is fully characterized. In the latter scenario, sensors are allowed to make random decisions as to whether they should remain silent and save power, or transmit and improve detection quality. It is then shown by an example that optimal sensor decisions may indeed be random

We study the transmission of two discrete memoryless correlated sources, consisting of a common and a private source, over a discrete memoryless multiterminal channel with two transmitters and two receivers. At the transmitter side, the common source is observed by both encoders but the private source can only be accessed by one encoder. At the receiver side, both decoders need to reconstruct the common source, but only one decoder needs to reconstruct the private source. We hence refer to this system by the asymmetric two-user source-channel coding system. We derive a universally achievable lossless joint source-channel coding (JSCC) error exponent pair for the two-user system by using a technique which generalizes Csiszar's type-packing lemma (1980) for the point-to-point (single-user) discrete memoryless source-channel system. We next investigate the largest convergence rate of asymptotic exponential decay of the system (overall) probability of erroneous transmission, i.e., the system JSCC error exponent. We obtain lower and upper bounds for the exponent. As a consequence, we establish a JSCC theorem with single-letter characterization and we show that the separation principle holds for the asymmetric two-user scenario. By introducing common randomization, we also provide a formula for the tandem (separate) source-channel coding error exponent. Numerical examples show that for a large class of systems consisting of two correlated sources and an asymmetric multiple-access channel with additive noise, the JSCC error exponent considerably outperforms the corresponding tandem coding error exponent.

Two different types of adaptive networks are considered for solving the centralized and distributed hy-pothesis testing problem. The performance of the two different types of networks is compared under differentperformance indices and training rules. It is shown that training rules based on the Neyman-Pearson criterion outperform error based training rules. Simulations are provided for data that are linearly and nonlinearly separable. I. INTRODUCTION The optimum Bayesian and Neyman-Pearson solution to the distributed decision fusion problem bearsstriking similarities to the structure of a neural network (NN), [28,29]. Moreover, NNs can, in principle learn arbitrary input-output mappings, provided that they are sufficiently smooth. These two facts motivate theuse of NNs for solving the centralized and distributed hypothesis testing problem. In selecting the properNN layout, one could argue that a perceptron-type NN can learn any input-output mapping, thus it can betrained to solve the hypothesis testing problem. However, the ability of a perceptron-type NN to learn an

The effect of quantization of prior probabilities in a collection of distributed Bayesian binary hypothesis testing problems over which the priors themselves vary is studied, with focus on conflicting agents. Conflict arises from differences in Bayes costs, even when all agents desire correct decisions and agree on the meaning of correct. In a setting with fusion of local binary decisions by majority rule, Nash equilibrium local decision strategies are found. Assuming that agents follow Nash equilibrium decision strategies, designing quantizers for prior probabilities becomes a strategic form game, we discuss its Nash equilibria. We also propose two different constrained quantizer design games, find Nash equilibrium quantizer designs, and compare performance. The system has deadweight loss: equilibrium decisions are not Pareto optimal.

A distributed hypothesis testing (HT) problem is considered, comprising two nodes and a unidirectional communication link. The receiving node is required to make a decision as to the probability distribution in effect. A binning process is used in order to minimize the probability of error, resulting in a new achievable error-exponent. A sub-class of HT problems with general hypotheses is defined, which contains many interesting and relevant problems. The advantage of the binning strategy in comparison to the non-binning approach is demonstrated by means of a binary symmetric example.