Bit-wise Sequential Coding with Feedback

In this paper, we study the problem of simulating a DMC channel from another DMC channel under an average-case and an exact model. We present several achievability and infeasibility results, with tight characterizations in special cases. In particular for the exact model, we fully characterize when a BSC channel can be simulated from a BEC channel when there is no shared randomness. We also provide infeasibility and achievability results for simulation of a binary channel from another binary channel in the case of no shared randomness. To do this, we use properties of Rényi capacity of a given order. We also introduce a notion of "channel diameter" which is shown to be additive and satisfy a data processing inequality.

In this paper, we put forward the tree-structured expectation propagation (TEP) algorithm for decoding block and convolutional low-density parity-check codes over any binary channel. We have already shown that TEP improves belief propagation (BP) over the binary erasure channel (BEC) by imposing marginal constraints over a set of pairs of variables that form a tree or a forest. The TEP decoder is a message-passing algorithm that sequentially builds a tree/forest of erased variables to capture additional information disregarded by the standard BP decoder, which leads to a noticeable reduction of the error rate for finite-length codes. In this paper, we show how the TEP can be extended to any channel, specifically to binary memoryless symmetric (BMS) channels. We particularly focus on how the TEP algorithm can be adapted for any channel model and, more importantly, how to choose the tree/forest to keep the gains observed for block and convolutional LDPC codes over the BEC.

In interactive coding, Alice and Bob wish to compute some function $f$ of their individual private inputs $x$ and $y$. They do this by engaging in a non-adaptive (fixed order, fixed length) protocol to jointly compute $f(x,y)$. The goal is to do this in an error-resilient way, such that even given some fraction of adversarial corruptions to the protocol, both parties still learn $f(x,y)$. In this work, we study the optimal error resilience of such a protocol in the face of adversarial bit flip or erasures. While the optimal error resilience of such a protocol over a large alphabet is well understood, the situation over the binary alphabet has remained open. In this work, we resolve this problem of determining the optimal error resilience over binary channels. In particular, we construct protocols achieving $\frac16$ error resilience over the binary bit flip channel and $\frac12$ error resilience over the binary erasure channel, for both of which matching upper bounds are known. We remark that the communication complexity of our binary bit flip protocol is polynomial in the size of the inputs, and the communication complexity of our binary erasure protocol is linear in the size of the minimal noiseless protocol computing $f$.

Recently, it has been verified that the Reed-Muller (RM) code, which has been predicted for a long time in the field of code theory, achieves capacity in a binary erasure channel (BEC) with a MAP (maximum a posteriori) decoder. Through weight distribution interpretation, it was also discovered that the RM code achieves a binary memoryless symmetric (BMS) channel capacity. A successive cancellation list (SCL) technique has been studied as a practical decoder instead of a MAP decoder. In this paper, the performance improvement potential that can be improved when a generalized concatenated code (GCC) with an outer code is designed to improve the practical SCL decoder performance is analyzed. When information bits are used as frozen bits in these code-word designs, we analyze an inter-code-word distance and the decoding outage probability of the designed code. We expect that our results can be used as a conceptual base for improving communication methods in the future development of channel encoding technology.

When several independent channels are coupled by a parity check constraint on their inputs, the mutual information between the input of one channel and the outputs of all other channels can be expressed as a combination of the mutual information between the input and the output of each individual channel. This concept is denoted as information combining. For binary-input symmetric discrete memoryless channels, we present bounds on the combined information which are only based on the mutual information of the channels. Furthermore, we show that these bounds cannot be further improved. Exact expressions are provided for the case that all channels are binary symmetric channels and for the case that all channels are binary erasure channels.

With the advent of 5G and technologies such as cloud computing, Internet-of-Things (IoT), etc, future communication networks will consist of a large number of heterogeneous devices connected together. A critical aspect will be ensuring that communication is not only fast and reliable, but also resilient to malicious attack. As networks increasingly adopt zero-trust principles in their security frameworks, it is important to consider such attacks from a zero-trust perspective. Motivated by this, we consider the problem of communicating a message reliably across a binary erasure channel (BEC(<inline-formula> <tex-math notation="LaTeX">$q$ </tex-math></inline-formula>)) or a binary symmetric channel (BSC(<inline-formula> <tex-math notation="LaTeX">$q$ </tex-math></inline-formula>)) against an adversary actively injecting additional erasures or flips at the channel&#x2019;s input. The adversary can corrupt up to a fraction <inline-formula> <tex-math notation="LaTeX">$p$ </tex-math></inline-formula> of the transmitted bits and knows the transmission scheme agreed upon by the communicating terminals. Further, he has the capability to causally snoop in on both the transmitter and the receiver in real time, i.e., if <inline-formula> <tex-math notation="LaTeX">${\mathbf {x}}=(x_{1},x_{2},\ldots, x_{n})$ </tex-math></inline-formula> and <inline-formula> <tex-math notation="LaTeX">${\mathbf {y}}=(y_{1},y_{2},\ldots, y_{n})$ </tex-math></inline-formula> denote the transmitted and the received codewords respectively, at each time <inline-formula> <tex-math notation="LaTeX">$k$ </tex-math></inline-formula>, he knows <inline-formula> <tex-math notation="LaTeX">$(x_{1},x_{2},\ldots, x_{k})$ </tex-math></inline-formula> and <inline-formula> <tex-math notation="LaTeX">$(y_{1},y_{2},\ldots, y_{k-1})$ </tex-math></inline-formula>. Using his side-information, the adversary is free to employ any attack that respects his constraint on the number of corruptible bits. We prove an information-theoretic tight capacity characterization as a function of <inline-formula> <tex-math notation="LaTeX">$p$ </tex-math></inline-formula> and <inline-formula> <tex-math notation="LaTeX">$q$ </tex-math></inline-formula> for (i) the erasure adversary with a BEC(<inline-formula> <tex-math notation="LaTeX">$q$ </tex-math></inline-formula>) and (ii) the bit-flip adversary with a BSC(<inline-formula> <tex-math notation="LaTeX">$q$ </tex-math></inline-formula>). A unique feature of our models is the compounding of stochastic and adversarial noise sources. Our analysis reveals the worst-case adversarial attacks for both models and proves the existence of coding schemes that achieve rates equal to the capacity for any adversarial attack. In the case of bit-flips, we show that, interestingly, when <inline-formula> <tex-math notation="LaTeX">$p$ </tex-math></inline-formula> is below a certain threshold (that depends on <inline-formula> <tex-math notation="LaTeX">$q$ </tex-math></inline-formula>), the adversary is no worse than an i.i.d. memory-less noise source.

An integer programming-based search technique for error-prone structures of LDPC codes

In a network application, channel coding alone is not sufficient to reliably transmit a message of finite length K from a source to one or more destinations as in, e.g., file transfer. To ensure that no data is lost, it must be combined with rateless erasure correcting schemes on a higher layer, such as a time-division multiple access (TDMA) system paired with automatic repeat request (ARQ) or random linear network coding (RLNC). We consider binary channel coding on a binary symmetric channel (BSC) and q-ary RLNC for erasure correction in a star network, where Y sources send messages to each other with the help of a central relay. In this scenario RLNC has been shown to have a throughput advantage over TDMA schemes as K→∞ and q→∞. In this paper we focus on finite block lengths and compare the expected throughputs of RLNC and TDMA. For a total message length of K bits, which can be subdivided into blocks of smaller size prior to channel coding, we obtain the channel code rate and the number of blocks that maximize the expected throughput of both RLNC and TDMA, and we find that TDMA is more throughput-efficient for small message lengths K and small q.

This article describes an encoding strategy through repetition codes for error protecting of the WiMedia payload. Such codes have usually their application rather constrained due to their low transmission rates over low error probabilities. A binary symmetric channel (BSC) was used to evaluate the system performance and a number of simulations were done to assess the influence of the parameters of the coded system. The chief idea is to select the maximum acceptable error probability and fade probability followed by the designing of the repetition coding scheme so as to maximize the transmission rate. Simulations results show that this pseudorandom adaptive technique presents better performance at low error probability, besides offering a tradeoff between rate and error probability.

On the fast-fading discrete memoryless channels (DMCs) with channel state information at the receiver a necessary and sufficient condition is determined for the tilted channel associated with an input distribution to be the product of a tilted channel state distribution and the tilted channel associated with the same input distribution on the non-fading DMC for the given channel state. On fading channels for which constituent channels associated with different channel states share a common Augustin capacity-achieving input distribution aforementioned necessary and sufficient condition is shown to hold, and a parametric form for the sphere packing exponent (SPE) is obtained in terms of the tilted channel state distribution and the tilted channel of the nonfading DMC for the given channel state. The SPE of fast-fading binary erasure and binary symmetric channels with channel state information at the receiver are analyzed as examples.

A simple statistical channel model is suggested to account for single position bit shifts that have been identified as one of the major impairments in magnetic recording. The authors investigate bounds on the capacity of this channel where the channel inputs are the (d,k) codes commonly used in magnetic recordings. For d>or=2, this channel is conveniently described in terms of phrase lengths where a bit shift causes a phrase either to shrink or to expand. The inherent correlation present in consecutive shift-affected phrases introduces memory into the channel model. Sequences of nondecreasing lower bounds and nonincreasing upper bounds on the capacity are evaluated and investigated for a variety of parameters. Lower bounds on the zero-error capacity are also studied. The channel model is extended to a concatenated scheme of a bit shift channel connected in tandem with the binary symmetric channel, thus capturing both major error generating mechanisms-the bit shifts as well as randomly generated errors. Lower and upper bounds on the capacity of this concatenated channel model with (d,k) input sequences are derived and discussed. >

Bounds on the capacities of binary symmetric and additive Gaussian channels with run-length limited two-level (binary, bipolar) inputs are presented and their tightness Is demonstrated for some interesting cases. Stationary input sequences which do not degrade capacity, are considered in the derivation of the bounds.

We derive upper and lower bounds on the encoding and decoding complexity of two capacity-achieving ensembles of irregular repeat-accumulate (IRA1 and IRA2) codes on the binary erasure channel (BEC). These bounds are expressed in terms of the gap between the channel capacity and the rate of a typical code from this ensemble for which reliable communications is achievable under message-passing iterative (MPI) decoding. The complexity of the ensemble of IRA1 codes grows like the negative logarithm of the gap to capacity. On the other hand, the complexity of the ensemble of IRA2 codes with any choice of the degree distribution grows at least like the inverse square root of the gap to capacity, and at most like the inverse of the gap to capacity.

We consider interactive communication performed over two simple types of noisy channels: binary error channels with noiseless feedback and binary erasure channels. In both cases, the noise model is adversarial. Assuming at most e-fraction of the bits can be corrupted, we show coding schemes that simulate any alternating interactive protocol with rate 1 − Θ(H(e)). All our simulations are simple, randomized, and computationally efficient.The rates of our coding schemes stand in contrast to the interactive communication rates supported by random or adversarial error channels without feedback, for which the best known coding schemes achieve rates of 1 − Θ([EQUATION]e) and 1 − Θ([EQUATION]e log log 1/e), respectively. As these rates are conjectured to be optimal, our result implies a large asymptotic gap between interactive communication rate over noisy channels with and without feedback. Such a gap has no equivalent in the standard one-way communication setting.

When estimating the bit error rate (BER) of communications systems using Monte Carlo (MC) simulation, importance sampling (IS) may be a powerful method for reducing simulation runtimes. The paper presents an IS technique combined with error event simulation in order to evaluate efficiently the performance of Viterbi decoding used with a binary nonsymmetric channel and with a binary erasure channel. Two biasing approaches are employed, stationary and nonstationary models. Comparative results between the two models and traditional MC methods are presented.