The optimal error resilience of interactive communication over binary channels

In this paper, we study the problem of simulating a DMC channel from another DMC channel under an average-case and an exact model. We present several achievability and infeasibility results, with tight characterizations in special cases. In particular for the exact model, we fully characterize when a BSC channel can be simulated from a BEC channel when there is no shared randomness. We also provide infeasibility and achievability results for simulation of a binary channel from another binary channel in the case of no shared randomness. To do this, we use properties of Rényi capacity of a given order. We also introduce a notion of "channel diameter" which is shown to be additive and satisfy a data processing inequality.

In this paper, we put forward the tree-structured expectation propagation (TEP) algorithm for decoding block and convolutional low-density parity-check codes over any binary channel. We have already shown that TEP improves belief propagation (BP) over the binary erasure channel (BEC) by imposing marginal constraints over a set of pairs of variables that form a tree or a forest. The TEP decoder is a message-passing algorithm that sequentially builds a tree/forest of erased variables to capture additional information disregarded by the standard BP decoder, which leads to a noticeable reduction of the error rate for finite-length codes. In this paper, we show how the TEP can be extended to any channel, specifically to binary memoryless symmetric (BMS) channels. We particularly focus on how the TEP algorithm can be adapted for any channel model and, more importantly, how to choose the tree/forest to keep the gains observed for block and convolutional LDPC codes over the BEC.

This paper considers the problem of bit-wise channel coding over a Binary Symmetric Channel (BSC) with feedback. While it is known that feedback does not increase the capacity of a memoryless channel, it is believed to simplify the coding schemes for some channels. The most significant one is the Binary Erasure Channel (BEC) where capacity is achieved by a simple sequential bit-wise repetition code under which each bit is (re-)transmitted until it is received. This sequential bit-wise feedback code has the added advantage that it can be used for streaming applications over BECs. In contrast, there is no known sequential bit-wise code with feedback that can achieve nonzero transmission rate for a BSC. For example, under Posterior Matching for a binary input channel with feedback, also known as Horstein scheme, each message is considered in its entirety in a block coding manner. This paper proposes a sequential feedback coding scheme with a nested bit-wise structure that generalizes repetition codes. This scheme is shown to achieve strictly positive rate for a large class of binary input channels including a BSC with arbitrary cross-over probability $p \in(0, \frac{1}{2})$ . The analysis relies on characterizing a lower bound on the step-wise Extrinsic Jensen Shannon divergence.

We consider interactive communication performed over two simple types of noisy channels: binary error channels with noiseless feedback and binary erasure channels. In both cases, the noise model is adversarial. Assuming at most e-fraction of the bits can be corrupted, we show coding schemes that simulate any alternating interactive protocol with rate 1 − Θ(H(e)). All our simulations are simple, randomized, and computationally efficient.The rates of our coding schemes stand in contrast to the interactive communication rates supported by random or adversarial error channels without feedback, for which the best known coding schemes achieve rates of 1 − Θ([EQUATION]e) and 1 − Θ([EQUATION]e log log 1/e), respectively. As these rates are conjectured to be optimal, our result implies a large asymptotic gap between interactive communication rate over noisy channels with and without feedback. Such a gap has no equivalent in the standard one-way communication setting.

When estimating the bit error rate (BER) of communications systems using Monte Carlo (MC) simulation, importance sampling (IS) may be a powerful method for reducing simulation runtimes. The paper presents an IS technique combined with error event simulation in order to evaluate efficiently the performance of Viterbi decoding used with a binary nonsymmetric channel and with a binary erasure channel. Two biasing approaches are employed, stationary and nonstationary models. Comparative results between the two models and traditional MC methods are presented.

This is the author's accepted manuscript. The final published article is available from the link below. Copyright @ 2008 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other users, including reprinting/ republishing this material for advertising or promotional purposes, creating new collective works for resale or redistribution to servers or lists, or reuse of any copyrighted components of this work in other works.

: In this paper, we discuss the Stochastic Resonance (SR) effect for the binary-input ternary-output Discrete Memoryless Channels (DMC). The DMC is of great interest for various digital communications channels, including acoustic underwater communication. We focus on the capacity maximization, brought about by additive Gaussian noise, for the DMCs. We analyze the capacity of the threshold binary communication channels in the presence of Additive Gaussian Noise (AGN). In the particular case of binary asymmetric channels with symmetric erasures (BAC/SE) we derive the optimum noise level in order to obtain the maximum capacity. The stochastic resonance for the general binary erasure channel is compared with those of binary symmetric channels. For the general case (2, n) DMC, the Pinsker and Helgert capacity bounds were used for the study of the stochastic resonance.

Overlap-free words and finite automata

An error-resilient variable-length arithmetic code is presented whose codewords are represented by binary digits. The input sequence is partitioned in subsequences, each of which is individually encoded using an arithmetic coding scheme with an integrated bit-stuffing technique that restricts the number of consecutive ones in the output sequence. An all-ones sequence of fixed length is appended to serve as a sync marker when the codewords are concatenated. The bit-stuffing technique ensures that the sync markers do not occur anywhere except at the boundaries between the codewords. Expressions for the optimal choice of the marker length and the block length are derived. The performance of the proposed code is determined in terms of redundancy and error resilience. An upper bound on the average error rate is derived and its tightness is confirmed with computer simulations. The proposed code shows to significantly suppress the error rate at the expense of a minimum increase in redundancy.

We introduce new polar coding schemes for independent non-identically distributed parallel binary discrete memoryless channels. The first scheme is developed for the case where underlying channels are time invariant (the case of a deterministic channel parameters), while the other schemes deal with a scenario where underlying channels change based on a distribution (the case of random channel parameters). For the former case, we also discuss the importance of the usage of an interleaver Q to enhance system reliability, and for the latter case, we model the channel behavior of binary erasure channels. It is shown that the proposed polar coding schemes achieve a symmetric capacity.

In this paper, transmission over a wireless channel is interpreted as a two-person zero-sum game, where the transmitter gambles against an unpredictable channel, controlled by nature. Mutual information is used as payoff function. Both discrete and continuous output channels are investigated. We use the fact that mutual information is a convex function of the channel matrix or noise distribution densities, respectively, and a concave function of the input distribution to deduce the existence of equilibrium points for certain channel strategies. The case that nature makes the channel useless with zero capacity is discussed in detail. For each, the discrete, continuous, and mixed discrete-continuous output channel, the capacity-achieving distribution is characterized by help of the Karush-Kuhn-Tucker conditions. The results cover a number of interesting examples like the binary asymmetric channel, the Z-channel, the binary asymmetric erasure channel, and the -ary symmetric channel. In each case, explicit forms of the optimum input distribution and the worst channel behavior are achieved. In the mixed discrete-continuous case, all convex combinations of some noise-free and maximum-noise distributions are considered as channel strategies. Equilibrium strategies are determined by extending the concept of entropy and mutual information to general absolutely continuous measures.

Given a noiseless protocol π0 computing a function f(x, y) of Alice and Bob’s private inputs x, y, the goal of interactive coding is to construct an error-resilient protocol π computing f such that even if some fraction of the communication is adversarially corrupted, both parties still learn f(x, y). Ideally, the resulting scheme π should be positive rate, computationally efficient, and achieve optimal error resilience.

This paper considers the transmission of QCIF resolution MPEG-4 video signals over hostile channels like wireless and mobile at transmission rate of 64 kbps. Although MPEG-4 incorporates some error resilience tools, this standard is still fragile for video transmission over time-variant deep fading channels. We investigated the significance of MPEG-4 video syntactical elements considering MPEG-4 encoded streams, QPSK modulated and delivered through Binary, AWGN and Rayleigh (50 Km/h) channels with SNR range between 0 and 40 dB. We have used four QCIF video test sequences, Foreman, News, Container and Coastguard, and enabled slice resynchronization and data partitioning modes in MPEG-4 video coding. The simulation results show that despite significance of particular elements varies with the channel SNR and with the particular test sequence, we could graphically identify two classes of significance at low channel SNRs. The most significant class includes elements such as VOP header, Video Packet header, DC Marker and DC Coefficients of INTRA-VOP & MB header. At high Rayleigh channel SNR, above 15 dB, AC coefficients of INTRA-VOPs and Motion Vectors were found to be the most significant elements. Any unequal error protection, data partitioning and grouping strategies covering a wide range of SNRs can then be designed based on the significance results presented in this paper.

We consider distributed parameter estimation using quantized observations in wireless sensor networks (WSN) operating in a noisy channel environment. Due to bandwidth constraints, each sensor quantizes its local observation into one bit of information. Previously, adaptive quantization(AQ) schemes were developed under the assumption of perfect communication links between the sensors and the fusion center (FC). In this paper we propose an adaptive quantization scheme for a WSN with channel links modeled as binary erasure channels. A first-order Hidden Markov Model (HMM) framework is introduced to model the adaptive quantization scheme. The introduction of a HMM framework aids in the systematic design of an estimator. To address the significant problem of bit erasures, we propose an Expectation-Maximization (EM) based estimator. Theoretical closed form solutions for the Cramer-Rao lower bounds are developed for the proposed estimation problem under certain assumptions. We analyze the performance of the proposed quantization scheme and estimator under different criteria. Numerical simulation results are shown for the proposed adaptive quantization and EM parameter estimation scheme under different scenarios. The simulation results indicate that the proposed quantization scheme and estimator are robust and can provide superior performance for erasure rates up to 10 %.

The invention of Fountain codes is a major advance in the field of error correcting codes. The goal of this work is to study and develop algorithms for source and channel coding using a family of Fountain codes known as Raptor codes. From an asymptotic point of view, the best currently known sum-product decoding algorithm for non binary alphabets has a high complexity that limits its use in practice. For binary channels, sum-product decoding algorithms have been extensively studied and are known to perform well. In the first part of this work, we develop a decoding algorithm for binary codes on non-binary channels based on a combination of sum-product and maximum-likelihood decoding. We apply this algorithm to Raptor codes on both symmetric and non-symmetric channels. Our algorithm shows the best performance in terms of complexity and error rate per symbol for blocks of finite length for symmetric channels. Then, we examine the performance of Raptor codes under sum-product decoding when the transmission is taking place on piecewise stationary memoryless channels and on channels with memory corrupted by noise. We develop algorithms for joint estimation and detection while simultaneously employing expectation maximization to estimate the noise, and sum-product algorithm to correct errors. We also develop a hard decision algorithm for Raptor codes on piecewise stationary memoryless channels. Finally, we generalize our joint LT estimation-decoding algorithms for Markov-modulated channels. In the third part of this work, we develop compression algorithms using Raptor codes. More specifically we introduce a lossless text compression algorithm, obtaining in this way competitive results compared to the existing classical approaches. Moreover, we propose distributed source coding algorithms based on the paradigm proposed by Slepian and Wolf.