Tree-Structured Expectation Propagation for LDPC Decoding over BMS Channels

This paper is concerned with the dynamics of spatially-coupled low-density parity-check (SC-LDPC) ensembles for transmission on general binary memoryless symmetric (BMS) channels under belief-propagation (BP) decoding. The decoding waves of such ensembles are found to exhibit solitonic behavior, propagating along the Tanner graphs at asymptotically constant velocities. A low-complexity approach termed interpolated density evolution (IDE) is proposed to predict the decoding wave velocities. In this approach, the densities of a decoding wave are approximated by interpolating between some fixed points of an uncoupled DE recursion with one-dimensional functions. Two transfer functions for updating these interpolation functions in the IDE recursion are established and a simple strategy is introduced to deal with the coexistence of multiple transition regions. In addition, a threshold analysis is developed based on two ansatzes, explaining why our approach can achieve a good trade-off between computational cost and accuracy, as illustrated with some numerical examples at the end of this paper.

Recently, it has been verified that the Reed-Muller (RM) code, which has been predicted for a long time in the field of code theory, achieves capacity in a binary erasure channel (BEC) with a MAP (maximum a posteriori) decoder. Through weight distribution interpretation, it was also discovered that the RM code achieves a binary memoryless symmetric (BMS) channel capacity. A successive cancellation list (SCL) technique has been studied as a practical decoder instead of a MAP decoder. In this paper, the performance improvement potential that can be improved when a generalized concatenated code (GCC) with an outer code is designed to improve the practical SCL decoder performance is analyzed. When information bits are used as frozen bits in these code-word designs, we analyze an inter-code-word distance and the decoding outage probability of the designed code. We expect that our results can be used as a conceptual base for improving communication methods in the future development of channel encoding technology.

Reed-Muller (RM) codes and polar codes are generated by the same matrix $G_m= \bigl[\begin{smallmatrix}1 & 0 \\ 1 & 1 \\ \end{smallmatrix}\bigr]^{\otimes m}$ but using different subset of rows. RM codes select simply rows having largest weights. Polar codes select instead rows having the largest conditional mutual information proceeding top to down in $G_m$; while this is a more elaborate and channel-dependent rule, the top-to-down ordering has the advantage of making the conditional mutual information polarize, giving directly a capacity-achieving code on any binary memoryless symmetric channel (BMSC). RM codes are yet to be proved to have such property. In this paper, we reconnect RM codes to polarization theory. It is shown that proceeding in the RM code ordering, i.e., not top-to-down but from the lightest to the heaviest rows in $G_m$, the conditional mutual information again polarizes. We further demonstrate that it does so faster than for polar codes. This implies that $G_m$ contains another code, different than the polar code and called here the twin code, that is provably capacity-achieving on any BMSC. This proves a necessary condition for RM codes to achieve capacity on BMSCs. It further gives a sufficient condition if the rows with the largest conditional mutual information correspond to the heaviest rows, i.e., if the twin code is the RM code. We show here that the two codes bare similarity with each other and give further evidence that they are likely the same.

In this paper, we study the problem of simulating a DMC channel from another DMC channel under an average-case and an exact model. We present several achievability and infeasibility results, with tight characterizations in special cases. In particular for the exact model, we fully characterize when a BSC channel can be simulated from a BEC channel when there is no shared randomness. We also provide infeasibility and achievability results for simulation of a binary channel from another binary channel in the case of no shared randomness. To do this, we use properties of Rényi capacity of a given order. We also introduce a notion of "channel diameter" which is shown to be additive and satisfy a data processing inequality.

This paper considers the problem of bit-wise channel coding over a Binary Symmetric Channel (BSC) with feedback. While it is known that feedback does not increase the capacity of a memoryless channel, it is believed to simplify the coding schemes for some channels. The most significant one is the Binary Erasure Channel (BEC) where capacity is achieved by a simple sequential bit-wise repetition code under which each bit is (re-)transmitted until it is received. This sequential bit-wise feedback code has the added advantage that it can be used for streaming applications over BECs. In contrast, there is no known sequential bit-wise code with feedback that can achieve nonzero transmission rate for a BSC. For example, under Posterior Matching for a binary input channel with feedback, also known as Horstein scheme, each message is considered in its entirety in a block coding manner. This paper proposes a sequential feedback coding scheme with a nested bit-wise structure that generalizes repetition codes. This scheme is shown to achieve strictly positive rate for a large class of binary input channels including a BSC with arbitrary cross-over probability $p \in(0, \frac{1}{2})$ . The analysis relies on characterizing a lower bound on the step-wise Extrinsic Jensen Shannon divergence.

In this correspondence, the performance of the nonlinear error-detecting code, described in CCIR Recommendation 476-2, when used over a memoryless binary symmetric channel (BSC), is derived. From this, it is apparent that the probability of accepting an erroneous block is not a monotonic function of the bit error probability, but rather reaches a peak of 5.653 percent when the bit error probability is 11.66 percent. Consequently, this code is not proper [1]-[6].

In interactive coding, Alice and Bob wish to compute some function $f$ of their individual private inputs $x$ and $y$. They do this by engaging in a non-adaptive (fixed order, fixed length) protocol to jointly compute $f(x,y)$. The goal is to do this in an error-resilient way, such that even given some fraction of adversarial corruptions to the protocol, both parties still learn $f(x,y)$. In this work, we study the optimal error resilience of such a protocol in the face of adversarial bit flip or erasures. While the optimal error resilience of such a protocol over a large alphabet is well understood, the situation over the binary alphabet has remained open. In this work, we resolve this problem of determining the optimal error resilience over binary channels. In particular, we construct protocols achieving $\frac16$ error resilience over the binary bit flip channel and $\frac12$ error resilience over the binary erasure channel, for both of which matching upper bounds are known. We remark that the communication complexity of our binary bit flip protocol is polynomial in the size of the inputs, and the communication complexity of our binary erasure protocol is linear in the size of the minimal noiseless protocol computing $f$.

A doubly multistage decoder (DMD) for low-density parity-check codes over ℤ2m, m>1, which fuses the multistage decoding approaches of Armand et al. and Varnica et al., is proposed. Two variants of the DMD are considered. The first (respectively, second) performs belief-propagation (BP) [respectively, offset min-sum (OMS)] decoding in each decoding stage and is referred to as DMD - BP (respectively, DMD - OMS). For the moderate-length codes considered in this study, the computer simulations show the DMD - BP (respectively, DMD - OMS) achieving coding gains of up to 0.43 dB (respectively, 0.67 dB) over standard BP decoding at a bit error rate of 10−6 on an additive-white-Gaussian-noise channel, while requiring significantly less computational power. Remarkably, DMD - OMS outperforms DMD - BP, yet has lower computational complexity than DMD - BP. Snapshots of the log-likelihood ratio densities of the decoded bits midway through the decoding process explain the superiority of the DMD over standard BP decoding.

In this paper, we discuss the belief propagation (BP) decoding of polar codes. The performance of polar codes for short lengths is not satisfactory. Therefore, motivated by this we propose a novel technique to improve the performance of polar codes under BP decoding. In order to enhance the reliability of the variable nodes’ propagated messages in BP decoding, the messages are multiplied by previous messages of variable nodes. It is also shown that the BP decoding can be performed on polar codes if the factor graph is constructed according to the parity check matrix of polar codes. Simulation results in binary-input additive white Gaussian noise channel show that using the proposed method a gain of 1–2 dB can be achieved. We also show that the complexity of proposed decoder is same as the complexity of standard BP decoder. Furthermore, we show that the proposed decoder requires about 10–25 iterations less than the standard BP decoder.

Using a finite field approach, a novel algebraic construction of low-density parity-check (LDPC) convolutional codes with fast encoding property is proposed. According to the matrices of quasi-cyclic (QC) codes constructed based on the multiplicative groups of finite fields and the algebraic property that a binary circulant matrix is isomorphic to a finite ring, we first generate a polynomial-form parity-check matrix of an LDPC convolutional code under a given rate over a given finite field. Then some related modifications are made upon the original polynomial-form matrix to obtain the new one with fast encoding property. Simulation results show that the proposed LDPC convolutional codes have good performance with the iterative belief propagation decoding algorithm.

We present the tree-structure expectation propagation (Tree-EP) algorithm to decode low-density parity-check (LDPC) codes over discrete memoryless channels (DMCs). EP generalizes belief propagation (BP) in two ways. First, it can be used with any exponential family distribution over the cliques in the graph. Second, it can impose additional constraints on the marginal distributions. We use this second property to impose pair-wise marginal constraints over pairs of variables connected to a check node of the LDPC code's Tanner graph. Thanks to these additional constraints, the Tree-EP marginal estimates for each variable in the graph are more accurate than those provided by BP. We also reformulate the Tree-EP algorithm for the binary erasure channel (BEC) as a peeling-type algorithm (TEP) and we show that the algorithm has the same computational complexity as BP and it decodes a higher fraction of errors. We describe the TEP decoding process by a set of differential equations that represents the expected residual graph evolution as a function of the code parameters. The solution of these equations is used to predict the TEP decoder performance in both the asymptotic regime and the finite-length regime over the BEC. While the asymptotic threshold of the TEP decoder is the same as the BP decoder for regular and optimized codes, we propose a scaling law (SL) for finite-length LDPC codes, which accurately approximates the TEP improved performance and facilitates its optimization.

In this paper, we propose a new type of code called Trellis-based Quasi-Cyclic (TQC)-LDPC convolutional code, which is a special case of protograph-based LDPC convolutional codes. The proposed TQC-LDPC convolutional code can be derived from any QC-LDPC block code by introducing trellis-based convolutional dependency to the code. The main advantage of the proposed TQC-LDPC convolutional code is that it allows reduced decoder complexity and input granularity (which is defined as the minimum number of input information bits the code requires to generate a codeword) while maintaining the same bit error-rate as the underlying QC-LDPC block code ensemble. We also propose two related power-efficient encoding methods to increase the code rate of the derived TQC-LDPC convolutional code. The newly derived short constraint length TQC-LDPC convolutional codes enable low complexity trellis-based decoders and one such decoder is proposed and described in this paper (namely, QC-Viterbi). The TQC-LDPC convolutional codes and the QC-Viterbi decoder are compared to conventional LDPC codes and Belief Propagation (BP) iterative decoders with respect to bit-error-rate (BER), signal-to-noise ratio (SNR), and decoder complexity. We show both numerically and through hardware implementation results that the proposed QC-Viterbi decoder outperforms the BP iterative decoders by at least 1 dB for same complexity and BER. Alternatively, the proposed QC-Viterbi decoder has 3 times lower complexity than the BP iterative decoder for the same SNR and BER. This low decoding complexity, low BER, and fine granularity makes it feasible for the proposed TQC-LDPC convolutional codes and associated trellis-based decoders to be efficiently implemented in high data rate, next generation mobile systems.

The codewords of a low-density parity-check (LDPC) convolutional code (LDPC-CC) are characterised into structured and non-structured. The number of the structured codewords is dominated by the size of the polynomial syndrome former matrix H T ( D ) , while the number of the non-structured ones depends on the particular monomials or polynomials in H T ( D ) . By evaluating the relationship of the codewords between the mother code and its super codes, the low weight non-structured codewords in the super codes can be eliminated by appropriately choosing the monomials or polynomials in H T ( D ) , resulting in improved distance spectrum of the mother code.

We propose a windowed decoding scheme for protograph-based LDPC convolutional codes (LDPC-CC) that allows us to efficiently trade-off decoding performance for gains in latency. We study the performance of regular LDPC-CC with the windowed decoding scheme. In particular, we show that the class of LDPC-CC proposed in the literature with good belief propagation performance is ill-suited for windowed decoding. Further, we establish properties of code ensembles with good windowed decoding performance over erasure channels with and without memory.

This work deals with coding systems based on sparse graph codes. The key issue we address is the relationship between iterative (in particular belief propagation) and maximum a posteriori decoding. We show that between the two there is a fundamental connection, which is reminiscent of the Maxwell construction in thermodynamics. The main objects we consider are EXIT-like functions. EXIT functions were originally introduced as handy tools for the design of iterative coding systems. It gradually became clear that EXIT functions possess several fundamental properties. Many of these properties, however, apply only to the erasure case. This motivates us to introduce GEXIT functions that coincide with EXIT functions over the erasure channel. In many aspects, GEXIT functions over general memoryless output-symmetric channels play the same role as EXIT functions do over the erasure channel. In particular, GEXIT functions are characterized by the general area theorem. As a first consequence, we demonstrate that in order for the rate of an ensemble of codes to approach the capacity under belief propagation decoding, the GEXIT functions of the component codes have to be matched perfectly. This statement was previously known as the matching condition for the erasure case. We then use these GEXIT functions to show that in the limit of large blocklengths a fundamental connection appears between belief propagation and maximum a posteriori decoding. A decoding algorithm, which we call Maxwell decoder, provides an operational interpretation of this relationship for the erasure case. Both the algorithm and the analysis of the decoder are the translation of the Maxwell construction from statistical mechanics to the context of probabilistic decoding. We take the first steps to extend this construction to general memoryless output-symmetric channels. More exactly, a general upper bound on the maximum a posteriori threshold for sparse graph codes is given. It is conjectured that the fundamental connection between belief propagation and maximum a posteriori decoding carries over to the general case.