Reed–Muller Codes: Theory and Algorithms

Reed-Muller (RM) codes are among the oldest, simplest and perhaps most ubiquitous family of codes. They are used in many areas of coding theory in both electrical engineering and computer science. Yet, many of their important properties are still under investigation. This work covers some of the developments regarding the weight enumerator and the capacity-achieving properties of RM codes, as well as some of the algorithmic developments. In particular, it discusses connections established between RM codes, thresholds of Boolean functions, polarization theory, hypercontractivity, and the techniques of approximating low weight codewords using lower degree polynomials (when codewords are viewed as evaluation vectors of degree r polynomials in m variables). It then overviews some of the algorithms for decoding RM codes, giving both algorithms with provable performance guarantees for every block length, as well as algorithms with state-of-the-art performances in practical regimes, which do not perform as well for large block length. Finally, some applications of RM codes in theoretical computer science and signal processing are given.

Reed-Muller (RM) codes and polar codes are generated by the same matrix $G_m= \bigl[\begin{smallmatrix}1 & 0 \\ 1 & 1 \\ \end{smallmatrix}\bigr]^{\otimes m}$ but using different subset of rows. RM codes select simply rows having largest weights. Polar codes select instead rows having the largest conditional mutual information proceeding top to down in $G_m$; while this is a more elaborate and channel-dependent rule, the top-to-down ordering has the advantage of making the conditional mutual information polarize, giving directly a capacity-achieving code on any binary memoryless symmetric channel (BMSC). RM codes are yet to be proved to have such property. In this paper, we reconnect RM codes to polarization theory. It is shown that proceeding in the RM code ordering, i.e., not top-to-down but from the lightest to the heaviest rows in $G_m$, the conditional mutual information again polarizes. We further demonstrate that it does so faster than for polar codes. This implies that $G_m$ contains another code, different than the polar code and called here the twin code, that is provably capacity-achieving on any BMSC. This proves a necessary condition for RM codes to achieve capacity on BMSCs. It further gives a sufficient condition if the rows with the largest conditional mutual information correspond to the heaviest rows, i.e., if the twin code is the RM code. We show here that the two codes bare similarity with each other and give further evidence that they are likely the same.

The main focus in this thesis is linear codes over rings. In the first part, we look at linear codes over Galois rings, and using the homogeneous weight, we improve upon Wilson's results about the prime power that divides the coefficients of the homogeneous weight enumerators of these codes. We also prove that our results are best possible. Our results about homogeneous weight enumerators of linear codes over Galois rings generalize the results that we have for the Lee weight enumerators of linear codes over the ring of integers modulo 4. We also consider other weight enumerators, in particular the complete weight enumerators of linear codes and we obtain MacWilliams-like identities for these weight enumerators considering different rings. These MacWilliams-like identities lead to MacWilliams identities for the Hamming weight enumerators of linear codes over rings. We also give a counter-example to show that we cannot have MacWilliams-like identities for the Euclidean weight enumerators of linear codes over the ring of integers modulo 4. We also look at Gray maps from Galois rings to the finite field of prime size. We first give an application of the distance preserving Gray maps to obtain good ternary codes, and then we give an inductive algebraic construction of a distance preserving Gray map from Galois rings with the homogeneous distance to the field of prime size with the Hamming distance as well as a combinatorial construction. By using the Gray maps, we obtain some results about the weight enumerators of linear codes over the ring of dual numbers. In the last part of this work, we consider the permutation invariance of binary codes and the connection with linearity over certain rings. Among the rings considered, we have different variations and extensions of the ring of dual numbers. In this context we consider the Reed-Muller codes and answer questions about permutation invariance of Reed-Muller codes under certain permutation groups and the linearity of Reed-Muller codes over these aforementioned rings.

Reed–Muller codes are one of the most well-studied families of codes; however, there are still open problems regarding their structure. Recently, a new ring-theoretic approach has emerged that provides a rather intuitive construction of these codes. This approach is centered around the notion of basic Reed–Muller codes. We recall that Reed–Muller codes over a prime field are radical powers of a corresponding group algebra. In this paper, we prove that basic Reed–Muller codes in the case of a nonprime field of arbitrary characteristic are distinct from radical powers. This implies the same result for regular codes. Also we show how to describe the inclusion graph of basic Reed–Muller codes and radical powers via simple arithmetic equations.

In this paper a class of nonprimitive cyclic codes quite similar in structure to the original Reed-Muller codes is presented. These codes, referred to herein as nonprimitive Reed-Muller codes, are shown to possess many of the properties of the primitive codes. Specifically, two major results are presented. First the code length, number of information symbols, and minimum distance are shown to be related by means of a parameter known as the order of the code. These relationships show that for given values of code length and rate the codes have relatively large minimum distances. It is also shown that the codes are subcodes of the BCH codes of the same length and guaranteed minimum distance; thus in general the codes are not as powerful as the BCH codes. However, for most interesting values of code length and rate the difference between the two types of codes is slight. The second result is the observation that the codes can be decoded with a variation of the original algorithm proposed by Reed for the Reed-Muller codes. In other words, they are L -step orthogonalizable. Because of their large minimum distances and the simplicity of their decoders, nonprimitive Reed-Muller codes seem attractive for use in error-control systems requiring multiple random-error correction.

Reed–Muller (RM) codes are known for their good maximum likelihood (ML) performance in the short block-length regime. Despite being one of the oldest classes of channel codes, finding a low complexity soft-input decoding scheme is still an open problem. In this work, we present a versatile decoding architecture for RM codes based on their rich automorphism group. The decoding algorithm can be seen as a generalization of multiple-bases belief propagation (MBBP) and may use any polar or RM decoder as constituent decoders. We provide extensive error-rate performance simulations for successive cancellation (SC)-, SC-list (SCL)- and belief propagation (BP)-based constituent decoders. We furthermore compare our results to existing decoding schemes and report a near-ML performance for the RM(3,7)-code (e.g., 0.04 dB away from the ML bound at BLER of 10 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">−3</sup> ) at a competitive computational cost. Moreover, we provide some insights into the automorphism subgroups of RM codes and SC decoding and, thereby, prove the theoretical limitations of this method with respect to polar codes.

Iterative decoding methods have gained interest, initiated by the results of the so-called "turbo" codes. The theoretical description of this decoding, however, seems to be difficult. Therefore, we study the iterative decoding of block codes. First, we discuss the iterative decoding algorithms developed by Gallager (1962), Battail et al. (1979), and Hagenauer et al. (1996). Based on their results, we propose a decoding algorithm which only uses parity check vectors of minimum weight. We give the relation of this iterative decoding to one-step majority-logic decoding, and interpret it as gradient optimization. It is shown that the used parity check set defines the region where the iterative decoding decides on a particular codeword. We make plausible that, in almost all cases, the iterative decoding converges to a codeword after some iterations. We derive a computationally efficient implementation using the minimal trellis representing the used parity check set. Simulations illustrate that our algorithm gives results close to soft decision maximum likelihood (SDML) decoding for many code classes like BCH codes. Reed-Muller codes, quadratic residue codes, double circulant codes, and cyclic finite geometry codes. We also present simulation results for product codes and parallel concatenated codes based on block codes.

This paper continues the series of studies of Boolean polynomials (Zhegalkin polynomials or algebraic normal forms (ANFs)). The investigation of properties of Boolean polynomials is an important topic of discrete mathematics and combinatorial analysis. Theoretical results in this field have wide practical applications. For example, a number of popular public-key cryptosystems are based on Reed–Muller codes, and the representation of these codes, as well as encryption and decryption algorithms are based on Boolean polynomials. The spectral properties of such codes are determined by the number of zeros of Boolean polynomials and are analyzed using the randomization lemma. Since the problem of determining the number of zeros Zg of the Boolean polynomial g(x) is NP-hard, the algorithms taking into account the “combinatorial structure of the polynomial” (even though they are search algorithms) are of practical interest. In this paper, such an algorithm based on the properties of the monomial matrix is proposed. For various types of polynomials, exact formulas for the number of roots and the expectation of the number of roots are obtained. A subclass of Boolean polynomials consisting of polynomials with separating variables is considered. A result that can be considered as a generalization of the randomization lemma is proved. The theoretical results provide a basis for estimating the applicability of polynomials for solving various practical problems.

Locally decodable codes (LDCs) allow any single encoded message symbol to be retrieved from a codeword with good probability by reading only a tiny number of codeword symbols, even if the codeword is partially corrupted. LDCs have surprisingly many applications in computer science and mathematics (we refer to [Yekhanin, 2012; Lovett, 2007] for extensive surveys). But despite their ubiquity, they are poorly understood. Of particular interest is the tradeoff between the codeword length N as a function of message length k when the query complexity - the number of probed codeword symbols - and alphabet size are constant. The Hadamard code is a 2-query LDC of length N = 2^O(k) and this length is optimal in the 2-query regime [Lovett, 2007]. For q ≥ 3, near-exponential gaps persist between the best-known upper and lower bounds. The family of Reed-Muller codes, which generalize the Hadamard code, were for a long time the best-known examples, giving q-query LDCs of length exp(O(k^{1/(q-1)})), until breakthrough constructions of matching vector LDCs of Yekhanin and Efremenko [Yekhanin, 2008; Efremenko, 2012].&#13;\nIn contrast with other combinatorial objects such as expander graphs, the probabilistic method has so far not been successfully used to beat the best explicit LDC constructions. In [Lovett, 2007], a probabilistic framework was given that could in principle yield best-possible LDCs, albeit non-constructively. A special instance of this framework connects LDCs with a probabilistic version of Szemerédi’s theorem. The setup for this is as follows: For a finite abelian group G of size N = |G|, let D ⊆ G be a random subset where each element is present with probability ρ independently of all others. For k ≥ 3 and ε ∈ (0,1), let E be the event that every subset A ⊆ G of size |A| ≥ ε |G| contains a proper k-term arithmetic progression with common difference in D. For fixed ε &gt; 0 and sufficiently large N, it is an open problem to determine the smallest value of ρ - denoted ρ_k - such that Pr[E] ≥ 1/2. In [Lovett, 2007] it is shown that there exist k-query LDCs of message length Ω(ρ_k N) and codeword length O(N). As such, Szemerédi’s theorem with random differences, in particular lower bounds on ρ_k, can be used to show the existence of LDCs. Conversely, this connection indirectly implies the best-known upper bounds on ρ_k for all k ≥ 3 [Lovett, 2007; Lovett, 2007]. However, a conjecture from [Lovett, 2007] states that over ℤ_N we have ρ_k ≤ O_k(N^{-1}log N) for all k, which would be best-possible. Truth of this conjecture would imply that over this group, Szemerédi’s theorem with random differences cannot give LDCs better than the Hadamard code. For finite fields, Altman [Lovett, 2007] showed that this is false. In particular, over 𝔽_pⁿ for p odd, he proved that ρ₃ ≥ Ω(p^{-n} n²); generally, ρ_k ≥ Ω(p^{-n} n^{k-1}) holds when p ≥ k+1 [Lovett, 2007]. In turn, these bounds are conjectured to be optimal for the finite-field setting, which would imply that over finite fields, Szemerédi’s theorem with random differences cannot give LDCs better than Reed-Muller codes. &#13;\nThe finite-field conjecture is motivated mainly by the possibility that so-called dual functions can be approximated well by polynomial phases, functions of the form e^{2π i P(x)/p} where P is a multivariate polynomial over 𝔽_p. We show that this is false. Using Yekhanin’s matching-vector-code construction, we give dual functions of order k over 𝔽_pⁿ that cannot be approximated in L_∞-distance by polynomial phases of degree k-1. This answers in the negative a natural finite-field analog of a problem of Frantzikinakis over ℕ [Lovett, 2007].

We introduce a new approach to proving that a sequence of deterministic linear codes achieves capacity on an erasure channel under maximum a posteriori decoding. Rather than relying on the precise structure of the codes, our method exploits code symmetry. In particular, the technique applies to any sequence of linear codes where the block lengths are strictly increasing, the code rates converge, and the permutation group of each code is doubly transitive. In a nutshell, we show that symmetry alone implies near-optimal performance. An important consequence of this result is that a sequence of Reed-Muller codes with increasing block length and converging rate achieves capacity. This possibility has been suggested previously in the literature, but it has only been proven for cases where the limiting code rate is 0 or 1. Moreover, these results extend naturally to affine-invariant codes and, thus, to all extended primitive narrow-sense BCH codes. This is used to resolve, in the affirmative, the existence question for capacity-achieving sequences of binary cyclic codes. The primary tools used in the proofs are the sharp threshold property for symmetric monotone boolean functions and the area theorem for extrinsic information transfer (EXIT) functions.

The correspondence between partial metrics and semivaluations

A forward error control coding can be viewed as an additional level of digital linear filtering (over the binary field) which introduces redundancy in the original digital data sequence. In application to CDMA communication, we study two major types of codes: block codes and convolutional codes. We emphasize low-rate first-order Reed-Muller (block) codes, orthogonal convolutional codes and some concatenated code constructions which are already used in CDMA systems.

Comparing with hard decision decoding algorithms, soft decoding has a lower probability of bit error but a higher computational complexity. As a maximum-likelihood soft decoding method, the <i>A</i>* algorithm is the most basic and widely used to minimize bit error probability. However, its average computational complexity strongly depends on a seed codeword and a heuristic function utilized during the decoding process. To efficiently reduce the computational complexity while maintaining the decoding accuracy theoretically and practically, this paper proposes an improved <i>A</i>* decoding algorithm consisting of two phases. The first phase applies the greedy list decoding to the linear block code to obtain a seed codeword. According to the seed, the second phase applies the improved <i>A</i>* algorithm to obtain the final decoding output. The heuristic function used in the <i>A</i>* algorithm is modified in two aspects: (1) use more information of partial decoded symbols to improve the accuracy of the function and (2) take advantage of Hamming distance to reduce the search space. Simulations on the <i>RM</i>(5, 2) Reed-Muller codes and [128, 64] binary extended BCH code show that this improved <i>A</i>* algorithm is more efficient in average decoding complexity than many other algorithms while maintaining the decoding accuracy.

This Research to Practice Full Paper discusses an approach to motivating graduate computer science and electrical engineering students in the study of quantum computing. The U.S. Congress and other governmental and industrial organizations have recognized that teaching Quantum Computing in university-level Computer Science and Electrical Engineering programs is strategically important for workforce development along with technology development. Success in this endeavor requires generating genuine interest in the topic and motivating potential students. Quantum Computing requires an understanding of relatively difficult related subjects including physics, math and computer science. Many computer science and electrical engineering students shy away from quantum computing because of its complexity. This paper discusses our success with generating interest in quantum computing among graduate students and motivating them through exploring, in related courses, some aspects of quantum computing that have the potential to generate discontinuities. The approach is also examined in terms of the Scientific Learning Cycle as discussed by Wankat & Oreovicz. Opportunities and challenges of adding Quantum Computing to Computer Science and Electrical Engineering programs were studied through a student small group project in a cybersecurity course. The work built on extensive research into the effectiveness of small group projects and the role of “play” in learning. Our small group project on quantum computing was introduced three years ago into a cybersecurity course which is required for all students in our university's Masters in Computer Science and Masters in Electrical Engineering programs. The basic goals of the project are to teach students some basics of quantum computing by allowing them to “play with it” and to alert students to the fact that quantum computing is a rapidly developing new field that they need to follow during at least the next decade. The small group project helped students to explore the potential of quantum computing to substantially undermine current Public Key Cryptography and, at the same time provide a new approach to encryption that is even more secure than Public Key Cryptography. A qualitative analysis of discussions and assigned tasks suggests that this strategy succeeded in generating significant interest in quantum computing among students. They became motivated and immersed in a difficult subject despite the requirement for investment of significant time and effort to understand a complex combination of physics, math and computer science.

Human-computer interaction for virtual-real fusion