Coding for the blackwell channel: a survey propagation approach

It has been shown that evolutionary algorithms are able to construct suitable search strategies for classes of Constraint Satisfaction Problems (CSPs) in Constraint Programming. This paper is an explanation of the use of multi-objective optimisation in contrast to simple additive weighting techniques with a view to develop search strategies to classes of CSPs. A hierarchical scheme is employed to select a candidate strategy from the Pareto frontier for final evaluation. The results demonstrate that multi-objective optimisation significantly outperforms the single objective scheme in the same number of objective evaluations. In situations where strategies developed for a class of problems fail to extend to unseen problem instances of the same class, it is found that the structure of the underlying CSPs do not resemble those employed in the training process.

A family of error-correcting codes is list-decodable from error fraction $p$ if, for every code in the family, the number of codewords in any Hamming ball of fractional radius $p$ is less than some integer $L$ that is independent of the code length. It is said to be list-recoverable for input $\ell$ if for every sufficiently large subset of codewords (of $L$ or more), there is a coordinate where the codewords take more than $\ell$ values. The parameter $L$ is said to be the list size in either case. The capacity, i.e., the largest possible rate for these notions as the $L \to \infty$, is known to be $1-h_q(p)$ for list-decoding, and $1-\log_q \ell$ for list-recovery, where $q$ is the alphabet of the code family. In this work, we study the of random linear codes for both list-decoding and list-recovery as the rate approaches capacity. We show the following claims hold with high probability over the choice of the code (below, $\epsilon > 0$ is the gap to capacity). (1) A random linear code of rate $1 - \log_q(\ell) - \epsilon$ requires $L \ge \ell^{\Omega(1/\epsilon)}$ for list-recovery from input $\ell$. This is surprisingly in contrast to completely random codes, where $L = O(\ell/\epsilon)$ suffices w.h.p. (2) A random linear code of rate $1 - h_q(p) - \epsilon$ requires $L \ge \lfloor h_q(p)/\epsilon+0.99 \rfloor$ for list-decoding from error fraction $p$, when $\epsilon$ is sufficiently small. (3) A random binary linear code of rate $1 - h_2(p) - \epsilon$ is list-decodable from average error fraction $p$ with with $L \leq \lfloor h_2(p)/\epsilon \rfloor + 2$. The second and third results together precisely pin down the sizes for binary random linear codes for both list-decoding and average-radius list-decoding to three possible values.

Decoding random linear codes is a well studied problem with many applications in complexity theory and cryptography. The security of almost all coding and LPN/LWE-based schemes relies on the assumption that it is hard to decode random linear codes. Recently, there has been progress in improving the running time of the best decoding algorithms for binary random codes. The ball collision technique of Bernstein, Lange and Peters lowered the complexity of Stern’s information set decoding algorithm to 20.0556n. Using representations this bound was improved to 20.0537n by May, Meurer and Thomae. We show how to further increase the number of representations and propose a new information set decoding algorithm with running time 20.0494n.KeywordsInformation Set DecodingRepresentation Technique

Constraint programming offers a declarative approach to solving problems modeled as constraint satisfaction problems (CSPs). However, the precise specification of a set of constraints is sometimes not available, but may have to be learned, for instance, from a set of examples of its solutions and non-solutions. In general, one may wish to learn generalized CSPs involving classical, fuzzy, weighted or probabilistic constraints, for example. This paper introduces a unifying framework for CSP learning. The framework is generic in that it can be instantiated to obtain specific formulations for learning classical, fuzzy, weighted or probabilistic CSPs. In particular, a new formulation for classical CSP learning, which minimizes the number of examples violated by candidate CSPs, is obtained by instantiating the framework. This formulation is equivalent to a simple pseudo-boolean optimization problem, thus being efficiently solvable using many optimization tools.

In this paper we consider constraint satisfaction problems where the set of constraint relations is fixed. Feder and Vardi (1998) identified three families of constraint satisfaction problems containing all known polynomially solvable problems. We introduce a new class of problems called para-primal problems, incomparable with the families identified by Feder and Vardi (1998) and we prove that any constraint problem in this class is decidable in polynomial time. As an application of this result we prove a complete classification for the complexity of constraint satisfaction problems under the assumption that the basis contains all the permutation relations. In the proofs, we make an intensive use of algebraic results from clone theory about the structure of para-primal and homogeneous algebras.

We introduce a generalization of the spreadsheet paradigm, called Knowledgesheet, for solving a class of constraint satisfaction problems. The traditional spreadsheet paradigm is based on attaching arithmetic expressions to individual cells and then evaluating them; our Knowledgesheet interface instead allows finite domain constraints to be attached to individual cells that are then solved to obtain a solution. This extension provides an easy-to-use interface for solving a large class of constraint satisfaction problems--those whose specification and solution conforms to a 2-dimensional structure, e.g., scheduling problems, timetabling problems, etc. A prototype for the Knowledgesheet has been developed and applied to solve many different types of problems.

Constraint Satisfaction with a Multi-Dimensional Domain

The expressiveness of First Order Logic (FOL) languages is severely counterbalanced by the complexity of matching formulas on a universe. Matching is an instance of the class of Constraint Satisfaction Problems (CSP), which have shown to undergo a phase transition with respect to two order parameters: constraint density and constraint tightness. This paper analyzes the problem of satisfying FOL Horn clauses in the light of these recent results. By means of an extensive experimental analysis, we show how Horn clause verification exhibits a typical phase transition with respect to the number of binary (or greater arity) predicates, and with respect to the ratio between the number of constants in the universe and the cardinality of the basic predicates extension.

The problem of analyzing the finite-length scaling behavior of sparse random linear codes is considered. Random linear codes with random generator matrices whose entries are picked according to i.i.d. Bernoulli distribution with parameter q = o(1) are called sparse. The parameter q is referred to as the sparsity of the random linear code. We develop a methodology to show the optimality of the scaling exponent of uniform random linear codes, i.e., q = 1/2, with high probability. The results are then extended to sparse random linear codes with sparsity q = Θ(n−1/2), where n is the code block length. The encoding complexity of such sparse random linear codes is reduced from O(n2), in uniform random linear codes, to O(n 3/2). It is also conjectured that q = log n/n is the lowest sparsity of random linear codes with optimal scaling exponent. The connection of these results to an open problem regarding finding binary polar codes with optimal scaling exponent are also discussed. In particular, we point out that as the size of the polarization kernel increases it can be used as the generator matrix for a code with optimal scaling exponent, without the need to do further polarization.

The survey propagation algorithm is the most effective information propagation algorithm for solving the 3-SAT problem. It can effectively solve the satisfiability problem when it converges. However, when the factor graph structure is complex, the algorithm often does not converge and the solution fails. In order to give a theoretical explanation to this phenomenon and to analyze the convergence of the survey propagation algorithm effectively, a connected treewidth model of the propositional formula was constructed by using the connected tree decomposition method, and the connected treewidth of the factor graph was calculated. The relationship between the connected treewidth and the convergence of the survey propagation algorithm is established, and the convergence judgment condition of the survey propagation algorithm based on the connected tree width is given. Through experimental analysis, the results show that the method is effective, which is of great significance for analyzing the convergence analysis of other information propagation algorithms.

A note on some collapse results of valued constraints

In the frame of classical constraint satisfaction problems (CSPs), the backtrack tree search, combined with learning methods, presents a double advantage: for static solving, it improves the search speed by avoiding redundant explorations; for dynamic solving (after a slight change of the problem) it reuses the previous searches to build a new solution quickly. Backtrack reasoning concludes the rejection of certain combinatorial choices. Nogood Recording memorizes these choices in order to not reproduce. We aim to use Nogood Recording in the wider scope of the CSP framework (VCSP) to enhance the branch and bound algorithm. Therefore, nogoods are used to increase the lower bound used by the branch and bound to prune the search. This issue leads to the definition of the Valued Nogoods and their use. This study focuses particularly on penalty and dynamic VCSPs which require special developments. However our results give an extension of the Nogood Recording to the general VCSP framework.

: The primary objective of this project has been the development of systems that reason in dynamic and open-ended environment and that use networks as their primary representation language. The focus of our research has been temporal reasoning, neural networks, truth maintenance, and default reasoning. This investigation has led to several basic results: the expressiveness of constraint networks was analyzed, tractable classes of constraint satisfaction problems were identified and effective processing techniques were developed.

In this paper, we unify survey propagation (SP) for constraint-satisfaction problems as "probabilistic token passing" algorithms. We show that the reduction of SP from belief propagation (BP) for general problems requires a modification of the BP message-passing rule, namely, introducing what we call a state-decoupling operation on the BP messages. This raises a question mark on the recent "folk belief" that SP is BP.

We investigate a systematic construction of regular low density parity check (LDPC) codes based on (γργ-1, ργ, ρ, γ{0,1}) combinatorial designs. The proposed (γ,ρ) regular LDPC ensemble has rate $${\left( {1 - \tfrac{1}{\rho }} \right)^\gamma }$$ girth ≥2γ+1, and exists for all γ≥2. The codes are a subset of Gallager's random ensemble, but contains a good combination of structure and (pseudo)randomness. In particular, the simple case of γ=2 results in a class of codes that are high-rate, systematic, quasi-cyclic, linear-time encodable and decodable, and free of length-4 and length-6 cycles. Analysis on distance spectrum shows that they are better than the Gallager ensemble of the same parameters. Simulation of the proposed codes on intersymbol interference channels show that they perform comparably to random LDPC codes. Unlike random codes, the proposed structured LDPC codes can lend themselves to a low-complexity implementation for high-speed applications.