Partition function expansion on region graphs and message-passing equations

Preface. 1 Introduction. 1.1 Two examples of combinatorial optimization. 1.2 Why study combinatorial optimization using statistical physics? 1.3 Textbooks. Bibliography. 2 Algorithms. 2.1 Pidgin Algol. 2.2 Iteration and recursion. 2.3 Divide-and-conquer. 2.4 Dynamic programming. 2.5 Backtracking. Bibliography. 3 Introduction to graphs. 3.1 Basic concepts and graph problems. 3.2 Basic graph algorithms. 3.3 Random graphs. Bibliography. 4 Introduction to complexity theory. 4.1 Turing machines. 4.2 Church's thesis. 4.3 Languages. 4.4 The halting problem. 4.5 Class P. 4.6 Class NP. 4.7 Definition of NP-completeness. 4.8 NP-complete problems. 4.9 Worst-case vs. typical-case complexity. Bibliography. 5 Statistical mechanics of the Ising model. 5.1 Phase transitions. 5.2 Some general notes on statistical mechanics. 5.3 The Curie-Weiss model of a ferromagnet. 5.4 The Ising model on a random graph. Bibliography. 6 Algorithms and numerical results for vertex covers. 6.1 Definitions. 6.2 Heuristic algorithms. 6.3 Branch-and-bound algorithm. 6.4 Results: Covering random graphs. 6.5 The leaf-removal algorithm. 6.6 Monte Carlo simulations. 6.7 Backbone. 6.8 Clustering of minimum vertex covers. Bibliography. 7 Statistical mechanics of vertex covers on a random graph. 7.1 Introduction. 7.2 The first-moment bound. 7.3 The hard-core lattice gas. 7.4 Replica approach. Bibliography. 8 The dynamics of vertex-cover algorithms. 8.1 The typical-case solution time of a complete algorithm. 8.2 The dynamics of generalized leaf-removal algorithms. 8.3 Random restart algorithms. Bibliography. 9 Towards new, statistical-mechanics motivated algorithms. 9.1 The cavity graph. 9.2 Warning propagation. 9.3 Belief propagation. 9.4 Survey propagation. 9.5 Numerical experiments on random graphs. Bibliography. 10 The satisfiability problem. 10.1 SAT algorithms. 10.2 Phase transitions in random K-SAT. 10.3 Typical-case dynamics of RandomWalkSAT. 10.4 Message-passing algorithms for SAT. Bibliography. 11 Optimization problems in physics. 11.1 Monte Carlo optimization. 11.2 Hysteric optimization. 11.3 Genetic algorithms. 11.4 Shortest paths and polymers in random media. 11.5 Maximum flows and random-field systems. 11.6 Submodular functions and free energy of Potts model. 11.7 Matchings and spin glasses. Bibliography. Index.

In this paper, we unify survey propagation (SP) for constraint-satisfaction problems as "probabilistic token passing" algorithms. We show that the reduction of SP from belief propagation (BP) for general problems requires a modification of the BP message-passing rule, namely, introducing what we call a state-decoupling operation on the BP messages. This raises a question mark on the recent "folk belief" that SP is BP.

Recently, Vontobel showed the relationship between Bethe free energy and annealed free energy for protograph factor graph ensembles. In this paper, annealed free energy of any random regular factor graph ensembles are connected to Bethe free energy. The annealed free energy is expressed as the solution of maximization problem whose stationary condition coincides with equations of belief propagation since the contribution to partition function of particular type of variable and factor nodes has similar form of minus Bethe free energy. It gives simple derivation of quenched free energy by using the replica method. It implies equivalence of the replica and cavity methods for any random irregular factor graph ensembles. As consequence, it is shown that the replica symmetric solution and annealed free energy are equal for regular ensemble.

The non-backtracking operator was recently shown to provide a significant improvement when used for spectral clustering of sparse networks. In this paper we analyze its spectral density on large random sparse graphs using a mapping to the correlation functions of a certain interacting quantum disordered system on the graph. On sparse, tree-like graphs, this can be solved efficiently by the cavity method and a belief propagation algorithm. We show that there exists a paramagnetic phase, leading to zero spectral density, that is stable outside a circle of radius , where ρ is the leading eigenvalue of the non-backtracking operator. We observe a second-order phase transition at the edge of this circle, between a zero and a non-zero spectral density. The fact that this phase transition is absent in the spectral density of other matrices commonly used for spectral clustering provides a physical justification of the performances of the non-backtracking operator in spectral clustering.

The application of Low-Density Generator Matrix (LDGM) Codes combined with Survey Propagation (SP) in steganography seems to be advantageous, since it is possible to approximate the coset leader directly. Thus, large codeword length can be applied, resulting in an embedding efficiency close to the theoretical upper bound of embedding efficiency. Since this approach is still quite complex, this paper deals with the application of Belief Propagation (BP) to LDGM Codes used to reduce the complexity of embedding while keeping the embedding efficiency constant.

In this paper, a bottom-up salient object detection method is proposed by modeling image as a random graph. The proposed method starts with portioning input image into superpixels and extracting color and spatial features for each superpixel. Then, a complete graph is constructed by employing superpixels as nodes. A high edge weight is assigned into a pair of superpixels if they have high similarity. Next, a random walk prior on nodes is assumed to generate the probability distribution on edges. On the other hand, a complete directed graph is created that each edge weight represents the probability for transmitting random walker from current node to next node. By considering a threshold and eliminating edges with higher probability than the threshold, a random graph is created to model input image. The inbound degree vector of a random graph is computed to determine the most salient nodes (regions). Finally, a propagation technique is used to form saliency map. Experimental results on two challenging datasets: MSRA10K and SED2 demonstrate the efficiency of the proposed unsupervised RG method in comparison with the state-of-the-art unsupervised methods.

We formalize the class of “sequential local and show that these algorithms fail to find satisfying assignments on random instances of the “Not-All-Equal-$K$-SAT” (NAE-$K$-SAT) problem if the number of message passing iterations is bounded by a function moderately growing in the number of variables and if the clause-to-variable ratio is above $(1+o_K(1)){2^{K-1}\over K}\ln^2 K$ for sufficiently large $K$. Sequential local algorithms are those that iteratively set variables based on some local information and/or local randomness and then recurse on the reduced instance. Our model captures some weak abstractions of natural algorithms such as Survey Propagation (SP)-guided as well as Belief Propagation (BP)-guided decimation algorithms---two widely studied message-passing--based algorithms---when the number of message-passing rounds in these algorithms is restricted to be growing only moderately with the number of variables. The approach underlying our paper is based on an intricate geometry of th...

We consider a one-step replica symmetry breaking description of the Edwards–Anderson spin glass model in 2D. The ingredients of this description are a Kikuchi approximation to the free energy and a second-level statistical model built on the extremal points of the Kikuchi approximation, which are also fixed points of a generalized belief propagation (GBP) scheme. We show that a generalized free energy can be constructed where these extremal points are exponentially weighted by their Kikuchi free energy and a Parisi parameter y, and that the Kikuchi approximation of this generalized free energy leads to second-level, one-step replica symmetry breaking (1RSB), GBP equations. We then proceed analogously to the Bethe approximation case for tree-like graphs, where it has been shown that 1RSB belief propagation equations admit a survey propagation solution. We discuss when and how the one-step-replica symmetry breaking GBP equations that we obtain also allow a simpler class of solutions which can be interpreted as a class of generalized survey propagation equations for the single instance graph case.

We introduce an efficient message passing scheme for solving Constraint Satisfaction Problems (CSPs), which uses stochastic perturbation of Belief Propagation (BP) and Survey Propagation (SP) messa...

Following this advice, we should look over the big questions, then doodle and sketch out some approaches. If you are an expert, then this is easy to do, but most people do not want to wait to become an expert before looking at interesting problems. Graph theory, our area of expertise, has many hard-to-solve questions. Some hark back to the recreational roots of the area yet still keep their mystery. These ‘‘acorns’’ can be planted on the backs of envelopes, on a blackboard, and over a coffee. Our goal is to collect some of these conjectures—arguably some of the most intriguing—in one place. We present ten conjectures in graph theory, and you can read about each one in at most ten minutes. As we live in the era of Twitter, all the conjectures we state are 140 characters or fewer (so ‘‘minute’’ here has a double meaning). We might even call these sketchy tweets, as we present examples for each conjecture that you can doodle on as you read. Hamming also references ambiguity: good researchers can work both on proving and disproving the same statement, so we approach the conjectures with an open mind. He also mentions that a good approach is to reframe the problem, and change the point of view. One example from Vizing’s Conjecture (which is discussed as our second-to-last conjecture in the following text), is the three-page paper [2] which, with a new way of thinking, reduced most of the published work of twenty years to a corollary of its main result! Given the size of modern graph theory, with its many smaller subfields (such as structural graph theory, random graphs, topological graph theory, graph algorithms, spectral graph theory, graph minors, and graph homomorphisms, to name a few), it would be impossible to list all, or even the bulk of the conjectures in the field. We are content instead to focus on a few family jewels, which have an intrinsic beauty and have provided some challenges for graph-theorists for at least two decades. There is something for everyone here, from undergraduate students taking their first course in graph theory, to seasoned researchers in the field. For additional reading on problems and conjectures in graph theory and other fields, see the Open Problem Garden maintained by IRMACS at Simon Fraser University [24]. We consider only finite and undirected graphs, with no multiple edges or loops (unless otherwise stated). We assume the reader has some basic familiarity with graphs and their terminology, including notions such as cycles, paths, complete graphs, complete bipartite graphs, vertex degrees, and connected graphs. We use the notation Cn for the cycle with n vertices, Pn for the path with n vertices, and Kn for the complete graph with n vertices. The complete bipartite graph with m and n vertices of the respective colors is denoted by Km,n. All the background we need can be found in any text in graph theory, such as those of Diestel [9] and West [41], or online (see for example [42]).

The H-free process, for some fixed graph H, is the random graph process defined by starting with an empty graph on n vertices and then adding edges one at a time, chosen uniformly at random subject to the constraint that no H subgraph is formed. Let G be the random maximal H-free graph obtained at the end of the process. When H is strictly 2-balanced, we show that for some c>0, with high probability as n→∞, the minimum degree in G is at least \(cn^{1-(v_{H}-2)/(e_{H}-1)}(\log n)^{1/(e_{H}-1)}\). This gives new lower bounds for the Turán numbers of certain bipartite graphs, such as the complete bipartite graphs K r,r with r≥5. When H is a complete graph K s with s≥5 we show that for some C>0, with high probability the independence number of G is at most \(Cn^{2/(s+1)}(\log n)^{1-1/(e_{H}-1)}\). This gives new lower bounds for Ramsey numbers R(s,t) for fixed s≥5 and t large. We also obtain new bounds for the independence number of G for other graphs H, including the case when H is a cycle. Our proofs use the differential equations method for random graph processes to analyse the evolution of the process, and give further information about the structure of the graphs obtained, including asymptotic formulae for a broad class of subgraph extension variables.

Finite size effects for the Ising model on random graphs with varying dilution

It is well known that tree-based theories can describe the properties of undirected clustered networks with extremely accurate results [S. Melnik etal., Phys. Rev. E 83, 036112 (2011)10.1103/PhysRevE.83.036112]. It is reasonable to suggest that a motif-based theory would be superior to a tree one, since additional neighbor correlations are encapsulated in the motif structure. In this paper, we examine bond percolation on random and real world networks using belief propagation in conjunction with edge-disjoint motif covers. We derive exact message passing expressions for cliques and chordless cycles of finite size. Our theoretical model gives good agreement with Monte Carlo simulation and offers a simple, yet substantial improvement on traditional message passing, showing that this approach is suitable to study the properties of random and empirical networks.

The research presented in this dissertation focuses on the use of platinum-based catalysts to enhance endothermic fuel cooling. Chapter 1 gives a brief introduction to the motivation for this work. Chapter 2 presents fundamental studies on the catalytic dehydrogenation of ethylene by size-selected Ptn (n = 4, 7, 8) clusters deposited onto thin film alumina supports. The model catalysts were probed by a combination of experimental and theoretical techniques including; temperature-programmed desorption and reaction (TPD/R), low energy ion scattering spectroscopy (ISS), X-ray photoelectron spectroscopy (XPS), plane wave density-functional theory (PW-DFT), and statistical mechanical theory. It is shown that the Pt clusters dehydrogenated approximately half of the initially adsorbed ethylene, leading to deactivation of the catalyst via (coking) carbon deposition. The catalytic activity was observed to be size-dependent and strongly correlated to the cluster structure, with Pt7 demonstrating the highest activity. In Chapter 3 the focus turns to selectively doping Pt7 clusters with boron. A combination of experiment and theory were used investigate the alkene-binding affinity of the bimetallic (PtnBm/alumina) model catalysts. A comparison of the theoretical and experimental results show that doping the Pt clusters with boron modifies the alkene-binding affinity and thus the tendency toward dehydrogenation to coke precursors. Chapter 4 describes a way to produce bimetallic (PtnBm/alumina) model catalysts by exposing prepared Ptn/alumina samples to diborane and heating. It is shown that the diborane exposure/hearting procedure results in the preferential binding of B to the Pt clusters.

Practical implementation of random binning is one of the key challenges in achieving the largest available rate regions for many multiuser channels. This paper explores the use of low-density parity-check (LDPC) like codes for a particular kind of deterministic broadcast channel called the Blackwell channel and illustrates that random linear codes can be used to construct practical binning schemes at rates close to the capacity region of the Blackwell channel. The key ingredient is an encoding algorithm known as which is a generalization of the well-known belief propagation algorithm for LDPC codes. Survey propagation has been previously devised for a class of constraint satisfaction problems called K-SAT. This paper shows that the encoding problem for the Blackwell channel contains the same features as the constraint satisfaction problem and that the survey propagation algorithm, when concatenated with an outer error correcting code, works well at rates close to the Blackwell channel capacity region