Distributed network control through sum product algorithm on graphs

The invention of Fountain codes is a major advance in the field of error correcting codes. The goal of this work is to study and develop algorithms for source and channel coding using a family of Fountain codes known as Raptor codes. From an asymptotic point of view, the best currently known sum-product decoding algorithm for non binary alphabets has a high complexity that limits its use in practice. For binary channels, sum-product decoding algorithms have been extensively studied and are known to perform well. In the first part of this work, we develop a decoding algorithm for binary codes on non-binary channels based on a combination of sum-product and maximum-likelihood decoding. We apply this algorithm to Raptor codes on both symmetric and non-symmetric channels. Our algorithm shows the best performance in terms of complexity and error rate per symbol for blocks of finite length for symmetric channels. Then, we examine the performance of Raptor codes under sum-product decoding when the transmission is taking place on piecewise stationary memoryless channels and on channels with memory corrupted by noise. We develop algorithms for joint estimation and detection while simultaneously employing expectation maximization to estimate the noise, and sum-product algorithm to correct errors. We also develop a hard decision algorithm for Raptor codes on piecewise stationary memoryless channels. Finally, we generalize our joint LT estimation-decoding algorithms for Markov-modulated channels. In the third part of this work, we develop compression algorithms using Raptor codes. More specifically we introduce a lossless text compression algorithm, obtaining in this way competitive results compared to the existing classical approaches. Moreover, we propose distributed source coding algorithms based on the paradigm proposed by Slepian and Wolf.

We consider the problem of downlink power control in wireless networks, consisting of multiple transmitter-receiver pairs communicating with each other over a single shared wireless medium. To mitigate the interference among concurrent transmissions, we leverage the network topology to create a graph neural network architecture, and we then use an unsupervised primal-dual counterfactual optimization approach to learn optimal power allocation decisions. We show how the counterfactual optimization technique allows us to guarantee a minimum rate constraint, which adapts to the network size, hence achieving the right balance between average and 5th percentile user rates throughout a range of network configurations.

Methods to extract information from the tracking of mobile objects/particles have broad interest in biological and physical sciences. Techniques based on simple criteria of proximity in time-consecutive snapshots are useful to identify the trajectories of the particles. However, they become problematic as the motility and/or the density of the particles increases due to uncertainties on the trajectories that particles followed during the images' acquisition time. Here, we report an efficient method for learning parameters of the dynamics of the particles from their positions in time-consecutive images. Our algorithm belongs to the class of message-passing algorithms, known in computer science, information theory, and statistical physics as belief propagation (BP). The algorithm is distributed, thus allowing parallel implementation suitable for computations on multiple machines without significant intermachine overhead. We test our method on the model example of particle tracking in turbulent flows, which is particularly challenging due to the strong transport that those flows produce. Our numerical experiments show that the BP algorithm compares in quality with exact Markov Chain Monte Carlo algorithms, yet BP is far superior in speed. We also suggest and analyze a random distance model that provides theoretical justification for BP accuracy. Methods developed here systematically formulate the problem of particle tracking and provide fast and reliable tools for the model's extensive range of applications.

3D shape matching based on meshed surfaces can be formulated as an energy function minimisation problem under a Markov random field (MRF) framework. However, to solve such a global optimisation problem is NP-hard. So research mainly focuses on approximation algorithms. One of the best known is belief propagation (BP), which has shown success in early vision and many other practical applications. In this paper, we investigate the application of both sum and max product algorithms of belief propagation to 3D shape matching. We also apply the 3D shape matching results to a 3D registration problem.

This chapter discusses fundamentals of information theory and channel coding (error correction coding ECC, Forward error correction FEC). First, key results of Shannon, including channel capacity and the relationship between transmit power and bandwidth, are discussed. Block codes multiply a block of data with an encoding matrix. For systematic codes, the resulting codeword consists of systematic bits and parity check bits. Decoding is achieved by multiplication with a parity check matrix and determination of the syndrome vector. Correction of errors within the correction sphere can be done, e.g., through lookup tables. Convolutional codes send the source data stream through shift registers and adders, a process described by a trellis diagram. Maximum-likelihood sequence estimation (Viterbi algorithm) determines the (transmit) bit sequence with best distance metric (Hamming or Euclidean) from the received signal. Methods for combining coding and higher-order modulation include trellis coded modulation (using set partitioning for code design) and bit-interleaved coded modulation (BICM). Turbo codes are close to capacity-achieving. They are based on transmitting differently interleaved convolutional codes; receivers use soft decoding of the constituent codes and iterative exchange of soft information (log-likelihood ratios). Low-density parity check codes (LDPCC) are decoded by belief propagation (message passing), where variable nodes and constraint nodes exchange beliefs about the codeword. In fading channels, (block or convolutional) interleaving is essential to provide diversity. From an information-theoretic point of view, in fading channels we can distinguish between the ergodic capacity and the outage capacity. If the channel state information is known at the transmitter, waterfilling is the optimum power allocation strategy. Automatic Repeat Request ARQ) and Hybrid ARQ (including chase combining or incremental redundancy) are described in the appendix.

Many different algorithms developed in statistical physics, coding theory, signal processing, and artificial intelligence can be expressed by graphical models and solved (either exactly or approximately) with iterative message-passing algorithms on the model. One quantity of interest in these algorithms is the partition function. In graphical models without cycle (trees), the partition function can be computed efficiently by means of message-passing algorithms, like GDL or the sum-product algorithm. In contrast, when graphical models contain cycles, the computation of the partition function is in general intractable. Our contributions in this dissertation are: deriving deterministic upper and lower bounds on partition function, and developing methods to approximate this quantity with Monte Carlo methods. Specifically, we obtain subtree-based upper and lower bounds which lead to theoretical results on optimality properties of the minimum entropy sub-tree and finally lead to a greedy algorithm. At last, we introduce and analyze a number of estimators that use Gibbs sampling to draw samples from different target distributions to estimate the value of the partition function. In one estimator, we demonstrate a novel strategy which combines Gibbs sampling and message-passing algorithms on trees.

Factor graphs are important models for succinctly representing probability distributions in machine learning, coding theory, and statistical physics. Several computational problems, such as computing marginals and partition functions, arise naturally when working with factor graphs. Belief propagation is a widely deployed iterative method for solving these problems. However, despite its significant empirical success, not much is known about the correctness and efficiency of belief propagation. Bethe approximation is an optimization-based framework for approximating partition functions. While it is known that the stationary points of the Bethe approximation coincide with the fixed points of belief propagation, in general, the relation between the Bethe approximation and the partition function is not well understood. It has been observed that for a few classes of factor graphs, the Bethe approximation always gives a lower bound to the partition function, which distinguishes them from the general case, where neither a lower bound, nor an upper bound holds universally. This has been rigorously proved for permanents and for attractive graphical models. Here we consider bipartite normal factor graphs and show that if the local constraints satisfy a certain analytic property, the Bethe approximation is a lower bound to the partition function. We arrive at this result by viewing factor graphs through the lens of polynomials. In this process, we reformulate the Bethe approximation as a polynomial optimization problem. Our sufficient condition for the lower bound property to hold is inspired by recent developments in the theory of real stable polynomials. We believe that this way of viewing factor graphs and its connection to real stability might lead to a better understanding of belief propagation and factor graphs in general.

For the tensor principal component analysis (tensor PCA) problem, we propose a new hierarchy of increasingly powerful algorithms with increasing runtime. Our hierarchy is analogous to the sum-of-squares (SOS) hierarchy but is instead inspired by statistical physics and related algorithms such as belief propagation and AMP (approximate message passing). Our level-ℓ algorithm can be thought of as a linearized message-passing algorithm that keeps track of ℓ-wise dependencies among the hidden variables. Specifically, our algorithms are spectral methods based on the Kikuchi Hessian , which generalizes the well-studied Bethe Hessian to the higher-order Kikuchi free energies. It is known that AMP, the flagship algorithm of statistical physics, has substantially worse performance than SOS for tensor PCA. In this work, we ‘redeem’ the statistical physics approach by showing that our hierarchy gives a polynomial-time algorithm matching the performance of SOS. Our hierarchy also yields a continuum of subexponential-time algorithms, and we prove that these achieve the same (conjecturally optimal) tradeoff between runtime and statistical power as SOS. Our proofs are much simpler than prior work, and also apply to the related problem of refuting random k -XOR formulas. The results we present here apply to tensor PCA for tensors of all orders, and to k -XOR when k is even. Our methods suggest a new avenue for systematically obtaining optimal algorithms for Bayesian inference problems, and our results constitute a step toward unifying the statistical physics and sum-of-squares approaches to algorithm design.

Noise is a major obstacle in the development of practical schemes for quantum computation and communication. Similar to the case of classical communication, this noise can be protected against by employing a code, which provides a means for encoding quantum states prior to transmission and allows for errors to be inferred, and hopefully corrected, by a decoder at the receiver. Unfortunately, designing good codes and decoders is typically a difficult problem. This thesis focuses on developing low-complexity heuristic approaches to three such problems: the design of modified belief propagation decoders for quantum low-density parity-check codes, the design of stabilizer codes for asymmetric channels, and the design of codeword stabilized codes. Quantum low-density parity-check codes are stabilizer codes with low-weight generators. Such codes permit low-complexity decoding via the use of belief propagation, which is an iterative message passing algorithm that takes place on a factor graph defined by the code. However, the performance of such a decoder is limited both by code structure and the degenerate nature of quantum errors. To overcome these limitations, at least in part, a number of modifications to belief propagation are developed. Central among these is the augmented decoder, which in the case of a decoding error, iteratively reattempts decoding using modified factor graph. This heuristic modification simply involves the duplication of a randomly selected subset of the graph’s check nodes, which are in one-to-one correspondence with the code’s stabilizer generators. Across a range of codes, it is shown that the decoders developed perform as well as or better than other modified decoders presented in literature. For a number of channels of physical interest, phase-flip errors occur far more frequently than bit-flip errors. When transmitting across these so-called asymmetric channels, the decoding error rate can be minimized by tailoring the code used to the channel. However, assessing the performance of codes on a given channel is made difficult by the #P-completeness of optimal decoding. To address this complexity, it is shown that the decoding error rate can be accurately approximated using only a small fraction of the possible errors caused by the channel. This approximation is then used to identify a number of cyclic stabilizer codes that perform well on two different asymmetric channels. To further build on this, a heuristic is demonstrated for assessing code performance based on the decoding error rate of an associated classical code. The complexity of calculating this classical error rate is relatively low, and it is shown that it can be used as the basis for a hill-climbing search algorithm. Such searches have yielded a large number of highly performant codes satisfying various structure constraints. The family of codeword stabilized codes encompasses both the stabilizer codes as well as many of the best known nonadditive codes. Constructing a standard form codeword stabilized code is a matter of selecting a simple undirected graph and a binary classical code. This makes designing optimal codes difficult as the number of possible graphs grows exponentially with code length, and the clique search required to construct the classical code is NP-hard. To address the exponential growth of the search space, a heuristic is developed for assessing graphs. This heuristic is then employed by a genetic algorithm that also makes use of a novel crossover operation based on spectral bisection, which is show to be superior to more standard crossover operations. With a graph selected, it is demonstrated that the complexity of the clique search required to construct the associated classical code can be mitigated through the use of a heuristic clique finding algorithm. A number of best known codes are presented that have been found using this approach.

Turbo Codes and Tail Biting Convolutional Codes are used for channel coding in LTE, but these codes are not pertinent in 5G. For Channel coding, advanced error correction techniques are required for the 5G. Polar codes are adopted for control information and LDPC codes are required for Data information for the 5G. Various coding techniques such as LDPC codes, Quasi Cyclic (QC)-LDPC codes, Non-Binary (NB)-LDPC codes, Spatially Coupled (SC)-LDPC codes, Irregular Repeat Accumulate (IRA) codes, Polar codes, Distributed CRC Aided (DCA) codes, and Fountain codes are illustrated. For Decoding Min-Sum Algorithm and Belief Propagation are used for LDPC codes. Min-Sum, OMS, NMS, BP methods are applied in QC-LDPC codes. SC, SCL, FSCL, CASCL, BP algorithms implemented for Polar codes. Simulation of QC-LDPC codes, Polar codes, and LDPC codes are generated by using MATLAB Software. The performance of all channel coding techniques with consideration to BER and BLER is mentioned.

This paper discusses the issues to consider when selecting a congestion control scheme for an integrated ground radio and satellite network engaging primarily in transmitting multimedia traffic, and then it proceeds to discuss effects of smoothing rendered on the input traffic as a congestion control scheme. We chose traffic shaping by smoothing rendered on the input traffic as the congestion control scheme to use for our mixed ground/satellite network due to the considerations of satellite's long propagation delay, high bandwidth of ground network, and the real-time nature of the traffic. Traffic shaping in our case, involves smoothing and policing the traffic by using the token and our original The simulation results show that in the case of congestion, virtual token bucket and multi-bucket algorithms, improve throughput significantly for the ground/satellite network, with virtual token buckets slightly outperforming multi-buckets. Our experimental multi-buckets algorithm is useful, because it is capable of rendering additional smoothing, and thus affording us more flexibility to examine the impact of smoothing. The fact that virtual token buckets outperform slightly the multi-buckets shows that more smoothing does not always render better throughput, and confirms that factors such as the tradeoffs between smoothing and network utilization, burstiness of the streams, and the number of congested hops traversed need to be considered when examining the usefulness of smoothing.

The major obstacle to shallow water acoustic communication is the interference of multipath signal results from surface and bottom reflections. Channel coding is indispensable in practical system to accommodate these adverse channel transmission characteristics due to the coding gain. In this paper, a channel model including time-varying fading, multipath and additive noise for the shallow water acoustic channels is built. Based on the channel model, the Probability Density Function (PDF) of initial decoding messages with Belief Propagation (BP) algorithm of LowDensity Parity-Check (LDPC) codes is deduced. The performance of the LDPC codes with BP algorithm is simulated, and the effects of multipath, fading velocity and channel interleaver on the decoding performance are studied. As the result, the following conclusions can be given: The performance of LDPC codes is excellent (Bit Error Rate (BER) is less than 10 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">-4</sup> ) over three-multipath shallow water acoustic channels, with 3~5 iterative decoding times and about 1000 bits code lengths; The BER of LDPC codes degrades with the increasing of the number of multipath; The BER of LDPC codes degrades slowly with the decreasing of the fading velocity, which shows the proposed channel code is not sensitive to the fading velocity of the channel.

Preface. 1 Introduction. 1.1 Two examples of combinatorial optimization. 1.2 Why study combinatorial optimization using statistical physics? 1.3 Textbooks. Bibliography. 2 Algorithms. 2.1 Pidgin Algol. 2.2 Iteration and recursion. 2.3 Divide-and-conquer. 2.4 Dynamic programming. 2.5 Backtracking. Bibliography. 3 Introduction to graphs. 3.1 Basic concepts and graph problems. 3.2 Basic graph algorithms. 3.3 Random graphs. Bibliography. 4 Introduction to complexity theory. 4.1 Turing machines. 4.2 Church's thesis. 4.3 Languages. 4.4 The halting problem. 4.5 Class P. 4.6 Class NP. 4.7 Definition of NP-completeness. 4.8 NP-complete problems. 4.9 Worst-case vs. typical-case complexity. Bibliography. 5 Statistical mechanics of the Ising model. 5.1 Phase transitions. 5.2 Some general notes on statistical mechanics. 5.3 The Curie-Weiss model of a ferromagnet. 5.4 The Ising model on a random graph. Bibliography. 6 Algorithms and numerical results for vertex covers. 6.1 Definitions. 6.2 Heuristic algorithms. 6.3 Branch-and-bound algorithm. 6.4 Results: Covering random graphs. 6.5 The leaf-removal algorithm. 6.6 Monte Carlo simulations. 6.7 Backbone. 6.8 Clustering of minimum vertex covers. Bibliography. 7 Statistical mechanics of vertex covers on a random graph. 7.1 Introduction. 7.2 The first-moment bound. 7.3 The hard-core lattice gas. 7.4 Replica approach. Bibliography. 8 The dynamics of vertex-cover algorithms. 8.1 The typical-case solution time of a complete algorithm. 8.2 The dynamics of generalized leaf-removal algorithms. 8.3 Random restart algorithms. Bibliography. 9 Towards new, statistical-mechanics motivated algorithms. 9.1 The cavity graph. 9.2 Warning propagation. 9.3 Belief propagation. 9.4 Survey propagation. 9.5 Numerical experiments on random graphs. Bibliography. 10 The satisfiability problem. 10.1 SAT algorithms. 10.2 Phase transitions in random K-SAT. 10.3 Typical-case dynamics of RandomWalkSAT. 10.4 Message-passing algorithms for SAT. Bibliography. 11 Optimization problems in physics. 11.1 Monte Carlo optimization. 11.2 Hysteric optimization. 11.3 Genetic algorithms. 11.4 Shortest paths and polymers in random media. 11.5 Maximum flows and random-field systems. 11.6 Submodular functions and free energy of Potts model. 11.7 Matchings and spin glasses. Bibliography. Index.

The paper investigates parameterized approximate message-passing schemes that are based on bounded inference and are inspired by Pearl's belief propagation algorithm (BP). We start with the bounded inference mini-clustering algorithm and then move to the iterative scheme called Iterative Join-Graph Propagation (IJGP), that combines both iteration and bounded inference. Algorithm IJGP belongs to the class of Generalized Belief Propagation algorithms, a framework that allowed connections with approximate algorithms from statistical physics and is shown empirically to surpass the performance of mini-clustering and belief propagation, as well as a number of other state-of-the-art algorithms on several classes of networks. We also provide insight into the accuracy of iterative BP and IJGP by relating these algorithms to well known classes of constraint propagation schemes.

The study of dynamical processes on complex networks constitutes a foundational domain bridging applied mathematics, statistical physics, systems theory, and data science. Temporal evolution, not static topology, determines the controllability, stability, and inference limits of real-world systems, from epidemics and neural circuits to power grids and social media. However, the methodological landscape remains fragmented, with distinct communities advancing separate formalisms for spreading, control, inference, and design. This review presents a unifying six-pillar framework for the analysis of network dynamics: (i) spectral and structural foundations; (ii) deterministic mean-field reductions; (iii) control and observability theory; (iv) adaptive and temporal networks; (v) probabilistic inference and belief propagation; (vi) multilayer and interdependent systems. Within each pillar, we delineate conceptual motivations, canonical models, analytical methodologies, and open challenges. Our corpus, selected via a PRISMA-guided screening of 134 mathematically substantive works (1997–2024), is organized to emphasize internal logic and cross-pillar connectivity. By mapping the field onto a coherent methodological spine, this survey aims to equip theorists and practitioners with a transferable toolkit for interpreting, designing, and controlling dynamic behavior on networks.