Inference in particle tracking experiments by passing messages between images

: Calculating the amount of information about a random function contained in another random function has important uses in communication theory. An expression for the mutual information for continuous time random processes has been given by Gelfand and Yaglom, Chiang, and Perez by generalizing Shannon's result in a natural way. Under a condition of absolute continuity of measures the continuous time expression has the same form as Shannon's result. For two Gaussian processes Gelfand and Yaglom express the mutual information in terms of a mean square estimation error. We generalize this result to diffusion processes and express the solution in a different form which is more naturally related to a corresponding filtering problem. We also use these results to calculate some information rates.

Sum product algorithm on graphs is a general message passing algorithm that unifies many algorithms in channel coding, signal processing, Bayesian inference, and statistical physics. We show that, by extending the underlying algebraic and graphical structures of sum product algorithm, it also provides a unifying perspective for important distributed algorithms in communication networks. Examples treated here include Bellman-Ford (1957, 1962) routing, traffic shaping, wireless network power control, and congestion control. Through local message passing in the form of sum product algorithm on graphs, each of these network control algorithms solves a corresponding global optimization problem. This common framework also leads to new distributed algorithms, such as joint optimization of power control and utility maximization through distributed gradient descent.

Many different algorithms developed in statistical physics, coding theory, signal processing, and artificial intelligence can be expressed by graphical models and solved (either exactly or approximately) with iterative message-passing algorithms on the model. One quantity of interest in these algorithms is the partition function. In graphical models without cycle (trees), the partition function can be computed efficiently by means of message-passing algorithms, like GDL or the sum-product algorithm. In contrast, when graphical models contain cycles, the computation of the partition function is in general intractable. Our contributions in this dissertation are: deriving deterministic upper and lower bounds on partition function, and developing methods to approximate this quantity with Monte Carlo methods. Specifically, we obtain subtree-based upper and lower bounds which lead to theoretical results on optimality properties of the minimum entropy sub-tree and finally lead to a greedy algorithm. At last, we introduce and analyze a number of estimators that use Gibbs sampling to draw samples from different target distributions to estimate the value of the partition function. In one estimator, we demonstrate a novel strategy which combines Gibbs sampling and message-passing algorithms on trees.

Factor graphs are important models for succinctly representing probability distributions in machine learning, coding theory, and statistical physics. Several computational problems, such as computing marginals and partition functions, arise naturally when working with factor graphs. Belief propagation is a widely deployed iterative method for solving these problems. However, despite its significant empirical success, not much is known about the correctness and efficiency of belief propagation. Bethe approximation is an optimization-based framework for approximating partition functions. While it is known that the stationary points of the Bethe approximation coincide with the fixed points of belief propagation, in general, the relation between the Bethe approximation and the partition function is not well understood. It has been observed that for a few classes of factor graphs, the Bethe approximation always gives a lower bound to the partition function, which distinguishes them from the general case, where neither a lower bound, nor an upper bound holds universally. This has been rigorously proved for permanents and for attractive graphical models. Here we consider bipartite normal factor graphs and show that if the local constraints satisfy a certain analytic property, the Bethe approximation is a lower bound to the partition function. We arrive at this result by viewing factor graphs through the lens of polynomials. In this process, we reformulate the Bethe approximation as a polynomial optimization problem. Our sufficient condition for the lower bound property to hold is inspired by recent developments in the theory of real stable polynomials. We believe that this way of viewing factor graphs and its connection to real stability might lead to a better understanding of belief propagation and factor graphs in general.

For the tensor principal component analysis (tensor PCA) problem, we propose a new hierarchy of increasingly powerful algorithms with increasing runtime. Our hierarchy is analogous to the sum-of-squares (SOS) hierarchy but is instead inspired by statistical physics and related algorithms such as belief propagation and AMP (approximate message passing). Our level-ℓ algorithm can be thought of as a linearized message-passing algorithm that keeps track of ℓ-wise dependencies among the hidden variables. Specifically, our algorithms are spectral methods based on the Kikuchi Hessian , which generalizes the well-studied Bethe Hessian to the higher-order Kikuchi free energies. It is known that AMP, the flagship algorithm of statistical physics, has substantially worse performance than SOS for tensor PCA. In this work, we ‘redeem’ the statistical physics approach by showing that our hierarchy gives a polynomial-time algorithm matching the performance of SOS. Our hierarchy also yields a continuum of subexponential-time algorithms, and we prove that these achieve the same (conjecturally optimal) tradeoff between runtime and statistical power as SOS. Our proofs are much simpler than prior work, and also apply to the related problem of refuting random k -XOR formulas. The results we present here apply to tensor PCA for tensors of all orders, and to k -XOR when k is even. Our methods suggest a new avenue for systematically obtaining optimal algorithms for Bayesian inference problems, and our results constitute a step toward unifying the statistical physics and sum-of-squares approaches to algorithm design.

The statistical description of particle dispersion in turbulent flow constitutes nowadays an active research area. Such a field is linked to the Lagrangian description of fluid turbulence and poses long standing questions on modeling of hydrodynamical forces on objects in unsteady nonuniform flows [1]. Even in the simplest conditions of highly diluted suspensions of particles whose sizes are smaller than the dissipative scale of the carrier flow, the presently available models are highly idealized. Therefore, theoretical results and predictions need systematic verifications with experimental measures. Particle tracking experiments are technically challenging for the high time and space resolution demanded and for the need to precisely estimate and control the intensity of turbulent flow. Our recent works have been devoted to comparisons between experiments and numerical simulations of simple dynamical particle models [2, 3]. We have widely investigated Lagrangian particle models evolving in a turbulent environment which is described from an Eulerian point of view. In this abstract we shortly review the methods employed: the models for particle dynamics together with the numerical methodology, and we detail on some recent progresses and results.

Data association, the problem of reasoning over correspondence between targets and measurements, is a fundamental problem in tracking. This paper presents a graphical model formulation of data association and applies an approximate inference method, belief propagation (BP), to obtain estimates of marginal association probabilities. We prove that BP is guaranteed to converge, and bound the number of iterations necessary. Experiments reveal a favourable comparison to prior methods in terms of accuracy and computational complexity.

This chapter discusses fundamentals of information theory and channel coding (error correction coding ECC, Forward error correction FEC). First, key results of Shannon, including channel capacity and the relationship between transmit power and bandwidth, are discussed. Block codes multiply a block of data with an encoding matrix. For systematic codes, the resulting codeword consists of systematic bits and parity check bits. Decoding is achieved by multiplication with a parity check matrix and determination of the syndrome vector. Correction of errors within the correction sphere can be done, e.g., through lookup tables. Convolutional codes send the source data stream through shift registers and adders, a process described by a trellis diagram. Maximum-likelihood sequence estimation (Viterbi algorithm) determines the (transmit) bit sequence with best distance metric (Hamming or Euclidean) from the received signal. Methods for combining coding and higher-order modulation include trellis coded modulation (using set partitioning for code design) and bit-interleaved coded modulation (BICM). Turbo codes are close to capacity-achieving. They are based on transmitting differently interleaved convolutional codes; receivers use soft decoding of the constituent codes and iterative exchange of soft information (log-likelihood ratios). Low-density parity check codes (LDPCC) are decoded by belief propagation (message passing), where variable nodes and constraint nodes exchange beliefs about the codeword. In fading channels, (block or convolutional) interleaving is essential to provide diversity. From an information-theoretic point of view, in fading channels we can distinguish between the ergodic capacity and the outage capacity. If the channel state information is known at the transmitter, waterfilling is the optimum power allocation strategy. Automatic Repeat Request ARQ) and Hybrid ARQ (including chase combining or incremental redundancy) are described in the appendix.

Hidden Markov processes (HMPs) are important objects of study in many areas of pure and applied mathematics, including information theory, probability theory, dynamical systems and statistical physics, with applications in electrical engineering, computer science and molecular biology. This collection of research and survey papers presents important new results and open problems, serving as a unifying gateway for researchers in these areas. Based on talks given at the Banff International Research Station Workshop, 2007, this volume addresses a central problem of the subject: computation of the Shannon entropy rate of an HMP. This is a key quantity in statistical physics and information theory, characterising the fundamental limit on compression and closely related to channel capacity, the limit on reliable communication. Also discussed, from a symbolic dynamics and thermodynamical viewpoint, is the problem of characterizing the mappings between dynamical systems which map Markov measures to Markov (or Gibbs) measures, and which allow for Markov lifts of Markov chains.

Numerical modeling of the beach process of marine plastics: A probabilistic and diagnostic approach with a particle tracking method

A decade on, the situation portrayed in the influential report of a deteriorating US science and technology ecosystem has, if anything, worsened.

Women and computing

Preface. 1 Introduction. 1.1 Two examples of combinatorial optimization. 1.2 Why study combinatorial optimization using statistical physics? 1.3 Textbooks. Bibliography. 2 Algorithms. 2.1 Pidgin Algol. 2.2 Iteration and recursion. 2.3 Divide-and-conquer. 2.4 Dynamic programming. 2.5 Backtracking. Bibliography. 3 Introduction to graphs. 3.1 Basic concepts and graph problems. 3.2 Basic graph algorithms. 3.3 Random graphs. Bibliography. 4 Introduction to complexity theory. 4.1 Turing machines. 4.2 Church's thesis. 4.3 Languages. 4.4 The halting problem. 4.5 Class P. 4.6 Class NP. 4.7 Definition of NP-completeness. 4.8 NP-complete problems. 4.9 Worst-case vs. typical-case complexity. Bibliography. 5 Statistical mechanics of the Ising model. 5.1 Phase transitions. 5.2 Some general notes on statistical mechanics. 5.3 The Curie-Weiss model of a ferromagnet. 5.4 The Ising model on a random graph. Bibliography. 6 Algorithms and numerical results for vertex covers. 6.1 Definitions. 6.2 Heuristic algorithms. 6.3 Branch-and-bound algorithm. 6.4 Results: Covering random graphs. 6.5 The leaf-removal algorithm. 6.6 Monte Carlo simulations. 6.7 Backbone. 6.8 Clustering of minimum vertex covers. Bibliography. 7 Statistical mechanics of vertex covers on a random graph. 7.1 Introduction. 7.2 The first-moment bound. 7.3 The hard-core lattice gas. 7.4 Replica approach. Bibliography. 8 The dynamics of vertex-cover algorithms. 8.1 The typical-case solution time of a complete algorithm. 8.2 The dynamics of generalized leaf-removal algorithms. 8.3 Random restart algorithms. Bibliography. 9 Towards new, statistical-mechanics motivated algorithms. 9.1 The cavity graph. 9.2 Warning propagation. 9.3 Belief propagation. 9.4 Survey propagation. 9.5 Numerical experiments on random graphs. Bibliography. 10 The satisfiability problem. 10.1 SAT algorithms. 10.2 Phase transitions in random K-SAT. 10.3 Typical-case dynamics of RandomWalkSAT. 10.4 Message-passing algorithms for SAT. Bibliography. 11 Optimization problems in physics. 11.1 Monte Carlo optimization. 11.2 Hysteric optimization. 11.3 Genetic algorithms. 11.4 Shortest paths and polymers in random media. 11.5 Maximum flows and random-field systems. 11.6 Submodular functions and free energy of Potts model. 11.7 Matchings and spin glasses. Bibliography. Index.

The paper investigates parameterized approximate message-passing schemes that are based on bounded inference and are inspired by Pearl's belief propagation algorithm (BP). We start with the bounded inference mini-clustering algorithm and then move to the iterative scheme called Iterative Join-Graph Propagation (IJGP), that combines both iteration and bounded inference. Algorithm IJGP belongs to the class of Generalized Belief Propagation algorithms, a framework that allowed connections with approximate algorithms from statistical physics and is shown empirically to surpass the performance of mini-clustering and belief propagation, as well as a number of other state-of-the-art algorithms on several classes of networks. We also provide insight into the accuracy of iterative BP and IJGP by relating these algorithms to well known classes of constraint propagation schemes.

The study of dynamical processes on complex networks constitutes a foundational domain bridging applied mathematics, statistical physics, systems theory, and data science. Temporal evolution, not static topology, determines the controllability, stability, and inference limits of real-world systems, from epidemics and neural circuits to power grids and social media. However, the methodological landscape remains fragmented, with distinct communities advancing separate formalisms for spreading, control, inference, and design. This review presents a unifying six-pillar framework for the analysis of network dynamics: (i) spectral and structural foundations; (ii) deterministic mean-field reductions; (iii) control and observability theory; (iv) adaptive and temporal networks; (v) probabilistic inference and belief propagation; (vi) multilayer and interdependent systems. Within each pillar, we delineate conceptual motivations, canonical models, analytical methodologies, and open challenges. Our corpus, selected via a PRISMA-guided screening of 134 mathematically substantive works (1997–2024), is organized to emphasize internal logic and cross-pillar connectivity. By mapping the field onto a coherent methodological spine, this survey aims to equip theorists and practitioners with a transferable toolkit for interpreting, designing, and controlling dynamic behavior on networks.