Exact sampling and approximate counting techniques

Low complexity state space representation and algorithms for closed queueing networks exact sampling

In Monte Carlo Markov Chain theory, a particle on a Markov chain is run for a ’long time’ until the particle is nearly in the stationary distribution. Coupling from the past is a methodology which allows samples to be taken exactly from the stationary distribution of the chain, but which requires that we have a completely coupling chain. One chain often used for the ferromagnetic Ising model is the Swendsen-Wang chain. We give the first method for running Swendsen-Wang as a completely coupling chain, thereby giving a procedure for taking exact samples from the ferromagnetic Ising model using Swendsen-Wang. Moreover, this chain completely couples in polynomial time when a parameter of the chain is inversely proportional to the maximum degree of the graph, or when this parameter is close to 1. This has two implications. First, the Swendsen-Wang chain is rapidly mixing over this set of parameter values, a fact recently (and independently) shown by Cooper and Frieze using path coupling. Second, over this set of parameter values the exact sampling procedure will have an expected running time which is polynomial. ∗This work was partially supported by an Office of Naval Research Fellowship and NSF grants CCR-9307391, CCR-9700029, and DMS-9505155, and ONR grant N00014-96-1-00500

New algorithm of exact sampling with directional threshold

The paper presents a class of numerical methods to compute the stationary distribution of Markov chains (MCs) with large and structured state spaces. A popular way of dealing with large state spaces in Markovian modeling and analysis is to employ Kronecker-based representations for the generator matrix and to exploit this matrix structure in numerical analysis methods. This paper presents various multilevel (ML) methods for a broad class of MCs with a hierarchcial Kronecker structure of the generator matrix. The particular ML methods are inspired by multigrid and aggregation-disaggregation techniques, and differ among each other by the type of multigrid cycle, the type of smoother, and the order of component aggregation they use. Numerical experiments demonstrate that so far ML methods with successive over-relaxation as smoother provide the most effective solvers for considerably large Markov chains modeled as HMMs with multiple macrostates.

Variable selection is very important in many fields, and for its resolution many procedures have been proposed and investigated. Among them are Bayesian methods that use Markov chain Monte-Carlo (MCMC) sampling algorithms. A problem with MCMC sampling, however, is that it cannot guarantee that the samples are exactly from the target distributions. This drawback is overcome by related methods known as perfect sampling algorithms. In this paper, we propose the use of two perfect sampling algorithms to perform variable selection within the Bayesian framework. They are the sandwiched coupling from the past (CFTP) algorithm and the Gibbs coupler. We focus our attention to scenarios where the model coefficients and noise variance are known. We indicate the condition under which the sandwiched CFTP can be applied. Most importantly, we design a detailed scheme to adapt the Gibbs coupler algorithm to variable selection. In addition, we discuss the possibilities of applying perfect sampling when the model coefficients and noise variance are unknown. Test results that show the performance of the algorithms are provided.

The standard Coupling From The Past (CFTP) algorithm is an interesting tool to sample from exact stationary distribution of a Markov chain. But it is very expensive in time consuming for large chains. There is a monotone version of CFTP, called MCFTP, that is less time consuming for monotone chains. In this work, we propose two techniques to get monotone chain allowing use of MCFTP: widening technique based on adding two fictitious states and clustering technique based on partitioning the state space in clusters. Usefulness and efficiency of our approaches are showed through a sample of Markov Chain Monte Carlo simulations.

We study new formulae based on Lya- punov exponents for entropy, mutual information, and capacity of finite state discrete time Markov chan- nels. We also develop a method for directly com- puting mutual information and entropy using contin- uous state space Markov chains. We show that the entropy rate for a symbol sequence is equal to the primary Lyapunov exponent for a product of random matrices. We then develop a continuous state space Markov chain formulation that allows us to directly compute entropy rates as expectations with respect to the Markov chain's stationary distribution. We also show that the stationary distribution is a continuous function of the input symbol dynamics. This continu- ity allows the channel capacity to be written in terms of Lyapunov exponents.

Let G = (V,E) be a weighted undirected graph with n vertices and m edges, and let d_G(u,v) be the length of the shortest path between u and v in G. In this paper we present a unified approach for obtaining algorithms for all pairs approximate shortest paths in weighted undirected graphs. For every integer k ≥ 2 we show that there is an O(n²+kn^{2-3/k}m^{2/k}) expected running time algorithm that computes a matrix M such that for every u,v ∈ V: d_G(u,v) ≤ M[u,v] ≤ (2+(k-2)/k)d_G(u,v). Previous algorithms obtained only specific approximation factors. Baswana and Kavitha [FOCS 2006, SICOMP 2010] presented a 2-approximation algorithm with expected running time of O(n²+m√ n) and a 7/3-approximation algorithm with expected running time of O(n²+m^{2/3}n). Their results improved upon the results of Cohen and Zwick [SODA 1997, JoA 2001] for graphs with m = o(n²). Kavitha [FSTTCS 2007, Algorithmica 2012] presented a 5/2-approximation algorithm with expected running time of O(n^{9/4}). For k = 2 and k = 3 our result gives the algorithms of Baswana and Kavitha. For k = 4, we get a 5/2-approximation algorithm with O(n^{5/4}m^{1/2}) expected running time. This improves upon the running time of O(n^{9/4}) due to Kavitha, when m = o(n²). Our unified approach reveals that all previous algorithms are a part of a family of algorithms that exhibit a smooth tradeoff between approximation of 2 and 3, and are not sporadic unrelated results. Moreover, our new algorithm uses, among other ideas, the celebrated approximate distance oracles of Thorup and Zwick [STOC 2001, JACM 2005] in a non standard way, which we believe is of independent interest, due to their extensive usage in a variety of applications.

Topology of the space of periodic ground states in the antiferromagnetic Ising and Potts models in selected spatial structures

Previous chapter Next chapter Full AccessProceedings Proceedings of the 2020 ACM-SIAM Symposium on Discrete Algorithms (SODA)Faster Deterministic and Las Vegas Algorithms for Offline Approximate Nearest Neighbors in High DimensionsJosh Alman, Timothy M. Chan, and Ryan WilliamsJosh Alman, Timothy M. Chan, and Ryan Williamspp.637 - 649Chapter DOI:https://doi.org/10.1137/1.9781611975994.39PDFBibTexSections ToolsAdd to favoritesExport CitationTrack CitationsEmail SectionsAboutAbstract We present a deterministic, truly subquadratic algorithm for offline (1 + ε)-approximate nearest or farthest neighbor search (in particular, the closest pair or diameter problem) in Hamming space in any dimension d ≤ nδ, for a sufficiently small constant δ > 0. The running time of the algorithm is roughly for nearest neighbors, or for farthest. The algorithm follows from a simple combination of expander walks, Chebyshev polynomials, and rectangular matrix multiplication. We also show how to eliminate errors in the previous Monte Carlo randomized algorithm of Alman, Chan, and Williams [FOCS’16] for offline approximate nearest or farthest neighbors, and obtain a Las Vegas randomized algorithm with expected running time . Finally, we note a simplification of Alman, Chan, and Williams' method and obtain a slightly improved Monte Carlo randomized algorithm with running time . As one application, we obtain improved deterministic and randomized (1 + ε)-approximation algorithms for MAX-SAT. Previous chapter Next chapter RelatedDetails Published:2020eISBN:978-1-61197-599-4 https://doi.org/10.1137/1.9781611975994Book Series Name:ProceedingsBook Code:PRDA20Book Pages:xxii + 3011

We consider the Similarity Sketching problem: Given a universe [u] = {0,..., u-1} we want a random function S mapping subsets A of [u] into vectors S(A) of size t, such that similarity is preserved. More precisely: Given subsets A,B of [u], define X_i = [S(A)[i] = S(B)[i]] and X = sum_{i in [t]} X_i. We want to have E[X] = t*J(A,B), where J(A,B) = |A intersect B|/|A union B| and furthermore to have strong concentration guarantees (i.e. Chernoff-style bounds) for X. This is a fundamental problem which has found numerous applications in data mining, large-scale classification, computer vision, similarity search, etc. via the classic MinHash algorithm. The vectors S(A) are also called sketches. The seminal t x MinHash algorithm uses t random hash functions h_1,..., h_t, and stores (min_{a in A} h_1(A),..., min_{a in A} h_t(A)) as the sketch of A. The main drawback of MinHash is, however, its O(t*|A|) running time, and finding a sketch with similar properties and faster running time has been the subject of several papers. Addressing this, Li et al. [NIPS12] introduced one permutation hashing (OPH), which creates a sketch of size t in O(t + |A|) time, but with the drawback that possibly some of the t entries are empty when |A| = O(t). One could argue that sketching is not necessary in this case, however the desire in most applications is to have one sketching procedure that works for sets of all sizes. Therefore, filling out these empty entries is the subject of several follow-up papers initiated by Shrivastava and Li [ICML14]. However, these densification schemes fail to provide good concentration bounds exactly in the case |A| = O(t), where they are needed. In this paper we present a new sketch which obtains essentially the best of both worlds. That is, a fast O(t log t + |A|) expected running time while getting the same strong concentration bounds as MinHash. Our new sketch can be seen as a mix between sampling with replacement and sampling without replacement. We demonstrate the power of our new sketch by considering popular applications in large-scale classification with linear SVM as introduced by Li et al. [NIPS11] as well as approximate similarity search using the LSH framework of Indyk and Motwani [STOC98]. In particular, for the j_1, j_2-approximate similarity search problem on a collection of n sets we obtain a data-structure with space usage O(n^{1+rho} + sum_{A in C} |A|) and O(n^rho * log n + |Q|) expected time for querying a set Q compared to a O(n^rho * log n * |Q|) expected query time of the classic result of Indyk and Motwani.

In 1996, Propp and Wilson introduced coupling from the past (CFTP), an algorithm for generating a sample from the exact stationary distribution of a Markov chain. In 1998, Fill proposed another so–called perfect sampling algorithm. These algorithms have enormous potential in Markov Chain Monte Carlo (MCMC) problems because they eliminate the need to monitor convergence and mixing of the chain. This article provides a brief introduction to the algorithms, with an emphasis on understanding rather than technical detail.

The set ℓ Pol of complex numbers ζ , which can be approximated in a polynomial time cζ ·n k depending from the precision 2−n, is a field. The operations of the field can be approximated in a time<c · n 2 and better relative to the time one needs to approximate the operands ζ 1 ,ζ 2 with precision 2−n. In the case of addition and multiplication it can be done for |ζ 1 |, |ζ 2 |<a with a c depending from a and the constants in the running time of the polynomial time approximation of ζ 1,ζ 2, but in the case of division the constant depends additionally from |ζ | . The same holds for the ring of the polynomials ℓ Pol [x ] and the field of the rational functions f ∈ ℓ Pol (x ). The constant factor in the polynomial time bound of the approximation of f (ζ ) depend additionally from the distance of ζ from the singularities of f . The field ℓ Pol is algebraically closed. In the case that the polynomials have constant coefficients there exist algorithms to approximate the roots of the polynomials in a linear time with the Newton method if the roots are simple and an isolation of the roots is precomputed [Sch82]. But this precomputation needs much time [ESY06]. The best known algorithms to solve this problem need a time O (m 5(τ +logm )2), m the degree of the polynomial and τ the length of the binary representation of the coefficients. The problem with not constant coefficients has been attacked on base of bitstreams in [Eig08], [ESY06]. In [Eig08] on can find an excellent overview of the state of art. In the case the coefficients are not fixed constants but given by approximations the situation is much more complicated because the precomputation depends from the precision of the approximation. It is clear that in general the approximation of a root of a polynomial with coefficients, which cannot be approximated in a time O (n k ), cannot be done in this time. Ker-I Ko discusses the approximation problems on base of the unary representation of the numbers [Ko91]. Polynomial time algorithms relative to dyadic and unary representation are obviously not the same. The motivation of this paper is the question how far the mentioned problems can be solved by global polynomial time algorithms. This are algorithms with a running time <c·n k with the constant c only depending from the constants of the polynomial time approximations of the arguments and the domain of the function to be computed. We prove that there exist analytic functions, which are computable on all finite domains in this sense and that analytic functions for which this is true can be represented by a power series in z.

Given an undirected graph G=(V,E) with node set V=[1,n], a subset $S\subseteq V$, and a rational vector $a\in {\rm\bf Q}^{S\cup E}$, the positive semidefinite matrix completion problem consists of determining whether there exists a real symmetric n × n positive semidefinite matrix X=(xij) satisfying xii=ai ($i\in S$) and xij=aij ($ij\in E$). Similarly, the Euclidean distance matrix completion problem asks for the existence of a Euclidean distance matrix completing a partially defined given matrix. It is not known whether these problems belong to NP. We show here that they can be solved in polynomial time when restricted to the graphs having a fixed minimum fill-in, the minimum fill-in of graph G being the minimum number of edges needed to be added to G in order to obtain a chordal graph. A simple combinatorial algorithm permits us to construct a completion in polynomial time in the chordal case. We also show that the completion problem is polynomially solvable for a class of graphs including wheels of fixed length (assuming all diagonal entries are specified). The running time of our algorithms is polynomially bounded in terms of n and the bitlength of the input a. We also observe that the matrix completion problem can be solved in polynomial time in the real number model for the class of graphs containing no homeomorph K4.

The results of this paper provide an Efficient Polynomial-Time Approximation Scheme (EPTAS) for approximating the genus (and non-orientable genus) of dense graphs. The running time of the algorithm is quadratic. Moreover, we extend the algorithm to output an embedding (rotation system), whose genus is arbitrarily close to the minimum genus. This second algorithm is an Efficient Polynomial-time Randomized Approximation Scheme (EPRAS) and the expected running time is also quadratic.