Determinantal Point Processes in Randomized Numerical Linear Algebra

A determinantal point process (DPP) is an ensemble of random nonnegative-integer-valued Radon measures [Formula: see text] on a space S with measure [Formula: see text], whose correlation functions are all given by determinants specified by an integral kernel K called the correlation kernel. We consider a pair of Hilbert spaces, [Formula: see text], which are assumed to be realized as [Formula: see text]-spaces, [Formula: see text], [Formula: see text], and introduce a bounded linear operator [Formula: see text] and its adjoint [Formula: see text]. We show that if [Formula: see text] is a partial isometry of locally Hilbert–Schmidt class, then we have a unique DPP [Formula: see text] associated with [Formula: see text]. In addition, if [Formula: see text] is also of locally Hilbert–Schmidt class, then we have a unique pair of DPPs, [Formula: see text], [Formula: see text]. We also give a practical framework which makes [Formula: see text] and [Formula: see text] satisfy the above conditions. Our framework to construct pairs of DPPs implies useful duality relations between DPPs making pairs. For a correlation kernel of a given DPP our formula can provide plural different expressions, which reveal different aspects of the DPP. In order to demonstrate these advantages of our framework as well as to show that the class of DPPs obtained by this method is large enough to study universal structures in a variety of DPPs, we report plenty of examples of DPPs in one-, two- and higher-dimensional spaces S, where several types of weak convergence from finite DPPs to infinite DPPs are given. One-parameter ([Formula: see text]) series of infinite DPPs on [Formula: see text] and [Formula: see text] are discussed, which we call the Euclidean and the Heisenberg families of DPPs, respectively, following the terminologies of Zelditch.

The probability hypothesis density (PHD) filter, which is used for multi-target tracking based on sensor measurements, relies on the propagation of the first-order moment, or intensity function, of a point process. This algorithm assumes that targets behave independently, an hypothesis which may not hold in practice due to potential target interactions. In this paper, we construct a second-order PHD filter based on determinantal point processes which are able to model repulsion between targets. Such processes are characterized by their first- and second-order moments, which allows the algorithm to propagate variance and covariance information in addition to first-order target count estimates. Our approach relies on posterior moment formulas for the estimation of a general hidden point process after a thinning operation and a superposition with a Poisson point process, and on suitable approximation formulas in the determinantal point process setting. The repulsive properties of determinantal point processes apply to the modeling of negative correlation between distinct measurement domains. Monte Carlo simulations with correlation estimates are provided.

Determinantal point processes (DPPs) have recently proved to be a useful class of models in several areas of statistics, including spatial statistics, statistical learning and telecommunications networks. They are models for repulsive (or regular, or inhibitive) point processes, in the sense that nearby points of the process tend to repel each other. We consider two ways to quantify the repulsiveness of a point process, both based on its second-order properties, and we address the question of how repulsive a stationary DPP can be. We determine the most repulsive stationary DPP, when the intensity is fixed, and for a given $R>0$ we investigate repulsiveness in the subclass of $R$-dependent stationary DPPs, that is, stationary DPPs with $R$-compactly supported kernels. Finally, in both the general case and the $R$-dependent case, we present some new parametric families of stationary DPPs that can cover a large range of DPPs, from the stationary Poisson process (the case of no interaction) to the most repulsive DPP.

With the explosive growth of video data, video summarization which converts long-time videos to key frame sequences has become an important task in information retrieval and machine learning. Determinantal point processes (DPPs) which are elegant probabilistic models have been successfully applied to video summarization. However, existing DPP-based video summarization methods suffer from poor efficiency of outputting a specified size summary or neglecting inherent sequential nature of videos. In this paper, we propose a new model in the DPP lineage named k-SDPP in vein of sequential determinantal point processes but with fixed user specified size k. Our k-SDPP partitions sampled frames of a video into segments where each segment is with constant number of video frames. Moreover, an efficient branch and bound method (BB) considering sequential nature of the frames is provided to optimally select k frames delegating the summary from the divided segments. Experimental results show that our proposed BB method outperforms not only k-DPP and sequential DPP (seqDPP) but also the partition and Markovian assumption based methods.

Goldman (2010) proved that the distribution of a stationary determinantal point process (DPP) Φ can be coupled with its reduced Palm version Φ0,! such that there exists a point process η where Φ=Φ0,!∪η in distribution and Φ0,!∩η=∅. The points of η characterize the repulsive nature of a typical point of Φ. In this paper we use the first-moment measure of η to study the repulsive behavior of DPPs in high dimensions. We show that many families of DPPs have the property that the total number of points in η converges in probability to 0 as the space dimension n→∞. We also prove that for some DPPs, there exists an R∗ such that the decay of the first-moment measure of η is slowest in a small annulus around the sphere of radius √nR∗. This R∗ can be interpreted as the asymptotic reach of repulsion of the DPP. Examples of classes of DPP models exhibiting this behavior are presented and an application to high-dimensional Boolean models is given.

On the correlation structure of some random point processes on the line

Determinantal point processes (DPPs) have garnered attention as an elegant probabilistic model of set diversity. They are useful for a number of subset selection tasks, including product recommendation. DPPs are parametrized by a positive semi-definite kernel matrix. In this work we present a new method for learning the DPP kernel from observed data using a low-rank factorization of this kernel. We show that this low-rank factorization enables a learning algorithm that is nearly an order of magnitude faster than previous approaches, while also providing for a method for computing product recommendation predictions that is far faster (up to 20x faster or more for large item catalogs) than previous techniques that involve a full-rank DPP kernel. Furthermore, we show that our method provides equivalent or sometimes better test log-likelihood than prior full-rank DPP approaches.

Imputing data is a critical issue for machine learning practitioners, including in the life sciences domain, where missing clinical data is a typical situation and the reliability of the imputation is of great importance. Currently, there is no canonical approach for imputation of clinical data and widely used algorithms introduce variance in the downstream classification. Here we propose novel imputation methods based on determinantal point processes (DPP) that enhance popular techniques such as the multivariate imputation by chained equations and MissForest. Their advantages are twofold: improving the quality of the imputed data demonstrated by increased accuracy of the downstream classification and providing deterministic and reliable imputations that remove the variance from the classification results. We experimentally demonstrate the advantages of our methods by performing extensive imputations on synthetic and real clinical data. We also perform quantum hardware experiments by applying the quantum circuits for DPP sampling since such quantum algorithms provide a computational advantage with respect to classical ones. We demonstrate competitive results with up to 10 qubits for small-scale imputation tasks on a state-of-the-art IBM quantum processor. Our classical and quantum methods improve the effectiveness and robustness of clinical data prediction modeling by providing better and more reliable data imputations. These improvements can add significant value in settings demanding high precision, such as in pharmaceutical drug trials where our approach can provide higher confidence in the predictions made.

We design fast algorithms for repeatedly sampling from strongly Rayleigh distributions, which include as special cases random spanning tree distributions and determinantal point processes. For a graph $G=(V,\ E)$, we show how to approximately sample uniformly random spanning trees from G in $O(|V|)$ <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> time per sample after an initial $O(|E|)$ time preprocessing. This is the first nearly-linear runtime in the output size, which is clearly optimal. For a determinantal point process on k-sized subsets of a ground set of n elements, defined via an $n\times n$ kernel matrix, we show how to approximately sample in ${\widetilde{O}}(k^{\omega})$ time after an initial ${\widetilde{O}}(nk^{\omega-1})$ time preprocessing, where $\omega\lt 2.372864$ is the matrix multiplication exponent. The time to compute just the weight of the output set is simply $\simeq k^{\omega}$, a natural barrier that suggests our runtime might be optimal for determinantal point processes as well. As a corollary, we even improve the state of the art for obtaining a single sample from a determinantal point process, from the prior runtime of ${\widetilde{O}}(\min\{nk^{2},\ n^{\omega}\})$ to ${\widetilde{O}}(nk^{\omega-1})$.In our main technical result, we achieve the optimal limit on domain sparsification for strongly Rayleigh distributions. In domain sparsification, sampling from a distribution $\mu$ on $\binom{[n]}{k}$ is reduced to sampling from related distributions on $\binom{[t]}{k}$ for $t\ll n$. We show that for strongly Rayleigh distributions, the domain size can be reduced to nearly linear in the output size $t={\widetilde{O}}(k)$, improving the state of the art from $t={\widetilde{O}}(k^{2})$ for general strongly Rayleigh distributions and the more specialized $t={\widetilde{O}}(k^{15})$ for sBanning tree distributions. Our reduction involves sampling from ${\widetilde{O}}(1)$ domain-sparsified distributions, all of which can be produced efficiently assuming approximate overestimates for marginals of $\mu$ are known and stored in a convenient data structure. Having access to marginals is the discrete analog of having access to the mean and covariance of a continuous distribution, or equivalently knowing “isotropy” for the distribution, the key behind optimal samplers in the continuous setting based on the famous Kannan-Lovász-Simonovits (KLS) conjecture. We view our result as analogous in spirit to the KLS conjecture and its consequences for sampling, but rather for discrete strongly Rayleigh measures. <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> Throughout, ${\widetilde{O}}(\cdot)$ hides polylogarithmic factors in n.

While the predictive performance of modern statistical dependency parsers relies heavily on the availability of expensive expert-annotated treebank data, not all annotations contribute equally to the training of the parsers. In this paper, we attempt to reduce the number of labeled examples needed to train a strong dependency parser using batch active learning (AL). In particular, we investigate whether enforcing diversity in the sampled batches, using determinantal point processes (DPPs), can improve over their diversity-agnostic counterparts. Simulation experiments on an English newswire corpus show that selecting diverse batches with DPPs is superior to strong selection strategies that do not enforce batch diversity, especially during the initial stages of the learning process. Additionally, our diversity-aware strategy is robust under a corpus duplication setting, where diversity-agnostic sampling strategies exhibit significant degradation.

The ability to convey relevant and diverse information is critical in multi-document summarization and yet remains elusive for neural seq-to-seq models whose outputs are often redundant and fail to correctly cover important details. In this work, we propose an attention mechanism which encourages greater focus on relevance and diversity. Attention weights are computed based on (proportional) probabilities given by Determinantal Point Processes (DPPs) defined on the set of content units to be summarized. DPPs have been successfully used in extractive summarisation, here we use them to select relevant and diverse content for neural abstractive summarisation. We integrate DPP-based attention with various seq-to-seq architectures ranging from CNNs to LSTMs, and Transformers. Experimental evaluation shows that our attention mechanism consistently improves summarization and delivers performance comparable with the state-of-the-art on the MultiNews dataset

In community-based question answering (CQA) platforms, it takes time for a user to get useful information from among many answers. Although one solution is an answer ranking method, the user still needs to read through the top-ranked answers carefully. This paper proposes a new task of selecting a diverse and non-redundant answer set rather than ranking the answers. Our method is based on determinantal point processes (DPPs), and it calculates the answer importance and similarity between answers by using BERT. We built a dataset focusing on a Japanese CQA site, and the experiments on this dataset demonstrated that the proposed method outperformed several baseline methods.

Reproducing kernels: Harmonic analysis and some of their applications

Submodular and supermodular functions have found wide applicability in machine learning, capturing notions such as diversity and regularity, respectively. These notions have deep consequences for optimization, and the problem of (approximately) optimizing submodular functions has received much attention. However, beyond optimization, these notions allow specifying expressive probabilistic models that can be used to quantify predictive uncertainty via marginal inference. Prominent, well-studied special cases include Ising models and determinantal point processes, but the general class of log-submodular and log-supermodular models is much richer and little studied. In this paper, we investigate the use of Markov chain Monte Carlo sampling to perform approximate inference in general log-submodular and log-supermodular models. In particular, we consider a simple Gibbs sampling procedure, and establish two sufficient conditions, the first guaranteeing polynomial-time, and the second fast (O(n log n)) mixing. We also evaluate the efficiency of the Gibbs sampler on three examples of such models, and compare against a recently proposed variational approach.

In this dissertation, new results on stochastic geometric models in high dimensional space are presented. We first concentrate on a particular class of repulsive point processes called determinantal point processes (DPPs). We establish a coupling of a DPP and its reduced Palm version showing the repulsive effect of a point of the point process. This is used for discussing the degree of repulsiveness in DPPs, including Ginibre point processes and other specific parametric models for DPPs. We then study this repulsion for stationary DPPs in high dimensional Euclidean space. It is shown that for many families of DPPs, a typical point has no repulsive effect with high probability for large space dimension n. It is also proved that for some DPPs there exists an R* such that the repulsive effect occurs at a distance of [square root] nR* with high probability for large n. This R* is interpreted as the asymptotic reach of repulsion of the DPP. Examples of DPPs exhibiting this behavior are presented and an application to high dimensional Boolean models is given. The second half of this dissertation examines zero cells of stationary Poisson tessellations. First, a stationary stochastic geometric model is proposed for analyzing one-bit data compression. The data is assumed to be an unconstrained stationary set, and each data point is compressed using one bit with respect to each hyperplane in a stationary and isotropic Poisson hyperplane tessellation. Size metrics of the zero cell of the tessellation are studied to determine how the intensity of hyperplanes must scale with dimension to ensure sufficient separation of different data by the hyperplanes or sufficient proximity of the data compressed together. The results have direct implications in compressive sensing and source coding. We then study the concentration of the norm of a random vector Y uniformly sampled in the centered zero cell of a stationary random tessellation in high dimensions. It is shown that for a stationary and isotropic Poisson-Voronoi tessellation, [mathematical equation] approaches one as the dimension approaches infinity. For a stationary and isotropic Poisson hyperplane tessellation, we prove that [mathematical equation] will be within a fixed range (R [subscript ℓ], R [subscript u]) with probability approaching one as dimension n tends to infinity.