A Gaussian Latent Variable Model for Incomplete Mixed Type Data

Purpose. The goal of this paper is to build a simulation environment that allows for the prediction of patient-specific tissue response by drawing samples from a generative model with a probability distribution. We propose a Gaussian Process (GP) regression approach to learn distributions over strain energy density functions including elastography, linear and hyperelastic models reported in the literature. Methods. We gather a total of 73 models characterising elastic properties of brain white matter, grey matter and abnormalities and express them as strain energy density functions. A multi-output GP is used to quantify means and confidence intervals across each anatomical region and model. We sample the GP distribution and use nonlinear optimisation to fit a Neo-Hookean meta-model to guarantee stable strain energy functions. We validate the Neo-Hookean meta-model by fitting known strain energy density functions from the literature and report optimisation cost. We also validate the ability of the GP to approximate elastic properties of tissue given a reference deformed state using simulation. Results. The GP was able to capture confidence intervals of varying strain ranges; the GP parameters and optimisation costs indicated a higher variability of hyperelastic models compared to elastography and linear models. Although one term is insufficient to fully capture hyperelastic models with higher number of terms, the resulting meta model is stable for real-time simulation within a wider range of stretches captured during mechanical characterisation of soft tissue. We demonstrated that our approach was able to approximate known elastic properties of tissue with a root-mean-squared error of 0.6 mm of node displacements when drawing six samples from a distribution of hyperelastic white matter. Conclusion. In this initial proof-of-concept, we demonstrated a GP-based approach to estimate the elastic behaviour of brain tissue through simulation by sampling a generative model comprising elastic models found in the literature.

MotivationThe negative binomial distribution has been shown to be a good model for counts data from both bulk and single-cell RNA-sequencing (RNA-seq). Gaussian process (GP) regression provides a useful non-parametric approach for modelling temporal or spatial changes in gene expression. However, currently available GP regression methods that implement negative binomial likelihood models do not scale to the increasingly large datasets being produced by single-cell and spatial transcriptomics.ResultsThe GPcounts package implements GP regression methods for modelling counts data using a negative binomial likelihood function. Computational efficiency is achieved through the use of variational Bayesian inference. The GP function models changes in the mean of the negative binomial likelihood through a logarithmic link function and the dispersion parameter is fitted by maximum likelihood. We validate the method on simulated time course data, showing better performance to identify changes in over-dispersed counts data than methods based on Gaussian or Poisson likelihoods. To demonstrate temporal inference, we apply GPcounts to single-cell RNA-seq datasets after pseudotime and branching inference. To demonstrate spatial inference, we apply GPcounts to data from the mouse olfactory bulb to identify spatially variable genes and compare to two published GP methods. We also provide the option of modelling additional dropout using a zero-inflated negative binomial. Our results show that GPcounts can be used to model temporal and spatial counts data in cases where simpler Gaussian and Poisson likelihoods are unrealistic.Availability and implementationGPcounts is implemented using the GPflow library in Python and is available at https://github.com/ManchesterBioinference/GPcounts along with the data, code and notebooks required to reproduce the results presented here. The version used for this paper is archived at https://doi.org/10.5281/zenodo.5027066.Supplementary informationSupplementary data are available at Bioinformatics online.

Gaussian processes (GPs) are a popular class of Bayesian nonparametric models, but its training can be computationally burdensome for massive training datasets. While there has been notable work on scaling up these models for big data, existing methods typically rely on a stationary GP assumption for approximation, and can thus perform poorly when the underlying response surface is nonstationary, that is, it has some regions of rapid change and other regions with little change. Such non-stationarity is, however, ubiquitous in real-world problems, including our motivating application for surrogate modeling of computer experiments. We propose a new Product of Sparse GP (ProSpar-GP) method for scalable GP modeling with massive nonstationary data. The ProSpar-GP makes use of a carefully-constructed product-of-experts formulation of sparse GP experts, where different experts are placed within local regions of non-stationarity. These GP experts are fit via a novel variational inference approach, which capitalizes on mini-batching and GPU acceleration for efficient optimization of inducing points and length-scale parameters for each expert. We further show that the ProSpar-GP is Kolmogorov-consistent, in that its generative distribution defines a valid stochastic process over the prediction space; such a property provides essential stability for variational inference, particularly in the presence of non-stationarity. We then demonstrate the improved performance of the ProSpar-GP over the state-of-the-art, in a suite of numerical experiments and an application for surrogate modeling of a satellite drag simulator. Supplemental materials for this article are available online.

Robots had a great impact on the manufacturing industry ever since the early seventies when companies such as KUKA and ABB started deploying their first industrial robots. These robots merely performed very specific tasks in specific ways within well-defined environments. Still, they proved to be very useful as they could exceed human performance at these tasks. However, in order to enable robots to enter our daily life, they need to become more versatile and need to operate in much less structured environments. This thesis is partly devoted to stretching these limitations by means of learning, namely imitation learning (IL) and inverse reinforcement learning (IRL). Reinforcement learning (RL) is a powerful approach to enable robots to solve a task in an unknown environment. The practitioner describes a desired behavior by specifying a reward function and the robot autonomously interacts with the environment in order to find a control policy that generates high accumulated reward. However, RL is not suitable for teaching new tasks by non-experts because specifying appropriate reward functions can be difficult. Demonstrating the desired behavior is often easier for non-experts. Imitation learning can be used in order to enable the robot to reproduce the demonstrations. However, without explicitly inferring and modeling the intentions of the demonstrations, it can become difficult to solve the task for unseen situations. Inverse reinforcement learning (IRL) therefore aims to infer a reward function from the demonstrations, such that optimizing this reward function yields the desired behavior even for different situations. This thesis introduces a unifying approach to solve the inverse reinforcement learning problem in the same way as the reinforcement learning problem. This is achieved by framing both problems as information projection problems, i.e., we strive to minimize the relative entropy between a probabilistic model of the robot behavior and a given desired distribution. Furthermore, a trust region on the robot behavior is used to stabilize the optimization. For inverse reinforcement learning, the desired distribution is implicitly given by the expert demonstrations. The resulting optimization can be efficiently solved using state-of-the-art reinforcement learning methods. For reinforcement learning, the log-likelihood of the desired distribution is given by the reward function. The resulting optimization problem corresponds to a standard reinforcement learning formulation, except for an additional objective of maximizing the entropy of the robot behavior. This entropy objective adds little overhead to the optimization, but can lead to better exploration and more diversified policies. Trust-region I-projections are not only useful for training robots, but can also be applied to other machine learning problems. I-projections are typically used for variational inference, in order to approximate an intractable distribution by a simpler model. However, the resulting optimization problems are usually optimized based on stochastic gradient descent which often suffers from high variance in the gradient estimates. As trust-region I-projections where shown to be effective for reinforcement learning and inverse reinforcement learning, this thesis also explores their use for variational inference. More specifically, trust-region I-projections are investigated for the problem of approximating an intractable distribution by a Gaussian mixture model (GMM) with an adaptive number of components. GMMs are highly desirable for variational inference because they can yield arbitrary accurate approximations while inference from GMMs is still relatively cheap. In order to make learning the GMM feasible, we derive a lower bound that enables us to decompose the objective function. The optimization can then be performed by iteratively updating individual components using a technique from reinforcement learning. The resulting method is capable of learning approximations of significantly higher quality than existing variational inference methods. Due to the similarity of the underlying optimization problems, the insights gained from our variational inference method are also useful for IL and IRL. Namely, a similar lower bound can be applied also for the I-projection formulation of imitation learning. However, whereas for variational inference the lower bound serves to decompose the objective function, for imitation learning it allows us to provide a reward signal to the robot that does not depend on its behavior. Compared to reward functions that are relative to the current behavior of the robot---which are typical for popular adversarial methods---behavior-independent reward functions have the advantages that we can show convergence even for greedy optimization. Furthermore, behavior-independent reward functions solve the inverse reinforcement learning problem, thereby closing the gap between imitation learning and IRL. However, algorithms derived from our non-adversarial formulation are actually very similar to existing AIL methods, and we can even show that adversarial inverse reinforcement learning (AIRL) is indeed an instance of our formulation. AIRL was derived from an adversarial formulation, and we point out several problems of that derivation. In contrast, we show that AIRL can be straightforwardly derived from out non-adversarial formulation. Furthermore, we demonstrate that the non-adversarial formulation can be also used to derive novel algorithms by presenting a non-adversarial method for offline imitation learning.

Some Thoughts About Data Type, Distribution, and Statistical Significance

Hidden information derived from probabilistic generative models of data distributions can be used to construct features for discriminative classifiers. This observation has motivated the development of approaches that attempt to couple generative and discriminative models together for classification. However, existing approaches typically feed features derived from generative models to discriminative classifiers, and do not refine the generative models or the feature mapping functions based on classification results. In this paper, we propose a coupling mechanism developed under the PAC-Bayes framework that can fine-tune the generative models and the feature mapping functions iteratively to improve the classifier's performance. In our approach, a stochastic feature mapping, which is a function over the random variables of a generative model, is derived to generate feature vectors for a stochastic classifier. We construct a stochastic classifier over the feature mapping and derive the PAC-Bayes generalization bound for the classifier, for both supervised and semi-supervised learning. This allows us to jointly learn the feature mapping and the classifier by minimizing the bound with an EM-like iterative algorithm using labeled and unlabeled data. The resulting framework integrates the learning of the discriminative classifier and the generative model and allows iterative fine-tuning of the generative models, and the feedforward feature mappings based on task performance feedback. Our experiments show, in three distinct applications, this new framework produces a general classification tool with state-of-the-art performance.

Variational inference has become one of the most widely used methods in latent variable modeling. In its basic form, variational inference employs a fully factorized variational distribution and minimizes its Kullback–Leibler divergence to the posterior. As the minimization can only be carried out approximately, this approximation induces a bias. In this paper, we revisit perturbation theory as a powerful way of improving the variational approximation. Perturbation theory relies on a form of Taylor expansion of the log marginal likelihood, vaguely in terms of the log ratio of the true posterior and its variational approximation. While first order terms give the classical variational bound, higher-order terms yield corrections that tighten it. However, traditional perturbation theory does not provide a lower bound, making it inapt for stochastic optimization. In this paper, we derive a similar yet alternative way of deriving corrections to the evidence lower bound that resemble perturbation theory, but that result in a valid bound. We show in experiments on Gaussian processes and variational autoencoders that the new bounds are more mass covering, and that the resulting posterior covariances are closer to the true posterior and lead to higher likelihoods on held-out data.

Gaussian Process (GP) offers a principled non-parametric framework for learning stochastic functions. The generalization capability of GPs depends heavily on the kernel function, which implicitly imposes the smoothness assumptions of the data. However, common feature-based kernel functions are inefficient to model the relational data, where the smoothness assumptions implied by the kernels are violated. To model the complex and non-differentiable functions over relational data, we propose a novel Graph Convolutional Kernel, which enables to incorporate relational structures to feature-based kernels to capture the statistical structure of data. To validate the effectiveness of proposed kernel function in modeling relational data, we introduce GP models with Graph Convolutional Kernel in two relational learning settings, i.e., unsupervised settings of link prediction and semi-supervised settings of object classification. The parameters of our GP models are optimized through the scalable variational inducing point method. However, the highly structured likelihood objective requires densely sampling from variational distributions, which is costly and makes its optimization challenging in the unsupervised settings. To tackle this challenge, we propose a Local Neighbor Sampling technique with a provable more efficient computational complexity. Experimental results on real-world datasets demonstrate that our model achieves state-of-the-art performance in two relational learning tasks.

Active inference provides a general framework for behavior and learning in autonomous agents. It states that an agent will attempt to minimize its variational free energy, defined in terms of beliefs over observations, internal states and policies. Traditionally, every aspect of a discrete active inference model must be specified by hand, i.e. by manually defining the hidden state space structure, as well as the required distributions such as likelihood and transition probabilities. Recently, efforts have been made to learn state space representations automatically from observations using deep neural networks. In this paper, we present a novel approach of learning state spaces using quantum physics-inspired tensor networks. The ability of tensor networks to represent the probabilistic nature of quantum states as well as to reduce large state spaces makes tensor networks a natural candidate for active inference. We show how tensor networks can be used as a generative model for sequential data. Furthermore, we show how one can obtain beliefs from such a generative model and how an active inference agent can use these to compute the expected free energy. Finally, we demonstrate our method on the classic T-maze environment.

Uncertainty propagation using infinite mixture of Gaussian processes and variational Bayesian inference

We investigate how well a variational Bayes (VB) method performs as an approximation for the demanding computations typically required in Gaussian process regression. We study a technique using so-called emph{inducing variables} to drastically reduce the computational load. The key question is: can we make equally good estimates with less computational effort? We show that this is indeed possible, provided that one carefully chooses emph{how many} and emph{which} inducing variables. If too few are chosen, the resulting model becomes too `smooth', leading to inaccurate estimates and an overestimation of uncertainty. However, with a sufficient number of well-chosen inducing variables, the VB method can be just as accurate as the original, much slower approach. Faster computation with retained accuracy is feasible, as long as the number of inducing variables is tailored to the complexity of the true underlying function. Besides providing a point estimate, we also want to know how certain we can be about it. A key advantage of the Bayesian approach is that it allows for emph{uncertainty quantification}. We analyse VB methods based on spectral features and examine how well these methods capture uncertainty. Surprisingly, these methods can yield reliable uncertainty estimates under certain conditions, which is not always true for other VB approaches. Thus, this method can offer not only accurate predictions but also trustworthy uncertainty estimates, provided the priors are well-matched to the underlying data. Finally, we address a practical challenge: how do you choose the right settings (hyperparameters) for a model when you don’t know how complex the real world is? We consider an emph{adaptive} VB method that adjusts itself to the complexity of the data without requiring prior knowledge. We combine a hierarchical Bayesian model with a variational approach, and demonstrate both theoretically and through experiments that this works well. The result is a fast method that automatically adapts to various types of underlying functions while still achieving optimal performance. Self-tuning VB methods are therefore possible and deliver strong results without requiring prior knowledge of the complexity of the data-generating mechanism.

Inversion of hierarchical Bayesian models using Gaussian processes

Many real-life processes can be described as simplified categorical (or discrete) data models, switching between a finite number of states or regimes. Two main computational problems can be considered in the context of their analysis: (a) dimension reduction (i.e., identification of the essential (discrete) degrees of freedom or categories), and (b) parameterization of reduced dynamical models (e.g., Markov chains or Bernoulli processes) that can be used for analysis and prediction. Existing analysis methods, addressing the above issues (a) and (b), are limited with respect to possibilities for analyzing the influences of exogenous factors and have an implicit assumption of the stationarity built in. A general framework for analysis and online prediction of categorical data dynamics is presented. In contrast to standard approaches of categorical data analysis (like generalized linear discrete models, e.g., logit and probit regression methods) that are based on the transformation of the discrete categorical data into continuous representation, the presented methods allow us to build the (auto) regressive models of the data and to find the reduced data representation directly in the discrete setting. This general framework is based on an extension of the principal component analysis method to structure-preserving dimension reduction of (nonstationary) discrete jump processes, combined with a data-based estimation of nonhomogeneous Markov chain models under the influence of external factors in the reduced representation. Efficiently parallelizable numerical methods for the solution of the resulting mixed discrete-continuous optimization problems are described, applicable to the analysis of systems with a large number of discrete categories. Described methods have a favorable scaling with the dimension (expressed via a number of discrete states). General applicability is illustrated on the generic toy model example and applied to two problems from computational sociology: (i) estimation of the time-dependent transition graphs describing the dynamics of political preferences of the mean German voter, and (ii) parameterization of the discrete jump process describing the transition between the employed and unemployed states of an average German individual. The resulting online predictions are compared to the ones obtained by standard methods of the time series analysis, and the influence of implicit statistical assumptions is discussed.

This chapter begins with a review of variational inference (VI) as a fast approximation alternative to Markov Chain Monte Carlo (MCMC) methods, solving an optimization problem for approximating the posterior. VI is scaled to stochastic variational inference and generalized to black-box variational inference (BBVI). Amortized VI leads to the variational auto-encoder (VAE) framework which is introduced using deep neural networks and graphical models and used for learning representations and generative modeling. Finally, we explore generative flows, the latent space manifold, and Riemannian geometry of generative models.

The relationship between flight performance and multi-physiological parameters under different flight operating patterns is unknown. This work proposes a Stacked Gaussian Process Network (SGPN) to reveal it. SGPN is a multi-layer network model formed by recursion from a regular Gaussian process and random disturbance. This work constructs an auxiliary variable strategy with the induced points to improve its learning efficiency, thus leading to a sparse SGPN model. In it, a Gaussian process acts as an activation function of each node, but the entire model is no longer a Gaussian process and thus very challenging to solve it. This work presents its solution via variational approximate inference. Experimental results of pilot flight performance evaluation show that the proposed model has stronger learning and generalization ability than its seven competitive peers. It is able to approximate non-linear coupling relationship between multi-physiological parameters and flight height differences.