Uncertainty propagation using infinite mixture of Gaussian processes and variational Bayesian inference

Global sensitivity analysis: A Bayesian learning based polynomial chaos approach

Non-parametric system identification with Gaussian Processes for underwater vehicles is explored in this research with the purpose of modelling autonomous underwater vehicle (AUV) dynamics with low amount of data. Multi-output Gaussian processes and its aptitude to model the dynamic system of an underactuated AUV without losing the relationships between tied outputs is used. The simulation of a first-principles model of a Remus 100 AUV is employed to capture data for the training and validation of the multi-output Gaussian processes. The metric and required procedure to carry out multi-output Gaussian processes for AUV with 6 degrees of freedom (DoF) is also shown in this paper. Multi-output Gaussian processes are compared with the popular technique of recurrent neural network show that Multi-output Gaussian processes manage to surpass RNN for non-parametric dynamic system identification in underwater vehicles with highly coupled DoF with the added benefit of providing a measurement of confidence.

This dissertation focuses on the development of multiple imputation models and algorithms for high-dimensional data with variable selection structures. Leveraging on the multivariate linear mixed-effects model with missing responses for clustered data, we incorporate the variable selection routines using spike-and-slab priors within the Bayesian variable selection framework. Specific choice of these priors allow us to "force'' variables of importance (e.g. design variables or variables known to play role in missingness mechanism) into the imputation models. Our ultimate goal is to improve computational speed by removing unnecessary variables. Markov chain Monte Carlo techniques have been designed to sample from the implied posterior distributions for model unknowns as well as missing data. A computationally efficient alternative, namely, variational inference algorithms have also been developed to overcome the computational burden of Markov chain Monte Carlo for sampling the posterior predictive distribution missing data. The continuous imputation model and algorithm can be easily modified through a calibration-based rounding strategy for multiple imputation of categorical variables. We also compare two imputation strategies, i.e., the sequential imputation and the joint modeling imputation, has been made in the presence of variable selection structures. We design a joint modeling based imputation framework using a nonparametric Bayesian Dirichlet process mixture model together with a sparse precision matrix model. Two computational algorithms have been carefully designed: the Gibbs sampler and the variational Bayesian inference algorithm. The proposed methodology can be applied to both continuous and mixed continuous and categorical missing data.

Compressive sensing is a sub-Nyquist sampling technique for efficient signal acquisition and reconstruction of sparse or compressible signals. In order to account for the sparsity of the underlying signal of interest, it is common to use sparsifying priors such as Bernoulli–Gaussian-inverse Gamma (BGiG) and Gaussian-inverse Gamma (GiG) priors on the components of the signal. With the introduction of variational Bayesian inference, the sparse Bayesian learning (SBL) methods for solving the inverse problem of compressive sensing have received significant interest as the SBL methods become more efficient in terms of execution time. In this paper, we consider the sparse signal recovery problem using compressive sensing and the variational Bayesian (VB) inference framework. More specifically, we consider two widely used Bayesian models of BGiG and GiG for modeling the underlying sparse signal for this problem. Although these two models have been widely used for sparse recovery problems under various signal structures, the question of which model can outperform the other for sparse signal recovery under no specific structure has yet to be fully addressed under the VB inference setting. Here, we study these two models specifically under VB inference in detail, provide some motivating examples regarding the issues in signal reconstruction that may occur under each model, perform comparisons and provide suggestions on how to improve the performance of each model.

Existing studies on sparse Bayesian learning for structural damage detection usually assume that the posterior probability density functions follow standard distributions which facilitate to circumvent the intractable integration problem of the evidence by means of numerical sampling or analytical derivation. Moreover, the uncertainties of each mode are usually quantified as a common parameter to simplify the calculation. These assumptions may not be realistic in practice. This study proposes a sparse Bayesian method for structural damage detection suitable for standard and nonstandard probability distributions. The uncertainty corresponding to each mode is assumed as different. Variational Bayesian inference is developed and the posterior probability density functions of each unknown are individually derived. The parameters are found to follow the gamma distribution, whereas the distribution of the damage index cannot be directly obtained because of the nonlinear relationship in its posterior probability density function. The delayed rejection adaptive Metropolis algorithm is then adopted to generate numerical samples of the damage index. The coupled damage index and parameters in the variational Bayesian inference are successively calculated via an iterative process. A laboratory-tested frame is utilised to verify the effectiveness of the proposed method. The results indicate that the sparse damage can be accurately detected. The proposed method has the advantage of high accuracy and broad applicability.

This paper presents a novel extension of multi-task Gaussian Cox processes for modeling multiple heterogeneous correlated tasks jointly, e.g., classification and regression, via multi-output Gaussian processes (MOGP). A MOGP prior over the parameters of the dedicated likelihoods for classification, regression and point process tasks can facilitate sharing of information between heterogeneous tasks, while allowing for nonparametric parameter estimation. To circumvent the non-conjugate Bayesian inference in the MOGP modulated heterogeneous multi-task framework, we employ the data augmentation technique and derive a mean-field approximation to realize closed-form iterative updates for estimating model parameters. We demonstrate the performance and inference on both 1D synthetic data as well as 2D urban data of Vancouver.

Multi-output Gaussian process (MGP) has been attracting increasing attention as a transfer learning method to model multiple outputs. Despite its high flexibility and generality, MGP still faces two critical challenges when applied to transfer learning. The first one is negative transfer, which occurs when there exists no shared information among the outputs. The second challenge is the input domain inconsistency, which is commonly studied in transfer learning yet not explored in MGP. In this paper, we propose a regularized MGP modeling framework with domain adaptation to overcome these challenges. More specifically, a sparse covariance matrix of MGP is constructed using convolution process, where penalization terms are added to adaptively select the most informative outputs for knowledge transfer. To deal with the domain inconsistency, a domain adaptation method is proposed by marginalizing inconsistent features and expanding missing features to align the input domains among different outputs. Statistical properties of the proposed method are provided to guarantee the performance practically and asymptotically. The proposed framework outperforms state-of-the-art benchmarks in comprehensive simulation studies and one real case study of a ceramic manufacturing process. The results demonstrate the effectiveness of our method in dealing with both the negative transfer and the domain inconsistency.

Multi-subject electroencephalography (EEG) classification involves the categorization of brain waves measured from multiple subjects, each of whom undergoes the same mental task. Common spatial patterns (CSP) or probabilistic CSP (PCSP) are widely used for extracting discriminative features from EEG, although they are trained on a subject-by-subject basis and inter-subject information is neglected. Moreover, the performance is degraded when only a few training samples are available for each subject. In this paper, we present a method for Bayesian CSP with Dirichlet process (DP) priors, where spatial patterns (corresponding to basis vectors) are simultaneously learned and clustered across subjects using variational Bayesian inference, which facilitates a flexible mixture model where the number of components are also learned. Spatial patterns in the same cluster share the hyperparameters of their prior distributions, which means information transfer is facilitated among subjects with similar spatial patterns. Numerical experiments using the BCI competition IV 2a dataset demonstrated the high performance of our method, compared with existing PCSP and Bayesian CSP methods with a single prior distribution.

To deal with the estimation problem of non-stationary channel in massive Multiple-Input Multiple-Output (MIMO) up-link, the 2D channels' sparse structure information in temporal-spatial domain is used, to design an iterative channel estimation algorithm based on Dirichlet Process (DP) and Variational Bayesian Inference (VBI), which can improve the accuracy under a lower pilot overhead and computation complexity. On account of that the stationary channel models is not suitable for massive MIMO systems anymore, a non-stationary channel prior model utilizing Dirichlet Process is constructed, which can map the physical spatial correlation channels to a probabilistic channel with the same sparse temporal vector. By applying VBI technology, a channel estimation iteration algorithm with low pilot overhead and complexity is designed. Experiment results show the proposed channel method has a better performance on the estimation accuracy than the state-of-art method, meanwhile it works robustly against the dynamic system key parameters.<br/><br/>

Supporting multiuser communication and localization is a typical scenario in Integrated sensing and communications (ISAC). However, the problem of multi-echo induced by multipath and multiuser makes it hard to determine the relationship between user equipments (UEs) and these echoes. Thus, applying traditional estimation algorithms at the radar receiver inevitably leads to weak communication and localization performances due to the mismatch between echoes and UEs. In this paper, aiming to achieve multiuser association and localization under doubly dispersive multipath channels, we construct an ISAC unified waveform based on the orthogonal delay-Doppler division multiplexing (ODDM) principle and develop an off-grid cluster sparse Bayesian learning estimation (OG-CSBL) algorithm. Particularly, we focus on the mono-static setup, where the base station (BS) expects to communicate with multiuser while sensing their locations. We utilize the high-resolution range profile (HRRP) to characterize the physical features of UEs and establish associations with their echoes by exploiting the inherent cluster structure. To estimate parameters, we design a hybrid Dirichlet process (DP)-Gaussian hierarchical prior distribution and propose a variational Bayesian inference (VBI)-EM strategy. Additionally, we develop a backtrack echo identification scheme to facilitate precise UE localization. Simulation results demonstrate that the proposed scheme achieves superior NMSE performance, offers meter-level localization accuracy, and obtains better BER performance in the complex multiuser coexistence scenario.

In this letter, we propose a novel ship detection method in synthetic aperture radar (SAR) imagery via variational Bayesian inference. First, we establish the ship detection probabilistic model which decomposes the SAR image as the sum of a sparse component associated with ships and a sea clutter component. Then, we introduce hierarchical priors of the latent variables in the model and use variational Bayesian inference to estimate the posterior distributions of the latent variables. The proposed method is an automatic iterative process without any sliding window. Experimental results accomplished over synthetic data and a RADARSAT-2 SAR image demonstrate that the proposed method can achieve state-of-the-art ship detection performance.

Many human genes express multiple transcript isoforms through alternative splicing, which greatly increases diversity of protein function. Although RNA sequencing (RNA-Seq) technologies have been widely used in measuring amounts of transcribed mRNA, accurate estimation of transcript isoform abundances from RNA-Seq data is challenging because reads often map to more than one transcript isoforms or paralogs whose sequences are similar to each other. We propose a statistical method to estimate transcript isoform abundances from RNA-Seq data. Our method can handle gapped alignments of reads against reference sequences so that it allows insertion or deletion errors within reads. The proposed method optimizes the number of transcript isoforms by variational Bayesian inference through an iterative procedure, and its convergence is guaranteed under a stopping criterion. On simulated datasets, our method outperformed the comparable quantification methods in inferring transcript isoform abundances, and at the same time its rate of convergence was faster than that of the expectation maximization algorithm. We also applied our method to RNA-Seq data of human cell line samples, and showed that our prediction result was more consistent among technical replicates than those of other methods. An implementation of our method is available at http://github.com/nariai/tigar nariai@megabank.tohoku.ac.jp Supplementary data are available at Bioinformatics online.

This letter addresses the recovery of hierarchical sparse signals in a Bayesian framework. Hierarchical sparse signals exhibit two levels of sparsity, i.e., block-sparsity among different blocks and internal sparsity within each individual block. As in sparse Bayesian learning, each component of the coefficient vector is firstly modeled as a Gaussian distributed variable with zero mean. To enforce the two-level hierarchical sparsity, the variance is further modeled by two classes of hidden variables controlling the block-sparsity and the internal sparsity, respectively. Finally, variational Bayesian inference is used to recover the coefficient vector from the noise corrupted data. Numerical simulation and experimental results show that the proposed method outperforms those recently reported recovery methods.

We propose two generic methods for improving semi-supervised learning (SSL). The first integrates weight perturbation (WP) into existing “consistency regularization” (CR) based methods. We implement WP by leveraging variational Bayesian inference (VBI). The second method proposes a novel consistency loss called “maximum uncertainty regularization” (MUR). While most consistency losses act on perturbations in the vicinity of each data point, MUR actively searches for “virtual” points situated beyond this region that cause the most uncertain class predictions. This allows MUR to impose smoothness on a wider area in the input-output manifold. Our experiments show clear improvements in classification errors of various CR based methods when they are combined with VBI or MUR or both.

A low-complexity algorithm based on variational Bayesian inference for MIMO channel estimation