• Home
  • Search
  • Conditional Deep Gaussian Processes: Empirical Bayes Hyperdata Learning.
  • Cite Icon4
  • https://doi.org/10.3390/e23111387Copy DOI Icon

Conditional Deep Gaussian Processes: Empirical Bayes Hyperdata Learning.

  • Oct 23, 2021
  • Entropy
  • Chi-Ken Lu +1 more
Show More
  • Abstract
  • Highlights & Summary
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

It is desirable to combine the expressive power of deep learning with Gaussian Process (GP) in one expressive Bayesian learning model. Deep kernel learning showed success as a deep network used for feature extraction. Then, a GP was used as the function model. Recently, it was suggested that, albeit training with marginal likelihood, the deterministic nature of a feature extractor might lead to overfitting, and replacement with a Bayesian network seemed to cure it. Here, we propose the conditional deep Gaussian process (DGP) in which the intermediate GPs in hierarchical composition are supported by the hyperdata and the exposed GP remains zero mean. Motivated by the inducing points in sparse GP, the hyperdata also play the role of function supports, but are hyperparameters rather than random variables. It follows our previous moment matching approach to approximate the marginal prior for conditional DGP with a GP carrying an effective kernel. Thus, as in empirical Bayes, the hyperdata are learned by optimizing the approximate marginal likelihood which implicitly depends on the hyperdata via the kernel. We show the equivalence with the deep kernel learning in the limit of dense hyperdata in latent space. However, the conditional DGP and the corresponding approximate inference enjoy the benefit of being more Bayesian than deep kernel learning. Preliminary extrapolation results demonstrate expressive power from the depth of hierarchy by exploiting the exact covariance and hyperdata learning, in comparison with GP kernel composition, DGP variational inference and deep kernel learning. We also address the non-Gaussian aspect of our model as well as way of upgrading to a full Bayes inference.

Similar Papers
  • PDF
  • Research Article
  • Citations34

Deep Bayesian Gaussian processes for uncertainty estimation in electronic health records

  • Oct 19, 2021
  • Scientific Reports
  • Yikuan Li +8
  • Research Article
  • Citations8

Deep Latent-Variable Kernel Learning

  • Oct 01, 2022
  • IEEE Transactions on Cybernetics
  • Haitao Liu +3
  • Research Article
  • Citations3

Probabilistic learning for pulsar classification

  • Oct 01, 2022
  • Journal of Cosmology and Astroparticle Physics
  • Sambatra Andrianomena
  • PDF
  • Research Article
  • Citations27

Resonant Frequency Modeling of Microstrip Antenna Based on Deep Kernel Learning

  • Jan 01, 2021
  • IEEE Access
  • Shudan Han +3
  • Research Article
  • Citations69

Bridging deep and multiple kernel learning: A review

  • Oct 07, 2020
  • Information Fusion
  • Tinghua Wang +2
  • Video Transcripts

The Promises and Pitfalls of Deep Kernel Learning

  • Jul 17, 2021
  • Underline Science Inc.
  • Carl Edward Rasmussen +2
  • Research Article
  • Citations29

Gaussian processes for missing value imputation

  • Apr 29, 2023
  • Knowledge-Based Systems
  • Bahram Jafrasteh +3
  • Book Chapter
  • Citations2

Deep Kernel Learning for Mortality Prediction in the Face of Temporal Shift

  • Jan 01, 2021
  • Miguel Rios +1
  • PDF
  • Research Article
  • Citations16

Deep kernel learning approach to engine emissions modeling

  • Jan 01, 2020
  • Data-Centric Engineering
  • Changmin Yu +8
  • Conference Article
  • Citations1

Automatically Designed Deep Gaussian Process for Turbomachinery Application

  • Jun 07, 2021
  • Yuan Jin +2
  • Research Article
  • Citations8

Uncertainty Analysis of Deep Kernel Learning Methods on Diabetic Retinopathy Grading

  • Jan 01, 2023
  • IEEE Access
  • Marlin Siebert +2
  • Research Article
  • Citations7

Deep Kernel Learning-Based Bayesian Optimization with Adaptive Kernel Functions

  • Jan 01, 2023
  • IFAC PapersOnLine
  • Xizhe Wang +3
  • Research Article
  • Citations23

Are you sure it’s an artifact? Artifact detection and uncertainty quantification in histological images

  • Dec 20, 2023
  • Computerized Medical Imaging and Graphics
  • Neel Kanwal +5
  • Research Article
  • Citations32

Empirical Assessment of Deep Gaussian Process Surrogate Models for Engineering Problems

  • Sep 18, 2020
  • Journal of Aircraft
  • Dushhyanth Rajaram +6
  • Research Article
  • Citations13

Few-Shot SAR Target Recognition Based on Deep Kernel Learning

  • Jan 01, 2022
  • IEEE Access
  • Ke Wang +3
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.