- Research Article
106
- 10.1016/j.asoc.2018.03.021
Physics-aware Gaussian processes in remote sensing
- Mar 22, 2018
- Applied Soft Computing
- Gustau Camps-Valls + 7 more +7
Physics-aware Gaussian processes in remote sensing
Earth observation from satellite sensory data pose challenging problems, where machine learning is currently a key player. In recent years, Gaussian Process (GP) regression and other kernel methods have excelled in biophysical parameter estimation tasks from space. GP regression is based on solid Bayesian statistics, and generally yield efficient and accurate parameter estimates. However, GPs are typically used for inverse modeling based on concurrent observations and in situ measurements only. Very often a forward model encoding the well-understood physical relations is available though. In this work, we review three GP models that respect and learn the physics of the underlying processes in the context of inverse modeling. First, we will introduce a Joint GP (JGP) model that combines in situ measurements and simulated data in a single GP model. Second, we present a latent force model (LFM) for GP modeling that encodes ordinary differential equations to blend data-driven modeling and physical models of the system. The LFM performs multi-output regression, adapts to the signal characteristics, is able to cope with missing data in the time series, and provides explicit latent functions that allow system analysis and evaluation. Finally, we present an Automatic Gaussian Process Emulator (AGAPE) that approximates the forward physical model via interpolation, reducing the number of necessary nodes. Empirical evidence of the performance of these models will be presented through illustrative examples of vegetation monitoring and atmospheric modeling.
Physics-aware Gaussian processes in remote sensing
Physics-aware Gaussian processes in remote sensing
Advances in Gaussian Processes for Earth Sciences: Physics-aware, interpretability and consistency
<p>Earth observation from remote sensing satellites allows us to monitor the processes occurring on the land cover, water bodies and the atmosphere, as well as their interactions. In the last decade machine learning has impacted the field enormously due to the unprecedented data deluge and emergence of complex problems that need to be tackled (semi)automatically. One of the main problems is to perform estimation of bio-geo-physical parameters from remote sensing observations. In this model inversion setting, Gaussian processes (GPs) are one of the preferred choices for model inversion, emulation, gap filling and data assimilation. GPs do not only provide accurate predictions but also allow for feature ranking, deriving confidence intervals, and error propagation and uncertainty quantification in a principled Bayesian inference framework.</p><p>Here we introduce GPs for data analysis in general and to address the forward-inverse problem posed in remote sensing in particular. GPs are typically used for inverse modelling based on concurrent observations and in situ measurements only, or to invert model simulations. We often rely on forward radiative transfer model (RTM) encoding the well-understood physical relations to either perform model inversion with machine learning, or to replace the RTM model with machine learning models, a process known as emulation. We review four novel GP models that respect and learn the physics, and deploy useful machine learning models for remote sensing parameter retrieval and model emulation tasks. First, we will introduce a Joint GP (JGP) model that combines in situ measurements and simulated data in a single GP model for inversion. Second, we present a latent force model (LFM) for GP modelling that encodes ordinary differential equations to blend data and physical models of the system. The LFM performs multi-output regression, can cope with missing data in the time series, and provides explicit latent functions that allow system analysis, evaluation and understanding. Third, we present an Automatic Gaussian Process Emulator (AGAPE) that approximates the forward physical model via interpolation, reducing the number of necessary nodes. Finally, we introduce a new GP model for data-driven regression that respects fundamental laws of physics via dependence-regularization, and provides consistency estimates. All models attain data-driven physics-aware modeling. Empirical evidence of performance of these models will be presented through illustrative examples of vegetation/land monitoring involving multispectral (Landsat, MODIS) and passive microwave (SMOS, SMAP) observations, as well as blending data with radiative transfer models, such as PROSAIL, SCOPE and MODTRAN.</p><p><br>References</p><p>"A Perspective on Gaussian Processes for Earth Observation". Camps-Valls et al. National Science Review 6 (4) :616-618, 2019</p><p>"Physics-aware Gaussian processes in remote sensing". Camps-Valls et al. Applied Soft Computing 68 :69-82, 2018</p><p>"A Survey on Gaussian Processes for Earth Observation Data Analysis: A Comprehensive Investigation". Camps-Valls et al. IEEE Geoscience and Remote Sensing Magazine 2016</p><p> </p>
Read moreGeneralized Gaussian process models
We propose a generalized Gaussian process model (GGPM), which is a unifying framework that encompasses many existing Gaussian process (GP) models, such as GP regression, classification, and counting. In the GGPM framework, the observation likelihood of the GP model is itself parameterized using the exponential family distribution. By deriving approximate inference algorithms for the generalized GP model, we are able to easily apply the same algorithm to all other GP models. Novel GP models are created by changing the parameterization of the likelihood function, which greatly simplifies their creation for task-specific output domains. We also derive a closed-form efficient Taylor approximation for inference on the model, and draw interesting connections with other model-specific closed-form approximations. Finally, using the GGPM, we create several new GP models and show their efficacy in building task-specific GP models for computer vision.
Read moreIntention-Aware Motion Modeling Using GP Priors With Conditional Kernels
For the challenging task of modeling actual complex motions, we propose a new class of Gaussian process (GP) models that are data-driven and also take into account prior knowledge of the motion intention. As a theoretical basis, we show that the GP regression is mathematically equivalent to regularized least-squares estimation for random functions with known prior means. Compared with the popular GP models in machine learning literature, the proposed GP motion model priors with conditional kernels have at least two advantages: 1) they are nonstationary and more applicable to represent complex motions by integrating the basic kinematic principles; 2) conditional kernels are further devised by incorporating the motion intent so that the resultant GP models are more versatile and would expectedly entail more accurate trajectory prediction. A superior property of the GP models with conditional kernels is found to improve the computational efficiency. Finally, illustrative examples are provided to show the superiority of the proposed motion models and to verify the theoretical results given in the paper.
Read moreEvolving Gaussian process models for predicting chaotic time-series
Gaussian process (GP) models are nowadays considered among the state-of-the-art tools in modern dynamic system identification. GP models are probabilistic, non-parametric models based on the principles of Bayesian probability. As a kernel methods they do not try to approximate the modelled system by fitting the parameters of the selected basis functions, but rather by searching for relationships among the measured data. While GP models are Bayesian models they are more robust to overfitting. Moreover, their output is normal distribution, expressed in terms of mean and variance. Due to these features they are used in various fields, e.g. model-based control, time-series prediction, modelling and estimation in engineering applications, etc. But, due to the matrix inversion calculation, whose computationally demand increases with the third power of the number of input data, the amount of training data is limited to at most a few thousand cases. Therefore GP models in principle are not applicable for modelling dynamic systems whose states evolve in time, such as chaotic time-series. In this paper we demonstrate an Evolving GP (EGP) models for predicting chaotic time-series. The EGP is iterative method which adapts model with information obtained from streaming data and concurrently optimizes hyperparameter values. To assess the viability of the EGP an empirical tests were carried out together with a comparative study of various evolving fuzzy methods on a benchmark chaotic time-series MacKey-Glass. The results indicate that the EGP can successfully identify MacKey-Glass chaotic time-series and demonstrate superior performance.
Read moreGaussian Process Regression with Soft Equality Constraints
This study introduces a novel Gaussian process (GP) regression framework that probabilistically enforces physical constraints, with a particular focus on equality conditions. The GP model is trained using the quantum-inspired Hamiltonian Monte Carlo (QHMC) algorithm, which efficiently samples from a wide range of distributions by allowing a particle’s mass matrix to vary according to a probability distribution. By integrating QHMC into the GP regression with probabilistic handling of the constraints, this approach balances the computational cost and accuracy in the resulting GP model, as the probabilistic nature of the method contributes to shorter execution times compared with existing GP-based approaches. Additionally, we introduce an adaptive learning algorithm to optimize the selection of constraint locations to further enhance the flexibility of the method. We demonstrate the effectiveness and robustness of our algorithm on synthetic examples, including 2-dimensional and 10-dimensional GP models under noisy conditions, as well as a practical application involving the reconstruction of a sparsely observed steady-state heat transport problem. The proposed approach reduces the posterior variance in the resulting model, achieving stable and accurate sampling results across all test cases while maintaining computational efficiency.
Read moreMachine-Learning Assisted X-Ray Spectroscopy for High- Throughput Characterization of Magnetic Materials
Machine-Learning Assisted X-Ray Spectroscopy for High- Throughput Characterization of Magnetic Materials
Simultaneous quantification of epistemic and aleatory uncertainty in GMPEs using Gaussian process regression
This paper presents a Bayesian non-parametric method based on Gaussian Process (GP) regression to derive ground-motion models for peak-ground parameters and response spectral ordinates. Due to its non-parametric nature there is no need to specify any fixed functional form as in parametric regression models. A GP defines a distribution over functions, which implicitly expresses the uncertainty over the underlying data generating process. An advantage of GP regression is that it is possible to capture the whole uncertainty involved in ground-motion modeling, both in terms of aleatory variability as well as epistemic uncertainty associated with the underlying functional form and data coverage. The distribution over functions is updated in a Bayesian way by computing the posterior distribution of the GP after observing ground-motion data, which in turn can be used to make predictions. The proposed GP regression models is evaluated on a subset of the RESORCE data base for the SIGMA project. The experiments show that GP models have a better generalization error than a simple parametric regression model. A visual assessment of different scenarios demonstrates that the inferred GP models are physically plausible.
Read moreClosed-Loop Control with Evolving Gaussian Process Models
This contribution presents a new development in the design of control system based on evolving Gaussian process (GP) models . GP models provide a probabilistic, nonparametric modelling approach for black-box identification of nonlinear dynamic systems. They can highlight areas of the input space where prediction quality is poor, due to the lack of data or its complexity, by indicating the higher variance around the predicted mean. GP models contain noticeably less coefficients to be optimised than commonly used parametric models. While GP models are Bayesian models, their output is normal distribution, expressed in terms of mean and variance. Latter can be interpreted as a confidence in prediction and used in many fields, especially in control system. Evolving GP model is the concept approach within which various ways of model adaptations can be used. Successful control system needs as much as possible data about process to be controlled . If the prior knowledge about the system to be controlled is scarce or the system varies with time or operating region, this control problem can be solved with an iterative method which adapts model with information obtained with streaming data and concurrently optimises hyperparameter values. This contribution provides: a survey of adaptive control algorithms for dynamic systems described in publications where GP models have been used for control design, a novel and improved closed-loop controller with evolving GP models and an example for the illustration of proposed control algorithm.
Read moreMachine Learning-Based Temperature Prediction for Runtime Thermal Management Across System Components
Elevated temperatures limit the peak performance of systems because of frequent interventions by thermal throttling. Non-uniform thermal states across system nodes also cause performance variation within seemingly equivalent nodes leading to significant degradation of overall performance. In this paper we present a framework for creating a lightweight thermal prediction system suitable for run-time management decisions. We pursue two avenues to explore optimized lightweight thermal predictors. First, we use feature selection algorithms to improve the performance of previously designed machine learning methods. Second, we develop alternative methods using neural network and linear regression-based methods to perform a comprehensive comparative study of prediction methods. We show that our optimized models achieve improved performance with better prediction accuracy and lower overhead as compared with the Gaussian process model proposed previously. Specifically we present a reduced version of the Gaussian process model, a neural network-based model, and a linear regression-based model. Using the optimization methods, we are able to reduce the average prediction errors in the Gaussian process from 4.2°C to 2.9°C. We also show that the newly developed models using neural network and Lasso linear regression have average prediction errors of 2.9°C and 3.8°C respectively. The prediction overheads are 0.22, 0.097, and 0.026 ms per prediction for reduced Gaussian process, neural network, and Lasso linear regression models, respectively, compared with 0.57 ms per prediction for the previous Gaussian process model. We have implemented our proposed thermal prediction models on a two-node system configuration to help identify the optimal task placement. The task placement identified by the models reduces the average system temperature by up to 11.9°C without any performance degradation. Furthermore, these models respectively achieve 75, 82.5, and 74.17 percent success rates in correctly pointing to those task placements with better thermal response, compared with 72.5 percent success for the original model in achieving the same objective. Finally, we extended our analysis to a 16-node system and we were able to train models and execute them in real time to guide task migration and achieve on average 17 percent reduction in the overall system cooling power.
Read moreForecasting Building Energy Demand under Uncertainty Using Gaussian Process Regression: Feature Selection, Baseline Prediction, Parametric Analysis and a Web-based Tool
Gaussian process (GP) regression has drawn growing attention in the application of building energy demand prediction. This paper demonstrates the process of developing GP models for baseline prediction and parametric analysis. In particular, this study proposes the method of feature selection based on characteristic lengthscale in covariance function, as well as the methods of anomaly detection and parametric analysis utilizing the predictive distribution of a GP regression. Two case studies are used to illustrate the processes in detail. The results show that successful feature selection can improve predictive accuracy and reduce computational cost. In baseline prediction, the outcome of a GP regression gives a confidence range, which provides valuable information for anomaly detection. In parametric analysis, the additional variance in the output caused by intentionally varying the range of a control variable gives an estimate of its impact on energy demand. Another contribution of this study is the development of a web-based tool, which allows users to build GP models without knowledge of the details GPs or programming skills.
Read moreDetecting differential growth of microbial populations with Gaussian process regression
Microbial growth curves are used to study differential effects of media, genetics, and stress on microbial population growth. Consequently, many modeling frameworks exist to capture microbial population growth measurements. However, current models are designed to quantify growth under conditions for which growth has a specific functional form. Extensions to these models are required to quantify the effects of perturbations, which often exhibit nonstandard growth curves. Rather than assume specific functional forms for experimental perturbations, we developed a general and robust model of microbial population growth curves using Gaussian process (GP) regression. GP regression modeling of high-resolution time-series growth data enables accurate quantification of population growth and allows explicit control of effects from other covariates such as genetic background. This framework substantially outperforms commonly used microbial population growth models, particularly when modeling growth data from environmentally stressed populations. We apply the GP growth model and develop statistical tests to quantify the differential effects of environmental perturbations on microbial growth across a large compendium of genotypes in archaea and yeast. This method accurately identifies known transcriptional regulators and implicates novel regulators of growth under standard and stress conditions in the model archaeal organism Halobacterium salinarum. For yeast, our method correctly identifies known phenotypes for a diversity of genetic backgrounds under cyclohexamide stress and also detects previously unidentified oxidative stress sensitivity across a subset of strains. Together, these results demonstrate that the GP models are interpretable, recapitulating biological knowledge of growth response while providing new insights into the relevant parameters affecting microbial population growth.
Read moreQuantitative prediction of subjective pain intensity from whole-brain fMRI data using Gaussian processes
Quantitative prediction of subjective pain intensity from whole-brain fMRI data using Gaussian processes
Music genre classification using Gaussian Process models
In this paper we introduce Gaussian Process (GP) models for music genre classification. Gaussian Processes are widely used for various regression and classification tasks, but there are relatively few studies where GPs are applied in the audio signal processing systems. The GP models are non-parametric discriminative classifiers similar to the well known SVMs in terms of usage. In contrast to SVMs, however, GP models produce truly probabilistic output and allow for kernel function parameters to be learned from the training data. In this work we compare the performance of GP models and SVMs as music genre classifiers using the ISMIR 2004 database. Audio preprocessing is the same for both cases and is based on Constant-Q spectrograms. The experimental results using linear as well as exponential kernel functions and different amounts of training data show that GP models always outperform SVMs with up to 5.6% absolute difference in the classification accuracy.
Read morePrognosis Using Gaussian Process Model
This chapter introduces a nonparametric approach for longitudinal data modeling and prediction. This approach is based on the Gaussian process (GP) model. GP is a stochastic process and can be viewed as a distribution over functions with a continuous domain, e.g. time or space. The chapter also introduces the structure of the GP model. It discusses the estimation and prediction methods for the GP model, examines the pairwise GP model. The chapter presents the extension of a single output GP model to the general Multiple Output Gaussian Process (MOGP) model, which plays a critical role in longitudinal data prediction. It discusses the time-to-failure distribution based on the MOGP model. The chapter provides the basic structure of MOGP and explains the MOGP based prediction method. The Gaussian process, also known as Kriging method, is a very flexible yet effective nonparametric model to describe smooth functional data.
Read more