Physics-Aware Gaussian Processes for Earth Observation

For the challenging task of modeling actual complex motions, we propose a new class of Gaussian process (GP) models that are data-driven and also take into account prior knowledge of the motion intention. As a theoretical basis, we show that the GP regression is mathematically equivalent to regularized least-squares estimation for random functions with known prior means. Compared with the popular GP models in machine learning literature, the proposed GP motion model priors with conditional kernels have at least two advantages: 1) they are nonstationary and more applicable to represent complex motions by integrating the basic kinematic principles; 2) conditional kernels are further devised by incorporating the motion intent so that the resultant GP models are more versatile and would expectedly entail more accurate trajectory prediction. A superior property of the GP models with conditional kernels is found to improve the computational efficiency. Finally, illustrative examples are provided to show the superiority of the proposed motion models and to verify the theoretical results given in the paper.

This contribution presents a new development in the design of control system based on evolving Gaussian process (GP) models . GP models provide a probabilistic, nonparametric modelling approach for black-box identification of nonlinear dynamic systems. They can highlight areas of the input space where prediction quality is poor, due to the lack of data or its complexity, by indicating the higher variance around the predicted mean. GP models contain noticeably less coefficients to be optimised than commonly used parametric models. While GP models are Bayesian models, their output is normal distribution, expressed in terms of mean and variance. Latter can be interpreted as a confidence in prediction and used in many fields, especially in control system. Evolving GP model is the concept approach within which various ways of model adaptations can be used. Successful control system needs as much as possible data about process to be controlled . If the prior knowledge about the system to be controlled is scarce or the system varies with time or operating region, this control problem can be solved with an iterative method which adapts model with information obtained with streaming data and concurrently optimises hyperparameter values. This contribution provides: a survey of adaptive control algorithms for dynamic systems described in publications where GP models have been used for control design, a novel and improved closed-loop controller with evolving GP models and an example for the illustration of proposed control algorithm.

Elevated temperatures limit the peak performance of systems because of frequent interventions by thermal throttling. Non-uniform thermal states across system nodes also cause performance variation within seemingly equivalent nodes leading to significant degradation of overall performance. In this paper we present a framework for creating a lightweight thermal prediction system suitable for run-time management decisions. We pursue two avenues to explore optimized lightweight thermal predictors. First, we use feature selection algorithms to improve the performance of previously designed machine learning methods. Second, we develop alternative methods using neural network and linear regression-based methods to perform a comprehensive comparative study of prediction methods. We show that our optimized models achieve improved performance with better prediction accuracy and lower overhead as compared with the Gaussian process model proposed previously. Specifically we present a reduced version of the Gaussian process model, a neural network–based model, and a linear regression–based model. Using the optimization methods, we are able to reduce the average prediction errors in the Gaussian process from $4.2^\circ$ C to $2.9^\circ$ C. We also show that the newly developed models using neural network and Lasso linear regression have average prediction errors of $2.9^\circ$ C and $3.8^\circ$ C respectively. The prediction overheads are 0.22, 0.097, and 0.026 ms per prediction for reduced Gaussian process, neural network, and Lasso linear regression models, respectively, compared with 0.57 ms per prediction for the previous Gaussian process model. We have implemented our proposed thermal prediction models on a two-node system configuration to help identify the optimal task placement. The task placement identified by the models reduces the average system temperature by up to $11.9^\circ$ C without any performance degradation. Furthermore, these models respectively achieve 75, 82.5, and 74.17 percent success rates in correctly pointing to those task placements with better thermal response, compared with 72.5 percent success for the original model in achieving the same objective. Finally, we extended our analysis to a 16-node system and we were able to train models and execute them in real time to guide task migration and achieve on average 17 percent reduction in the overall system cooling power.

Quantitative prediction of subjective pain intensity from whole-brain fMRI data using Gaussian processes

Gaussian Process regression is a kernel method successfully adopted in many real-life applications. Recently, there is a growing interest on extending this method to non-Euclidean input spaces, like the one considered in this paper, consisting of probability measures. Although a Positive Definite kernel can be defined by using a suitable distance—the Wasserstein distance— the common procedure for learning the Gaussian Process model can fail due to numerical issues, arising earlier and more frequently than in the case of an Euclidean input space and, as demonstrated, impossible to avoid by adding artificial noise (nugget effect) as usually done. This paper uncovers the main reason of these issues, that is a non-stationarity relation between the Wasserstein-based squared exponential kernel and its Euclidean counterpart. As a relevant result, we learn a Gaussian Process model by assuming the input space as Euclidean and then use an algebraic transformation, based on the uncovered relation, to transform it into a non-stationary and Wasserstein-based Gaussian Process model over probability measures. This algebraic transformation is simpler than log-exp maps used on data belonging to Riemannian manifolds and recently extended to consider the pseudo-Riemannian structure of an input space equipped with the Wasserstein distance.

The Gaussian process is a widely used model for regression problems in supervised machine learning. However, predicting new inputs via a Gaussian process model becomes computationally inefficient when training a large data set. This paper proposes a fast quantum algorithm for prediction based on the Gaussian process regression. The proposed quantum algorithm consists of two subalgorithms: the first one aims to efficiently prepare the squared exponential covariance matrices and covariance functions vector with annihilation and creation operators; the other is to obtain predictive mean values and covariance values for new inputs. Evidence is also shown that the proposed quantum Gaussian process regression algorithm can achieve quadratic speedup over the classical counterpart.

Mechanical property prediction plays a crucial role in the steel industry, enabling better materials selection and enhancing production efficiency. In this study, we propose a novel framework that facilitates optimal materials selection for steel’s alloying elements, based on accurate predictions of the Jominy hardness curve. Leveraging Gaussian Process (GP) regression, we provide probabilistic predictions of steel hardenability characteristics from alloy element composition. Taking it a step further, our framework incorporates these accurate predictions into a constrained optimization process, yielding optimal compositions that reduce overall spending while meeting performance specifications. Through data obtained from 1080 steel samples, our GP regression model exhibits high accuracy, achieving an RMSE of 1.37 and showcasing significant improvements in the field. Moreover, our constrained optimization utilizing the GP model and historical market data reveals an average cost reduction of 18% on alloying element expenses, highlighting the tangible cost-saving potential of this approach. By leveraging Gaussian Process (GP) regression, we not only achieve accurate predictions of the Jominy hardness curve based on alloy element composition, but we also introduce a crucial element of uncertainty quantification. This empowers us to place trust in the results of our optimization process, ensuring robust and reliable materials selection. The integration of GP regression and optimization provides a powerful tool for achieving cost-effective materials selection and marks a significant advancement compared to existing studies. This research underscores the promise of machine learning in the steel industry, demonstrating its ability to yield substantial cost savings and enhance decision-making in materials selection.

In the task of 3-D topography measurement by fringe projection profilometry (FPP), it is crucial to establish mapping from the phase map to the 3-D coordinates, known as 3-D calibration. The traditional methods are prone to select some specific functions to fit the phase-to-coordinates’ relationship, which needs to make a compromise between measurement accuracy and efficiency. This article proposes a novel calibration method based on the Gaussian process (GP) regression to solve this problem. In this work, according to the geometric and other systemic constraints, a pixel-dependent semiparameterized calibration model is derived to guarantee the efficiency of computation and data storage. Based on the spatial correlations of the calibration data, the GP regression method is applied to enhance the fitting ability and flexibility of the calibration model without any specific functions and parameters. The GP regression method is also applied to remove random noise from the phase map, which further improves the accuracy of 3-D coordinates. The experimental results of measuring a whiteboard and a double ball bar demonstrate the superiority of the proposed GP-based calibration model in terms of accuracy and robustness when compared with the traditional models.

Nonparametric Gaussian processes regression (GPR) is an important tool in machine learning, can be applied in identifying nonlinear models from experimental data, especially, the prediction of mean and variance present the useful advantage. However, when dealing with the large number of training data for the prediction of a complex dynamics system, GPR is not suitable to implement in real-time learning systems. To reduce the computation effort, local learning algorithm is introduced to improve the global Gaussian processes (GP) model in this paper. In this paper, a convenient and effective method for building local model network is proposed and then local GP for weighted regression is performed. The proposed local GPR method is implemented on a simulated example of online identification and prediction fast for a complex dynamic system of wheeled mobile robot.

Due to the geological body uncertainty, the identification of the surrounding rock parameters in the tunnel construction process is of great significance to the calculation of tunnel stability. The ubiquitous-joint model and three-dimensional numerical simulation have advantages in the parameter identification of surrounding rock with weak planes, but conventional methods have certain problems, such as a large number of parameters and large time consumption. To solve the problems, this study combines the orthogonal design, Gaussian process (GP) regression, and difference evolution (DE) optimization, and it constructs the parameters identification method of the jointed surrounding rock. The calculation process of parameters identification of a tunnel jointed surrounding rock based on the GP optimized by the DE includes the following steps. First, a three-dimensional numerical simulation based on the ubiquitous-joint model is conducted according to the orthogonal and uniform design parameters combing schemes, where the model input consists of jointed rock parameters and model output is the information on the surrounding rock displacement and stress. Then, the GP regress model optimized by DE is trained by the data samples. Finally, the GP model is integrated into the DE algorithm, and the absolute differences in the displacement and stress between calculated and monitored values are used as the objective function, while the parameters of the jointed surrounding rock are used as variables and identified. The proposed method is verified by the experiments with a joint rock surface in the Dadongshan tunnel, which is located in Dalian, China. The obtained calculation and analysis results are as follows: CR = 0.9, F = 0.6, NP = 100, and the difference strategy DE/Best/1 is recommended. The results of the back analysis are compared with the field monitored values, and the relative error is 4.58%, which is satisfactory. The algorithm influencing factors are also discussed, and it is found that the local correlation coefficient σf and noise standard deviation σn affected the prediction accuracy of the GP model. The results show that the proposed method is feasible and can achieve high identification precision. The study provides an effective reference for parameter identification of jointed surrounding rock in a tunnel.

Gaussian Process Models for Mortality Rates and Improvement Factors

Prediction of pricing and hedging errors for equity linked warrants with Gaussian process models☆

When solving a multi-objective optimization problem using Evolutionary Algorithms, the diversity loss can occur as the evolution process is made. This is particularly significant in Pareto-based strategies where a diversity mechanism is required to maintain a set of solutions well distributed in the Pareto Front extension. Therefore, algorithms are required with the ability to keep a good balance between exploration and exploitation. To address this challenge, a new algorithm is proposed considering past generations to establish trends in population movement, and in this way, to find better Pareto solutions. The proposal, Gaussian Process Regression-based Evolutionary Algorithm (GPR-EA), employs Differential Evolution operators and polynomial mutation. A Gaussian Process model is used to form predictions about the new population in particular generations. The experiments were performed on 15 well-known test functions: UF1-I0 and ZDTI-4, 6. The GPR-EA comparisons with nine algorithms regarding two metrics are presented, evidencing that the proposal outperforms the other algorithms in most problems.

Device sizing is a challenging problem for analog circuit design. Traditional methods depend on domain knowledge and intensive simulations to search for feasible parameters. Recent studies apply the Bayesian optimization (BO) and a Gaussian process (GP) model in analog circuit synthesis to improve efficiency. The BO framework automatically selects the parameter candidates by inferring the surrogate GP model. However, naive BO employs a sequential updating strategy which is inefficient in a multicore environment. Besides, the widely used GP model requires costly high fidelity data, which are obtained from fine simulations. In this article, we propose a constrained batch BO approach with a multifidelity (MF) model to solve the above difficulties. The batch BO exploits parallel computing and selects promising parameters by multiple acquisition function ensemble. In addition, the MF GP model adapts the low fidelity data obtained from coarse simulations. Specifically, the proposed method incorporates information gain in a weighted clustering algorithm to refine the parameter candidates. As a result, the proposed method maintains the candidates’ quality and diversity, which speeds up the optimization convergence. In the experiments, we demonstrate the efficiency of the proposed approach on three real-world circuits. The results show that our approach reduces the simulation costs by at least 54.6% compared to the state-of-the-art baselines.

The Gaussian process (GP) model provides a powerful methodology for calibrating a computer model in the presence of model uncertainties. However, if the data contain systematic experimental errors, then the GP model can lead to an unnecessarily complex adjustment of the computer model. In this work, we introduce an adjustment procedure that brings the computer model closer to the data by making minimal changes to it. This is achieved by applying a lasso-based variable selection on the systematic experimental error terms while fitting the GP model. Two real examples and simulations are presented to demonstrate the advantages of the proposed approach. This article has supplementary material available online.