Multi‐Resolution Spatial Methods on the Sphere: Efficient Prediction for Global Data

This contribution presents a new development in the design of control system based on evolving Gaussian process (GP) models . GP models provide a probabilistic, nonparametric modelling approach for black-box identification of nonlinear dynamic systems. They can highlight areas of the input space where prediction quality is poor, due to the lack of data or its complexity, by indicating the higher variance around the predicted mean. GP models contain noticeably less coefficients to be optimised than commonly used parametric models. While GP models are Bayesian models, their output is normal distribution, expressed in terms of mean and variance. Latter can be interpreted as a confidence in prediction and used in many fields, especially in control system. Evolving GP model is the concept approach within which various ways of model adaptations can be used. Successful control system needs as much as possible data about process to be controlled . If the prior knowledge about the system to be controlled is scarce or the system varies with time or operating region, this control problem can be solved with an iterative method which adapts model with information obtained with streaming data and concurrently optimises hyperparameter values. This contribution provides: a survey of adaptive control algorithms for dynamic systems described in publications where GP models have been used for control design, a novel and improved closed-loop controller with evolving GP models and an example for the illustration of proposed control algorithm.

Elevated temperatures limit the peak performance of systems because of frequent interventions by thermal throttling. Non-uniform thermal states across system nodes also cause performance variation within seemingly equivalent nodes leading to significant degradation of overall performance. In this paper we present a framework for creating a lightweight thermal prediction system suitable for run-time management decisions. We pursue two avenues to explore optimized lightweight thermal predictors. First, we use feature selection algorithms to improve the performance of previously designed machine learning methods. Second, we develop alternative methods using neural network and linear regression-based methods to perform a comprehensive comparative study of prediction methods. We show that our optimized models achieve improved performance with better prediction accuracy and lower overhead as compared with the Gaussian process model proposed previously. Specifically we present a reduced version of the Gaussian process model, a neural network–based model, and a linear regression–based model. Using the optimization methods, we are able to reduce the average prediction errors in the Gaussian process from $4.2^\circ$ C to $2.9^\circ$ C. We also show that the newly developed models using neural network and Lasso linear regression have average prediction errors of $2.9^\circ$ C and $3.8^\circ$ C respectively. The prediction overheads are 0.22, 0.097, and 0.026 ms per prediction for reduced Gaussian process, neural network, and Lasso linear regression models, respectively, compared with 0.57 ms per prediction for the previous Gaussian process model. We have implemented our proposed thermal prediction models on a two-node system configuration to help identify the optimal task placement. The task placement identified by the models reduces the average system temperature by up to $11.9^\circ$ C without any performance degradation. Furthermore, these models respectively achieve 75, 82.5, and 74.17 percent success rates in correctly pointing to those task placements with better thermal response, compared with 72.5 percent success for the original model in achieving the same objective. Finally, we extended our analysis to a 16-node system and we were able to train models and execute them in real time to guide task migration and achieve on average 17 percent reduction in the overall system cooling power.

Earth observation from satellite sensory data pose challenging problems, where machine learning is currently a key player. In recent years, Gaussian Process (GP) regression and other kernel methods have excelled in biophysical parameter estimation tasks from space. GP regression is based on solid Bayesian statistics, and generally yield efficient and accurate parameter estimates. However, GPs are typically used for inverse modeling based on concurrent observations and in situ measurements only. Very often a forward model encoding the well-understood physical relations is available though. In this work, we review three GP models that respect and learn the physics of the underlying processes in the context of inverse modeling. First, we will introduce a Joint GP (JGP) model that combines in situ measurements and simulated data in a single GP model. Second, we present a latent force model (LFM) for GP modeling that encodes ordinary differential equations to blend data-driven modeling and physical models of the system. The LFM performs multi-output regression, adapts to the signal characteristics, is able to cope with missing data in the time series, and provides explicit latent functions that allow system analysis and evaluation. Finally, we present an Automatic Gaussian Process Emulator (AGAPE) that approximates the forward physical model via interpolation, reducing the number of necessary nodes. Empirical evidence of the performance of these models will be presented through illustrative examples of vegetation monitoring and atmospheric modeling.

Quantitative prediction of subjective pain intensity from whole-brain fMRI data using Gaussian processes

For the challenging task of modeling actual complex motions, we propose a new class of Gaussian process (GP) models that are data-driven and also take into account prior knowledge of the motion intention. As a theoretical basis, we show that the GP regression is mathematically equivalent to regularized least-squares estimation for random functions with known prior means. Compared with the popular GP models in machine learning literature, the proposed GP motion model priors with conditional kernels have at least two advantages: 1) they are nonstationary and more applicable to represent complex motions by integrating the basic kinematic principles; 2) conditional kernels are further devised by incorporating the motion intent so that the resultant GP models are more versatile and would expectedly entail more accurate trajectory prediction. A superior property of the GP models with conditional kernels is found to improve the computational efficiency. Finally, illustrative examples are provided to show the superiority of the proposed motion models and to verify the theoretical results given in the paper.

Prediction of pricing and hedging errors for equity linked warrants with Gaussian process models☆

Enhanced urban PM2.5 prediction: Applying quadtree division and time-series transformer with WRF-chem

Device sizing is a challenging problem for analog circuit design. Traditional methods depend on domain knowledge and intensive simulations to search for feasible parameters. Recent studies apply the Bayesian optimization (BO) and a Gaussian process (GP) model in analog circuit synthesis to improve efficiency. The BO framework automatically selects the parameter candidates by inferring the surrogate GP model. However, naive BO employs a sequential updating strategy which is inefficient in a multicore environment. Besides, the widely used GP model requires costly high fidelity data, which are obtained from fine simulations. In this article, we propose a constrained batch BO approach with a multifidelity (MF) model to solve the above difficulties. The batch BO exploits parallel computing and selects promising parameters by multiple acquisition function ensemble. In addition, the MF GP model adapts the low fidelity data obtained from coarse simulations. Specifically, the proposed method incorporates information gain in a weighted clustering algorithm to refine the parameter candidates. As a result, the proposed method maintains the candidates’ quality and diversity, which speeds up the optimization convergence. In the experiments, we demonstrate the efficiency of the proposed approach on three real-world circuits. The results show that our approach reduces the simulation costs by at least 54.6% compared to the state-of-the-art baselines.

The Gaussian process (GP) model provides a powerful methodology for calibrating a computer model in the presence of model uncertainties. However, if the data contain systematic experimental errors, then the GP model can lead to an unnecessarily complex adjustment of the computer model. In this work, we introduce an adjustment procedure that brings the computer model closer to the data by making minimal changes to it. This is achieved by applying a lasso-based variable selection on the systematic experimental error terms while fitting the GP model. Two real examples and simulations are presented to demonstrate the advantages of the proposed approach. This article has supplementary material available online.

Dynamic reliability is defined as the probability that an engineered system successfully performs the predefined functionality over a certain period of time considering time-variant operation condition and component deterioration. In practice, it is still a major challenge to conduce dynamic reliability analysis due to the prohibitively high computational costs. In this study, a confidence-based meta-modeling approach is proposed for efficient sensitivity-free dynamic reliability analysis, referred to as double-loop adaptive sampling (DLAS). In DLAS a Gaussian process (GP) model is constructed to approximate extreme system responses over time, so that Monte Carlo simulation (MCS) can be employed directly to estimate dynamic reliability. A qualitative confidence measure is proposed to evaluate the accuracy of dynamic reliability estimation while using the MCS approach based on developed GP models. To improve the confidence, a double-loop adaptive sampling scheme is developed to efficiently update the GP model in a sequential manner, by considering system input variables and time concurrently in double sampling loops. The model updating process can be terminated once the user defined confidence target is satisfied. The DLAS approach does not require computationally expensive sensitivity analysis, thus substantially improves the efficiency of dynamic reliability assessment. Two case studies are used to demonstrate the effectiveness of DLAS for dynamic reliability analysis.

This article describes a method for modelling non-linear dynamic systems from measurement data. The method merges the linear local model blending approach in the velocity-based linearisation form with Bayesian Gaussian process (GP) modelling. The new Fixed-Structure GP (FSGP) model has a predetermined linear model structure with varying and probabilistic parameters represented by GP models. These models have several advantages for the modelling of local model parameters as they give us adequate results, even with small data sets. Furthermore, they provide a measure of the confidence in the prediction of the varying parameters and information about the dependence of the parameters on individual inputs. The FSGP model can be applied for the extended local linear equivalence class of non-linear systems. The obtained non-linear system model can be, for example, used for control-system design. The proposed modelling method is illustrated with a simple example of non-linear system modelling for control design.

There is uncertainty in the neuromusculoskeletal system, and deterministic models cannot describe this significant presence of uncertainty, affecting the accuracy of model predictions. In this paper, a knee joint angle prediction model based on surface electromyography (sEMG) signals is proposed. To address the instability of EMG signals and the uncertainty of the neuromusculoskeletal system, a non-parametric probabilistic model is developed using a Gaussian process model combined with the physiological properties of muscle activation. Since the neuromusculoskeletal system is a dynamic system, the Gaussian process model is further combined with a non-linear autoregressive with eXogenous inputs (NARX) model to create a Gaussian process autoregression model. In this paper, the normalized root mean square error (NRMSE) and the correlation coefficient (CC) are compared between the joint angle prediction results of the Gaussian process autoregressive model prediction and the actual joint angle under three test scenarios: speed-dependent, multi-speed and speed-independent. The mean of NRMSE and the mean of CC for all test scenarios in the healthy subjects dataset and the hemiplegic patients dataset outperform the results of the Gaussian process model, with significant differences (p < 0.05 and p < 0.05, p < 0.05 and p < 0.05). From the perspective of uncertainty, a non-parametric probabilistic model for joint angle prediction is established by using Gaussian process autoregressive model to achieve accurate prediction of human movement.

For multi-input and multi-output (MIMO) systems, its nonlinearity and strong coupling make the control problem more complicated. This paper introduces a new PID control strategy based on Gaussian process (GP) model to solve the problem of dynamic nonlinearity and strong coupling in complex industrial fields. This processing method is a combination of partial least squares (PLS) as an external framework and GP for internal processing. Firstly, the dynamic PLS (DPLS) is used to extract the input and output variables to eliminate the correlation, and the obtained feature matrix is used as the input of the GP to establish the model. Therefore, the MIMO control problem in the original space is transformed into multiple single-input and single-output (SISO) control problems in the latent space. Finally, a numerical example are used to illustrate the efficiency of the proposed control method.

Storage system performance modeling is crucial for efficient use of heterogeneous shared resources on leadership class computers. Variability in application performance, particularly variability arising from concurrent applications sharing I/O resources, is a major hurdle in the development of accurate performance models. We adopt a deep learning approach based on conditional variational auto encoders (CVAE) for I/O performance modeling, and use it to quantify performance variability. We illustrate our approach using the data collected on Edison, a production supercomputing system at the National Energy Research Scientific Computing Center (NERSC). The CVAE approach is investigated by comparing it to a previously proposed sensitivity-based Gaussian process (GP) model. We find that the CVAE model performs slightly better than the GP model in cases where training and testing data come from different applications, since CVAE can inherently leverage the whole data from multiple applications whereas GP partitions the data and builds separate models for each partition. Hence, the CVAE offers an alternative modeling approach that does not need preprocessing; it has enough flexibility to handle data from a wide variety of applications without changing the inference approach.

A new data-driven system identification method, called KL-GP, is proposed for spatiotemporal system. It combines Karhunen-Loève (KL) decomposition and Gaussian process (GP) models. As the nonlinear spatial-temporal spatiotemporal system has strong spatiotemporal characteristics, KL decomposition with good characteristics is employed for time/space separation and dimension reduction. Then the spatiotemporal output is expanded onto a low-dimensional KL space with temporal coefficients. GP models are employed to build up the temporal relation using these coefficients. In addition, a healthy spatial-temporal model that has accuracy predictions is always unknown in practice. GP provides an estimate of the variance of its predicted output. Using this characteristic, active data in the spatiotemporal system region can be found out for the model improvement. This enables the spatiotemporal system model to be updated without high computational demand. Simulation results of spatiotemporal system are presented to demonstrate the effectiveness of this KLGP modeling method.