Machine Learning-Based Temperature Prediction for Runtime Thermal Management Across System Components

This contribution presents a new development in the design of control system based on evolving Gaussian process (GP) models . GP models provide a probabilistic, nonparametric modelling approach for black-box identification of nonlinear dynamic systems. They can highlight areas of the input space where prediction quality is poor, due to the lack of data or its complexity, by indicating the higher variance around the predicted mean. GP models contain noticeably less coefficients to be optimised than commonly used parametric models. While GP models are Bayesian models, their output is normal distribution, expressed in terms of mean and variance. Latter can be interpreted as a confidence in prediction and used in many fields, especially in control system. Evolving GP model is the concept approach within which various ways of model adaptations can be used. Successful control system needs as much as possible data about process to be controlled . If the prior knowledge about the system to be controlled is scarce or the system varies with time or operating region, this control problem can be solved with an iterative method which adapts model with information obtained with streaming data and concurrently optimises hyperparameter values. This contribution provides: a survey of adaptive control algorithms for dynamic systems described in publications where GP models have been used for control design, a novel and improved closed-loop controller with evolving GP models and an example for the illustration of proposed control algorithm.

Quantitative prediction of subjective pain intensity from whole-brain fMRI data using Gaussian processes

Prediction of pricing and hedging errors for equity linked warrants with Gaussian process models☆

Earth observation from satellite sensory data pose challenging problems, where machine learning is currently a key player. In recent years, Gaussian Process (GP) regression and other kernel methods have excelled in biophysical parameter estimation tasks from space. GP regression is based on solid Bayesian statistics, and generally yield efficient and accurate parameter estimates. However, GPs are typically used for inverse modeling based on concurrent observations and in situ measurements only. Very often a forward model encoding the well-understood physical relations is available though. In this work, we review three GP models that respect and learn the physics of the underlying processes in the context of inverse modeling. First, we will introduce a Joint GP (JGP) model that combines in situ measurements and simulated data in a single GP model. Second, we present a latent force model (LFM) for GP modeling that encodes ordinary differential equations to blend data-driven modeling and physical models of the system. The LFM performs multi-output regression, adapts to the signal characteristics, is able to cope with missing data in the time series, and provides explicit latent functions that allow system analysis and evaluation. Finally, we present an Automatic Gaussian Process Emulator (AGAPE) that approximates the forward physical model via interpolation, reducing the number of necessary nodes. Empirical evidence of the performance of these models will be presented through illustrative examples of vegetation monitoring and atmospheric modeling.

For the challenging task of modeling actual complex motions, we propose a new class of Gaussian process (GP) models that are data-driven and also take into account prior knowledge of the motion intention. As a theoretical basis, we show that the GP regression is mathematically equivalent to regularized least-squares estimation for random functions with known prior means. Compared with the popular GP models in machine learning literature, the proposed GP motion model priors with conditional kernels have at least two advantages: 1) they are nonstationary and more applicable to represent complex motions by integrating the basic kinematic principles; 2) conditional kernels are further devised by incorporating the motion intent so that the resultant GP models are more versatile and would expectedly entail more accurate trajectory prediction. A superior property of the GP models with conditional kernels is found to improve the computational efficiency. Finally, illustrative examples are provided to show the superiority of the proposed motion models and to verify the theoretical results given in the paper.

This article describes a method for modelling non-linear dynamic systems from measurement data. The method merges the linear local model blending approach in the velocity-based linearisation form with Bayesian Gaussian process (GP) modelling. The new Fixed-Structure GP (FSGP) model has a predetermined linear model structure with varying and probabilistic parameters represented by GP models. These models have several advantages for the modelling of local model parameters as they give us adequate results, even with small data sets. Furthermore, they provide a measure of the confidence in the prediction of the varying parameters and information about the dependence of the parameters on individual inputs. The FSGP model can be applied for the extended local linear equivalence class of non-linear systems. The obtained non-linear system model can be, for example, used for control-system design. The proposed modelling method is illustrated with a simple example of non-linear system modelling for control design.

A new data-driven system identification method, called KL-GP, is proposed for spatiotemporal system. It combines Karhunen-Loève (KL) decomposition and Gaussian process (GP) models. As the nonlinear spatial-temporal spatiotemporal system has strong spatiotemporal characteristics, KL decomposition with good characteristics is employed for time/space separation and dimension reduction. Then the spatiotemporal output is expanded onto a low-dimensional KL space with temporal coefficients. GP models are employed to build up the temporal relation using these coefficients. In addition, a healthy spatial-temporal model that has accuracy predictions is always unknown in practice. GP provides an estimate of the variance of its predicted output. Using this characteristic, active data in the spatiotemporal system region can be found out for the model improvement. This enables the spatiotemporal system model to be updated without high computational demand. Simulation results of spatiotemporal system are presented to demonstrate the effectiveness of this KLGP modeling method.

Device sizing is a challenging problem for analog circuit design. Traditional methods depend on domain knowledge and intensive simulations to search for feasible parameters. Recent studies apply the Bayesian optimization (BO) and a Gaussian process (GP) model in analog circuit synthesis to improve efficiency. The BO framework automatically selects the parameter candidates by inferring the surrogate GP model. However, naive BO employs a sequential updating strategy which is inefficient in a multicore environment. Besides, the widely used GP model requires costly high fidelity data, which are obtained from fine simulations. In this article, we propose a constrained batch BO approach with a multifidelity (MF) model to solve the above difficulties. The batch BO exploits parallel computing and selects promising parameters by multiple acquisition function ensemble. In addition, the MF GP model adapts the low fidelity data obtained from coarse simulations. Specifically, the proposed method incorporates information gain in a weighted clustering algorithm to refine the parameter candidates. As a result, the proposed method maintains the candidates’ quality and diversity, which speeds up the optimization convergence. In the experiments, we demonstrate the efficiency of the proposed approach on three real-world circuits. The results show that our approach reduces the simulation costs by at least 54.6% compared to the state-of-the-art baselines.

In this work, probabilistic damage quantification under varying loading conditions in a non-stationary, guided-wave environment is being tackled via the synergistic integration between Time-varying Autoregressive (TAR) models and Gaussian Process regression models (GPRMs). Applying these TAR-GPRMs onto an aluminum coupon with simulated damage under different loading conditions fitted with piezoelectric sensors/actuators, the TAR models show the capability of capturing the non-stationarity in the ultrasonic guided-wave signals generated by the actuators under varying plate-loading conditions. They also provide insights to the maintainer by showing when in time the guided-wave time series deviate the most. In order to take advantage of that, a time instant selection algorithm was developed to allow flexibility in choosing the time instant(s) at which probabilistic damage quantification should be done. Finally, this quantification task is tackled by GPRMs, in which multiple GPRMs are trained using the TAR model parameters under varying conditions, and then used to predict damage size and/or loading state. While this framework is much more powerful in terms of tapping into the dynamics of how guided-waves change with varying conditions compared to simpler forms of GPRMs (such as damage index-trained GPRMs), training of TAR-GPRMs is far more complex. The advantages and challenges associated with the proposed TAR-GPRM approach is presented herein along with potential open areas for research in this regard.

There is uncertainty in the neuromusculoskeletal system, and deterministic models cannot describe this significant presence of uncertainty, affecting the accuracy of model predictions. In this paper, a knee joint angle prediction model based on surface electromyography (sEMG) signals is proposed. To address the instability of EMG signals and the uncertainty of the neuromusculoskeletal system, a non-parametric probabilistic model is developed using a Gaussian process model combined with the physiological properties of muscle activation. Since the neuromusculoskeletal system is a dynamic system, the Gaussian process model is further combined with a non-linear autoregressive with eXogenous inputs (NARX) model to create a Gaussian process autoregression model. In this paper, the normalized root mean square error (NRMSE) and the correlation coefficient (CC) are compared between the joint angle prediction results of the Gaussian process autoregressive model prediction and the actual joint angle under three test scenarios: speed-dependent, multi-speed and speed-independent. The mean of NRMSE and the mean of CC for all test scenarios in the healthy subjects dataset and the hemiplegic patients dataset outperform the results of the Gaussian process model, with significant differences (p < 0.05 and p < 0.05, p < 0.05 and p < 0.05). From the perspective of uncertainty, a non-parametric probabilistic model for joint angle prediction is established by using Gaussian process autoregressive model to achieve accurate prediction of human movement.

The Gaussian process (GP) model provides a powerful methodology for calibrating a computer model in the presence of model uncertainties. However, if the data contain systematic experimental errors, then the GP model can lead to an unnecessarily complex adjustment of the computer model. In this work, we introduce an adjustment procedure that brings the computer model closer to the data by making minimal changes to it. This is achieved by applying a lasso-based variable selection on the systematic experimental error terms while fitting the GP model. Two real examples and simulations are presented to demonstrate the advantages of the proposed approach. This article has supplementary material available online.

Dynamic reliability is defined as the probability that an engineered system successfully performs the predefined functionality over a certain period of time considering time-variant operation condition and component deterioration. In practice, it is still a major challenge to conduce dynamic reliability analysis due to the prohibitively high computational costs. In this study, a confidence-based meta-modeling approach is proposed for efficient sensitivity-free dynamic reliability analysis, referred to as double-loop adaptive sampling (DLAS). In DLAS a Gaussian process (GP) model is constructed to approximate extreme system responses over time, so that Monte Carlo simulation (MCS) can be employed directly to estimate dynamic reliability. A qualitative confidence measure is proposed to evaluate the accuracy of dynamic reliability estimation while using the MCS approach based on developed GP models. To improve the confidence, a double-loop adaptive sampling scheme is developed to efficiently update the GP model in a sequential manner, by considering system input variables and time concurrently in double sampling loops. The model updating process can be terminated once the user defined confidence target is satisfied. The DLAS approach does not require computationally expensive sensitivity analysis, thus substantially improves the efficiency of dynamic reliability assessment. Two case studies are used to demonstrate the effectiveness of DLAS for dynamic reliability analysis.

For multi-input and multi-output (MIMO) systems, its nonlinearity and strong coupling make the control problem more complicated. This paper introduces a new PID control strategy based on Gaussian process (GP) model to solve the problem of dynamic nonlinearity and strong coupling in complex industrial fields. This processing method is a combination of partial least squares (PLS) as an external framework and GP for internal processing. Firstly, the dynamic PLS (DPLS) is used to extract the input and output variables to eliminate the correlation, and the obtained feature matrix is used as the input of the GP to establish the model. Therefore, the MIMO control problem in the original space is transformed into multiple single-input and single-output (SISO) control problems in the latent space. Finally, a numerical example are used to illustrate the efficiency of the proposed control method.

Storage system performance modeling is crucial for efficient use of heterogeneous shared resources on leadership class computers. Variability in application performance, particularly variability arising from concurrent applications sharing I/O resources, is a major hurdle in the development of accurate performance models. We adopt a deep learning approach based on conditional variational auto encoders (CVAE) for I/O performance modeling, and use it to quantify performance variability. We illustrate our approach using the data collected on Edison, a production supercomputing system at the National Energy Research Scientific Computing Center (NERSC). The CVAE approach is investigated by comparing it to a previously proposed sensitivity-based Gaussian process (GP) model. We find that the CVAE model performs slightly better than the GP model in cases where training and testing data come from different applications, since CVAE can inherently leverage the whole data from multiple applications whereas GP partitions the data and builds separate models for each partition. Hence, the CVAE offers an alternative modeling approach that does not need preprocessing; it has enough flexibility to handle data from a wide variety of applications without changing the inference approach.

In this paper, we utilize the probability density function of the data to estimate the minimum variance lower bound (MVLB) of a nonlinear system. For this purpose, the Gaussian Process (GP) model has been used. With this model, given a new input and based on past observations, we naturally obtained the variance of the predictive distribution of the future output, which enabled us to estimate MVLB as well as estimation uncertainty. Also, an advantage of the proposed method over others is its ability to estimate MVLB recursively. The application of this method to the real-life dynamic process (experimental four-tank process) indicates that this approach gives very credible estimates of the MVLB.