Temporal Gaussian Process Regression in Logarithmic Time

Mechanical property prediction plays a crucial role in the steel industry, enabling better materials selection and enhancing production efficiency. In this study, we propose a novel framework that facilitates optimal materials selection for steel’s alloying elements, based on accurate predictions of the Jominy hardness curve. Leveraging Gaussian Process (GP) regression, we provide probabilistic predictions of steel hardenability characteristics from alloy element composition. Taking it a step further, our framework incorporates these accurate predictions into a constrained optimization process, yielding optimal compositions that reduce overall spending while meeting performance specifications. Through data obtained from 1080 steel samples, our GP regression model exhibits high accuracy, achieving an RMSE of 1.37 and showcasing significant improvements in the field. Moreover, our constrained optimization utilizing the GP model and historical market data reveals an average cost reduction of 18% on alloying element expenses, highlighting the tangible cost-saving potential of this approach. By leveraging Gaussian Process (GP) regression, we not only achieve accurate predictions of the Jominy hardness curve based on alloy element composition, but we also introduce a crucial element of uncertainty quantification. This empowers us to place trust in the results of our optimization process, ensuring robust and reliable materials selection. The integration of GP regression and optimization provides a powerful tool for achieving cost-effective materials selection and marks a significant advancement compared to existing studies. This research underscores the promise of machine learning in the steel industry, demonstrating its ability to yield substantial cost savings and enhance decision-making in materials selection.

Earth observation from satellite sensory data pose challenging problems, where machine learning is currently a key player. In recent years, Gaussian Process (GP) regression and other kernel methods have excelled in biophysical parameter estimation tasks from space. GP regression is based on solid Bayesian statistics, and generally yield efficient and accurate parameter estimates. However, GPs are typically used for inverse modeling based on concurrent observations and in situ measurements only. Very often a forward model encoding the well-understood physical relations is available though. In this work, we review three GP models that respect and learn the physics of the underlying processes in the context of inverse modeling. First, we will introduce a Joint GP (JGP) model that combines in situ measurements and simulated data in a single GP model. Second, we present a latent force model (LFM) for GP modeling that encodes ordinary differential equations to blend data-driven modeling and physical models of the system. The LFM performs multi-output regression, adapts to the signal characteristics, is able to cope with missing data in the time series, and provides explicit latent functions that allow system analysis and evaluation. Finally, we present an Automatic Gaussian Process Emulator (AGAPE) that approximates the forward physical model via interpolation, reducing the number of necessary nodes. Empirical evidence of the performance of these models will be presented through illustrative examples of vegetation monitoring and atmospheric modeling.

Gaussian Process Regression (GPR) is a powerful Bayesian method. However, the performance of GPR can be significantly degraded when the training data are contaminated by outliers, including target outliers and input outliers. Although there are some variants of GPR (e.g., GPR with Student-t likelihood (GPRT)) aiming to handle outliers, most of the variants focus on handling the target outliers while little effort has been done to deal with the input outliers. In contrast, in this work, we aim to handle both the target outliers and the input outliers at the same time. Specifically, we replace the Gaussian noise in GPR with independent Student-t noise to cope with the target outliers. Moreover, to enhance the robustness w.r.t. the input outliers, we use a Student-t Process prior instead of the common Gaussian Process prior, leading to Student-t Process Regression with Student-t Likelihood (TPRT). We theoretically show that TPRT is more robust to both input and target outliers than GPR and GPRT, and prove that both GPR and GPRT are special cases of TPRT. Various experiments demonstrate that TPRT outperforms GPR and its variants on both synthetic and real datasets.

This paper describes a real-time data collection framework and an adaptive machining learning method for constructing a real-time energy prediction model for a machine tool. To effectively establish the energy consumption pattern of a machine tool over time, the energy prediction model is continuously updated with new measurement data to account for time-varying effects of the machine tool, such as tool wear and machine tool deterioration. In this work, a real-time data collection and processing framework is developed to retrieve raw data from a milling machine tool and its sensors and convert them into relevant input features. The extracted input features are then used to construct the energy prediction model using Gaussian Process (GP) regression. To update the GP regression model with real-time streaming data, we investigate the use of sparse representation of the covariance matrix to reduce the computational and storage demands of the GP regression. We compare computational efficiency of sparse GP to that of full GP regression model and show the effectiveness of the sparse GP regression model for tracking the variation in the energy consumption pattern of the target machine.

Human motion intent learning based motion assistance control for a wearable exoskeleton

BackgroundThe analysis of gene expression from time series underpins many biological studies. Two basic forms of analysis recur for data of this type: removing inactive (quiet) genes from the study and determining which genes are differentially expressed. Often these analysis stages are applied disregarding the fact that the data is drawn from a time series. In this paper we propose a simple model for accounting for the underlying temporal nature of the data based on a Gaussian process.ResultsWe review Gaussian process (GP) regression for estimating the continuous trajectories underlying in gene expression time-series. We present a simple approach which can be used to filter quiet genes, or for the case of time series in the form of expression ratios, quantify differential expression. We assess via ROC curves the rankings produced by our regression framework and compare them to a recently proposed hierarchical Bayesian model for the analysis of gene expression time-series (BATS). We compare on both simulated and experimental data showing that the proposed approach considerably outperforms the current state of the art.ConclusionsGaussian processes offer an attractive trade-off between efficiency and usability for the analysis of microarray time series. The Gaussian process framework offers a natural way of handling biological replicates and missing values and provides confidence intervals along the estimated curves of gene expression. Therefore, we believe Gaussian processes should be a standard tool in the analysis of gene expression time series.

We explore how the big-three computing paradigms---symmetric multiprocessor, graphical processing units (GPUs), and cluster computing---can together be brought to bear on large-data Gaussian processes (GP) regression problems via a careful implementation of a newly developed local approximation scheme. Our methodological contribution focuses primarily on GPU computation, as this requires the most care and also provides the largest performance boost. However, in our empirical work we study the relative merits of all three paradigms to determine how best to combine them. The paper concludes with two case studies. One is a real data fluid-dynamics computer experiment which benefits from the local nature of our approximation; the second is a synthetic example designed to find the largest data set for which (accurate) GP emulation can be performed on a commensurate predictive set in under an hour.

Kepler-78b is a transiting planet that is 1.2 times the radius of Earth and orbits a young, active K dwarf every 8 hours. The mass of Kepler-78b has been independently reported by two teams based on radial velocity measurements using the HIRES and HARPS-N spectrographs. Due to the active nature of the host star, a stellar activity model is required to distinguish and isolate the planetary signal in radial velocity data. Whereas previous studies tested parametric stellar activity models, we modeled this system using nonparametric Gaussian process (GP) regression. We produced a GP regression of relevant Kepler photometry. We then use the posterior parameter distribution for our photometric fit as a prior for our simultaneous GP + Keplerian orbit models of the radial velocity datasets. We tested three simple kernel functions for our GP regressions. Based on a Bayesian likelihood analysis, we selected a quasi-periodic kernel model with GP hyperparameters coupled between the two RV datasets, giving a Doppler amplitude of 1.86 $\pm$ 0.25 m s$^{-1}$ and supporting our belief that the correlated noise we are modeling is astrophysical. The corresponding mass of 1.87 $^{+0.27}_{-0.26}$ M$_{\oplus}$ is consistent with that measured in previous studies, and more robust due to our nonparametric signal estimation. Based on our mass and the radius measurement from transit photometry, Kepler-78b has a bulk density of 6.0$^{+1.9}_{-1.4}$ g cm$^{-3}$. We estimate that Kepler-78b is 32$\pm$26% iron using a two-component rock-iron model. This is consistent with an Earth-like composition, with uncertainty spanning Moon-like to Mercury-like compositions.

Gaussian process (GP) regression is a nonparametric regression method that can be used to predict continuous quantities. Here, we show that the same technique can be applied to a class of phase imaging techniques based on measurements of intensity at multiple propagation distances, i.e. the transport of intensity equation (TIE). In this paper, we demonstrate how to apply GP regression to estimate the first intensity derivative along the direction of propagation and incorporate non-uniform propagation distance sampling. The low-frequency artifacts that often occur in phase recovery using traditional methods can be significantly suppressed by the proposed GP TIE method. The method is shown to be stable with moderate amounts of Gaussian noise. We validate the method experimentally by recovering the phase of human cheek cells in a bright field microscope and show better performance as compared to other TIE reconstruction methods.

We present a novel computational approach for extracting localized signals from smooth background distributions. We focus on datasets that can be naturally presented as binned integer counts, demonstrating our procedure on the CERN open dataset with the Higgs boson signature, from the ATLAS collaboration at the Large Hadron Collider. Our approach is based on Gaussian Process (GP) regression — a powerful and flexible machine learning technique which has allowed us to model the background without specifying its functional form explicitly and separately measure the background and signal contributions in a robust and reproducible manner. Unlike functional fits, our GP-regression-based approach does not need to be constantly updated as more data becomes available. We discuss how to select the GP kernel type, considering trade-offs between kernel complexity and its ability to capture the features of the background distribution. We show that our GP framework can be used to detect the Higgs boson resonance in the data with more statistical significance than a polynomial fit specifically tailored to the dataset. Finally, we use Markov Chain Monte Carlo (MCMC) sampling to confirm the statistical significance of the extracted Higgs signature.

The Expected Value of Perfect Partial Information (EVPPI) is a decision‐theoretic measure of the ‘cost’ of parametric uncertainty in decision making used principally in health economic decision making. Despite this decision‐theoretic grounding, the uptake of EVPPI calculations in practice has been slow. This is in part due to the prohibitive computational time required to estimate the EVPPI via Monte Carlo simulations. However, recent developments have demonstrated that the EVPPI can be estimated by non‐parametric regression methods, which have significantly decreased the computation time required to approximate the EVPPI. Under certain circumstances, high‐dimensional Gaussian Process (GP) regression is suggested, but this can still be prohibitively expensive. Applying fast computation methods developed in spatial statistics using Integrated Nested Laplace Approximations (INLA) and projecting from a high‐dimensional into a low‐dimensional input space allows us to decrease the computation time for fitting these high‐dimensional GP, often substantially. We demonstrate that the EVPPI calculated using our method for GP regression is in line with the standard GP regression method and that despite the apparent methodological complexity of this new method, R functions are available in the package BCEA to implement it simply and efficiently. © 2016 The Authors. Statistics in Medicine Published by John Wiley & Sons Ltd.

Nonnegativity-enforced Gaussian process regression

Automated rock recognition is a key step for building a fully autonomous mine. When characterizing rock types from drill performance data, the main challenge is that there is not an obvious one-to-one correspondence between the two. In this paper, a hybrid rock recognition approach is proposed which combines Gaussian Process (GP) regression with clustering. Drill performance data is also known as Measurement While Drilling (MWD) data and a rock hardness measure - Adjusted Penetration Rate (APR) is extracted using the raw data in discrete drill holes. GP regression is then applied to create a more dense APR distribution, followed by clustering which produces discrete class labels. No initial labelling is needed. Comparisons are made with alternative measures of rock hardness from MWD data as well as state-of-the-art GP classification. Experimental results from an actual mine site show the effectiveness of our proposed approach.

Gaussian Process (GP) regression has seen widespread use in robotics due to its generality, simplicity of use, and the utility of Bayesian predictions. The predominant implementation of GP regression is a nonparametric kernel-based approach, as it enables fitting of arbitrary nonlinear functions. However, this approach suffers from two main drawbacks: (1) it is computationally inefficient, as computation scales poorly with the number of samples; and (2) it can be data inefficient, as encoding prior knowledge that can aid the model through the choice of kernel and associated hyperparameters is often challenging and unintuitive. In this work, we propose ALPaCA, an algorithm for efficient Bayesian regression which addresses these issues. ALPaCA uses a dataset of sample functions to learn a domain-specific, finite-dimensional feature encoding, as well as a prior over the associated weights, such that Bayesian linear regression in this feature space yields accurate online predictions of the posterior predictive density. These features are neural networks, which are trained via a meta-learning (or “learning-to-learn”) approach. ALPaCA extracts all prior information directly from the dataset, rather than restricting prior information to the choice of kernel hyperparameters. Furthermore, by operating in the weight space, it substantially reduces sample complexity. We investigate the performance of ALPaCA on two simple regression problems, two simulated robotic systems, and on a lane-change driving task performed by humans. We find our approach outperforms kernel-based GP regression, as well as state of the art meta-learning approaches, thereby providing a promising plug-in tool for many regression tasks in robotics where scalability and data-efficiency are important.

In this letter, we propose a novel cosmological model-independent method to determine cosmic curvature, combining the recent measurements of transverse and line-of-sight directions in the baryon acoustic oscillations (BAO) with cosmic chronometers (CC) datasets. Considering that the CC dataset is discrete and includes only 32 H(z) measurements, we apply Gaussian process (GP) regression to fit the CC dataset and reconstruct them. Our methodology, which does not need the calibration or selection of any cosmological model, provide multiple measurements of spatial curvature (ΩK) at different redshifts (depending on the redshift coverage of BAO dataset). For combination of all BAO data, we find that the constraint result on cosmic curvature is ΩK=−0.096−0.195+0.190 with 1σ observational uncertainty. Although the measured ΩK is in good agreement with zero cosmic curvature within 1σ confidence level, our result revels the fact of a closed universe. More importantly, our results show that the obtained ΩK measurements are almost unaffected by different priors of the Hubble constant. This could help solve the issue of the Hubble tension that may be caused by inconsistencies in the spatial curvature between the early and late universes.