Trained Quantum Neural Networks are Gaussian Processes

Recent years have witnessed an increasing interest in the correspondence between infinitely wide networks and Gaussian processes. Despite the effectiveness and elegance of the current neural network Gaussian process theory, to the best of our knowledge, all the neural network Gaussian processes (NNGPs) are essentially induced by increasing width. However, in the era of deep learning, what concerns us more regarding a neural network is its depth as well as how depth impacts the behaviors of a network. Inspired by a width-depth symmetry consideration, we use a shortcut network to show that increasing the depth of a neural network can also give rise to a Gaussian process, which is a valuable addition to the existing theory and contributes to revealing the true picture of deep learning. Beyond the proposed Gaussian process by depth, we theoretically characterize its uniform tightness property and the smallest eigenvalue of the Gaussian process kernel. These characterizations can not only enhance our understanding of the proposed depth-induced Gaussian process but also pave the way for future applications. Lastly, we examine the performance of the proposed Gaussian process by regression experiments on two benchmark datasets.

Prediction of pricing and hedging errors for equity linked warrants with Gaussian process models☆

Predictive analytics can forecast trends and determines statistical probabilities and to act upon fraud and security threats for big data applications such as business trading, fraud detection, crime investigation, banking, insurance, enterprise security, government, healthcare, e-commerce, and telecommunications. Predictive analytics as a service (PAaaS) framework is proposed in our earlier work. One solution based upon ensemble model that uses Gaussian process with varying hyper-parameters is also given in our earlier works. Test results proved that the third hyper-parameter values yielded a good result with less error rate and more variance which is reliable for a predictive model. This paper presents solution based upon ensemble model that uses best out of prediction algorithms such as artificial neural networks (ANN), auto-regression algorithm (ARX) and Gaussian process (GP). Feature engineering methods such as recursive feature elimination that uses random forest algorithm is used for attribute selection. Performance measures NRMSE and COD are used to analyze the model. Test results proved that neural networks performed well when compared to regression and Gaussian process.

Bayesian optimization (BO) can accelerate materials discovery by guiding expensive experiments toward the most promising processing conditions. We systematically compare five BO surrogate and framework combinations (Gaussian processes in Ax, Gaussian processes and Monte-Carlo neural networks in BayBE, random forests in Lolopy, and tree-structured Parzen (TPE) estimators in Hyperopt) on three benchmarks that mimic common materials design tasks (a discrete solid-electrolyte composition space, a hybrid discrete/continuous laminate-composite design problem solved with micromechanics modeling, and the continuous Ishigami analytic function which is a standard optimization benchmark). Each BO surrogate is paired with posterior mean, probability of improvement, and expected improvement acquisition functions and run for 100 trials from randomized initial samples with uniform random search providing a control. Across five random seeds per setting, BayBE’s Gaussian-process surrogate with expected improvement consistently reached ≥95 % of the known optimum in the fewest evaluations, while Lolopy’s random forest matched or exceeded GP performance on purely categorical or mixed spaces at a higher computational cost. Posterior mean alone often stagnated at local optima, underscoring the need for exploration, whereas probability and expected improvement balanced exploration and exploitation leading to better optimization in fewer trials. Execution times ranged from milliseconds for TPE to minutes for neural-network and random-forest surrogates. These results establish baseline expectations for BO in automated materials laboratories and highlight expected improvement with Gaussian processes as a reliable first choice, with random forests offering a strong alternative when categorical variables dominate. The benchmark suite and code are released to facilitate future surrogate, acquisition, and constraint-handling research in data-driven materials optimization.

Prediction of Length-of-day Variations Based on Gaussian Processes

Pressure gradient (PG) in liquid-liquid flow is one of the key components to design an energy-efficient transportation system for wellbores. This study aims to develop five robust machine learning (ML) algorithms and their fusions for a wide range of flow patterns (FP) regimes. The MLs include Support Vector Machine (SVM), Gaussian Process (GP), Random Forest (RF), Artificial Neural Network (ANN), k-Nearest Neighbor (kNN), and fusions of these five MLs. A total of eleven hundred experimental data points for nine FPs (two stratified and seven dispersed patterns) in horizontal wellbores are used to develop the MLs. The MLs' performance is evaluated using the metrics including mean absolute percentage error (MAPE), median absolute percentage error (MdAPE), coefficient of variation of root mean squared error (CV-RMSE), and adjusted coefficient of determination. The evaluation metrics are cross-validated using a repeated train-test split strategy. Seven important predictor variables are identified using a supervised feature selection approach: oil and water velocities, FP, input diameter, oil and water density, and oil viscosity. The results show that the high PG prediction accuracy can be achieved using GP compared to other MLs except for the ML-fusions (p < 0.05). A Friedman's test and Wilcoxon Sign-Rank post hoc analysis with Bonferroni correction show that PG prediction errors using GP are significantly lower than using the ANN model (p < 0.05). The values are 18.44 % and 23.9 % for CV-RMSE, 11.6 % and 10.06 % for MAPE, and 7.5 % and 6.75 % for MdAPE, using ANN and GP, respectively. While the previous studies mostly used ANN to demonstrate the capability of MLs to predict PG over the mechanistic or correlation-based models, the present research has shown that GP is even better than ANN using a wide range of FPs and a large data set. • Machine-learning algorithms are developed to predict the pressure gradient. • The Gaussian Process model produces significantly lower prediction error. • The median absolute percentage error using Gaussian Process is 6.75 %. • More than 95 % variation of pressure gradient can be explained using seven variables.

This talk addresses the challenge of generating high-resolution wind climatology maps for Switzerland, a region characterized by sparse measurement stations and complex mountainous terrain. Accurately mapping wind patterns in such areas is inherently difficult due to the nonlinear and rapidly changing nature of wind flow, compounded by diverse and abrupt relief. Existing numerical models also lack the spatial granularity and temporal resolution necessary for comprehensive wind mapping.Our objective is twofold. Firstly, we aim to create more detailed wind maps that closely align with observations. Leveraging comprehensive orography maps, topographic descriptors, and numerical model outputs, we seek to downscale wind maps to achieve finer spatial resolution. Secondly, we intend to utilize these downscaled wind maps to compute wind climatology maps, focusing on maximum wind gusts and mean wind velocity, both hourly and daily. These wind climatology maps are invaluable for a plethora of stakeholders such as renewable energy planning, infrastructure development, and environmental monitoring, aiding in informed resource allocation and decision-making.We propose employing machine learning techniques, specifically Gaussian Processes (GPs) and Neural Processes (NPs), to address these challenges. Both present a compelling approach for wind map downscaling as both learn from small datasets and sparse observations to interpolate wind data and generalize to unseen locations. GPs offer a probabilistic framework to model complex relationships between inputs and outputs. By pairing sparse observations, high-resolution topographic descriptors and knowledge of the dynamics via a numerical model output, GPs can infer wind patterns across the territory with improved accuracy and spatial detail. Additionally, GPs inherently provide uncertainty estimates crucial for determining confidence in predictions. Finally, GPs rely on explicit prior knowledge and modeling assumptions, making their predictions interpretable. NPs, on the other hand, leverage neural networks to learn complex, non-linear relationships between inputs and outputs. Furthermore, NPs are scalable and highly efficient in handling large-scale datasets, and their model-less nature allow for more flexibility in modelling complex distributions.Integration of these machine learning techniques with domain-specific knowledge and data sources will enable the development of robust and accurate models for generating high-resolution wind climatology maps. These maps will closely align with observational data, advancing our understanding of local wind patterns and their impact on various applications. This approach aims to foster interdisciplinary collaboration and innovation in this field.Note: Drafting the initial version of this abstract has been aided by AI tools.

The computational limitations of complex numerical models have led to adoption of statistical emulators across a variety of problems in science and engineering disciplines to circumvent the high computational costs associated with numerical simulations. In flood modelling, many hydraulic and hydrodynamic numerical models, especially when operating at high spatiotemporal resolutions, have prohibitively high computational costs for tasks requiring the instantaneous generation of very large numbers of simulation results. This study examines the appropriateness and robustness of Gaussian Process (GP) models to emulate the results from a hydraulic inundation model. The developed GPs produce real-time predictions based on the simulation output from LISFLOOD-FP numerical model. An efficient dimensionality reduction scheme is developed to tackle the high dimensionality of the output space and is combined with the GPs to investigate the predictive performance of the proposed emulator for estimation of the inundation depth. The developed GP-based framework is capable of robust and straightforward quantification of the uncertainty associated with the predictions, without requiring additional model evaluations and simulations. Further, this study explores the computational advantages of using a GP-based emulator over alternative methodologies such as neural networks, by undertaking a comparative analysis. For the case study data presented in this paper, the GP model was found to accurately reproduce water depths and inundation extent by classification and produce computational speedups of approximately 10,000 times compared with the original simulator, and 80 times for a neural network-based emulator.

Given the increasing impact of aircraft noise for certification and commercialization purposes, a great effort has been made to decrease the acoustic emissions of newer aircraft. A great number of airports around the world limit noise emissions during the landing and takeoff phases, which contributes to increase the importance of controlling acoustic emissions of high-lift devices. Typically, simulations on aeroacustics are extremely expensive, because they involve fluid dynamics computa- tions in the transient regime. Not rarely, a single run of the computer code may require dozens of thousands of CPUs. Thus, direct optimization in aeroacoustics is often unfeasible for most of practical applications. A popular approach that has been currently adopted to overcome such difficulties involves creating a cheap-to- compute, yet accurate, surrogate (or metamodel) for the computation-intensive function on which optimization is to be performed. This work aims at using and analyzing the performance of various metamodeling techniques in slat noise simula- tions, carried out using a 2D discrete vortices method. Three classes of metamodels, were tested: (i) first and second order polynomial regressions, (ii) artificial neural networks, and (iii) Gaussian stochastic processes (GaSP). For the neural networks model, various models were created in order to verify which combinations of numbers of neurons and layers yielded better results. For the GaSP models, different covariance functions and base functions were tested. All metamodels were constructed based on data generated according to a Latin hypercube design with 28 points in a four-dimensional input space consisting in angle of attack, slat deflection angle and the two dimensions of slat position. The performance of each model was assessed with respect to the root mean square cross-validation prediction error. Results for the performed simulations indicated that the Gaussian stochastic process with the power expo- nential covariance function and a first order base function produced the best metamodel.

Analysis of geospatial data has traditionally been model-based, with a mean model, customarily specified as a linear regression on the covariates, and a Gaussian process covariance model, encoding the spatial dependence. While nonlinear machine learning algorithms like neural networks are increasingly being used for spatial analysis, current approaches depart from the model-based setup and cannot explicitly incorporate spatial covariance. We propose NN-GLS, embedding neural networks directly within the traditional Gaussian process (GP) geostatistical model to accommodate nonlinear mean functions while retaining all other advantages of GP, like explicit modeling of the spatial covariance and predicting at new locations via kriging. In NN-GLS, estimation of the neural network parameters for the nonlinear mean of the Gaussian Process explicitly accounts for the spatial covariance through use of the generalized least squares (GLS) loss, thus, extending the linear case. We show that NN-GLS admits a representation as a special type of graph neural network (GNN). This connection facilitates the use of standard neural network computational techniques for irregular geospatial data, enabling novel and scalable mini-batching, backpropagation, and kriging schemes. We provide methodology to obtain uncertainty bounds for estimation and predictions from NN-GLS. Theoretically, we show that NN-GLS will be consistent for irregularly observed spatially correlated data processes. We also provide a finite sample concentration rate, which quantifies the need to accurately model the spatial covariance in neural networks for dependent data. To our knowledge, these are the first large-sample results for any neural network algorithm for irregular spatial data. We demonstrate the methodology through numerous simulations and an application to air pollution modeling. We develop a software implementation of NN-GLS in the Python package geospaNN. Supplementary materials for this article are available online, including a standardized description of the materials available for reproducing the work.

Elevated temperatures limit the peak performance of systems because of frequent interventions by thermal throttling. Non-uniform thermal states across system nodes also cause performance variation within seemingly equivalent nodes leading to significant degradation of overall performance. In this paper we present a framework for creating a lightweight thermal prediction system suitable for run-time management decisions. We pursue two avenues to explore optimized lightweight thermal predictors. First, we use feature selection algorithms to improve the performance of previously designed machine learning methods. Second, we develop alternative methods using neural network and linear regression-based methods to perform a comprehensive comparative study of prediction methods. We show that our optimized models achieve improved performance with better prediction accuracy and lower overhead as compared with the Gaussian process model proposed previously. Specifically we present a reduced version of the Gaussian process model, a neural network–based model, and a linear regression–based model. Using the optimization methods, we are able to reduce the average prediction errors in the Gaussian process from $4.2^\circ$ C to $2.9^\circ$ C. We also show that the newly developed models using neural network and Lasso linear regression have average prediction errors of $2.9^\circ$ C and $3.8^\circ$ C respectively. The prediction overheads are 0.22, 0.097, and 0.026 ms per prediction for reduced Gaussian process, neural network, and Lasso linear regression models, respectively, compared with 0.57 ms per prediction for the previous Gaussian process model. We have implemented our proposed thermal prediction models on a two-node system configuration to help identify the optimal task placement. The task placement identified by the models reduces the average system temperature by up to $11.9^\circ$ C without any performance degradation. Furthermore, these models respectively achieve 75, 82.5, and 74.17 percent success rates in correctly pointing to those task placements with better thermal response, compared with 72.5 percent success for the original model in achieving the same objective. Finally, we extended our analysis to a 16-node system and we were able to train models and execute them in real time to guide task migration and achieve on average 17 percent reduction in the overall system cooling power.

We propose a theoretical understanding of neural networks in terms of Wilsonian effective field theory. The correspondence relies on the fact that many asymptotic neural networks are drawn from Gaussian processes (GPs), the analog of non-interacting field theories. Moving away from the asymptotic limit yields a non-Gaussian process (NGP) and corresponds to turning on particle interactions, allowing for the computation of correlation functions of neural network outputs with Feynman diagrams. Minimal NGP likelihoods are determined by the most relevant non-Gaussian terms, according to the flow in their coefficients induced by the Wilsonian renormalization group. This yields a direct connection between overparameterization and simplicity of neural network likelihoods. Whether the coefficients are constants or functions may be understood in terms of GP limit symmetries, as expected from ’t Hooft’s technical naturalness. General theoretical calculations are matched to neural network experiments in the simplest class of models allowing the correspondence. Our formalism is valid for any of the many architectures that becomes a GP in an asymptotic limit, a property preserved under certain types of training.

The Department of Energy relies on complex physics simulations for prediction in domains like cosmology, nuclear theory, and materials science. These simulations are often extremely computationally intensive, with some requiring days or weeks for a single simulation. In order to assure their accuracy, these models are calibrated against observational data in order to estimate inputs and systematic biases. Because of their great computational complexity, this process typically requires the construction of an emulator, a fast approximation to the simulation. In this paper, two emulator approaches are compared: Gaussian process regression and neural networks. Their emulation accuracy and calibration performance on three real problems of Department of Energy interest is considered. On these problems, the Gaussian process emulator tends to be more accurate with narrower, but still well‐calibrated uncertainty estimates. The neural network emulator is accurate, but tends to have large uncertainty on its predictions. As a result, calibration with the Gaussian process emulator produces more constrained posteriors that still perform well in prediction.

In recent years, artificial intelligence has been widely used in the field of meteorology. However, it is perhaps less known that AI-based tools have been used for over 30 years in the modeling of pollutant dispersion in the atmosphere.In the early 1990s, Slovenia and other parts of Europe faced significant air pollution problems caused by SO₂ emissions from coal-fired power plants without desulfurization systems. Due to the highly complex terrain, simple physical Gaussian dispersion models were not suitable for reconstructing the dispersion of SO₂ in the atmosphere, while complex numerical Lagrangian particle dispersion models were still in development.Therefore, as early as 1992, we modeled air pollution using a Multilayer Perceptron Artificial Neural Network (MLPANN). We developed the world's first comprehensive method for selecting training samples, features, and the topology of the neural network, enabling the model to learn from measured meteorological variables and pollutant concentrations around the power plant to predict SO₂ concentrations at a selected location in advance.This type of artificial neural network remains the foundation for various derived structures used for machine learning from data even today.In the following decades, we expanded the use of MLPANN to ozone prediction. Researchers from other Slovenian research groups also worked on this topic. Meanwhile, the use of these and similar tools in pollutant dispersion modelling began to grow on a global scale.Together with Brazilian researchers, we successfully applied MLPANN for predicting diffuse solar radiation. In this field, we also managed to develop a spatially transferable model, which is not a common capability for artificial neural network-based models.For the classification of wind fields based on measurements from ground-based meteorological stations, we used a Kohonen neural network.Later, we added another tool called "Gaussian processes" (which, like MLPANN, is a universal approximator) and the tool "decision trees." We expanded the use to point-based forecasting of basic meteorological variables. In recent years, with high computational capabilities available even without supercomputers, we have been using these tools for surrogate models that can represent or predict pollutant concentration fields in the atmosphere, not only for individual points but for entire areas around pollution sources.Other Slovenian groups from Slovenian Meteorological Agency, the Department of Meteorology, and the Faculty of Computer Science have, in recent years, used machine learning for global medium-range forecasts of daily averages of meteorological variables and for post-processing weather forecasts from physical models.In the presentation, we will explain the basic principles of building models based on artificial neural networks in a way that is understandable to laypeople. For experts in the field, we will showcase numerous examples of their application. For the latter group, we hope these examples will inspire others to expand the use of these modern tools even further.

Fault detection based on Gaussian process latent variable models