Data-driven torque identification of turboprop engines using optimized feedforward neural networks

Single-hidden-layer feed-forward quantum neural network based on Grover learning

The water distribution network is one of the most pivotal services for communities, and that’s why its effective and secure operation is crucial to the growth of national and global economies. To this end, the paramount objective of this study is to devise an automated self-adaptive model for deterioration prediction of saltwater pipes. The developed model is envisioned on coupling deep learning network with Bayesian optimization (HBO-DL) for forecasting the condition of different material categories of saltwater pipes stepping on their pip-related, soil-related, operational-related, and environmental-related features. In this regard, Bayesian optimization is leveraged to amplify the training mechanism of deep learning neural network through iterative optimization of its hyper parameters. The developed model is validated through several folds of validation that encompass performance evaluation, statistical analysis, graphical comparison, and unified ranking. The conducted comparative analysis evinced that the developed HBO-DL model managed to significantly perform better than feed forward neural network, support vector machines and Gaussian process regression by 76.85%, 73.31% and 79.08%, respectively. The developed prediction model can stand as a practical and useful tool to forecast the condition performance of water networks which aids municipalities in designing optimum intervention plans evading socioeconomic losses elicited from pipe failures.

Abstarct Engineering design problems typically require optimizing a quality measure by finding the right combination of controllable input parameters. In Additive Manufacturing (AM), the output characteristics of the process can often be non-stationary functions of the process parameters. Bayesian Optimization (BO) is a methodology to optimize such “black-box” functions, i.e., the input–output relationship is unknown and expensive to compute. Optimization tasks involving “black-box” functions widely use BO with Gaussian Process (GP) regression surrogate model. Using GPs with standard kernels is insufficient for modeling non-stationary functions, while GPs with non-stationary kernels are typically over-parameterized. On the other hand, a Deep Gaussian Process (DGP) can overcome GPs’ shortcomings by considering a composition of multiple GPs. Inference in a DGP is challenging due to its structure resulting in a non-Gaussian posterior, and using DGP as a surrogate model for BO is not straightforward. Stochastic Imputation (SI)-based inference is promising in speed and accuracy for BO. This work proposes a bootstrap aggregation-based procedure to effectively utilize the SI-based inference for BO with a DGP surrogate model. The proposed BO algorithm DGP-SI-BO is faster and empirically better than the state-of-the-art BO method in optimizing non-stationary functions. Several analytical test functions and a case study in metal AM simulation demonstrate the applicability of the proposed method.

Hyper-Parameter Optimization (HPO) occupies a fundamental role in Deep Learning systems due to the number of hyper-parameters (HPs) to be set. The state-of-the-art of HPO methods are Grid Search, Random Search and Bayesian Optimization. The first two methods try all possible combinations and random combination of the HPs values, respectively. This is performed in a blind manner, without any information for choosing the new set of HPs values. Bayesian Optimization (BO), instead, keeps track of past results and uses them to build a probabilistic model mapping HPs into a probability density of the objective function. Bayesian Optimization builds a surrogate probabilistic model of the objective function, finds the HPs values that perform best on the surrogate model and updates it with new results. In this paper, we improve BO applied to Deep Neural Network (DNN) by adding an analysis of the results of the network on training and validation sets. This analysis is performed by exploiting rule-based programming, and in particular by using Probabilistic Logic Programming. The resulting system, called Symbolic DNN-Tuner, logically evaluates the results obtained from the training and the validation phase and, by applying symbolic tuning rules, fixes the network architecture, and its HPs, therefore improving performance. We also show the effectiveness of the proposed approach, by an experimental evaluation on literature and real-life datasets.

Liquid-cooled heat sink design for a multilevel inverter switch with considerations for heat spreading and manufacturability

We develop a probabilistic model for determining the location of dc-link faults in MT-HVdc networks using discrete wavelet transforms (DWTs), Bayesian optimization, and multilayer artificial neural networks (ANNs) based on local information. Likewise, feedforward neural networks (FFNNs) are trained using the Levenberg-Marquardt backpropagation (LMBP) method, which multi-stage BO optimizes for efficiency. During training, the feature vectors at the sending terminal of the dc link are selected based on the norm values of the observed waveforms at various frequency bands. The multilayer ANN is trained using a comprehensive set of offline data that takes the denoising scheme into account. This choice not only helps to reduce the computational load but also provides better accuracy. An overall percentage error of 0.5144% is observed for the proposed algorithm when tested against fault resistances ranging from 10 to 485 Ω. The simulation results show that the proposed method can accurately estimate the fault site to a precision of 485 Ω and is more robust.

We use experimental and simulation data to recalibrate the standard Spalart–Allmaras model. Free-shear flow, the buffer layer, the log layer, and flows with adverse pressure gradients are targeted. In this process, the recalibration does not affect untargeted flows. Our approach uses Bayesian optimization and feedforward neural networks. The recalibrated model is implemented in two codes and tested in 11 flows: the majority of which are outside the training dataset and have geometries that are distinctly different from those in the training dataset. We show that the data-enabled recalibration offers improvements while preserving the model’s existing good behavior. In particular, our recalibration improves the model’s behavior in separated flows while preserving its existing good behaviors in flat-plate boundary-layer flows and channel flows. Further analysis indicates that the improvements in separated flow are mainly due to the recalibrated function and the resulting, more precise representation of the “slingshot” effect.

Bayesian optimization (BO) has demonstrated potential for optimizing control performance in data-limited settings, especially for systems with unknown dynamics or unmodeled performance objectives. The BO algorithm efficiently trades-off exploration and exploitation by leveraging uncertainty estimates using surrogate models. These surrogates are usually learned using data collected from the target dynamical system to be optimized. Intuitively, the convergence rate of BO is better for surrogate models that can accurately predict the target system performance. In classical BO, initial surrogate models are constructed using very limited data points, and therefore rarely yield accurate predictions of system performance. In this paper, we propose the use of meta-learning to generate an initial surrogate model based on data collected from performance optimization tasks performed on a variety of systems that are different to the target system. To this end, we employ deep kernel networks (DKNs) which are simple to train and which comprise encoded Gaussian process models that integrate seamlessly with classical BO. The effectiveness of our proposed DKN-BO approach for speeding up control system performance optimization is demonstrated using a well-studied nonlinear system with unknown dynamics and an unmodeled performance function.

Previous studies have suggested the use of reduced-order models calibrated by means of high-fidelity load simulations as means for computationally inexpensive wind turbine load assessments; the so far best performing surrogate modelling approach in terms of balance between accuracy and computational cost has been the polynomial chaos expansion (PCE). Regarding the growing interest in advanced machine learning applications, the potential of using Artificial Neural-Network (ANN) based surrogate models for improved simplified load assessment is investigated in this study. Different ANN model architectures have been evaluated and compared to other types of surrogate models (PCE and quadratic response surface). The results show that a feedforward neural network with two hidden layers and 11 neurons per layer, trained with the Levenberg Marquardt backpropagation algorithm is able to estimate blade root flapwise damage-equivalent loads (DEL) more accurately and faster than a PCE trained on the same data set. Further research will focus on further model improvements by applying different training techniques, as well as expanding the work with more load components.

There is a wave of interest in using physics-informed neural networks for solving differential equations. Most of the existing methods are based on feed-forward networks, while recurrent neural networks solvers have not been extensively explored. We introduce a reservoir computing (RC) architecture, an echo-state recurrent neural network capable of discovering approximate solutions that satisfy ordinary differential equations (ODEs). We suggest an approach to calculate time derivatives of recurrent neural network outputs without using back-propagation. The internal weights of an RC are fixed, while only a linear output layer is trained, yielding efficient training. However, RC performance strongly depends on finding the optimal hyper-parameters, which is a computationally expensive process. We use Bayesian optimization to discover optimal sets in a high-dimensional hyper-parameter space efficiently and numerically show that one set is robust and can be transferred to solve an ODE for different initial conditions and time ranges. A closed-form formula for the optimal output weights is derived to solve first-order linear equations in a one-shot backpropagation-free learning process. We extend the RC approach by solving nonlinear systems of ODEs using a hybrid optimization method consisting of gradient descent and Bayesian optimization. Evaluation of linear and nonlinear systems of equations demonstrates the efficiency of the RC ODE solver.

Non-intrusive surrogate modeling for parametrized time-dependent partial differential equations using convolutional autoencoders

The aim of this endeavor is to apply artificial intelligence to provide original solutions for very complicated and challenging physical model. The results of chemical reaction involving Marangoni convection, gyrotactic microbes and thermo-bioconvection in water-based THNF (trihybrid nanofluid) flow across a sheet with heat source, thermal radiation, thermophoretic particle deposition is assessed in this work using a feed-forward neural network in conjunction with back-propagation Bayesian regularization optimization (FFNN-BPBRO). The suggested Bayesian-Regularized Neural Network model for assessing mass and heat transportation in ternary nanofluids including motile microorganisms can be used to improve thermal performance in biomedical systems, microfluidic devices, energy storage, and cooling technologies. It also offers important insights for wastewater treatment, bio-convective transport, and nanofluid-based industrial processes, where precise prediction and thermodynamic optimization are critical for increasing efficiency, stability, and sustainability. As the Peclet number increases, the microorganism profile decreases.

A neural network–based method is developed to fast optimize EMC3-EIRENE input parameters, enabling EMC3-EIRENE to produce synthetic data that closely match experimental measurements on Wendelstein 7-X (W7-X). Initially, an EMC3-EIRENE simulation database covering a range of key input parameters is generated. Trained on this database, a feed-forward neural network (FNN) surrogate model efficiently maps EMC3-EIRENE input parameters to synthetic signals corresponding to experimentally observed physical quantities. Subsequently, the trained surrogate model is incorporated into a Bayesian inference framework with Dynamic Nested Sampling to infer posterior distributions of the EMC3-EIRENE input parameters. In this step, the FNN-predicted synthetic data are compared with the experimental data, and the likelihood function explicitly accounts for the measurement uncertainties of the selected diagnostics. EMC3-EIRENE simulations using the maximum a posteriori (MAP) estimates derived from these posterior distributions reproduce experimental measurements with satisfactory accuracy. This neural network-based method significantly reduces computational costs and the need for manual parameter tuning, and it can be generalized to other similar modeling codes.

Bayesian Optimization Analysis of Containment-Venting Operation in a Boiling Water Reactor Severe Accident

Activity-based transportation models simulate demand and supply as a complex system and therefore large set of parameters need to be adjusted. One such model is Preday activity-based model that requires adjusting a large set of parameters for its calibration on new urban environments. Hence, the calibration process is time demanding, and due to costly simulations, various optimisation methods with dimensionality reduction and stochastic approximation are adopted. This study adopts Bayesian Optimisation for Likelihood-free Inference (BOLFI) method for calibrating the Preday activity-based model to a new urban area. Unlike the traditional variant of the method that uses Gaussian Process as a surrogate model for approximating the likelihood function through modelling discrepancy, we apply a composite surrogate model that encompasses Random Forest surrogate model for modelling the discrepancy and Gaussian Mixture Model for estimating the its density. The results show that the proposed method benefits the extension and improves the general applicability to high-dimensional settings without losing the efficiency of the Bayesian Optimisation in sampling new samples towards the global optima.