Automated Model Tuning for Multifidelity Uncertainty Propagation in Trajectory Simulation

History matching is a critical step within the reservoir management process to synchronize the simulation model with the production data. The history-matched model can be used for planning optimum field development and performing optimization and uncertainty quantifications. We present a novel history matching workflow based on a Bayesian framework that accommodates subsurface uncertainties. Our workflow involves three different model resolutions within the Bayesian framework: 1) a coarse low-fidelity model to update the prior range, 2) a fine low-fidelity model to represent the high-fidelity model, and 3) a high-fidelity model to re-construct the real response. The low-fidelity model is constructed by a multivariate polynomial function, while the high-fidelity model is based on the reservoir simulation model. We firstly develop a coarse low-fidelity model using a two-level Design of Experiment (DoE), which aims to provide a better prior. We secondly use Latin Hypercube Sampling (LHS) to construct the fine low-fidelity model to be deployed in the Bayesian runs, where we use the Metropolis-Hastings algorithm. Finally, the posterior is fed into the high-fidelity model to evaluate the matching quality. This work demonstrates the importance of including uncertainties in history matching. Bayesian provides a robust framework to allow uncertainty quantification within the reservoir history matching. Under uniform prior, the convergence of the Bayesian is very sensitive to the parameter ranges. When the solution is far from the mean of the parameter ranges, the Bayesian introduces bios and deviates from the observed data. Our results show that updating the prior from the coarse low-fidelity model accelerates the Bayesian convergence and improves the matching convergence. Bayesian requires a huge number of runs to produce an accurate posterior. Running the high-fidelity model multiple times is expensive. Our workflow tackles this problem by deploying a fine low-fidelity model to represent the high-fidelity model in the main runs. This fine low-fidelity model is fast to run, while it honors the physics and accuracy of the high-fidelity model. We also use ANOVA sensitivity analysis to measure the importance of each parameter. The ranking gives awareness to the significant ones that may contribute to the matching accuracy. We demonstrate our workflow for a geothermal reservoir with static and operational uncertainties. Our workflow produces accurate matching of thermal recovery factor and produced-enthalpy rate with physically-consistent posteriors. We present a novel workflow to account for uncertainty in reservoir history matching involving multi-resolution interaction. The proposed method is generic and can be readily applied within existing history-matching workflows in reservoir simulation.

Aim/Purpose: Traditional hyperparameter tuning methods’ inefficiency and low accuracy in diabetes prediction. Application and evaluation of Bayesian optimization aim to increase the accuracy of diabetes prediction models. Background: Bayesian optimization (BO) has emerged as a powerful technique for hyperparameter tuning in machine learning models with its methodical approach for optimizing complex functions with expensive evaluations. Selecting the optimal hyperparameters for models can significantly increase prediction accuracy in the healthcare sector, particularly for diabetes prediction, hence improving patient outcomes and resource management. Methodology: Among the well-known diabetes databases used in the study is the Pima Indian Diabetes Database. Among the machine learning models developed and whose hyperparameters are modified via Bayesian optimization are Support Vector Machines (SVM), Random Forest, and Gradient Boosting Machines (GBM). The optimization process is compared with traditional tuning methods to assess improvements in model performance. Contribution: Problems include the increasing prevalence of diabetes worldwide and the role vital function prediction models play in early diagnosis and therapy. Especially when dealing with enormous datasets common in healthcare, grid search, and random search are sometimes computationally taxing and inefficient. However, Bayesian optimization promises a more practical and economical approach by selecting hyperparameters iteratively based on earlier evaluations. Findings: Bayesian optimization not only yields faster calculation times but also outperforms traditional methods. More precisely, models adapted by Bayesian optimization show greater sensitivity and specificity, which are crucial for accurate and prompt diabetes diagnosis. Recommendation for Researchers: By highlighting the specific elements and network characteristics that are behind the anomalies it discovers, on a real-time or large scale dataset. Future Research: This work can be improvised using several recent artificial intelligence algorithms with the integration of IoT on real-time datasets.

Hyperparameter optimization (HPO) has been well-developed and evolved into a well-established research topic over the decades. With the success and wide application of deep learning, HPO has garnered increased attention, particularly within the realm of machine learning model training and inference. The primary objective is to mitigate the challenges associated with manual hyperparameter tuning, which can be ad-hoc, reliant on human expertise, and consequently hinders reproducibility while inflating deployment costs. Recognizing the growing significance of HPO, this paper surveyed classical HPO methods, approaches for accelerating the optimization process, HPO in an online setting (dynamic algorithm configuration, DAC), and when there is more than one objective to optimize (multi-objective HPO). Acceleration strategies were categorized into multi-fidelity, bandit-based, and early stopping; DAC algorithms encompassed gradient-based, population-based, and reinforcement learning-based methods; multi-objective HPO can be approached via scalarization, metaheuristics, and model-based algorithms tailored for multi-objective situation. A tabulated overview of popular frameworks and tools for HPO was provided, catering to the interests of practitioners.

Predicting Prostate Radiotherapy Toxicity in a Phase III Trial Using Clinical/Dosimetric Variables in Logistic Regression and Random Forest Models

CNN has gained great success in many applications but the major design hurdles for deploying CNN on driver assistance systems or ADAS are limited computation, memory resource, and power budget. Recently, there has been greater exploration into small DNN architectures, such as SqueezeNet and SqueezeNext architectures. In this paper, the proposed Shallow SqueezeNext architecture for driver assistance systems achieves better model size with a good model accuracy and speed in comparison to baseline SqueezeNet and SqueezeNext architectures. The proposed architecture is compact, efficient and flexible in terms of model size and accuracy with minimum tradeoffs and less penalty. The proposed Shallow SqueezeNext uses SqueezeNext architecture as its motivation and foundation. The proposed architecture is developed with intention for implementation or deployment on a real-time autonomous system platform and to keep the model size less than 5 MB. Due to its extremely small model size, 0.370 MB with a competitive model accuracy of 82.44 %, decent both training and testing model speed of 7 seconds, it can be successfully deployed on ADAS, driver assistance systems or a real time autonomous system platform such as BlueBox2.0 by NXP. The proposed Shallow SqueezeNext architecture is trained and tested from scratch on CIFAR-10 dataset for developing a dataset specific trained model.

From wearables to powerful smart devices, modern automatic speech recognition (ASR) models run on a variety of edge devices with different computational budgets. To navigate the Pareto front of model accuracy vs model size, researchers are trapped in a dilemma of optimizing model accuracy by training and fine-tuning models for each individual edge device while keeping the training GPU-hours tractable. In this paper, we propose Omni-sparsity DNN, where a single neural network can be pruned to generate optimized model for a large range of model sizes. We develop training strategies for Omni-sparsity DNN that allows it to find models along the Pareto front of word-error-rate (WER) vs model size while keeping the training GPU-hours to no more than that of training one singular model. We demonstrate the Omni-sparsity DNN with streaming E2E ASR models. Our results show great saving on training time and resources with similar or better accuracy on LibriSpeech compared to individually pruned sparse models: 2%-6.6% better WER on Test-other.

Hyperparameter tuning is crucial for optimising deep learning models. Traditional methods like grid search and random search can be time consuming and computationally expensive. This study explores space filling methodology for hyperparameter tuning, aiming to explore the hyperparameter space for optimal configurations. Hyperparameters are tuned for ResNet models using CIFAR-10 dataset and compared MaxPro space filling methodology with randomised search and grid search in terms of accuracy and computational costs. A greater accuracy was observed for the hyperparameters selected based on space filling methodology as compared to the randomised search while the runtime is similar for both. By systematically exploring the hyperparameter space, space filling offers an efficient approach to hyperparameter tuning, enabling faster model development. The findings underscore space filling methodology's potential for effective hyperparameter tuning, especially in resource constrained scenarios.

The travel time distribution (TTD) is a lumped representation of groundwater discharge and solute export responding to rainfall. It reflects the mixing process of water parcels and solute particles of different ages and characterizes reactive transport progress in hillslope aquifers. As a result of the mixing process, groundwater leaving the system at a certain time is an integration of multiple water parcels of different ages from different historical rainfall events. Under nonstationary rainfall input condition, the TTD varies with transit groundwater flow, leading to the time-variant TTD. Most methods for estimating time-variant TTD are constrained by requiring either the long-term continuous hydrogeochemical data or the intensive computations. This study introduces a multi-fidelity model to overcome these limitations and evaluate time-variant TTD numerically. In this multi-fidelity model, groundwater age distribution model is taken as the high-fidelity model, and particle tracking model without random walk is taken as the low-fidelity model. Non-parametric regression by non-linear Gaussian process is applied to correlate the two models and then build up the multi-fidelity model. The advantage of the multi-fidelity model is that it combines the accuracy of high-fidelity model and the computational efficiency of low-fidelity model. Moreover, in groundwater and solute transport model with low P\'eclet number, as the spatial scale of the model increases, the number of particles required for multi-fidelity model is reduced significantly compared to random walk particle tracking model. The correlation between high and low-fidelity models is demonstrated in a one dimensional pulse injection case. In a two dimensional hypothetical model, convergence analysis indicates that the multi-fidelity model converges well when increasing the number of high-fidelity models. Error analysis also confirms the good performance of the multi-fidelity model.

We address the problem of expensive multi-objective optimization using a machine learning assisted model of evolutionary computation. Specifically, we formulate a meta-objective function tailored to the framework of MOEA/D, which can be solved by means of supervised regression learning using the Support Vector Machine (SVM) algorithm. The learned model constitutes the knowledge which can be then utilized to guide the evolution process within MOEA/D so as to reach better regions in the search space more quickly. Simulation results on a variety of benchmark problems show that the machine-learning enhanced MOEA/D is able to obtain better estimation of Pareto fronts when the allowed computational budget, measured in terms of number of objective function evaluation, is scarce.KeywordsEvolutionary multi-objective optimizationsupport vector machineexpensive optimizationsurrogate modelling

Remote estimation of chlorophyll concentration in hyper-eutrophic aquatic systems: Model tuning and accuracy optimization

The present work reports an investigation on the use of adaptive metamodels based on radial basis functions (RBFs) for aeroacoustic applications of highly innovative configurations. The relevance of the topic lies on the paramount importance of metamodelling techniques within the design optimisation process of disruptive aircraft layouts. Indeed, the air traffic growth, consequently the hard environmental constraints imposed by regulations, will make a technological breakthrough, an imperative need within few years. As a consequence, the engineering community is paying particular attention to the development of innovative techniques for the design of unconventional configurations. For this class of applications, the designer cannot successfully rely on historical data or low-fidelity models, and the expensive direct simulations remain the only valuable design strategy. In this regard, it can be demonstrated that the use of surrogate models, i.e., metamodels, significantly reduces the computing costs, especially in view of a robust approach to the optimised design. In order to further improve the efficiency of metamodel-based techniques, dynamic approaches based on hyperparameter optimisation and adaptive sampling procedures have been recently developed. The case study presented here pertains the exploiting of dynamic RBF-based metamodels for noise shielding applications. The analysis of the metamodel performances and its convergence properties shows how the final number of direct simulations is significantly reduced by the hyperparameter optimisation algorithm, still strongly depending on the choice of the RBF kernel function.

Multi-fidelity surrogate modeling (MFSM) methods are gaining recognition for their effectiveness in addressing simulation-based design challenges. Prior approaches have typically relied on recursive techniques, combining a limited number of high-fidelity (HF) samples with multiple low-fidelity (LF) datasets structured in hierarchical levels to generate a precise HF approximation model. However, challenges arise when dealing with non-level LF datasets, where the fidelity levels of LF models are indistinguishable across the design space. In such cases, conventional methods employing recursive frameworks may lead to inefficient LF dataset utilization and substantial computational costs. To address these challenges, this work proposes the extended hierarchical Kriging (EHK) method, designed to simultaneously incorporate multiple non-level LF datasets for improved HF model construction, regardless of minor differences in fidelity levels. This method leverages a unique Bayesian-based MFSM framework, simultaneously combining non-level LF models using scaling factors to construct a global trend model. During model processing, unknown scaling factors are implicitly estimated through hyperparameter optimization, resulting in minimal computational costs during model processing, regardless of the number of LF datasets integrated, while maintaining the necessary accuracy in the resulting HF model. The advantages of the proposed EHK method are validated against state-of-the-art MFSM methods through various analytical examples and an engineering case study involving the construction of an aerodynamic database for the KP-2 eVTOL aircraft under various flying conditions. The results demonstrated the superiority of the proposed method in terms of computational cost and accuracy when generating aerodynamic models from the given multi-fidelity datasets.

Multilevel and multifidelity uncertainty quantification for cardiovascular hemodynamics

Recently, renewable energy (RE) has become popular due to its benefits, such as being inexpensive, low-carbon, ecologically friendly, steady, and reliable. The RE sources are gradually combined with non-renewable energy (NRE) sources into electric grids to satisfy energy demands. Since energy utilization is highly related to national energy policy, energy prediction using artificial intelligence (AI) and deep learning (DL) based models can be employed for energy prediction on RE and NRE power resources. Predicting energy consumption of RE and NRE sources using effective models becomes necessary. With this motivation, this study presents a new multimodal fusion-based predictive tool for energy consumption prediction (MDLFM-ECP) of RE and NRE power sources. Actual data may influence the prediction performance of the results in prediction approaches. The proposed MDLFM-ECP technique involves pre-processing, fusion-based prediction, and hyperparameter optimization. In addition, the MDLFM-ECP technique involves the fusion of four deep learning (DL) models, namely long short-term memory (LSTM), bidirectional LSTM (Bi-LSTM), deep belief network (DBN), and gated recurrent unit (GRU). Moreover, the chaotic cat swarm optimization (CCSO) algorithm is applied to tune the hyperparameters of the DL models. The design of the CCSO algorithm for optimal hyperparameter tuning of the DL models, showing the novelty of the work. A series of simulations took place to validate the superior performance of the proposed method, and the simulation outcome emphasized the improved results of the MDLFM-ECP technique over the recent approaches with minimum overall mean absolute percentage error of 3.58%.

To study the mechanical deformation characteristics and anti-explosion mechanisms of steel-structure protective doors under chemical explosion shock wave loads, numerical simulations of loads and door damage were carried out using the AUTODYN and LS-DYNA software based on model tuning with actual field test results. The finite element simulation results were compared with the test results to verify the accuracy of the simulation model and material parameters. A parametric analysis was carried out on the influencing factors of the anti-explosion performance of the beam–plate steel structure protective door under typical shock wave loads. The impact of the material strength and geometry of each part of the protective door on its anti-explosion performance was studied. The results showed that the protective door sustained a uniform shock wave load and that increasing the steel strength of the skeleton could significantly reduce the maximum response displacement of the protective door. The steel strength increase of the inner and outer panels had little or a negligible effect on the anti-explosion performance of the protective door. The geometric dimensions of different parts of the protective door had different effects on the anti-explosion performance. Increasing the skeleton height had the most significant effect on the anti-explosion performance. The skeleton’s I-steel flange thickness and the inner and outer panel thicknesses had less significant effects.