An adaptive sequential sampling method based on ANN_MCD and RF: Application in geotechnical problems

Reliability analysis of complex geotechnical engineering is time-consuming since its performance function is highly nonlinear and implicit. In this paper, an adaptive sequential sampling metamodeling-based method is proposed to deal with such problems. Gaussian process regression (GPR), utilized to approximate the real performance function, is constructed by the initial design of experiments (DOEs). Based on the geometric meaning of the most probable point (MPP) in the first-order reliability method (FORM), the potential MPP, which is a point infinitely close to the limit-state surface and with the minimum distance to the origin while considering a distance constraint, is searched and added to the DOE to refine the GPR. Then, the Monte Carlo simulation (MCS) is adopted to evaluate the failure probability by the refined GPR. The above two procedures are repeated until the stopping criterion is reached. Three examples, including one mathematical example and two geotechnical engineering problems, are analyzed. The results show the proposed method requires fewer performance function calls and is an efficient, accurate, and robust reliability analysis method.

This paper develops a new modified Bayesian Kriging (MBKG) surrogate modeling method for problems in which simulation analyses are inherently noisy and thus standard Kriging approaches fail to properly represent the responses. The purpose is to develop a method that can be used to carry out reliability analysis to predict probability of failure. The formulation of the MBKG surrogate modeling method is presented, and the full conditional distributions of the unknown MBKG parameters are presented. Using the full conditional distributions with a Gibbs sampling algorithm, Markov chain Monte Carlo is used to fit the MBKG surrogate model. A sequential sampling method that uses the posterior credible sets for inserting new design of experiment (DoE) sample points is proposed. The sequential sampling method is developed in such a way that the newly added DoE sample points will provide the maximum amount of information possible to the MBKG surrogate model, making it an efficient and effective way to reduce the number of DoE sample points needed. Therefore, the proposed method improves the posterior distribution of the probability of failure efficiently. To demonstrate the developed MBKG and sequential sampling methods, a 2-D mathematical example with added random noise is used. It is shown how, with the use of the sequential sample method, the posterior distribution of the probability of failure converges to capture the true probability of failure. A 3-D multibody dynamics (MBD) engineering block-car example illustrates an application of the new method to a simple engineering example for which standard Kriging methods fail.

Investigating adaptive sampling strategies for optimal building energy performance using artificial neural networks and kriging surrogate models

Surrogate modeling of phase equilibrium calculations using adaptive sampling

Surrogate SDOF models for probabilistic performance assessment of multistory buildings: Methodology and application for steel special moment frames

Surrogate models have been used to replace computationally expensive analysis models in engineering design problems. However, time-dependent variables and historical data are usually ignored in the surrogate modeling process. For instance, in a dam network design, using hydraulic simulations to estimate the water flow is computationally expensive, and the data is in the form of time series. So, we need time-dependent surrogate models to replace these simulations and manage this computational complexity. In this paper, we describe surrogate models to predict the amount of water flow into a reservoir. The challenge is that the flow is a time-dependent variable, and we need to incorporate time series into surrogate models. Thus, there are three contributions: (1) using surrogate modeling to predict flow for dam network design, (2) incorporating time series analysis in surrogate models for water network design, (3) using an ensemble of surrogates to increase the accuracy of prediction. We also demonstrate how to integrate surrogate models and machine learning with time series analysis for more accurate and faster prediction. Due to the availability of data, we use the Buffalo Reservoir in the Red River Basin as an example. Based on the time series data for flow, evaporation, precipitation, and maximum and minimum temperature, three surrogate models are used to examine the impact of integrating time series into surrogate models. These are multivariate autoregressive integrated moving average (MARIMA), a classic time series analysis method; artificial neural network (ANN), and random forest (RF) methods, two machine learning surrogate models. We use seven different time lags as features within an RF model, as an ensemble of surrogate models, and predict the flow for seven-time steps ahead. We successfully incorporate the time series data and particularly the concept of the time lag within surrogate models. We show that RF as the ensemble of surrogates provides more accurate predictions than the other two surrogate models. Although this method has been demonstrated for the Red River Basin, it could also be applied to designing anything in which time-dependent flow is an issue, for example, in biomedical applications, the management of manufacturing processes and product sales as well as any products in which fluid flow is an issue.

Slope reliability analysis has always been a research focus for geotechnical engineers. Currently, most studies use stationary random fields to simulate the spatial variability of soil parameters. However, many field test data show that soil parameters such as cohesion and friction angle often exhibit obvious nonstationary characteristics along the burial depth direction. Moreover, the slope reliability analysis method based on Monte Carlo simulation requires a combination of the FEM and a large number of computational resources, resulting in extremely low computational efficiency. Therefore, this paper proposes a convolutional neural network (CNN)-based surrogate model for slope reliability analysis, emphasizing the nonstationary spatial variability of cohesion and friction angle. First, stationary and nonstationary random fields considering the spatial variability of shear strength parameters are generated, and sample data sets are collected using an automated computational program. Next, a CNN surrogate model is established to predict the factor of safety (FOS), which is trained using nine sample data sets containing stationary and nonstationary random fields. Finally, the model performance is validated using six slope models with different shear strength parameters. The results show that the CNN surrogate model can adapt to the data characteristics of different random fields and also shows good predictive performance in handling unseen sample data sets. The predicted FOS values show a high degree of consistency with the results calculated by the FEM, with the computation time being significantly reduced.

The reliability analysis methods based on the Kriging model have been explored to update the design of experiments (DoE), but the number of calls to the performance function is high. In this paper, a new active learning method combining the weight information entropy function (WH) and the adaptive candidate sample pool is proposed to improve the reliability analysis efficiency. Based on the information entropy function (H), the learning function WH is constructed to consider Kriging variance and the joint probability density function (PDF). In this proposed method, the sample points can be assigned different weight by the learning function WH according to the important degrees of sample points. The point which not only nears the limit state function (LSF), but also has a high probability density function value and large Kriging variance is assigned more weight than others. To select the sample points with lower confidence level, the adaptive candidate sample pool is generated by Markov Chain Monte Carlo (MCMC) simulation. The Kriging model can be updated efficiently by the proposed method. Four numerical examples and an engineering example with implicit performance function are used to verify the efficiency and accuracy of the proposed method. The results show that the proposed method can significantly improve the computational efficiency of the reliability analysis without losing accuracy.

The reliable and efficient estimation of uncertainty in artificial intelligence (AI) models poses an ongoing challenge in many fields such as radiation therapy. AI models are intended to automate manual steps involved in the treatment planning workflow. We focus in this study on dose prediction models that predict an optimal dose trade-off for each new patient for a specific treatment modality. They can guide physicians in the optimization, be part of automatic treatment plan generation or support decision in treatmentindication. Most common uncertainty estimation methods are based on Bayesian approximations, like Monte Carlo dropout (MCDO) or Deep ensembling (DE). These two techniques, however, have a high inference time (i.e., require multiple inference passes) and might not work for detecting out-of-distribution (OOD) data (i.e., overlapping uncertainty estimate for in-distribution (ID) and OOD). In this study, we present a direct uncertainty estimation method and apply it for a dose prediction U-Net architecture. It can be used to flag OOD data and give information on the quality of the doseprediction. Our method consists in the addition of a branch decoding from the bottleneck which reconstructs the CT scan given as input. The input reconstruction error can be used as a surrogate of the model uncertainty. For the proof-of-concept, our method is applied to proton therapy dose prediction in head and neck cancer patients. A dataset of 60 oropharyngeal patients was used to train the network using a nested cross-validation approach with 11 folds (training: 50 patients, validation: 5 patients, test: 5 patients). For the OOD experiment, we used 10 extra patients with a different head and neck sub-location. Accuracy, time-gain, and OOD detection are analyzed for our method in this particular application and compared with the popular MCDO andDE. The additional branch did not reduce the accuracy of the dose prediction model. The median absolute error is close to zero for the target volumes and less than 1% of the dose prescription for organs at risk. Our input reconstruction method showed a higher Pearson correlation coefficient with the prediction error (0.620) than DE (0.447) and MCDO (between 0.599 and 0.612). Moreover, our method allows an easier identification of OOD (no overlap for ID and OOD data and a Z-score of 34.05). The uncertainty is estimated simultaneously to the regression task, therefore requires less time and computationalresources. This study shows that the error in the CT scan reconstruction can be used as a surrogate of the uncertainty of the model. The Pearson correlation coefficient with the dose prediction error is slightly higher than state-of-the-art techniques. OOD data can be more easily detected and the uncertainty metric is computed simultaneously to the regression task, therefore faster than MCDO or DE. The code and pretrained model are available on the gitlab repository: https://gitlab.com/ai4miro/ct-reconstruction-for-uncertainty-quatification-of-hdunet.

An improved adaptive sampling and experiment design method for aerodynamic optimization

Column leaching tests are a common approach for evaluating the leaching behavior of contaminated soil and waste materials, which are often reused for various construction purposes. The observed breakthrough curves of the contaminants are affected by the intricate dynamics of solute transport, inter-phase mass transfer, and dispersion. Disentangling these interactions requires numerical models. However, inverse modeling and parameter sensitivity analysis are often time-consuming, especially when sorption/desorption kinetics are explicitly described by intra-particle diffusion, requiring the discretization along the column axis and inside the grains. To replace such computationally expensive models, we developed a machine-learning based surrogate model employing two disparate ensemble methods (stacking and weighted distance average) within the defined parameter range based on the German standard for column leaching tests. To optimize the surrogate model, adaptive sampling methods based on three distinct infill criteria are employed. These criteria include maximizing expected improvement, the Mahalanobis distance (exploitation), and maximizing standard deviation (exploration).The stacking surrogate model makes use of extremely randomized trees and random forest as base- and meta-model. The model shows a very good performance in emulating the behavior of the original numerical model (Relative Root Mean Squared Error = 0.09). Our proposed surrogate model has been applied to estimate the complete posterior parameter distribution using Markov Chain Monte Carlo simulation. The impact of individual input parameters on the predictions generated by the surrogate model was analyzed using SHapley Additive exPlanations methods.

Objectives: This study aims to predict slope stability using machine learning algorithms, considering geotechnical and geometric variables to improve the accuracy of safety factor estimation and mitigate risks associated with slope failures. Theoretical Background: Traditional limit equilibrium methods, such as Bishop's method, present limitations when dealing with complex environmental conditions. Recent literature has highlighted the potential of machine learning techniques, such as Random Forest, Neural Networks, and XGBoost, in modeling nonlinear and highly variable systems, which are typical in geotechnical problems. Method: A parametric dataset with 1,520 observations generated through GeoSlope simulations was used, containing variables such as cohesion, internal friction angle, height, slope angle, and water level. Four algorithms were implemented: Random Forest, Gradient Boosting, Artificial Neural Networks, and XGBoost. Model performance was evaluated through cross-validation, hyperparameter tuning, and metrics including R², MSE, and RMSE. Results and Discussion: The XGBoost model achieved the best performance (R² = 0.9841; RMSE = 0.1419), followed by Gradient Boosting and Neural Networks. Variable importance analysis indicated that cohesion and internal friction angle are the primary factors influencing slope stability. Research Implications: The findings reinforce the applicability of machine learning models as decision-support tools in geotechnical engineering projects, offering greater precision and efficiency compared to traditional methods. Originality/Value: This study advances the field by integrating multiple modern algorithms within a replicable modeling framework, demonstrating superior performance over previous approaches and expanding the scope for practical applications in geotechnical engineering.

Bounds-based structure reliability analysis of bogie frame under variable load cases

This paper presents the development and evaluation of Artificial Neural Networks (ANN) based models and optimally selected surrogate models to provide the day-ahead forecast of the hourly-averaged energy load of buildings, by relating it to eight weather parameters as well as the hour of the day. Although ANN and other surrogate models have been used to predict building energy loads in the past, there is a limited understanding of what type of model prescriptions impact their performance as well as how un-recorded impact factors (e.g., human behavior and building repair work) should be accounted for. Here, the recorded energy data of three university buildings, from 9/2013–12/2015, is cleaned and synchronized with the local weather data. The data is then classified into eight classes; weekends and weekdays of Fall/Winter/Spring/Summer semesters. Both Multi-Layer Perceptron (MLP) and Radial Basis Function (RBF) NNs are explored. Differing number of hidden layers and transfer function choices are also explored, leading to the choice of the hyperbolic-tangent-sigmoid transfer function and 60 hidden layers. Similarly, an automated surrogate modeling framework is used to select the best models from among a pool of Kriging, RBF, and SVR models. A baseline concept, that uses energy information from the previous day as an added input to the ANN, helps to account for otherwise unrecorded recent changes in the building behavior, leading to improvement in fidelity of up to 30%.

In order to reduce the time and resources devoted to design-space exploration during simulation-based design and optimization, the use of surrogate models, or metamodels, has been proposed in the literature. Key to the success of metamodeling efforts are the experimental design techniques used to generate the combinations of input variables at which the computer experiments are conducted. Several adaptive sampling techniques have been proposed to tailor the experimental designs to the specific application at hand, using the already-acquired data to guide further exploration of the input space, instead of using a fixed sampling scheme defined a priori. Though mixed results have been reported, it has been argued that adaptive sampling techniques can be more efficient, yielding better surrogate models with less sampling points. In this paper, we address the problem of adaptive sampling for single and multi-response metamodels, with a focus on Multi-stage Multi-response Bayesian Surrogate Models (MMBSM). We compare distance-optimal latin hypercube sampling, an entropy-based criterion and the maximum cross-validation variance criterion, originally proposed for one-dimensional output spaces and implemented in this paper for multi-dimensional output spaces. Our results indicate that, both for single and multi-response surrogate models, the entropy-based adaptive sampling approach leads to models that are more robust to the initial experimental design and at least as accurate (or better) when compared with other sampling techniques using the same number of sampling points.