Information-Theoretic Bayesian Active Learning for Surrogate Training and Inverse Modeling in Subsurface Transport Applications

We propose the Bayesian Active Learning by Disagreement (BALD) model for keyword spotting in handwritten documents. In the context of keyword spotting in handwritten documents, the background text is all regions in the document that do not contain the keywords. The model tries to learn certain characteristics of the keyword and background text in an active learning framework. It takes into account the local character level scores and global word level scores to distinguish keywords from non-keywords. We propose to apply the bayesian active learning strategy to identify the regions of sample space from which more meaningful labeled samples of keywords and non-keywords can be extracted. This work is an extension to our previous work which used a variational dynamic background model to model the large variations of background text. The approach has been tested on IAM dataset for English. The results show that a decent background model can be learned in a more quicker and efficient manner using the BALD framework. The approach outperforms our prior work and other state of the art approaches.

Complex step-based low-rank extended Kalman filtering for state-parameter estimation in subsurface transport models

Limited experimental data remains a key challenge in applying machine learning to drug discovery, particularly for cancer-related targets. In this study, we present a data-efficient active meta-deep learning framework to predict mitogen-activated protein kinase 1 (MAPK1) inhibitors, which are promising candidates for cancer-related therapies. Our approach integrates active learning (AL) with a meta-model that combines four deep architectures: a convolutional neural network, an attention, a graph convolutional network, and a graph neural network-attention, trained on molecular descriptors and graph-based representations. These models generate four probability-based features that feed into an attention-based meta-learner, improving predictive performance by 5.12% in the area under the precision-recall curve (AUPRC) and 5.48% in the Matthews correlation coefficient (MCC) using only 10% of the training data. Among the AL sampling strategies evaluated, entropy sampling showed competitive performance in selecting informative molecules for model improvement. Overall, our framework achieves an AUPRC of 0.835 ± 0.017 and MCC of 0.817 ± 0.017, on par with a traditional training method despite using only 26.7% of the training data. Compared to a conventional random forest model trained on brute-force, a 100% full training set, our approach shows a 10.6% improvement in AUPRC and modest gains in MCC, confirming the effectiveness of the proposed framework. Under severe class imbalance, balanced accuracy steadily increased across AL iterations, reaching values greater than 0.85 at the final iteration for all uncertainty-driven strategies. Molecular docking confirmed successful prioritization of the top four predicted compounds. Evaluation on an external MAPK1 data set demonstrated generalizability, with our approach achieving an AUPRC of 0.818 and an MCC of 0.403, comparable to the independent test set. These results highlight the potential of combining intelligent data selection with deep learning architectures through the meta-model to accelerate predictive performance in data-scarce drug discovery. Scientific contribution: This study contributes a novel, data-efficient active meta-deep learning framework for predicting MAPK1 inhibitors, addressing the challenge of limited experimental data in a cancer-specific target. By integrating AL with a meta-model composed of four deep architectures, the approach significantly enhances the predictive performance using only a fraction of the training data. The framework achieves superior metrics compared to traditional training methods, highlighting its potential to accelerate drug discovery in data-scarce settings.

We consider modeling approaches to characterize solute transport in porous media, integrating them into a unique theoretical and experimental framework for model evaluation and data interpretation. To date, development of (conservative and reactive chemical) transport models and formulation of model calibration methods grounded on sensitivity‐based collection of measurements have been pursued in parallel. Key questions that remain include: For a given set of measurements, which conceptual picture of the transport processes, as embodied in a mathematical model or models, is most appropriate? What are the most valuable space‐time locations for solute concentration measurements, depending on the model selected? How is model parameter uncertainty propagated to model output, and how does this propagation affect model calibration? We address these questions by merging parallel streams of research—model formulation, reduction, calibration, sensitivity analysis, and discrimination—offering our view on an emerging framework that guides (i) selection of an appropriate number and location of time‐dependent concentration measurements given a transport model and (ii) assessment (through discrimination criteria) of the relative benefit of applying any particular model from a set of several models. Our strategy is to employ metrics to quantify the relative contribution of each uncertain model parameter to the variability of the model output. We evaluate these metrics through construction of a surrogate (or “meta”) transport model that has the additional benefit of enabling sensitivity analysis and model calibration at a highly reduced computational cost. We demonstrate the applicability of this framework, focusing on transport of reactive chemicals in laboratory‐scale porous media.

This effort develops a surrogate modeling approach for predicting the effects of manufacturing variations on blade unsteadiness and performance of a transonic turbine. CFD results from a set of 105 as-manufactured turbine blade geometries are used to train and validate the surrogate models. Blade geometry variation is characterized with point clouds created from a structured light optical measurement system and as-measured CFD grids are generated through mesh morphing of the nominal design grid data. Results from a Reynolds-averaged Navier-Stokes flow solver with the two-equation Wilcox turbulence model are used as training and validation data. Principal Component Analysis (PCA) of the measured airfoil geometry variations is used to create reduced basis of independent surrogate model parameters. Results of the surrogate are compared to the CFD results. It is shown that the surrogate model typically captures between 60% and 80% of the full CFD predicted variance. Three new approaches are introduced to improve the accuracy of the surrogate. A zonal PCA approach is defined which improves surrogate accuracy by focusing on key regions of the airfoil. A training point reduction strategy is proposed that is based on the kd-tree nearest neighbor search algorithm and reduces the required training points by 25%. A second reduction approach uses k-means clustering to effectively select training points from the 105 blade population and is used to reduce the required training points by up to 66%.

This effort develops a surrogate modeling approach for predicting the effects of manufacturing variations on performance and unsteady loading of a transonic turbine. Computational fluid dynamics (CFD) results from a set of 105 as-manufactured turbine blade geometries are used to train and validate the surrogate models. Blade geometry variation is characterized with point clouds gathered from a structured light, optical measurement system and as-measured CFD grids are generated through mesh morphing of the nominal design grid data. Principal component analysis (PCA) of the measured airfoil geometry variations is used to create a reduced basis of independent surrogate model parameters. It is shown that the surrogate model typically captures between 60% and 80% of the CFD predicted variance. Three new approaches are introduced to improve surrogate effectiveness. First, a zonal PCA approach is defined which investigates surrogate accuracy when limiting analysis to key regions of the airfoil. Second, a training point reduction strategy is proposed that is based on the k–d tree nearest neighbor search algorithm and reduces the required training points up to 38% while only having a small impact on accuracy. Finally, an alternate reduction approach uses k-means clustering to effectively select training points and reduces the required training points up to 66% with a small impact on accuracy.

Background Mining high-throughput screening (HTS) assays is key for enhancing decisions in the area of drug repositioning and drug discovery. However, many challenges are encountered in the process of developing suitable and accurate methods for extracting useful information from these assays. Virtual screening and a wide variety of databases, methods and solutions proposed to-date, did not completely overcome these challenges. This study is based on a multi-label classification (MLC) technique for modeling correlations between several HTS assays, meaning that a single prediction represents a subset of assigned correlated labels instead of one label. Thus, the devised method provides an increased probability for more accurate predictions of compounds that were not tested in particular assays. Results Here we present DRABAL, a novel MLC solution that incorporates structure learning of a Bayesian network as a step to model dependency between the HTS assays. In this study, DRABAL was used to process more than 1.4 million interactions of over 400,000 compounds and analyze the existing relationships between five large HTS assays from the PubChem BioAssay Database. Compared to different MLC methods, DRABAL significantly improves the F1Score by about 22%, on average. We further illustrated usefulness and utility of DRABAL through screening FDA approved drugs and reported ones that have a high probability to interact with several targets, thus enabling drug-multi-target repositioning. Specifically DRABAL suggests the Thiabendazole drug as a common activator of the NCP1 and Rab-9A proteins, both of which are designed to identify treatment modalities for the Niemannâ Pick type C disease. Conclusion We developed a novel MLC solution based on a Bayesian active learning framework to overcome the challenge of lacking fully labeled training data and exploit actual dependencies between the HTS assays. The solution is motivated by the need to model dependencies between existing experimental confirmatory HTS assays and improve prediction performance. We have pursued extensive experiments over several HTS assays and have shown the advantages of DRABAL. The datasets and programs can be downloaded from https://figshare.com/articles/DRABAL/3309562 . Graphical abstract .

In microwave device and circuit design, many simulations are often needed to find a set of designs that satisfy one or multiple specifications chosen by the designer upfront: the feasible region. A novel Bayesian active learning framework is presented to accurately identify the feasible region with a low number of simulations. The technique leverages on a stochastic model to obtain an efficient and automated procedure. A suitable application example validates the proposed technique and shows its effectiveness to rapidly obtain many suitable designs.

For engineering, there are two major challenges in reliability analysis. First, to ensure the accuracy of simulation results, mechanical products are usually defined implicitly by complex numerical models that require time-consuming. Second, the mechanical products are fortunately designed with a large safety margin, which leads to a low failure probability. This paper proposes an efficient and high-precision adaptive active learning algorithm based on the Kriging surrogate model to deal with the problems with low failure probability and time-consuming numerical models. In order to solve the problem with multiple failure regions, the adaptive kernel-density estimation is introduced and improved. Meanwhile, a new criterion for selecting points based on the current Kriging model is proposed to improve the computational efficiency. The criterion for choosing the best sampling points considers not only the probability of misjudging the sign of the response value at a point by the Kriging model but also the distribution information at that point. In order to prevent the distance between the selected training points from too close, the correlation between training points is limited to avoid information redundancy and improve the computation efficiency of the algorithm. Finally, the efficiency and accuracy of the proposed method are verified compared with other algorithms through two academic examples and one engineering application.

The focus of this paper is on the development of validated models for wind turbine blades. Validation of these models is a comprehensive undertaking which requires carefully designing and executing experiments, proposing appropriate physics-based models, and applying correlation techniques to improve these models based on the test data. This paper will cover each of these three aspects of model validation, although the focus is on the third – model calibration. The result of the validation process is an understanding of the credibility of the model when used to make analytical predictions. These general ideas will be applied to a wind turbine blade designed, tested, and modeled at Sandia National Laboratories. The key points of the paper include discussions of the tests which are needed, the required level of detail in these tests to validate models of varying detail, and mathematical techniques for improving blade models. Results from investigations into calibrating simplified blade models are presented. I. Introduction HERE are a number of reasons why one desires to develop models of wind turbine blades, and in each case one wants to ensure that these models are useful for the intended purpose. For example, correctly predicting failure in blades using a model can reduce the need for numerous costly tests including the fabrication of additional blades for a test-based failure prediction approach. An additional benefit of modeling and simulation is that the time required to complete the design and fabrication cycle can be reduced significantly when validated models are used to evaluate key aspects of the design that would otherwise require testing. Additionally, modern blades are large and costly – a validated predictive tool would be useful for assessing larger blades of the future. An important step in ensuring that a model is useful for the purpose of the analysis, that is ensuring that a model accurately predicts the behavior of interest, is a process called model validation. A validated model is one in which an analyst or designer can place a great deal of confidence – one can use this model to accurately predict performance. The validation process incorporates both testing and analysis. A set of calibration experiments are designed which provide enough data to improve the model so that the observations from the test and the corresponding predictions from the analysis are suitably correlated. In the next step, additional “validation experiments” are conducted in order to ensure that the model is predictive for the conditions of the validation experiments. If the validation experiments can be predicted, then the model is considered validated, otherwise additional experiments must be performed to provide data for further improvement of the model. It is important to note that a model which has been calibrated to match the test data is not necessarily a valid model. The process of calibrating a model is called model updating, while model validation includes the additional step of performing validation experiments. The main objective of the paper is to detail a general model validation process applied to wind turbine blades designed and tested at Sandia National Laboratories. The key points to be covered include those related to 1) testing (experiment design), 2) analysis (model development), and 3) comparison of test-analysis data for use in model calibration. Key points are covered in each of these areas; however, the focus of this paper is on model calibration. Optimization of the model parameters incorporating various types of blade test observations, including modal testing and static testing, is a novel development. As an example encompassing the key points, a program aimed at improving current modeling capabilities for a research-sized wind turbine blade is discussed in detail.

Active learning has recently attracted increasing attention in the task of person re-identification, due to its unique scalability that not only maximally reduces the annotation cost but also retains the satisfying performance. Although some preliminary active learning methods have been explored in scalable person re-identification task, they have the following two problems: 1) the inefficiency in the selection process of image pairs due to the huge search space, and 2) the ineffectiveness caused by ignoring the impact of unlabeled data in model training. Considering that, we propose a Multi-grained Active Semi-Supervised learning framework, named MASS, to address the scalable person re-identification problem existing in the practical scenarios. Specifically, we firstly design a cluster-scatter procedure to alleviate the inefficiency problem, which consists of two components: cluster step and scatter step. The cluster step shrinks the search space into individual small clusters by a coarse-grained clustering method, and the subsequent scatter step further mines the hard distinguished image pairs from unlabelled set to purify the learned clusters by a novel centrality-based adaptive purification strategy. Afterward, we introduce a customized purification loss for the purified clustering, which utilizes the complementary information in both labeled and unlabeled data to optimize the model for solving the ineffectiveness problem. The cluster-scatter procedure and the model optimization are performed in an iterative fashion to achieve the promising performance while greatly reducing the annotation cost. Extensive experimental results have demonstrated that MASS can even achieve a competitive performance with fully supervised methods in the case of extremely less annotation requirements.

Pipeline corrosion poses significant challenges and risks to the energy industry and its mitigation requires extensive and reliable predictive modeling. Corrosion models based on computational fluid dynamics (CFD) stands as a desirable candidate for its detailed physical characterization and modeling flexibility, but its applications in practical industrial settings is limited by the high computational cost and laborious manual operation in the modeling and sampling process. To address these challenges, we propose a Bayesian active learning method. The method consists of a surrogate model formulated using Gaussian process regression (GPR) to provide rapid model prediction as well as uncertainty quantification, and an adaptive sampling scheme to automate and accelerate the data collection process. Careful dimension reduction guided by both physics and data is also carried out to significantly simplify the sampling space. The capability of the overall method for efficient and automated sampling and surrogate modeling is demonstrated on an example case of corrosion predictive modeling and can be leveraged in industrial applications at a much larger scale.

Ensuring the sustainable operation of solid‐oxide fuel cells (SOFCs) requires an understanding of the components' lifespan. Multiphase‐field simulation studies play a major role in understanding the underlying microstructural changes and the resulting property alterations in SOFCs over time. The primary challenge in such simulations lies in identifying a suitable model and defining its parametrization. This study presents an Active Learning framework combined with Bayesian Optimization to identify optimal model parameters to simulate the aging of nickel‐gadolinium doped ceria (Ni‐GDC) anodes. The study overcomes incompleteness and inconsistency of literature data, and navigates the complex, high‐dimensional parameter space, by leveraging experimental microstructure data and the power of the AL framework. The successful parameter search enables simulation studies of Ni‐GDC anode aging and performance during long‐term SOFC operation. This approach improves the accuracy of phase‐field simulations and offers a versatile tool for broader applications in SOFC development, predicting material behavior under various operational conditions.

Ab initio protein docking represents a major challenge for optimizing a noisy and costly "black box"-like function in a high-dimensional space. Despite progress in this field, there is a lack of rigorous uncertainty quantification (UQ). To fill the gap, we introduce a novel algorithm, Bayesian active learning (BAL), for optimization and UQ of such black-box functions with applications to flexible protein docking. BAL directly models the posterior distribution of the global optimum (i.e., native structures) with active sampling and posterior estimation iteratively feeding each other. Furthermore, it uses complex normal modes to span a homogeneous, Euclidean conformation space suitable for high-dimensional optimization and constructs funnel-like energy models for quality estimation of encounter complexes. Over a protein-docking benchmark set and a CAPRI set including homology docking, we establish that BAL significantly improves against starting points from rigid docking and refinements by particle swarm optimization, providing a top-3 near-native prediction for one third targets. Quality assessment empowered with UQ leads to tight quality intervals with half range around 25% of the actual interface root-mean-square deviation and confidence level at 85%. BAL's estimated probability of a prediction being near-native achieves binary classification AUROC at 0.93 and area under the precision recall curve over 0.60 (compared to 0.50 and 0.14, respectively, by chance), which also improves ranking predictions. This study represents the first UQ solution for protein docking, with rigorous theoretical frameworks and comprehensive empirical assessments.

Predicting combined tidal and pluvial flood inundation using a machine learning surrogate model