Model-based adaptive spatial sampling for occurrence map construction

Adaptive sampling methods, often used in combination with Markov state models, are becoming increasingly popular for speeding up rare events in simulation such as molecular dynamics (MD) without biasing the system dynamics. Several adaptive sampling strategies have been proposed, but it is not clear which methods perform better for different physical systems. In this work, we present a systematic evaluation of selected adaptive sampling strategies on a wide selection of fast folding proteins. The adaptive sampling strategies were emulated using models constructed on already existing MD trajectories. We provide theoretical limits for the sampling speed-up and compare the performance of different strategies with and without using some a priori knowledge of the system. The results show that for different goals, different adaptive sampling strategies are optimal. In order to sample slow dynamical processes such as protein folding without a priori knowledge of the system, a strategy based on the identification of a set of metastable regions is consistently the most efficient, while a strategy based on the identification of microstates performs better if the goal is to explore newer regions of the conformational space. Interestingly, the maximum speed-up achievable for the adaptive sampling of slow processes increases for proteins with longer folding times, encouraging the application of these methods for the characterization of slower processes, beyond the fast-folding proteins considered here.

Non-intrusive reduced order modeling methods (ROMs) have been widely used as a computationally cheap method of modeling aerodynamic flows. A problem with the construction of these models is that a large number of samples are required to achieve global accuracy leading to a high cost of obtaining samples to create the ROMs. In this work, a novel adaptive sampling strategy is proposed that uses the uncertainty estimate of the high-dimensional prediction of the ROM as a sampling objective. The high-dimensional uncertainty estimate is obtained by performing Monte Carlo simulations to propagate the uncertainty of the low-dimensional space obtained using a multi-task Gaussian process model. The proposed algorithms are demonstrated on a two- and fourteen-dimensional aerodynamics modeling test case and compared to the performance of Halton sequences. The results of the two-dimensional case illustrate that the adaptive sampling strategies can achieve up to a 15% improvement over Halton sequences in terms of the mean relative error for the same number of samples. In the fourteen-dimensional test case, the adaptive sampling strategies achieve up to a 7.6% improvement over the Halton sequences at the same number of samples. All sampling strategies have comparable performance once the computational budget is exhausted, achieving a mean relative error of approximately 0.050 and 0.085 in the two- and fourteen-dimensional test cases, respectively. The adaptive sampling strategies can also better predict important features of the pressure distribution, such as shock waves and suction peaks.

Reduced-order models (ROMs) become increasingly popular in industrial design and optimization processes, since they allow to approximate expensive high fidelity computational fluid dynamics (CFD) simulations in near real-time. The quality of ROM predictions highly depends on the placement samples in the spanned parameter space. Adaptive sampling strategies allow to identify regions of interest, which feature e.g. nonlinear responses with respect to the parameters, and therefore enable the sensible placement of new samples. By introducing more samples in these regions, the ROM prediction accuracy should increase. In this contribution we investigate different adaptive sampling strategies based on cross-validation, Gaussian mean-squared error, two methods exploiting the CFD residual and a two manifold embedding methods. The performance of those strategies is evaluated and measured by their ability to successfully identify the regions of interest and the resulting sample placement in terms of different quantitative statistical values. We further discuss the reduction of the ROM prediction error over the adaptive sampling iterations and show that depending on the adaptive sampling strategy, the number of required samples can be reduced by 35–44% without deteriorating model quality compared to a Halton sequence sampling plan.

Investigating adaptive sampling strategies for optimal building energy performance using artificial neural networks and kriging surrogate models

We consider the regularized empirical risk minimization (ERM) of linear predictors, which arises in a variety of problems in machine learning and statistics. After reformulating the original ERM as a bilinear saddle‐point problem, we can apply stochastic primal–dual methods to solve it. Sampling the primal or dual coordinates with a fixed nonuniform distribution is usually employed to accelerate the convergence of the algorithm, but such a strategy only exploits the global information of the objective function. To capture its local structures, we propose an adaptive importance sampling strategy that chooses the coordinates based on delicately designed nonuniform and nonstationary distributions. When our adaptive coordinate sampling strategy is applied to the Stochastic Primal‐Dual Coordinate (SPDC), we prove that the resulting algorithm enjoys linear convergence. Moreover, we show that the ideal form of our adaptive sampling exhibits strictly sharper convergence rate under certain conditions compared with the vanilla SPDC. We also extend our sampling strategy to other algorithms including Doubly Stochastic Primal‐Dual Coordinate (DSPDC) and Stochastic Primal‐Dual with O(1) per‐iteration cost and Variance Reduction (SPD1‐VR), where both primal and dual coordinates are randomly sampled. Our experiment results show that the proposed strategy significantly improves the convergence performance of the methods when compared with existing sampling strategies.

The problem of approximating smooth, multivariate functions from sample points arises in many applications in scientific computing, e.g., in computational Uncertainty Quantification (UQ) for science and engineering. In these applications, the target function may represent a desired quantity of interest of a parameterized Partial Differential Equation (PDE). Due to the large cost of solving such problems, where each sample is computed by solving a PDE, sample efficiency is a key concern in these applications. Recently, there has been increasing focus on the use of Deep Neural Networks (DNN) and Deep Learning (DL) for learning such functions from data. In this work, we propose an adaptive sampling strategy, CAS4DL (Christoffel Adaptive Sampling for Deep Learning) to increase the sample efficiency of DL for multivariate function approximation. Our novel approach is based on interpreting the second to last layer of a DNN as a dictionary of functions defined by the nodes on that layer. With this viewpoint, we then define an adaptive sampling strategy motivated by adaptive sampling schemes recently proposed for linear approximation schemes, wherein samples are drawn randomly with respect to the Christoffel function of the subspace spanned by this dictionary. We present numerical experiments comparing CAS4DL with standard Monte Carlo (MC) sampling. Our results demonstrate that CAS4DL often yields substantial savings in the number of samples required to achieve a given accuracy, particularly in the case of smooth activation functions, and it shows a better stability in comparison to MC. These results therefore are a promising step towards fully adapting DL towards scientific computing applications.

In recent decades, surrogate-based optimization (SBO) has been developed to replace costly models with cheap surrogates to improve efficiency. In this article, an adaptive parallel infill strategy is proposed to balance exploration and exploitation over the design space during the optimization process of SBO. Within this method, an inaccurate search strategy is adopted to optimize the surrogate models, thereby helping to locate the exploitation point. An elite archive is exploited to store superior sampling points for batch sampling, while a customized batch size determination strategy is introduced. The proposed SBO method with its adaptive parallel sampling strategy is tested on six unconstrained and five constrained analytical cases with the optimization results compared to state-of-the-art optimization algorithms. The optimization of a 582-bar tower truss system is also performed and utilized to verify the proposed SBO method. The proposed SBO with the adaptive parallel sampling strategy shows excellent performance and better stability.

BackgroundDrug-target interaction (DTI) prediction plays a pivotal role in drug discovery and drug repositioning, enabling the identification of potential drug candidates. However, most previous approaches often do not fully utilize the complementary relationships among multiple biological networks, which limits their ability to learn more consistent representations. Additionally, the selection strategy of negative samples significantly affects the performance of contrastive learning methods.ResultsIn this study, we propose CCL-ASPS, a novel deep learning model that incorporates Collaborative Contrastive Learning (CCL) and Adaptive Self-Paced Sampling strategy (ASPS) for drug-target interaction prediction. CCL-ASPS leverages multiple networks to learn the fused embeddings of drugs and targets, ensuring their consistent representations from individual networks. Furthermore, ASPS dynamically selects more informative negative sample pairs for contrastive learning. Experiment results on the established dataset demonstrate that CCL-ASPS achieves significant improvements compared to current state-of-the-art methods. Moreover, ablation experiments confirm the contributions of the proposed CCL and ASPS strategies.ConclusionsBy integrating Collaborative Contrastive Learning and Adaptive Self-Paced Sampling, the proposed CCL-ASPS effectively addresses the limitations of previous methods. This study demonstrates that CCL-ASPS achieves notable improvements in DTI predictive performance compared to current state-of-the-art approaches. The case study and cold start experiments further illustrate the capability of CCL-ASPS to effectively predict previously unknown DTI, potentially facilitating the identification of new drug-target interactions.

Multivariate statistical process control techniques have been widely used for online monitoring in various applications. However, when monitoring high-dimensional processes, practical resource constraints such as limited data transmission bandwidth, number of sensors, and sensor battery life often restrict our ability to collect observations from all data streams. In this paper, we propose a neural network-based adaptive sampling strategy for monitoring high-dimensional processes. To adaptively select data streams for observation, it is essential to dynamically assess the importance of each stream. Accordingly, we propose learning an importance function through a neural network. By incorporating domain knowledge of process monitoring, a monotonic neural network is constructed. A key challenge in training this network is the lack of ground truth for importance assessment. To address this challenge, we simulate training data and design a novel loss function. Compared to existing adaptive sampling strategies, our method does not rely on heuristic techniques and offers enhanced scalability. With the proposed adaptive sampling strategy, a generic monitoring scheme is then developed for the monitoring of high-dimensional processes. We have conducted thorough numerical simulations and a case study, which demonstrate that our method significantly reduces detection delays and increases the likelihood of observing shifted data streams.

Stochastic gradient descent (SGD) usually samples training data based on the uniform distribution, which may not be a good choice because of the high variance of its stochastic gradient. Thus, importance sampling methods are considered in the literature to improve the performance. Most previous work on SGD-based methods with importance sampling requires the knowledge of Lipschitz constants of all component gradients, which are in general difficult to estimate. In this paper, we study an adaptive importance sampling method for common SGD-based methods by exploiting the local first-order information without knowing any Lipschitz constants. In particular, we periodically changes the sampling distribution by only utilizing the gradient norms in the past few iterations. We prove that our adaptive importance sampling non-asymptotically reduces the variance of the stochastic gradients in SGD, and thus better convergence bounds than that for vanilla SGD can be obtained. We extend this sampling method to several other widely used stochastic gradient algorithms including SGD with momentum and ADAM. Experiments on common convex learning problems and deep neural networks illustrate notably enhanced performance using the adaptive sampling strategy.

Metamodels aim to approximate characteristics of functions or systems from the knowledge extracted on only a finite number of samples. In recent years kriging has emerged as a widely applied metamodeling technique for resource-intensive computational experiments. However its prediction quality is highly dependent on the size and distribution of the given training points. Hence, in order to build proficient kriging models with as few samples as possible adaptive sampling strategies have gained considerable attention. These techniques aim to find pertinent points in an iterative manner based on information extracted from the current metamodel. A review of adaptive schemes for kriging proposed in the literature is presented in this article. The objective is to provide the reader with an overview of the main principles of adaptive techniques, and insightful details to pertinently employ available tools depending on the application at hand. In this context commonly applied strategies are compared with regards to their characteristics and approximation capabilities. In light of these experiments, it is found that the success of a scheme depends on the features of a specific problem and the goal of the analysis. In order to facilitate the entry into adaptive sampling a guide is provided. All experiments described herein are replicable using a provided open source toolbox.

Spatial adaptive sampling in multiscale simulation

To achieve efficient and cost-effective sensing coverage of the vast under-sampled 3D aquatic volume, intelligent adaptive sampling strategies involving a team of Autonomous Underwater Vehicles (AUVs) endowed with underwater wireless communication capabilities become essential. Given a 3D field of interest to sample, the AUVs should coordinate to take measurements using minimal resources (time or energy) in order to reconstruct the field at an onshore station with admissible error. A novel distributed adaptive sampling solution that can minimize the sampling cost (in terms of time or energy expenditure) is proposed along with underwater acoustic communication protocols that facilitate the coordination of the vehicles. The proposed solution operates in two distinct phases in which it employs random compressive sensing (Phase I) and adaptive sampling (Phase II). Phase I captures the spatial distribution of the field of interest while Phase II tracks the temporal variation of the same. A distributed framework for multi-vehicle adaptive sampling that facilitates the movement of data between AUVs and enables compute intensive adaptive sampling algorithms is proposed. Simulation results on real data traces show that the proposed adaptive sampling solution significantly outperforms existing solutions in terms of reconstruction accuracy and energy expenditure.

A surrogate modeling strategy, using effective interpolation and sampling methods, facilitates a reduction in the number of computational fluid dynamics simulations required to construct an aerodynamic model to a specified accuracy. In this paper, two adaptive sampling strategies are compared for generating surrogate models, based on Kriging and radial basis function interpolation, respectively. The relationships between the two model formulations are discussed, and three test cases are considered, including analytic functions and recovery of aerodynamic coefficients for two example applications: longitudinal flight mechanics analysis for the DLR-F12 aircraft and structural loads analysis of an RAE2822 airfoil. For the airfoil example, models of , , and were constructed with the two sampling strategies using Euler/boundary-layer-coupled computational fluid dynamics and a three-dimensional flight envelope of incidence, Mach, and Reynolds number. The two sampling approaches direct some samples toward exploration of the domain by minimizing model uncertainty and some toward refinement of local nonlinearities, by adapting to model curvature or extrema. The results provide some evidence that, for certain functions, in certain scenarios, each update scheme could be useful. Both methods were at least better than traditional space-filling sampling for all the test cases presented.

Currently, there are debates on the accuracy of vision transformers (ViTs) compared to ConvNets for image processing tasks. Image quality assessment (IQA) and particularly 360-IQA is lacking insights regarding their performances and robustness compared to the widely used ConvNets. This paper aims to investigate transfer learning from two pre-trained versions of ViTs and two ConveNets (ResNet-50 and EfficientNet-B3) for 360-degree image quality assessment with a focus on (i) the prediction accuracy and generalization ability and (ii) their adaptation to the specific characteristics of 360-degree images. Furthermore, the influence of adaptive patches sampling compared to simply using equirectangular content is analyzed with each architecture. Experimental findings on publicly available datasets (OIQA, CVIQ and MVAQD) show the superiority of ResNet-50 over ViTs and EfficientNet-B3 while requiring less computational time. Also, the base version of ViTs outperforms the larger one. Finally, except for CVIQ, both ViTs and ConveNets benefit from the adaptive sampling strategy, depicting the interest of taking 360-degree characteristics into account.