MOOSE ProbML: Parallelized probabilistic machine learning and uncertainty quantification for computational energy applications

Artificial intelligence (AI) and machine learning (ML) have transformed numerous domains by enabling systems to learn from data and make intelligent decisions. Within the field of ML, probabilistic machine learning has gained significant attention due to its ability to capture uncertainty and provide predictions based on probabilities. This essay explores the concept of probabilistic machine learning and its applications in AI. We delve into the fundamentals of machine learning, discuss probabilistic modeling, explore various probabilistic machine learning techniques, highlight the advantages of probabilistic approaches, and examine real-world applications. Additionally, we address the challenges associated with probabilistic machine learning and discuss future directions in this exciting field.

The utilization of learning analytics to identify at-risk students for early intervention has exhibited promising results. However, most predictive models utilized to address this issue have been based on non-probabilistic machine learning models. In response, this study incorporated probabilistic machine learning for two reasons: (1) to facilitate the inclusion of domain knowledge, and (2) to enable the quantification of uncertainty in model parameters and predictions. The study developed a five-stage, probabilistic logistic regression model to identify at-risk students at different stages throughout the academic calendar. Rather than predicting a student’s final or exam mark, the model was focused on predicting the at-risk probabilities for subsequent assessments—specifically, the probability of a student failing an upcoming assessment. The model incorporated student engagement data from Moodle, as well as demographic and student performance data. The study’s findings indicate that the significance and certainty of student engagement and demographic variables decreased after incorporating student-performance variables, such as assignments and tests. The most effective week for identifying at-risk students was found to be week 6, when the accuracy was 92.81%. Furthermore, the average level of uncertainty exhibited by the models decreased by 60% from stage 3 to 5, indicating more reliable predictions at later than earlier stages. The study highlights the potential of a probabilistic machine learning model to aid instructors and practitioners in identifying at-risk students, and thereby to enhance academic outcomes.

Machine learning is a powerful data-driven technology for regression and classification tasks in numerous branches of science and technology. With probabilistic machine learning, we can approximate the probability density function of a given digital data set, unlocking the sampling of new and realistic data. We implement probabilistic machine learning as a statistical-based data augmentation tool to effectively improve the performance of a classifier on scarce data sets. Limited data are the norm in most practical situations, as is the case with core samples retrieved from unconventional oil and gas reservoirs. Obtaining and properly labeling rock sample images is a manual and time-consuming task. This leads to scarce data sets that make the lithologic classification of rock samples challenging. We illustrate the technology on images taken from well cores from the Argentinean Neuquén and Austral Basins, where by introducing generated samples in the training stage of a machine-learning classifier, we achieve a 5% higher accuracy with respect to the baseline obtained using the same number of samples from the original data set. We reach these results by proposing a lightweight workflow that can be run on very limited hardware. Our workflow is an alternative to classical data augmentation strategies for dealing with incomplete information, such as geometric transformations.

In the setting of learning indexed families, probabilistic learning under monotonicity constraints is more powerful than deterministic learning under monotonicity constraints even if the probability is close to 1 provided the learning machines are restricted to proper or class preserving hypothesis spaces (cf. [19]). In this paper, we investigate the relation between probabilistic learning and oracle identification under monotonicity constraints. In particular, we deal with the question how much “additional information” provided by oracles is necessary in order to compensate the additional power of probabilistic learning.If the oracle machines have access to K-oracle, then they can compensate the power of monotonic (conservative) probabilistic machines completely, provided the probability p is greater than 2/3 (1/2). Furthermore, we show that for every recursively enumerable oracle A, there exists a learning problem which is strong-monotonically learnable by an oracle machine having access to A, but not conservatively or monotonically learnable with any probability p < 0. A similar result holds for Peano-complete oracles. However, probabilistic learning under monotonicity constraints is ”rich“ enough to encode every recursively enumerable set in a characteristic learning problem, i.e., for every recursively enumerable set A, and every p < 2/3, there exists a learning problem la which is monotonically learnable with probability p, and monotonically learnable with oracle B if and only if A is Turing-reducible to B. The same result holds for conservative probabilistic learning with p < 1/2, and strong-monotonic learning with probability p = 2/3. In particular, it follows that probabilistic learning under monotonicity constraints cannot be characterized in terms of oracle identification. Moreover, we close an open problem that appeared in [19] by showing that the probabilistic hierarchies of class preserving monotonic and conservative probabilistic learning are dense.Finally, we show that these probability bounds are strict, i.e., in the case of monotonic probabilistic learning with probability p = 2/3, conservative probabilistic learning with probability p = 1/2, and strong-monotonic probabilistic learning with probability p = 1/2, K is not suficient to compensate the power of probabilistic learning under monotonicity constraints.KeywordsRecursive FunctionInductive InferenceInductive LogicProbabilistic InferenceHypothesis SpaceThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

IntroductionMechanical Thrombectomy (MT) is the standard-of-care in the interventional management of Acute Ischemic Stroke (AIS). The NVQI-QOD AIS Thrombectomy Registry documents detailed patient characteristics, pre-operative imaging, procedural metrics, and post-operative...

Efficient seismic fragility analysis method utilizing ground motion clustering and probabilistic machine learning

&amp;lt;p&amp;gt;As the number of underground activities increase, the need for better understanding of the geospatial properties become more and more essential for correct engineering designs and optimal decision making. However, gathering subsurface data is still an extremely costly and imprecise endeavour. Geological modelling has played a crucial role for years helping to understand and correlate the complex geometries encountered underground but single deterministic models fail to capture all possible configurations given the limited data. Probabilistic machine learning allows to integrate domain knowledge and observations of the physical world on a rigorous and consistent manner. Inferences to the probabilistic model implements an automatic learning-from-observations process.&amp;lt;/p&amp;gt;&amp;lt;p&amp;gt;&amp;amp;#160;&amp;lt;/p&amp;gt;&amp;lt;p&amp;gt;In this work, we show how by embedding state-of-the-art implicit interpolants into probabilistic frameworks, we can integrate the information of distinct data sets in one single common earth model. We will present results from a minimal working example to introduce Bayesian statistics, to full 3-D probabilistic inversions. All the models used for this demonstration are implemented in the open-source library GemPy ( www.gempy.org) allowing full reproducibility of the results.&amp;lt;/p&amp;gt;&amp;lt;p&amp;gt;&amp;amp;#160;&amp;lt;/p&amp;gt;

Predicting undergraduate student evaluations of teaching using probabilistic machine learning: The importance of motivational climate

For a multiple-fluid system, CG models capable of accurately predicting the interfacial properties as a function of curvature are still lacking. In this work, we propose a new probabilistic machine learning (ML) model for learning CG potentials for binary fluids. The water-hexane mixture is selected as a typical immiscible binary liquid-liquid system. We develop a new CG force field (FF) using the Shinoda-DeVane-Klein (SDK) FF framework and compute parameters in this CG FF using the proposed probabilistic ML method. It is shown that a standard response-surface approach does not provide a unique set of parameters, as it results in a loss function with multiple shallow minima. To address this challenge, we develop a probabilistic ML approach where we compute the probability density function (PDF) of parameters that minimize the loss function. The PDF has a well-defined peak corresponding to a unique set of parameters in the CG FF that reproduces the desired properties of a liquid-liquid interface. We compare the performance of the new CG FF with several existing FFs for the water-hexane mixture, including two atomistic and three CG FFs with respect to modeling the interface structure and thermodynamic properties. It is demonstrated that the new FF significantly improves the CG model prediction of both the interfacial tension and structure for the water-hexane mixture.

Data-driven probabilistic machine learning in sustainable smart energy/smart energy systems: Key developments, challenges, and future research opportunities in the context of smart grid paradigm

A machine learning approach for the surrogate modeling of uncertain distributed process engineering models

Every year, tropical hurricanes affect North and Central American wildlife and people. The ability to forecast hurricanes is essential in order to minimize the risks and vulnerabilities in North and Central America. Machine learning is a newly tool that has been applied to make predictions about different phenomena. We present an original framework utilizing Machine Learning with the purpose of developing models that give insights into the complex relationship between the land–atmosphere–ocean system and tropical hurricanes. We study the activity variations in each Atlantic hurricane category as tabulated and classified by NOAA from 1950 to 2021. By applying wavelet analysis, we find that category 2–4 hurricanes formed during the positive phase of the quasi-quinquennial oscillation. In addition, our wavelet analyses show that super Atlantic hurricanes of category 5 strength were formed only during the positive phase of the decadal oscillation. The patterns obtained for each Atlantic hurricane category, clustered historical hurricane records in high and null tropical hurricane activity seasons. Using the observational patterns obtained by wavelet analysis, we created a long-term probabilistic Bayesian Machine Learning forecast for each of the Atlantic hurricane categories. Our results imply that if all such natural activity patterns and the tendencies for Atlantic hurricanes continue and persist, the next groups of hurricanes over the Atlantic basin will begin between 2023 ± 1 and 2025 ± 1, 2023 ± 1 and 2025 ± 1, 2025 ± 1 and 2028 ± 1, 2026 ± 2 and 2031 ± 3, for hurricane strength categories 2 to 5, respectively. Our results further point out that in the case of the super hurricanes of the Atlantic of category 5, they develop in five geographic areas with hot deep waters that are rather very well defined: (I) the east coast of the United States, (II) the Northeast of Mexico, (III) the Caribbean Sea, (IV) the Central American coast, and (V) the north of the Greater Antilles.

Machine learning (ML) is increasingly utilized in materials and structural analysis. However, for an ML model to perform effectively, it requires a large amount of high-quality data for training. When experimental or simulation data are insufficient, training data can be sourced from literature. However, literature data often come from diverse sources (e.g., different testing equipment and materials) and lack standardization, leading to significant data variability and missing information. This work aims to develop a physics-guided machine learning model that accounts for these data variations and missing data. We focus on the probabilistic fatigue life prediction of additively manufactured (AM) Ti-6Al-4V, considering the influence of process parameters. Fatigue data for AM Ti-6Al-4V, both in as-built and annealed conditions, were collected from extensive literature, capturing various process parameters (e.g., scanning speed, laser power, hatch space, layer thickness, heat temperature, and heat time). A Probabilistic Physics-Guided Neural Network (PPgNN) is proposed to model these data, with the following novelties: (1) Handling Data Variability: Given the wide range of data sources, there are significant variations for the same variables. The PPgNN can be trained on noisy literature data and predict both the mean and variance of fatigue life. (2) Addressing Missing Data: Due to inconsistencies in how data are reported, some process parameters are available in some literature but missing in others. The model can be trained with incomplete datasets and adapt when additional information becomes available. (3) Incorporating Domain Knowledge: Existing fatigue analysis knowledge is integrated into the ML model to guide learning and prediction. The PPgNN includes constraints on model parameters to ensure the generated probabilistic stress-life (P-S-N) curves have physics-consistent curvature and non-constant variance. The results demonstrate that PPgNN effectively manages uncertainties arising from data variations and missing information, producing physics-consistent probabilistic predictions. A global sensitivity analysis using the Sobol sensitivity was also performed to assess the importance of the process parameters.

Probabilistic graphical models are a fundamental tool in probabilistic modeling, machine learning and artificial intelligence. They allow us to integrate in a natural way expert knowledge, physical modeling, heterogeneous and correlated data and quantities of interest. For exactly this reason, multiple sources of model uncertainty are inherent within the modular structure of the graphical model. In this paper we develop information-theoretic, robust uncertainty quantification methods and non-parametric stress tests for directed graphical models to assess the effect and the propagation through the graph of multi-sourced model uncertainties to quantities of interest. These methods allow us to rank the different sources of uncertainty and correct the graphical model by targeting its most impactful components with respect to the quantities of interest. Thus, from a machine learning perspective, we provide a mathematically rigorous approach to correctability that guarantees a systematic selection for improvement of components of a graphical model while controlling potential new errors created in the process in other parts of the model. We demonstrate our methods in two physico-chemical examples, namely quantum scale-informed chemical kinetics and materials screening to improve the efficiency of fuel cells.

Chapter 6 - Machine learning for thyroid cancer diagnosis