Machine Learning, Probabilistic Inference, System Identification and Control

Artificial intelligence (AI) and machine learning (ML) have transformed numerous domains by enabling systems to learn from data and make intelligent decisions. Within the field of ML, probabilistic machine learning has gained significant attention due to its ability to capture uncertainty and provide predictions based on probabilities. This essay explores the concept of probabilistic machine learning and its applications in AI. We delve into the fundamentals of machine learning, discuss probabilistic modeling, explore various probabilistic machine learning techniques, highlight the advantages of probabilistic approaches, and examine real-world applications. Additionally, we address the challenges associated with probabilistic machine learning and discuss future directions in this exciting field.

Efficient seismic fragility analysis method utilizing ground motion clustering and probabilistic machine learning

The utilization of learning analytics to identify at-risk students for early intervention has exhibited promising results. However, most predictive models utilized to address this issue have been based on non-probabilistic machine learning models. In response, this study incorporated probabilistic machine learning for two reasons: (1) to facilitate the inclusion of domain knowledge, and (2) to enable the quantification of uncertainty in model parameters and predictions. The study developed a five-stage, probabilistic logistic regression model to identify at-risk students at different stages throughout the academic calendar. Rather than predicting a student’s final or exam mark, the model was focused on predicting the at-risk probabilities for subsequent assessments—specifically, the probability of a student failing an upcoming assessment. The model incorporated student engagement data from Moodle, as well as demographic and student performance data. The study’s findings indicate that the significance and certainty of student engagement and demographic variables decreased after incorporating student-performance variables, such as assignments and tests. The most effective week for identifying at-risk students was found to be week 6, when the accuracy was 92.81%. Furthermore, the average level of uncertainty exhibited by the models decreased by 60% from stage 3 to 5, indicating more reliable predictions at later than earlier stages. The study highlights the potential of a probabilistic machine learning model to aid instructors and practitioners in identifying at-risk students, and thereby to enhance academic outcomes.

Machine learning is a powerful data-driven technology for regression and classification tasks in numerous branches of science and technology. With probabilistic machine learning, we can approximate the probability density function of a given digital data set, unlocking the sampling of new and realistic data. We implement probabilistic machine learning as a statistical-based data augmentation tool to effectively improve the performance of a classifier on scarce data sets. Limited data are the norm in most practical situations, as is the case with core samples retrieved from unconventional oil and gas reservoirs. Obtaining and properly labeling rock sample images is a manual and time-consuming task. This leads to scarce data sets that make the lithologic classification of rock samples challenging. We illustrate the technology on images taken from well cores from the Argentinean Neuquén and Austral Basins, where by introducing generated samples in the training stage of a machine-learning classifier, we achieve a 5% higher accuracy with respect to the baseline obtained using the same number of samples from the original data set. We reach these results by proposing a lightweight workflow that can be run on very limited hardware. Our workflow is an alternative to classical data augmentation strategies for dealing with incomplete information, such as geometric transformations.

Probabilistic machine learning and data-driven methods gradually show their high efficiency in solving the forward and inverse problems of partial differential equations (PDEs). This paper will focus on investigating the forward problem of solving time-dependent nonlinear delay PDEs with multi-delays based on multi-prior numerical Gaussian processes (MP-NGPs), which are constructed by us to solve complex PDEs that may involve fractional operators, multi-delays and different types of boundary conditions. We also quantify the uncertainty of the prediction solution by the posterior distribution of the predicted solution. The core of MP-NGPs is to discretize time firstly, then a Gaussian process regression based on multi-priors is considered at each time step to obtain the solution of the next time step, and this procedure is repeated until the last time step. Different types of boundary conditions are studied in this paper, which include Dirichlet, Neumann and mixed boundary conditions. Several numerical tests are provided to show that the methods considered in this paper work well in solving nonlinear time-dependent PDEs with delay, where delay partial differential equations, delay partial integro-differential equations and delay fractional partial differential equations are considered. Furthermore, in order to improve the accuracy of the algorithm, we construct Runge–Kutta methods under the frame of multi-prior numerical Gaussian processes. The results of the numerical experiments prove that the prediction accuracy of the algorithm is obviously improved when the Runge–Kutta methods are employed.

For a multiple-fluid system, CG models capable of accurately predicting the interfacial properties as a function of curvature are still lacking. In this work, we propose a new probabilistic machine learning (ML) model for learning CG potentials for binary fluids. The water-hexane mixture is selected as a typical immiscible binary liquid-liquid system. We develop a new CG force field (FF) using the Shinoda-DeVane-Klein (SDK) FF framework and compute parameters in this CG FF using the proposed probabilistic ML method. It is shown that a standard response-surface approach does not provide a unique set of parameters, as it results in a loss function with multiple shallow minima. To address this challenge, we develop a probabilistic ML approach where we compute the probability density function (PDF) of parameters that minimize the loss function. The PDF has a well-defined peak corresponding to a unique set of parameters in the CG FF that reproduces the desired properties of a liquid-liquid interface. We compare the performance of the new CG FF with several existing FFs for the water-hexane mixture, including two atomistic and three CG FFs with respect to modeling the interface structure and thermodynamic properties. It is demonstrated that the new FF significantly improves the CG model prediction of both the interfacial tension and structure for the water-hexane mixture.

Data-driven probabilistic machine learning in sustainable smart energy/smart energy systems: Key developments, challenges, and future research opportunities in the context of smart grid paradigm

Gaussian Processes (GPs) are a powerful modelling framework incorporating kernels and Bayesian inference, and are recognised as stateof-the-art for many machine learning tasks. Despite this, GPs have seen few applications in natural language processing (notwithstanding several recent papers by the authors). We argue that the GP framework offers many benefits over commonly used machine learning frameworks, such as linear models (logistic regression, least squares regression) and support vector machines. Moreover, GPs are extremely flexible and can be incorporated into larger graphical models, forming an important additional tool for probabilistic inference. Notably, GPs are one of the few models which support analytic Bayesian inference, avoiding the many approximation errors that plague approximate inference techniques in common use for Bayesian models (e.g. MCMC, variational Bayes).1 GPs accurately model not just the underlying task, but also the uncertainty in the predictions, such that uncertainty can be propagated through pipelines of probabilistic components. Overall, GPs provide an elegant, flexible and simple means of probabilistic inference and are well overdue for consideration of the NLP community. This tutorial will focus primarily on regression and classification, both fundamental techniques of wide-spread use in the NLP community. Within NLP, linear models are near ubiquitous, because they provide good results for many tasks, support efficient inference (including dynamic programming in structured prediction) and support simple parameter interpretation. However, linear models are inherently limited in the types of relationships between variables they can model. Often

The need for a large number of costly experimental tests is a major challenge in the certification of composite parts in the aerospace industry. While computational simulation, specifically the finite element method (FEM), can accelerate this process through analysis supported by testing, the time-consuming calibration of composite failure models remains a drawback. This often requires an engineer with years of experience to perform trial-and-error simulations. To address this, an automated machine learning framework has been proposed to simulate and calibrate the failure models of composites in an accelerated and low-cost manner. In this framework, FEM parameters are calibrated using probabilistic machine learning in a step-by-step and targeted approach. The single-edge notch tension (SENT) test was chosen as a case study and simulated explicitly with predefined initial parameters. The result was compared with experiments to calculate the calibration error. Then, Gaussian process regression (GPR) was used in an in-house developed program to automatically select new failure parameters based on initial values and the calibration error in the previous step. The program continues this process until the error becomes negligible. For validation, the calibrated model was used to predict the behavior of a center-notch tension (CNT) test. The presented automated framework provides a robust and reliable calibrated model in a significantly faster manner. It also reduces a substantial amount of computational costs, eliminates the need for high expertise, and saves valuable engineering time.

A machine learning approach for the surrogate modeling of uncertain distributed process engineering models

Every year, tropical hurricanes affect North and Central American wildlife and people. The ability to forecast hurricanes is essential in order to minimize the risks and vulnerabilities in North and Central America. Machine learning is a newly tool that has been applied to make predictions about different phenomena. We present an original framework utilizing Machine Learning with the purpose of developing models that give insights into the complex relationship between the land–atmosphere–ocean system and tropical hurricanes. We study the activity variations in each Atlantic hurricane category as tabulated and classified by NOAA from 1950 to 2021. By applying wavelet analysis, we find that category 2–4 hurricanes formed during the positive phase of the quasi-quinquennial oscillation. In addition, our wavelet analyses show that super Atlantic hurricanes of category 5 strength were formed only during the positive phase of the decadal oscillation. The patterns obtained for each Atlantic hurricane category, clustered historical hurricane records in high and null tropical hurricane activity seasons. Using the observational patterns obtained by wavelet analysis, we created a long-term probabilistic Bayesian Machine Learning forecast for each of the Atlantic hurricane categories. Our results imply that if all such natural activity patterns and the tendencies for Atlantic hurricanes continue and persist, the next groups of hurricanes over the Atlantic basin will begin between 2023 ± 1 and 2025 ± 1, 2023 ± 1 and 2025 ± 1, 2025 ± 1 and 2028 ± 1, 2026 ± 2 and 2031 ± 3, for hurricane strength categories 2 to 5, respectively. Our results further point out that in the case of the super hurricanes of the Atlantic of category 5, they develop in five geographic areas with hot deep waters that are rather very well defined: (I) the east coast of the United States, (II) the Northeast of Mexico, (III) the Caribbean Sea, (IV) the Central American coast, and (V) the north of the Greater Antilles.

Machine learning (ML) is increasingly utilized in materials and structural analysis. However, for an ML model to perform effectively, it requires a large amount of high-quality data for training. When experimental or simulation data are insufficient, training data can be sourced from literature. However, literature data often come from diverse sources (e.g., different testing equipment and materials) and lack standardization, leading to significant data variability and missing information. This work aims to develop a physics-guided machine learning model that accounts for these data variations and missing data. We focus on the probabilistic fatigue life prediction of additively manufactured (AM) Ti-6Al-4V, considering the influence of process parameters. Fatigue data for AM Ti-6Al-4V, both in as-built and annealed conditions, were collected from extensive literature, capturing various process parameters (e.g., scanning speed, laser power, hatch space, layer thickness, heat temperature, and heat time). A Probabilistic Physics-Guided Neural Network (PPgNN) is proposed to model these data, with the following novelties: (1) Handling Data Variability: Given the wide range of data sources, there are significant variations for the same variables. The PPgNN can be trained on noisy literature data and predict both the mean and variance of fatigue life. (2) Addressing Missing Data: Due to inconsistencies in how data are reported, some process parameters are available in some literature but missing in others. The model can be trained with incomplete datasets and adapt when additional information becomes available. (3) Incorporating Domain Knowledge: Existing fatigue analysis knowledge is integrated into the ML model to guide learning and prediction. The PPgNN includes constraints on model parameters to ensure the generated probabilistic stress-life (P-S-N) curves have physics-consistent curvature and non-constant variance. The results demonstrate that PPgNN effectively manages uncertainties arising from data variations and missing information, producing physics-consistent probabilistic predictions. A global sensitivity analysis using the Sobol sensitivity was also performed to assess the importance of the process parameters.

What mathematicians, scientists, engineers, and statisticians mean by “inverse problem” differs. For a statistician, an inverse problem is an inference or estimation problem. The data are finite in number and contain errors, as they do in classical estimation or inference problems, and the unknown typically is infinite-dimensional, as it is in nonparametric regression. The additional complication in an inverse problem is that the data are only indirectly related to the unknown. Standard statistical concepts, questions, and considerations such as bias, variance, mean-squared error, identifiability, consistency, efficiency, and various forms of optimality apply to inverse problems. This article discusses inverse problems as statistical estimation and inference problems, and points to the literature for a variety of techniques and results.

Chapter 6 - Machine learning for thyroid cancer diagnosis

<p>As the number of underground activities increase, the need for better understanding of the geospatial properties become more and more essential for correct engineering designs and optimal decision making. However, gathering subsurface data is still an extremely costly and imprecise endeavour. Geological modelling has played a crucial role for years helping to understand and correlate the complex geometries encountered underground but single deterministic models fail to capture all possible configurations given the limited data. Probabilistic machine learning allows to integrate domain knowledge and observations of the physical world on a rigorous and consistent manner. Inferences to the probabilistic model implements an automatic learning-from-observations process.</p><p> </p><p>In this work, we show how by embedding state-of-the-art implicit interpolants into probabilistic frameworks, we can integrate the information of distinct data sets in one single common earth model. We will present results from a minimal working example to introduce Bayesian statistics, to full 3-D probabilistic inversions. All the models used for this demonstration are implemented in the open-source library GemPy ( www.gempy.org) allowing full reproducibility of the results.</p><p> </p>