Sequential stratified splitting for efficient Monte Carlo integration

Bayesian methods are rapidly becoming popular tools for making statistical inference in various fields of science including biology, engineering, finance, and genetics. One of the key aspects of Bayesian inferential method is its logical foundation that provides a coherent framework to utilize not only empirical but also scientific information available to a researcher. Prior knowledge arising from scientific background, expert judgment, or previously collected data is used to build a prior distribution which is then combined with current data via the likelihood function to characterize the current state of knowledge using the so-called posterior distribution. Bayesian methods allow the use of models of complex physical phenomena that were previously too difficult to estimate (e.g., using asymptotic approximations). Bayesian methods offer a means of more fully understanding issues that are central to many practical problems by allowing researchers to build integrated models based on hierarchical conditional distributions that can be estimated even with limited amounts of data. Furthermore, advances in numerical integration methods, particularly those based on Monte Carlo methods, have made it possible to compute the optimal Bayes estimators. However, there is a reasonably wide gap between the background of the empirically trained scientists and the full weight of Bayesian statistical inference. Hence, one of the goals of this chapter is to bridge the gap by offering elementary to advanced concepts that emphasize linkages between standard approaches and full probability modeling via Bayesian methods.

This article reviews applications of Bayesian inference and machine learning (ML) in nuclear fusion research. Current and next-generation nuclear fusion experiments require analysis and modelling efforts that integrate different models consistently and exploit information found across heterogeneous data sources in an efficient manner. Model-based Bayesian inference provides a framework well suited for the interpretation of observed data given physics and probabilistic assumptions, also for very complex systems, thanks to its rigorous and straightforward treatment of uncertainties and modelling hypothesis. On the other hand, ML, in particular neural networks and deep learning models, are based on black-box statistical models and allow the handling of large volumes of data and computation very efficiently. For this reason, approaches which make use of ML and Bayesian inference separately and also in conjunction are of particular interest for today’s experiments and are the main topic of this review. This article also presents an approach where physics-based Bayesian inference and black-box ML play along, mitigating each other’s drawbacks: the former is made more efficient, the latter more interpretable.

The swift progression and expansion of machine learning (ML) have not gone unnoticed within the realm of statistical mechanics. In particular, ML techniques have attracted attention by the classical density-functional theory (DFT) community, as they enable automatic discovery of free-energy functionals to determine the equilibrium-density profile of a many-particle system. Within classical DFT, the external potential accounts for the interaction of the many-particle system with an external field, thus, affecting the density distribution. In this context, we introduce a statistical-learning framework to infer the external potential exerted on a classical many-particle system. We combine a Bayesian inference approach with the classical DFT apparatus to reconstruct the external potential, yielding a probabilistic description of the external-potential functional form with inherent uncertainty quantification. Our framework is exemplified with a grand-canonical one-dimensional classical particle ensemble with excluded volume interactions in a confined geometry. The required training dataset is generated using a Monte Carlo (MC) simulation where the external potential is applied to the grand-canonical ensemble. The resulting particle coordinates from the MC simulation are fed into the learning framework to uncover the external potential. This eventually allows us to characterize the equilibrium density profile of the system by using the tools of DFT. Our approach benchmarks the inferred density against the exact one calculated through the DFT formulation with the true external potential. The proposed Bayesian procedure accurately infers the external potential and the density profile. We also highlight the external-potential uncertainty quantification conditioned on the amount of available simulated data. The seemingly simple case study introduced in this work might serve as a prototype for studying a wide variety of applications, including adsorption, wetting, and capillarity, to name a few.

Statistical Models and Predictive Algorithms in Sports Betting Bayesian Approaches, Machine Learning, and Monte Carlo Simulations

BackgroundLate‐onset Alzheimer’s disease (AD) arises from interactions between genetic (nature) and environmental/behavioral (nurture) factors, but their relative contributions are not well defined. With the development of sensitive cognitive tasks for early detection and the growing applications of machine learning to AD research, there are unprecedented opportunities to uncover the most salient genetic and environmental/behavioral factors for preclinical AD risk. Our previous studies show that generalization performance––the ability to apply prior learning to new contexts––is a cognitive risk marker for preclinical AD. Using a novel technique combining Bayesian machine learning and bootstrap imputation, the present study identified among various candidate genetic and environmental/behavioral factors which most strongly predicted generalization performance in a deeply‐phenotyped cohort of cognitively unimpaired older African Americans.Method472 older African American participants from the Pathways to Healthy Aging in African Americans cohort study (Mage=68.34 years, SD=6.99; Meducation=13.93 years, SD=2.36; MMMSE=27.65, SD=1.95) responded to demographic, health, and lifestyle questionnaires; provided a saliva sample for the following AD risk genotyping: ABCA7 rs115550680, APOE‐ε4 status, ABCA7 rs3764650; and completed a cognitive battery including a generalization task (Concurrent Discrimination and Transfer Task). We employed Bayesian Additive Regression Trees (a.k.a., BART) with a bootstrap imputation‐based variable selection procedure to extract the top three variables in terms of relative contribution to generalization performance.ResultAmong 25 candidate variables, the top three ranked variables related to generalization included ABCA7 rs115550680, est.=3.29 [95% CI 0.42, 6.19]; APOE‐ε4 status, est.=2.41 [95% CI 0.37, 4.46]; and their performance on the training phase of the generalization task, est.=0.92, [95% CI 0.86, 0.99], above and beyond other demographic, health, lifestyle, and cognitive variables.ConclusionUsing novel machine learning techniques and a sensitive cognitive risk marker, we identified that generalization performance is most strongly driven by genetic vulnerability to AD among older African Americans. These findings suggest a greater contribution of genetic factors (nature) compared to environmental/behavioral factors (nurture) to late‐onset AD risk within this population.

Contrasting human and mouse embryonic stem cell self‐renewal using Bayesian network machine learning

The incidence of low-energy acetabular fractures has increased. However, the structural factors for these fractures remain unclear. The objective of this study was to extract trabecular bone architecture and proximal femur geometry (PFG) measures from clinical computed tomography (CT) images to (1) identify possible structural risk factors of acetabular fractures, and (2) to discriminate fracture cases from controls using machine learning methods. CT images of 107 acetabular fracture subjects (25 females, 82 males) and 107 age-gender matched controls were examined. Three volumes of interest, one at the acetabulum and two at the femoral head, were extracted to calculate bone volume fraction (BV/TV), gray-level co-occurrence matrix and histogram of the gray values (GV). The PFG was defined by neck shaft angle and femoral neck axis length. Relationships between the variables were assessed by statistical mean comparisons and correlation analyses. Bayesian logistic regression and Elastic net machine learning models were implemented for classification. We found lower BV/TV at the femoral head (0.51 vs. 0.55, p = 0.012) and lower mean GV at both the acetabulum (98.81 vs. 115.33, p < 0.001) and femoral head (150.63 vs. 163.47, p = 0.005) of fracture subjects when compared to their matched controls. The trabeculae within the femoral heads of the acetabular fracture sides differed in structure, density and texture from the corresponding control sides of the fracture subjects. Moreover, the PFG and trabecular architectural variables, alone and in combination, were able to discriminate fracture cases from controls (area under the receiver operating characteristics curve 0.70 to 0.79). In conclusion, lower density in the acetabulum and femoral head with abnormal trabecular structure and texture at the femoral head, appear to be risk factors for low-energy acetabular fractures.

Electroencephalography (EEG) and magnetoencephalog?raphy (MEG) are the most common noninvasive brain-imaging techniques for monitoring electrical brain activity and inferring brain function. The central goal of EEG/MEG analysis is to extract informative brain spatiotemporal?spectral patterns or to infer functional connectivity between different brain areas, which is directly useful for neuroscience or clinical investigations. Due to its potentially complex nature [such as nonstationarity, high dimensionality, subject variability, and low signal-to-noise ratio (SNR)], EEG/MEG signal processing poses some great challenges for researchers. These challenges can be addressed in a principled manner via Bayesian machine learning (BML). BML is an emerging field that integrates Bayesian statistics, variational methods, and machine-learning techniques to solve various problems from regression, prediction, outlier detection, feature extraction, and classification. BML has recently gained increasing attention and widespread successes in signal processing and big-data analytics, such as in source reconstruction, compressed sensing, and information fusion. To review recent advances and to foster new research ideas, we provide a tutorial on several important emerging BML research topics in EEG/MEG signal processing and present representative examples in EEG/MEG applications.

The complex compositional space of high entropy alloys (HEAs) has shown a great potential to reduce the cost and further increase the catalytic activity for hydrogen evolution reaction (HER) by compositional optimization. Without uncovering the specifics of the HER mechanism on a given HEA surface, it is unfeasible to apply compositional modifications to enhance the performance and save costs. In this work, a combination of density functional theory and Bayesian machine learning is used to demonstrate the unique catalytic mechanism of IrPdPtRhRu HEA catalysts for HER. At high coverage of underpotential-deposited hydrogen, a d-band investigation of the active sites of the HEA surface is conducted to elucidate the superior catalytic performance through electronic interactions between elements. At low coverage, a novel Bayesian learning with oversampling approach is then outlined to optimize the HEA composition for performance improvement and cost reduction. This approach proves more efficacious and efficient and yields higher-quality structures with less training set bias compared with neural-network optimization. The proposed HEA optimization theoretically outperforms benchmark Pt catalysts' overpotential by ≈40% at a 15% reduced synthesis cost comparing to the equiatomic ratio HEA.

Kepler has revolutionized our understanding of both exoplanets and their host stars. Asteroseismology is a valuable tool in the characterization of stars and Kepler is an excellent observing facility to perform asteroseismology. Here we select a sample of 35 Kepler solar-type stars which host transiting exoplanets (or planet candidates) with detected solar-like oscillations. Using available Kepler short cadence data up to Quarter 16 we create power spectra optimized for asteroseismology of solar-type stars. We identify modes of oscillation and estimate mode frequencies by 'peak bagging' using a Bayesian Markov Chain Monte Carlo framework. In addition, we expand the methodology of quality assurance using a Bayesian unsupervised machine learning approach. We report the measured frequencies of the modes of oscillation for all 35 stars and frequency ratios commonly used in detailed asteroseismic modelling. Due to the high correlations associated with frequency ratios we report the covariance matrix of all frequencies measured and frequency ratios calculated. These frequencies, frequency ratios, and covariance matrices can be used to obtain tight constraint on the fundamental parameters of these planet-hosting stars.

ObjectiveAutomated air‐conduction pure‐tone audiograms through Bayesian estimation and machine learning (ML) classification have recently been proposed in the literature. Although such ML‐based audiometry approaches represent a significant addition to the field, they remain unsuited for daily clinical settings, in particular for listeners with asymmetric or conductive hearing loss, severe hearing loss, or cochlear dead zones. The goal here is to expand on previously proposed ML approaches and assess the performance of this improved ML audiometry for a large sample of listeners with a wide range of hearing status.MethodsFirst, we describe the changes made to the ML method through the addition of: (1) safety limits to test listeners with a wide range of hearing status, (2) transient responses to cater for cochlear dead zones or nonmeasurable thresholds, and importantly, (3) automated contralateral masking to test listeners with asymmetric or conductive hearing loss. Next, we compared the performance of this improved ML audiometry with conventional and manual audiometry in a large cohort (n = 109 subjects) of both normal‐hearing and hearing‐impaired listeners.ResultsOur results showed that for all audiometric frequencies tested, no significant difference was found between hearing thresholds obtained using manual audiometry on a clinical audiometer as compared to both the manual and automated improved ML methods. Furthermore, the test–retest difference was not significant with the automated improved ML method for each audiometric frequency tested. Finally, when examining cross‐clinic reliability measures, significant differences were found for most audiometric frequencies tested.ConclusionsTogether, our results validate the use of this improved ML‐based method in adult clinical tests for air‐conduction audiometry.

A brain-computer interface (BCI) is a system that enables control of devices or communication with other persons, only through cerebral activity, without using muscles. The main application for BCIs is assistive technology for disabled persons. Examples for devices that can be controlled by BCIs are artificial limbs, spelling devices, or environment control systems. BCI research has seen renewed interest in recent years, and it has been convincingly shown that communication via a BCI is in principle feasible. However, present day systems still have shortcomings that prevent their widespread application. In part, these shortcomings are caused by limitations in the functionality of the pattern recognition algorithms used for discriminating brain signals in BCIs. Moreover, BCIs are often tested exclusively with able-bodied persons instead of conducting tests with the target user group, namely disabled persons. The goal of this thesis is to extend the functionality of pattern recognition algorithms for BCI systems and to move towards systems that are helpful for disabled users. We discuss extensions of linear discriminant analysis (LDA), which is a simple but efficient method for pattern recognition. In particular, a framework from Bayesian machine learning, the so-called evidence framework, is applied to LDA. An algorithm is obtained that learns classifiers quickly, robustly, and fully automatically. An extension of this algorithm allows to automatically reduce the number of sensors needed for acquisition of brain signals. More specifically, the algorithm allows to perform electrode selection. The algorithm for electrode selection is based on a concept known as automatic relevance determination (ARD) in Bayesian machine learning. The last part of the algorithmic development in this thesis concerns methods for computing accurate estimates of class probabilities in LDA-like classifiers. These probabilities are used to build a BCI that dynamically adapts the amount of acquired data, so that a preset, approximate bound on the probability of misclassifications is not exceeded. To test the algorithms described in this thesis, a BCI specifically tailored for disabled persons is introduced. The system uses electroencephalogram (EEG) signals and is based on the P300 evoked potential. Datasets recorded from five disabled and four able-bodied subjects are used to show that the Bayesian version of LDA outperforms plain LDA in terms of classification accuracy. Also, the impact of different static electrode configurations on classification accuracy is tested. In addition, experiments with the same datasets demonstrate that the algorithm for electrode selection is computationally efficient, yields physiologically plausible results, and improves classification accuracy over static electrode configurations. The classification accuracy is further improved by dynamically adapting the amount of acquired data. Besides the datasets recorded from disabled and able-bodied subjects, benchmark datasets from BCI competitions are used to show that the algorithms discussed in this thesis are competitive with state-of-the-art electroencephalogram (EEG) classification algorithms. While the experiments in this thesis are uniquely performed with P300 datasets, the presented algorithms might also be useful for other types of BCI systems based on the EEG. This is the case because functionalities such as robust and automatic computation of classifiers, electrode selection, and estimation of class probabilities are useful in many BCI systems. Seen from a more general point of view, many applications that rely on the classification of cerebral activity could possibly benefit from the methods developed in this thesis. Among the potential applications are interrogative polygraphy ("lie detection") and clinical applications, for example coma outcome prognosis and depth of anesthesia monitoring.

Chapter 56 Computationally intensive methods for integration in econometrics

Chronic knee conditions and musculoskeletal injuries are prevalent and pose significant challenges to global healthcare systems. Current diagnostic methods, such as MRI and X-ray, provide valuable insights but are limited by high costs and the inability to continuously monitor knee health. In response to these challenges, the proposed work makes the following contributions. Firstly, a novel wearable knee health monitoring system is developed, incorporating a flexible composite piezoelectric film that captures subtle patellar displacements as early indicators of knee joint abnormalities. Secondly, a Bayesian machine learning framework incorporating both expert knowledge and diagnostic data is proposed to address the potential limitations of current machine learning-based diagnostic methods, such as low accuracy and poor interpretability in medical practice. This framework integrates causal discovery algorithms, clinical insights, and medical expertise to identify the key relationships between machine learning models and medical principles, ultimately improving diagnostic accuracy. Finally, the proposed system achieves an 83.3% success rate in distinguishing between healthy, sub-healthy, and unhealthy knee states, outperforming traditional methods with an accuracy improvement of 6.4% to 17%. Furthermore, the system integrates a wireless sensor network and user-friendly software, enabling convenient monitoring and diagnosis in both clinical and home settings. The main innovation of this study lies in the design of a wearable sensor network structure based on patellar activity and a medical knowledge-guided Bayesian framework to enable a three-state classification diagnosis, namely, healthy, sub-healthy, and unhealthy. This system is particularly suited for knee health assessments in outpatient clinics, community settings, and home environments..

Technology Focus Machine-learning methods have gained tremendous attention in the last decade. These methods are tackling and solving complex problems with impressive results in many different areas. Buzzwords such as “big data” and “deep learning” are now part of our everyday vocabulary. The underlying idea behind machine learning is that computers can identify patterns and learn from data with minimal human intervention. This is not very different from the notion of automatic history matching. Actually, the similarities are more profound: Both machine learning and history-matching methods use the same mathematical tools, such as linear algebra, optimization, Bayesian inference, and Monte Carlo sampling. In fact, I believe that machine learning is a natural research direction to improve our current history-matching methods. At this point, some readers must be thinking about the risks of automated processes. They may very well claim that there are no surrogates for experience, a thorough understanding of the reservoir production mechanisms, and attention to the practical aspects and limitations of the models. After all, if machines can learn, humans can think. I am not trying to say otherwise. My only point is that we cannot ignore the recent developments in machine learning and we must not turn a blind eye to what computer-aided strategies can offer. If used responsibly, they can help us achieve better results in less time. To conclude, I would like to point out that machine-learning methods may also find very interesting applications in data-driven production forecasts. In fact, data-driven methods are not new in the petroleum literature; see, for example, decline-curve analysis. These methods are used currently as complements (and often alternatives) to model-driven approaches. Machine-learning methods have much to offer in this regard, especially in cases with an abundance of data. Recommended additional reading at OnePetro: www.onepetro.org. SPE 191473 An Automatic History-Matching Work Flow for Unconventional Coupling MCMC and Nonintrusive EDFM Methods by Wei Yu, The University of Texas at Austin, et al. SPE 191516 Robust Uncertainty Quantification Through Integration of Distributed Gauss-Newton Optimization With Gaussian Mixture Modeling and Parallelized Sampling Algorithms by Guohua Gao, Shell, et al. SPE 191655 An Efficient Probabilistic Assisted History-Matching Tool Using Gaussian Process Proxy Models: Application to Coalbed Methane Reservoir by Sachin Rana, The Pennsylvania State University, et al.