Lithofacies prediction from core images using Bayesian neural networks

This paper presents a novel texture segmentation method using neural networks and a Markov random field (MRF) model. Multi-scale wavelet coefficients are used as input for the neural networks. The output of the neural network is modeled as a posterior probability. Initially, the multi-scale texture segmentation is performed by the posterior probabilities from the neural networks and MAP (maximum a posterior) classification. Then the MAP segmentation maps are produced at all scales. In order to obtain the more improved segmentation result at the finest scale, our proposed method fuses the multi-scale MAP segmentations sequentially from coarse to fine scales. This is done by computing the MAP segmentation given the segmentation map at one scale and a priori knowledge regarding contextual information which is extracted from the adjacent coarser scale segmentation. In this fusion process, the MRF prior distribution and Gibbs sampler are used, where the MRF model serves as the smoothness constraint and the Gibbs sampler acts as the MAP classifier.

Timely information on lake ice cover conditions is critical in support of commercial shipping, winter-road transportation, and winter leisure activities such as snowmobiling and ice fishing. Monitoring of lake ice extent and ice phenology (i.e. dates associated with freeze-up/break-up and ice duration) is also valuable for improving numerical weather prediction (NWP) and for climate monitoring. The availability of free synthetic aperture radar (SAR) data from the European Space Agency’s Sentinel-1 A/B constellation provides an unprecedented opportunity to develop operational algorithms for lake ice cover mapping at a temporal frequency not available until now from SAR missions (ca. every 1–5 days depending on latitude). For NWP, rapid and accurate mapping of ice cover and open water areas in lake-rich regions is required. However, the classification of SAR imagery using traditional machine learning (ML) approaches is challenging due to the large data volume generated by imaging systems such as Sentinel-1 SAR as well as the complexity of radar signatures as a function of sensor and target characteristics. For lakes specifically, radar backscatter can vary greatly with incidence angle, polarization and surface properties (e.g. calm and wind-roughed open water, new thin ice, surface roughness due to the presence of pressure ridges, ice type, melting of ice and on-ice snow). To address the challenge of large-scale lake ice mapping, we investigate the GPU-boosted deep neural network approach that is efficient at handling big complex data. In this paper, we design a novel maximum a posterior (MAP) approach combining a convolutional neural network (CNN) and conditional random field (CRF) for better addressing the challenges of operational lake ice mapping. The proposed approach is tested on 17 Sentinel-1 dual-polarization (VV and VH) SAR images, where eight are used for training and validation and nine used for out-of-sample testing. To identify the optimal network architecture, an independent validation set is used to evaluate the performance of a series of six CNNs with increasing model complexity. The best model overcomes the noise effect in Sentinel-1 SAR imagery and the lake ice signature ambiguity issue; it achieves average classification accuracies of 97.10% and 97.14% for open water and ice, respectively, on the validation set. Moreover, the best model outperforms the last-ranked model by about 2% in terms of mean overall accuracy (OA), demonstrating the improvement and importance of model selection. The use of CRF can consistently improve the CNN by reducing the artefacts and noise effect in ice maps, outperforming the CNN model when used alone by about 3% in terms of mean OA on the validation set. The proposed CNN-CRF approach also achieves high accuracy on the nine test scenes, achieving a mean OA of 93.10%, demonstrating strong generalization capability that is important for SAR lake ice mapping.

Quality measures for the evaluation of machine learning architectures on the quantification of epistemic and aleatoric uncertainties in complex dynamical systems

Daily scale streamflow forecasting in multiple stream orders of Cauvery River, India: Application of advanced ensemble and deep learning models

The intermittence and fluctuation of renewable energy bring significant uncertainty to the power system, which enormously increases the operational risks of the power system. The development of efficient interval prediction models can provide data support for decision making and help improve the economy and reliability of energy interconnection operation. The performance of Bayesian deep learning models and Bayesian shallow neural networks in short-term interval prediction of photovoltaic power is compared in this study. Specifically, an LSTM Approximate Bayesian Neural Network model (ABNN-I) is built on the basis of the deep learning and Monte Carlo Dropout method. Meanwhile, a Feedforward Bayesian Neural Network (ABNN-II) model is introduced by Feedforward Neural Network and the Markov Chain Monte Carlo method. To better compare and verify the interval prediction capability of the ABNN models, a novel clustering method with three-dimensional features which include the number of peaks and valleys, the average power value, and the non-stationary measurement coefficient is proposed for generating sunny and non-sunny clustering sets, respectively. Results show that the ABNN-I model has an excellent performance in the field of photovoltaic short-term interval forecasting. At a 95% confidence level, the interval coverage from ABNN-I to ABNN-II can be increased by up to 3.1% and the average width of the interval can be reduced by 56%. Therefore, with the help of the high computational capacity of deep learning and the inherent ability to quantify uncertainty of the interval forecast from Bayesian methods, this research provides high-quality interval prediction results for photovoltaic power prediction and solves the problem of difficult modeling for over-fitting that exists in the training process, especially on the non-sunny clustering sets.

With the advances in computational power and numerical methods, analysts can now rely on first-principle simulations to predict ultra-fine details in a variety of applications. Advances in machine learning (ML) have produced algorithms that can now learn high-level abstractions via hierarchical models. This project aims to leverage advances in ML techniques and the available high-resolution simulation data to develop a novel modeling and simulation (M\&S) methodology for reactor safety analysis. While application-agnostic ML techniques are available, complex physics constraints need to be incorporated into ML techniques to build ML-based closures for computationally efficient predictive simulations. This project intends to develop a physics-guided data-driven multi-scale methodology for M\&S of advanced reactors. The project focuses on thermal fluid (T/F) phenomena, which play major roles in advanced reactor safety. Specifically, we propose a data-driven coarse-mesh turbulence model based on local flow features for the transient analysis of thermal mixing and stratification in a sodium-cooled fast reactor (SFR). The model has a coarse-mesh setup to ensure computational efficiency, while it is trained by fine-mesh computational fluid dynamics (CFD) data with Reynolds-averaged Navier-Stokes (RANS) turbulence model to ensure accuracy. Three different neural networks are developed and tested for loss-of-flow transients in the hot pool of SFR, i.e. the densely connected convolutional neural network (DCNN), long-short-term-memory network based on proper orthogonal decomposition (POD-LSTM), and the DCNN informed by LSTM (DCNN-LSTM). The performances of these three neural networks are evaluated based on baseline models. The DCNN-LSTM model has been chosen for further hyperparameter optimization. Furthermore, based on a simplified two-dimensional case, uncertainty quantification (UQ) of the developed ML-based closure are investigated with three methods, i.e. Monte Carlo dropout, deep ensemble, and Bayesian neural network. The developed ML-based turbulent viscosity closure relation based on deep ensemble is then integrated into the system analysis module SAM and serves as a term in the conservation equations. Such a SAM-ML based procedure guarantees that the obtained results are consistent with the physical constraints of the thermal-fluid system. The SAM-ML simulation on the same loss-of-flow transient showed comparable accuracy with the CFD simulation but with a much coarser mesh setup. Last but not least, the ML-based closure improvement with the support of higher-fidelity data from large eddy simulation (LES) is discussed. As a first step towards this direction, a baseline LES simulation is performed to obtain comparable data with RANS results. Based on the early results, future investigation on further improving the ML-based closure is discussed. We believe the developed approach that combines scientific machine learning with nuclear system analysis code can benefit the advanced reactor community as more accurate safety analyses will better characterize reactor safety margins and reduce licensing efforts.

Landslide disaster risk reduction necessitates the investigation of different geotechnical causal factors for slope failures. Machine learning (ML) techniques have been proposed to study causal factors across many application areas. However, the development of ensemble ML techniques for identifying the geotechnical causal factors for slope failures and their subsequent prediction has lacked in literature. The primary goal of this research is to develop and evaluate novel feature selection methods for identifying causal factors for slope failures and assess the potential of ensemble and individual ML techniques for slope failure prediction. Twenty-one geotechnical causal factors were obtained from 60 sites (both landslide and non-landslide) spread across a landslide-prone area in Mandi, India. Relevant causal factors were evaluated by developing a novel ensemble feature selection method that involved an average of different individual feature selection methods like correlation, information-gain, gain-ratio, OneR, and F-ratio. Furthermore, different ensemble ML techniques (Random Forest (RF), AdaBoost (AB), Bagging, Stacking, and Voting) and individual ML techniques (Bayesian network (BN), decision tree (DT), multilayer perceptron (MLP), and support vector machine (SVM)) were calibrated to 70% of the locations and tested on 30% of the sites. The ensemble feature selection method yielded six major contributing parameters to slope failures: relative compaction, porosity, saturated permeability, slope angle, angle of the internal friction, and in-situ moisture content. Furthermore, the ensemble RF and AB techniques performed the best compared to other ensemble and individual ML techniques on test data. The present study discusses the implications of different causal factors for slope failure prediction.

The drilling process can result in irregular measurements due to unconsolidated geological formations, affecting the accuracy of wireline logging devices. This impacts the precision of elastic log measurements, such as velocity and density profiles, which are essential for reservoir characterization. The reliability of the wireline-logging tool is crucial in preventing inaccuracies when assessing reservoir rock properties. Previous studies have focused on applying machine learning (ML) techniques to wireline logging, but these methods have limited applicability, particularly for outlier detection and log reconstruction. In response, this study integrates both supervised and unsupervised ML techniques to enhance the accuracy of elastic log responses in reservoir characterization. Initially, density-based spatial clustering of applications with noise was applied for outlier detection, followed by feature selection to identify correlated logs for reconstructing the density log. A random forest regression model, optimized with particle swarm optimization (PSO), was then trained using the selected features. The comparative analysis showed a significant improvement in porosity estimation from the reconstructed density log compared to core data. Specifically, the comparison between core and original bulk density porosity yielded an R2 of 0.95 and a root mean squared error (RMSE) of 0.012. In contrast, the comparison between core and the rebuilt density log porosity resulted in an R2 of 0.98 and an RMSE of 0.007. The integration of advanced ML techniques with PSO-optimized random forest models represents a considerable advancement in the field of reservoir characterization. This approach enhances accuracy but also saves time and reduces manual effort, highlighting considerable potential for the advancement of methods in petroleum exploration and production.

Laramide basement blocks of the U.S. Rocky Mountain Province have long been considered the sediment source of the Paleocene–Eocene Wilcox Group, implying that large-scale progradation driven by Laramide uplift induced a transition from carbonate to clastic deposition in the Gulf of Mexico. Detrital zircon U-Pb age spectra from 10 samples collected from the Wilcox Group in south Texas reveal a complex grain assemblage with major age populations at ca. 1800–1300 Ma and 350–50 Ma and minor populations at 1300–950 Ma, 3250–1850 Ma, and 900–400 Ma. Modal data for Wilcox Group sand grain populations from the same region suggest a diverse provenance, including sources from the Cordilleran magmatic arc and subordinate reworked sedimentary rocks. Sediment was primarily derived from a mix of crystalline basement blocks in the U.S. southern Rocky Mountains and northern Mexico, the Cordilleran arc of western Mexico and the southwestern United States, and Mesozoic–Paleogene magmatic rocks of northern Mexico. Secondary sources include recycled strata from Sevier-Laramide basins in the southwestern United States and northern Mexico. Comparisons of detrital zircon U-Pb age spectra show that the Lower Wilcox subunit contains significant magmatic arc–derived sediment, whereas the Upper Wilcox subunit is more enriched in basement and recycled sedimentary detritus. Therefore, the relatively quartzose composition of the bulk detrital assemblages throughout the Wilcox Group of south Texas is not an accurate reflection of the range of volcanic and basement sources indicated by the associated U-Pb ages, suggesting significant attrition of unstable grains during long-distance (>500–1000 km) transport.

Lithology identification of geological beds in the subsurface is fundamental in reservoir characterization. Recently, automated log analysis has an increasing demand in reservoir research and the oil industry. In this context, Machine Learning (ML) techniques arise as a surrogate model to provide lithology identification in a fast way. However, to achieve suitable performance, ML techniques require the adjustment of some parameters, and that can become a hard task, depending on the difficulty of the problem to be solved. This paper presents an Artificial Neural Network (ANN), assisted by an adaptive Differential Evolution (DE) algorithm to classify petrophysical data in the Southern Provence Basin. The main contribution is searching for a competent ANN configuration, including architecture, activation functions, regularization, and training algorithms. The proposed approach outperformed four classifiers and two results previously published. The computational methodology proposed here is able to assist in the classification of petrophysical data, helping to improve the procedure of reservoir characterization and the idealization of the development of production.

The latest technological advancements in the development and production of sensors have led to their increased usage in marine science, thus expanding data volume and rates within the field. The extensive data collection efforts to monitor and maintain the health of marine environments supports the efforts in data driven learning, which can help policy makers in making effective decisions. Machine learning techniques show a lot of promise for improving the quality and scope of marine research by detecting implicit patterns and hidden trends, especially in big datasets that are difficult to analyze with traditional methods. Machine learning is extensively used on marine science data collected in various regions, but it has not been applied in a significant way to data generated in the Gulf of Mexico (GOM). Machine learning methods using ocean science data are showing encouraging results and thus are drawing interest from data science researchers and marine scientists to further the research. The purpose of this paper is to review the existing approaches in studying GOM data, the state of the art in machine learning techniques as applied to the GOM, and propose solutions to GOM data problems. We review several issues faced by marine environments in GOM in addition to climate change and its effects. We also present machine learning techniques and methods used elsewhere to address similar problems and propose applications to problems in the GOM. We find that Harmful Algal Blooms (HABs), hypoxia, and sea-level rises have not received as much attention as other climate change problems and within the machine learning literature, the impacts on estuaries and coastal systems, as well as oyster mortality (also major problems for the GOM) have been understudied – we identify those as important areas for improvement. We anticipate this manuscript will act as a baseline for data science researchers and marine scientists to solve problems in the GOM collaboratively and/or independently.

In recent years, planners have started considering Machine Learning (ML) techniques as an alternative to discrete choice models (CM). ML techniques are primarily data-driven and typically achieve better prediction accuracy compared to CM. However, it is hypothesized that since the ML techniques do not have the strong grounding to economic theory as the CMs, they may not perform well in contexts that are radically different from the ‘training’ scenario. It is also hypothesized that the relative prediction performance may be affected by the metrics used for comparing the models. This research aims to test these two hypotheses empirically by modelling vehicle ownership choices using household survey data from Dhaka, Bangladesh collected in 2004, 2010 and 2019. The performances of CM (multinomial logit) and ML techniques (neural networks and gradient boosting trees) have been compared using log-likelihood and mean absolute percentage error of market shares. The results indicate that the multinomial logit model (MNL) with a piecewise linear transformation of the household income, has the best performance in terms of log-likelihood and mean absolute percentage error of market shares. This is followed by Neural Networks (NN) and Gradient Boosting Trees (GBT). The results thus provide empirical evidence that the ML techniques do not consistently outperform CM. Moreover, the difference in the performance of the models further increases if the prediction scenario is substantially different. This reinforces the hypothesis that CMs, with their behavioural underpinning, are better suited for long-term forecasting than data-driven ML approaches, especially if the population and network attributes are expected to change substantially. These findings will be useful for planners and policy makers in the selection of the appropriate tool for forecasting travel demand.

The Gulf of Mexico is an ecologically and economically important marine ecosystem that is affected by a variety of natural and anthropogeni c pressures. These complex and interacting pressures, together with the dynamic environment of the Gulf, present challenges for the effective management of its resources. The recent adoption of Bayesian networks to ecology allows for the discovery and quan tification of complex interactions from data after making only a few assumptions about observations of the system. In this study, we apply Bayesian network models, with different levels of structural complexity and a varying number of hidden variables to account for uncertainty when modelling ecosystem dynamics. From these models, we predict focal ecosystem components within the Gulf of Mexico. The predictive ability of the models varied with their structure. The model that performed best was parameterized through data- driven learning techniques and accounted for multispecies associations and their interactions with human and natural pressures over time. Then, we altered sea surface temperature in the best performing model to explore the response of different variables to increased temperature. The magnitude and even direction of predicted responses varied by ecosystem components due to heterogeneity in driving factors and their spatial overlap. Our findings suggest that due to varying species sensitivity to drivers, changes in temperature will potentially lead to trade- offs in terms of population productivity. We were able to discover meaningful interactions between ecosystem components and their environment and show how sensitive these relationships are to climate perturbations, which increases our understanding of the potential future response of the system to increasing temperature. Our findings demonstrate that accounting for additional sources of variation, by incorporating multiple interactions and press ures in the model layout, has the potential for gaining deeper insights into the structure and dynamics of ecosystems.

Studies of intrinsically disordered proteins that lack a stable tertiary structure but still have important biological functions critically rely on computational methods that predict this property based on sequence information. Although a number of fairly successful models for prediction of protein disorder were developed over the last decade, the quality of their predictions is limited by available cases of confirmed disorders. To more reliably estimate protein disorder from protein sequences, an iterative algorithm is proposed that integrates predictions of multiple disorder models without relying on any protein sequences with confirmed disorder annotation. The iterative method alternately provides the maximum a posterior (MAP) estimation of disorder prediction and the maximum-likelihood (ML) estimation of quality of multiple disorder predictors. Experiments on data used at the Critical Assessment of Techniques for Protein Structure Prediction (CASP7 and CASP8) have shown the effectiveness of the proposed algorithm.

On the relative expressiveness of Bayesian and neural networks