Uncertainty-aware Mixed-variable Machine Learning for Materials Design

It is imperative to comprehend the cyclical variations inherent in liquid methane engines (LMEs) across both design and operational domains. The theoretical thermal efficiency of LMEs is high at higher compression ratios, but the combustion instability also increases. Obtaining relevant metrics from bench experiments is difficult and time-consuming; therefore, in this study, we model tabular data using Conditional GAN (CTGAN) to model the tabular data and generated more virtual samples based on the experimental results of the key metrics (peak pressure, maximum pressure rise rate, and average effective pressure). Through this, a machine learning model was proposed that couples a random forest (RF) model with a Bayesian optimization machine learning model for predicting cyclic variation. The findings indicate that the Bayesian-optimized RF model demonstrates superiority in predicting the metrics with greater accuracy and reliability compared to the gradient boosting (XGBoost) and support vector machine (SVM) models. The R2 value of the former model is consistently greater than 0.75, and the root mean square error (RMSE) is typically lower than 0.3. This paper highlights the promising potential of the Bayesian-optimized RF model in predicting unknown cyclic parameters.

Given the increasing importance of machine learning (ML) in our lives, several algorithmic fairness techniques have been proposed to mitigate biases in the outcomes of the ML models. However, most of these techniques are specialized to cater to a single family of ML models and a specific definition of fairness, limiting their adaptibility in practice. We introduce a general constrained Bayesian optimization (BO) framework to optimize the performance of any ML model while enforcing one or multiple fairness constraints. BO is a model-agnostic optimization method that has been successfully applied to automatically tune the hyperparameters of ML models. We apply BO with fairness constraints to a range of popular models, including random forests, gradient boosting, and neural networks, showing that we can obtain accurate and fair solutions by acting solely on the hyperparameters. We also show empirically that our approach is competitive with specialized techniques that enforce model-specific fairness constraints, and outperforms preprocessing methods that learn fair representations of the input data. Moreover, our method can be used in synergy with such specialized fairness techniques to tune their hyperparameters. Finally, we study the relationship between fairness and the hyperparameters selected by BO. We observe a correlation between regularization and unbiased models, explaining why acting on the hyperparameters leads to ML models that generalize well and are fair.

Optimized machine learning models for predicting crown convergence of plateau mountain tunnels

Propofol is a widely used sedative-hypnotic agent for critically-ill patients requiring invasive mechanical ventilation (IMV). Despite its clinical benefits, propofol is associated with increased risks of hypertriglyceridemia. Early identification of patients at risk for propofol-associated hypertriglyceridemia is crucial for optimizing sedation strategies and preventing adverse outcomes. Machine learning (ML) models offer a promising approach for predicting individualized patient risks of propofol-associated hypertriglyceridemia. We propose the development of a ML model aimed at predicting the risk of propofol-associated hypertriglyceridemia in ICU patients receiving IMV. The study will utilize retrospective data from four Mayo Clinic sites. Nested cross-validation (CV) will be employed, with a 10-fold inner CV loop for model tuning and selection as well as an outer loop using leave-one-site-out CV for external validation. Feature selection will be conducted using Boruta and LASSO-penalized logistic regression. Data preprocessing steps include missing data imputation, feature scaling, and dimensionality reduction techniques. Six ML algorithms will be tuned and evaluated. Bayesian optimization will be used for hyperparameter selection. Global model explainability will be assessed using permutation importance, and local model explainability will be assessed using SHapley Additive exPlanations (SHAP). The proposed ML model aims to provide a reliable and interpretable tool for clinicians to predict the risk of propofol-associated hypertriglyceridemia in ICU patients. The final model will be deployed in a web-based clinical risk calculator. The model development process and performance measures obtained during nested cross-validation will be described in a study publication to be disseminated in a peer-reviewed journal. The proposed study has received ethics approval from the Mayo Clinic Institutional Review Board (IRB #23-007416). Robust external validation using a nested cross-validation (CV) framework will help assess the generalizability of models produced from the modeling pipeline across different hospital settings.A diverse set of machine learning (ML) algorithms and advanced hyperparameter tuning techniques will be employed to identify the most optimal model configuration.Integration of feature explainability will enhance the clinical applicability of the ML models by providing transparency in predictions, which can improve clinician trust and encourage adoption.Reliance on retrospective data may introduce biases due to inconsistent or erroneous data collection, and the computational intensity of the validation approach may limit replication and future model expansion in resource-constrained settings.

Aim/Purpose: Traditional hyperparameter tuning methods’ inefficiency and low accuracy in diabetes prediction. Application and evaluation of Bayesian optimization aim to increase the accuracy of diabetes prediction models. Background: Bayesian optimization (BO) has emerged as a powerful technique for hyperparameter tuning in machine learning models with its methodical approach for optimizing complex functions with expensive evaluations. Selecting the optimal hyperparameters for models can significantly increase prediction accuracy in the healthcare sector, particularly for diabetes prediction, hence improving patient outcomes and resource management. Methodology: Among the well-known diabetes databases used in the study is the Pima Indian Diabetes Database. Among the machine learning models developed and whose hyperparameters are modified via Bayesian optimization are Support Vector Machines (SVM), Random Forest, and Gradient Boosting Machines (GBM). The optimization process is compared with traditional tuning methods to assess improvements in model performance. Contribution: Problems include the increasing prevalence of diabetes worldwide and the role vital function prediction models play in early diagnosis and therapy. Especially when dealing with enormous datasets common in healthcare, grid search, and random search are sometimes computationally taxing and inefficient. However, Bayesian optimization promises a more practical and economical approach by selecting hyperparameters iteratively based on earlier evaluations. Findings: Bayesian optimization not only yields faster calculation times but also outperforms traditional methods. More precisely, models adapted by Bayesian optimization show greater sensitivity and specificity, which are crucial for accurate and prompt diabetes diagnosis. Recommendation for Researchers: By highlighting the specific elements and network characteristics that are behind the anomalies it discovers, on a real-time or large scale dataset. Future Research: This work can be improvised using several recent artificial intelligence algorithms with the integration of IoT on real-time datasets.

BackgroundNeoadjuvant immunotherapy has been shown to improve survival in patients with gastric cancer. This study sought to develop and validate a radiomics-based machine learning (ML) model for patients with locally advanced gastric cancer (LAGC), specifically to predict whether patients will achieve a major pathological response (MPR) following neoadjuvant immunotherapy. With its predictive capabilities, this tool shows promise for enhancing clinical decision-making processes in the future.MethodsThis study utilized a multicenter cohort design, retrospectively gathering clinical data and computed tomography (CT) images from 268 patients diagnosed with advanced gastric cancer who underwent neoadjuvant immunotherapy between January 2019 and December 2023 from two medical centers. Radiomic features were extracted from CT images, and a multi-step feature selection procedure was applied to identify the top 20 representative features. Nine ML algorithms were implemented to build prediction models, with the optimal algorithm selected for the final prediction model. The hyperparameters of the chosen model were fine-tuned using Bayesian optimization and grid search. The performance of the model was evaluated using several metrics, including the area under the curve (AUC), accuracy, and Cohen’s kappa coefficient.ResultsThree cohorts were included in this study: the development cohort (DC, n = 86), the internal validation cohort (IVC, n = 59), and the external validation cohort (EVC, n = 52). Nine ML models were developed using DC cases. Among these, an optimized Bayesian-LightGBM model, demonstrated robust predictive performance for MPR following neoadjuvant immunotherapy in LAGC patients across all cohorts. Specifically, within DC, the LightGBM model attained an AUC of 0.828, an overall accuracy of 0.791, a Cohen’s kappa coefficient of 0.552, a sensitivity of 0.742, a specificity of 0.818, a positive predictive value (PPV) of 0.586, a negative predictive value (NPV) of 0.867, a Matthews correlation coefficient (MCC) of 0.473, and a balanced accuracy of 0.780. Comparable performance metrics were validated in both the IVC and the EVC, with AUC values of 0.777 and 0.714, and overall accuracies of 0.729 and 0.654, respectively. These results suggested good fitness and generalization of the Bayesian-LightGBM model. Shapley Additive Explanations (SHAP) analysis identified significant radiomic features contributing to the model’s predictive capability. The SHAP values of the features wavelet.LLH_gldm_SmallDependenceLowGrayLevelEmphasis, wavelet.HHL_glrlm_RunVariance, and wavelet.LLH_glszm_LargeAreaHighGrayLevelEmphasis were ranked among the top three, highlighting their significant contribution to the model’s predictive performance. In contrast to existing radiomic models that exclusively focus on neoadjuvant chemotherapy, our model integrates both neoadjuvant immunotherapy and chemotherapy, thereby offering more precise predictive capabilities.ConclusionThe radiomics-based ML model demonstrated significant efficacy in predicting the pathological response to neoadjuvant immunotherapy in LAGC patients, thereby providing a foundation for personalized treatment strategies.

For most machine learning models, the mapping from the hyper-parameter set to the model's generalization error can be regarded as a complex black box function. Particle swarm optimization (PSO) methods cannot be directly used in the problem of hyper-parameters estimation since the mathematical formulation of the mapping from hyper-parameters to loss function or generalization accuracy is unclear. Functions with high evaluation costs can be solved by Bayesian optimization (BO) which converting the optimization of hyper-parameters into the optimization of an acquisition function. The proposed method in this paper uses the particle swarm method to optimize the acquisition function in the BO to get better hyper-parameters. The performances of proposed method in both of the classification and regression models are evaluated and demonstrated.

Risk and susceptibility mapping of groundwater salinity (GWS) are challenging tasks for groundwater quality monitoring and management. Advancement of accurate prediction systems is essential for the identification of vulnerable areas in order to raise awareness about the potential salinity susceptibility and protect the groundwater and top-soil in due time. In this study, three machine learning models of Stochastic Gradient Boosting (StoGB), Rotation Forest (RotFor), and Bayesian Generalized Linear Model (Bayesglm) are developed for building prediction models and their performance evaluated in the delineation of salinity susceptibility maps. Both natural and human effective factors (16 features) were used as predictors for groundwater salinity modeling and were randomly divided into the training (80%) and testing (20%) datasets. The models were evaluated using testing datasets after calibration using the selected features by recursive feature elimination (RFE) method. The RFE indicated that modeling with 8 features had better performance among 1 to 16 features (Accuracy = 0.87). Results of the groundwater salinity prediction highlighted that StoGB had a good performance, whereas the RotFor and Bayesglm had an excellent performance based on the Kappa values (>0.85). Although spatial prediction of the models was different, all of the models indicated that central parts of the region have a very high susceptibility which matches with agricultural areas, lithology map, the locations with low depth to groundwater, low slope, and elevation. Additionally, areas near to the Maharlu lake and locations with a high decline in groundwater are also located in the very high susceptibility zone, which can confirm the effects of saltwater intrusion. The susceptibility maps produced in this study are of utmost importance for water security and sustainable agriculture.

This study presents a comprehensive investigation into the interplay between machine learning (ML) models, morphological features, and outdoor thermal comfort (OTC) across three key indices: Universal Thermal Climate Index (UTCI), Physiological Equivalent Temperature (PET), and Predicted Mean Vote (PMV). Based on a comprehensive field measurement for 173 urban canyons, proper dataset for summer outdoor thermal condition was provided. Concurrently, six distinct ML models were evaluated and optimized using Bayesian optimization (BO) technique, considering performance indicators like weighted accuracy, F1-Score, precision, and recall. Notable trends emerged, with the CatBoost Classifier demonstrating superior performance in UTCI prediction, the Random Forest classifier excelling in PET estimation, and the XGBoost Classifier achieving optimal PMV prediction. Furthermore, the study delved into the influence of morphological features on OTC, prioritizing factors using SHAP values. Results consistently identified 90-degree orientation, street width, and 180-degree orientation as pivotal factors influencing OTC, with varying degrees of sensitivity across different classifications of thermal stress. Analysis of binary SHAP values unveiled intricate relationships between urban features and OTC indices, emphasizing the critical influence of street orientation on regulating outdoor thermal environments for UTCI and PET scenarios. Surprisingly, street width emerged as the foremost influential factor within the PMV index, challenging established trends and highlighting the complexity of thermal comfort modeling. Additionally, current research delineates the multifaceted impact of street width on microclimate dynamics, enriching our understanding of urban thermal dynamics and emphasizing its role in mitigating thermal stress within urban environments.

Shale caprocks, often organic-rich, play a significant role in hydrogen sequestration, but their wettability is typically altered from water-wet to H2-wet due to the organic matter, compromising their sealing performance. Silica nanofluids can restore water-wet conditions, which is vital for effective containment of H2 in geological formations. However, laboratory experiments to test the effects of nanomaterials on wettability and interfacial tension (IFT) are resource-intensive and time-consuming. To address this challenge, data-driven machine learning (ML) models offer a promising alternative, allowing for the prediction of key UHS parameters—such as advancing and receding contact angles without the need for extensive laboratory testing. We have applied multiple ML models, including gradient boosting, extreme gradient boosting (XGboost), Catboost, random forest, and extra trees to predict the wettability of organic rich shale samples treated with silica nanofluids. The ML models were trained on a comprehensive dataset generated through laboratory experiments conducted under a wide range of pressure (0.5 – 1600psi) and temperature (298-323 K) conditions, using brine (1 wt.% KCl + 2 wt.% NaCl) and varying silica nanofluid concentrations (0.05–1.0 wt.%). Advanced visualization techniques, including swarm maps, box plots, and heatmap plots, were utilized to analyze the experimental dataset thoroughly. To enhance the generalization capabilities of the ML models, we employed Bayesian optimization for hyperparameter tuning, along with k-fold cross-validation and grid-search optimization techniques. Both statistical and graphical analyses were performed to show the reliability and performance of the models. The results demonstrated that the implemented ML models, particularly the XGboost and Catboost, accurately predicted the wettability behavior under various operating conditions. The training and testing average R² scores were above 0.998 and 0.949, respectively, confirming the high accuracy of the predictions. These findings highlight the reliability of the ML algorithms in predicting the complex interaction between shale, brine, and H2. Further investigation into the feature importance revealed that nanomaterials had the greatest influence on the wettability of the ternary system. The silica nanofluid concentrations played a key role in modifying wettability, with varying impacts on the receding contact angles, emphasizing the effect of nanomaterial treatment. The model's predictions were consistent with experimental results, suggesting that geological conditions, including pressure and temperature, have a profound influence on rock mineral wettability, directly impacting H2 geo-storage capacity. This study is the first to employ data-driven ML models to predict critical UHS parameters in organic-rich shales treated with silica nanofluids. The ability of the ML models to predict UHS parameters across a wide range of conditions provides a valuable tool for optimizing UHS system designs, offering a more efficient and scalable approach compared to traditional experimental methods.

Inverse design of composite metal oxide optical materials based on deep transfer learning and global optimization

Oil viscosity is an important parameter for estimating the flow behavior of crude oil under varying pressure and temperature regimes encountered in the subsurface and surface components of the petroleum production system. Although experimental data provides the most accurate characterization of this property, obtaining such data in the laboratory is often time-consuming and expensive. Alternatively, one of several empirical correlations can be used, albeit with often poor generalization to samples from differing thermodynamic systems or crude oil types. In this paper, we use a data-driven approach based on machine learning (ML) algorithms to create high-performing models for predicting the dead oil and bubblepoint oil viscosity of Nigerian crudes. Random Forest (RF) and Gradient Boosting (GB) ensembles, as well as Multi-Layer Perceptron (MLP) and Radial Basis Function (RBF) networks, are compared to several widely used correlations, and their performance is evaluated on a large dataset of crude oil samples collected from various Niger Delta oil fields. The ML models are built with minimal and easily accessible input parameters shared by the correlations, allowing for an objective assessment of their superiority to these correlations given the same input framework. These are reservoir temperature and oil API gravity for the dead oil viscosity modeling, and the solution gas-oil ratio and dead oil viscosity for the bubblepoint oil viscosity modeling. Furthermore, the model's reliability is tested by partitioning the dataset into training, validating, and testing sets, and optimal model configurations are achieved through Bayesian Optimization. On the dead oil viscosity modeling task, we achieve a modest improvement over the best-performing correlation developed by Al-Khafaji (1987). The RF model's R2 and mean absolute error (MAE) scores on the testing data are 0.831 and 3.423, respectively, versus 0.809 and 4.439 for the correlation method. However, for the bubblepoint oil viscosity modeling, the improvement recorded using the RF model over the best-performing correlation developed by Beggs and Robinson (1975) is remarkable. In this case, the R2 and MAE scores for the RF model are 0.924 and 0.441, respectively, compared to 0.574 and 0.813 for the correlation method. As a result, our ML models have outperformed conventional correlations for modeling oil viscosity. Ultimately, oil viscosity estimates from the developed ML models can provide a much more accurate characterization of the fluid behavior of Niger Delta crudes than the conventional correlations can. Furthermore, the ML models allow for continuous improvement of their generalization ability by introducing new data from various sources to the models.

Background Accurate prediction of outcomes following a heart transplant is critical to explaining risks and benefits to patients and decision-making when considering potential organ offers. Given the large number of potential variables to be considered, this task may be most efficiently performed using machine learning (ML). Purpose We trained and tested different ML algorithms to accurately predict outcomes following a cardiac transplant using the United Network of Organ Sharing (UNOS) database. Methods We included 67 939 adult and pediatric patients enrolled in the UNOS database between January 1994 and December 2016 who underwent cardiac transplantation (median age 53 [IQR 38 – 60], 72.7% males). In our models, as an input, we included 114 features that have been collected from recipients and donors prior to transplant. The primary outcome was all-cause mortality at one-year post-transplant. We evaluated three different ML methods: XGBoost, Random Forest (RF) and L2 regularized logistic regression. Algorithms were trained and tested using shuffled 10-fold cross-validation (CV) as well as rolling CV. In the rolling CV, to mimic prospective procedure, ML models were trained by incrementally adding patients according to transplant year and testing models on the data in the following year. The hyperparameters, controlling the learning process, were tuned using Bayesian optimization. Prognostic accuracy for one-year all-cause mortality was characterized using the area under the receiver-operating characteristic curve (AUC). Results In total, 8,394 patients died within 1 year of transplant. We observed a substantial difference in prognostic accuracy between the shuffled 10-fold CV and the rolling CV. In the 10-fold CV, XGBoost and RF achieved high predictive performance with AUC of 0.848 (95% CI: 0.842–0.854) and 0.891 (95% CI: 0.886–0.896), respectively. In the rolling CV, which is a more realistic setting, AUC dropped to 0.673 (95% CI: 0.661–0.684) for XGBoost and 0.670 (0.657–0.683) for RF. Predictive performance of L2 regularized logistic regression remained stable across the two CV procedures, achieving AUC 0.669 (95% CI: 0.662–0.676) in the 10-fold shuffled CV and 0.665 (95% CI: 0.649–0.680) in the rolling CV procedure (Figure). Conclusions Our study suggests that ML models could be used to predict mortality in the first year post-transplant. We also show that the choice of CV procedure is crucial for evaluating ML models, particularly in data collected over a long period of time. The difference between the shuffled and rolling CV in the predictive performance of the tree-based ML models might indicate temporal dataset shift. In the rolling CV, all three methods achieved similar predictive performance. Funding Acknowledgement Type of funding sources: Public grant(s) – National budget only. Main funding source(s): Research Foundation Flanders (FWO)

Multi-objective materials bayesian optimization with active learning of design constraints: Design of ductile refractory multi-principal-element alloys

This study proposes an investment portfolio optimization system based on the AutoML framework, which integrates Bayesian hyperparameter optimization and machine learning techniques to achieve accurate prediction of financial market volatility and dynamic asset allocation. The system first extracts multi-scale features (including short-term, medium-term, and long-term returns, volatility, trading volume, and momentum indicators) based on historical stock price data, and uses a Bayesian optimized support vector regression (SVR) model to predict the volatility for the next 20 days; At the asset allocation level, the system dynamically adjusts the weights of the investment portfolio based on the predicted volatility, controlling the target volatility at 20% while constraining the leverage ratio within the range of 0-150%. The empirical results show that: 1) The volatility prediction model has excellent performance.