Deep learning hyperparameter optimization: Application to electricity and heat demand prediction for buildings

ABSTRACTThe prediction of pollutants removal efficiency from the generated effluent of a treatment plant is valuable and can reduce the time, sampling and energy required during performance assessment. The present study aims to predict the effect of different input parameters on the treatment efficiency of the developed microbial‐based anaerobic process for textile effluent using machine leaning algorithms. The decolourisation and chemical oxygen demand (COD) reduction of the treated effluent were predicted on the basis of the three different input parameters pH, COD and colour value of the textile wastewater. The effectiveness of different machine learning algorithms, support vector machines (SVM), random forest (RF), gradient boost regressor (GBR), AdaBoost, extreme gradient boosting (XGB) regressor and voting regressor, were evaluated based on the correlation coefficient (R2) value. The results revealed that the RF achieved the highest accuracy for decolourisation (training data R2: ∼0.85 and test data R2: ∼0.84) as well as COD reduction (training data R2: ∼0.87 and test data R2: ∼0.94) compared to the other algorithms. These results were validated experimentally, confirming that RF can be used as a tool to predict the performance efficiency of a microbial‐based treatment system.

Enhancing biomass Pyrolysis: Predictive insights from process simulation integrated with interpretable Machine learning models

Background Machine learning (ML) algorithm support vector machine (SVM) performed better than Thrombolysis in Myocardial Infarction (TIMI) score for ASIAN STEMI patients. However, Deep Learning (DL) effectiveness in the multiethnic ASIAN population has yet to be determined. DL has automatic learning of the feature from a given dataset without the need to conduct feature selection. However, the selected features by the algorithm is black box. Identifying features associated with mortality is essential to recognize characteristics of patients with high risk for better patient management. Purpose To develop a DL algorithm for in-hospital mortality in multiethnic STEMI patients using predictors identified from the SVM algorithm. To investigate DL performance constructed using predictors from SVM feature extraction and expert-recommended predictors. Methods We constructed four algorithms; a) DL and SVM algorithms with predictors identified from the SVM variable importance b) DL and SVM using predictors based on expert recommendation. We used registry data from the National Cardiovascular Disease Database of 11397 patient's. Fifty parameters including demographics, cardiovascular risk, medications and clinical variables were considered. The Area under the curve (AUC) is the performance evaluation metric. Algorithms were validated against the TIMI and tested using the same validation data. SVM variable importance with backward elimination was used to select and rank important variables. Results DL algorithms outperform SVM and TIMI on the validation dataset; i) DL with SVM selected predictors (15 predictors, AUC = 0.97), ii) DL with expert-recommended predictors (16 predictors, AUC = 0.96), iii) SVM with selected predictors (15 predictors, AUC = 0.92), iv) SVM with expert-recommended predictors (AUC = 0.89) and TIMI (AUC = 0.82). Common predictors across SVM feature selection, expert-recommendation and TIMI are: age, heart rate, Killip class, fasting blood glucose, systolic blood pressure, comorbid diseases and ST-elevation. SVM feature selection also identified diuretics, PCI and pharmacotherapy drugs as predictors that improve mortality prediction in STEMI patients. Our findings suggest that the TIMI score underestimates patients risk of mortality. DL algorithm using selected predictors classified 35% of nonsurvival patients as high risk (risk probabilities >50%) compared to only 12.7% nonsurvival patients by TIMI (score >5) (Figure below). Conclusions In the ASIAN population, patients with STEMI can be better classified using the DL algorithm compared to the ML and TIMI score. Combining ML feature selection with DL allows the identification of distinct factors in a unique ASIAN population for better mortality prediction than relying solely on an expert recommendation as it is a very subjective approach. Continuous validation on population-specific algorithms using DL and ML is needed before implementing in a real clinical setting. Funding Acknowledgement Type of funding sources: Public grant(s) – National budget only. Main funding source(s): Technology Development Fund 1 TIMI performance on validation setDL performance on validation set

Outage duration prediction under typhoon disaster with stacking ensemble learning

BackgroundMachine learning (ML) algorithm support vector machine (SVM) performed better than Thrombolysis in Myocardial Infarction (TIMI) score for ASIAN STEMI patients. However, Deep Learning (DL) effectiveness in the multiethnic ASIAN population has yet to be determined. DL has automatic learning of the feature from a given dataset without the need to conduct feature selection. However, the selected features by the algorithm is black box. Identifying features associated with mortality is essential to recognize characteristics of patients with high risk for better patient management.PurposeTo develop a DL algorithm for in-hospital mortality in multiethnic STEMI patients using predictors identified from the SVM algorithm. To investigate DL performance constructed using predictors from SVM feature extraction and expert-recommended predictors.MethodsWe constructed four algorithms; a) DL and SVM algorithms with predictors identified from the SVM variable importance b) DL and SVM using predictors based on expert recommendation. We used registry data from the National Cardiovascular Disease Database of 11397 patient's. Fifty parameters including demographics, cardiovascular risk, medications and clinical variables were considered. The Area under the curve (AUC) is the performance evaluation metric. Algorithms were validated against the TIMI and tested using the same validation data. SVM variable importance with backward elimination was used to select and rank important variables.ResultsDL algorithms outperform SVM and TIMI on the validation dataset; i) DL with SVM selected predictors (15 predictors, AUC = 0.97), ii) DL with expert-recommended predictors (16 predictors, AUC = 0.96), iii) SVM with selected predictors (15 predictors, AUC = 0.92), iv) SVM with expert-recommended predictors (AUC = 0.89) and TIMI (AUC = 0.82). Common predictors across SVM feature selection, expert-recommendation and TIMI are: age, heart rate, Killip class, fasting blood glucose, systolic blood pressure, comorbid diseases and ST-elevation. SVM feature selection also identified diuretics, PCI and pharmacotherapy drugs as predictors that improve mortality prediction in STEMI patients. Our findings suggest that the TIMI score underestimates patients risk of mortality. DL algorithm using selected predictors classified 35% of nonsurvival patients as high risk (risk probabilities >50%) compared to only 12.7% nonsurvival patients by TIMI (score >5) (Figure below).ConclusionsIn the ASIAN population, patients with STEMI can be better classified using the DL algorithm compared to the ML and TIMI score. Combining ML feature selection with DL allows the identification of distinct factors in a unique ASIAN population for better mortality prediction than relying solely on an expert recommendation as it is a very subjective approach. Continuous validation on population-specific algorithms using DL and ML is needed before implementing in a real clinical setting.Funding AcknowledgementType of funding sources: Public grant(s) – National budget only. Main funding source(s): Technology Development Fund 1TIMI performance on validation setDL performance on validation set

The aim of this study is to develop a predictive model utilizing deep learning and machine learning techniques that will inform clinical decision-making by predicting the 1-year postoperative recovery of patients with lumbar disk herniation. The clinical data of 470 inpatients who underwent tubular microdiscectomy (TMD) between January 2018 and January 2021 were retrospectively analyzed as variables. The dataset was randomly divided into a training set (n = 329) and a test set (n = 141) using a 10-fold cross-validation technique. Various deep learning and machine learning algorithms including Random Forests, Extreme Gradient Boosting, Support Vector Machines, Extra Trees, K-Nearest Neighbors, Logistic Regression, Light Gradient Boosting Machine, and MLP (Artificial Neural Networks) were employed to develop predictive models for the recovery of patients with lumbar disk herniation 1 year after surgery. The cure rate score of lumbar JOA score 1 year after TMD was used as an outcome indicator. The primary evaluation metric was the area under the receiver operating characteristic curve (AUC), with additional measures including decision curve analysis (DCA), accuracy, sensitivity, specificity, and others. The heat map of the correlation matrix revealed low inter-feature correlation. The predictive model employing both machine learning and deep learning algorithms was constructed using 15 variables after feature engineering. Among the eight algorithms utilized, the MLP algorithm demonstrated the best performance. Our study findings demonstrate that the MLP algorithm provides superior predictive performance for the recovery of patients with lumbar disk herniation 1 year after surgery.

This study applies machine learning (ML) techniques to classify fertile [for porphyry Cu and (or) Au systems] and barren adakites using geochemical data from New Brunswick, Canada. It emphasizes that not all intrusive units, including adakites, are inherently fertile and should not be directly used as the heat source evidence layer in mineral prospectivity mapping without prior analysis. Adakites play a crucial role in mineral exploration by helping distinguish between fertile and barren intrusive units, which significantly influence ore-forming processes. A dataset of 99 fertile and 66 barren adakites was analyzed using seven ML models: support vector machine (SVM), neural network, random forest (RF), decision tree, AdaBoost, gradient boosting, and logistic regression. These models were applied to classify 829 adakite samples from around the world into fertile and barren categories, with performance evaluated using area under the curve (AUC), classification accuracy, F1 score, precision, recall, and Matthews correlation coefficient (MCC). SVM achieved the highest performance (AUC = 0.91), followed by gradient boosting (0.90) and RF (0.89). For model validation, 160 globally recognized fertile adakites were selected from the dataset based on well-documented fertility characteristics. Among the tested models, SVM demonstrated the highest classification accuracy (93.75%), underscoring its effectiveness in distinguishing fertile from barren adakites for mineral prospectivity mapping. Statistical analysis and feature selection identified middle rare earth elements (REEs), including Gd and Dy, with Hf, as key indicators of fertility. A comprehensive analysis of 1596 scatter plots, generated from 57 geochemical variables, was conducted using linear discriminant analysis (LDA) to determine the most effective variable pairs for distinguishing fertile and barren adakites. The most informative scatter plots featured element vs. element combinations (e.g., Ga vs. Dy, Ga vs. Gd, and Pr vs. Gd), followed by element vs. major oxide (e.g., Fe2O3T vs. Gd and Al2O3 vs. Hf) and ratio vs. element (e.g., La/Sm vs. Gd, Rb/Sr vs. Hf) plots, whereas major oxide vs. major oxide, ratio vs. ratio, and major oxide vs. ratio plots had limited discriminatory power.

ObjectiveThis study aims to develop and compare different models to predict the Length of Stay (LoS) and the Prolonged Length of Stay (PLoS) of inpatients admitted through the emergency department (ED) in general patient settings. This aim is not only to promote any specific model but rather to suggest a decision-supporting tool (i.e., a prediction framework).MethodsWe analyzed a dataset of patients admitted through the ED to the “Sant”Orsola Malpighi University Hospital of Bologna, Italy, between January 1 and October 26, 2022. PLoS was defined as any hospitalization with LoS longer than 6 days. We deployed six classification algorithms for predicting PLoS: Random Forest (RF), Support Vector Machines (SVM), Gradient Boosting (GB), AdaBoost, K-Nearest Neighbors (KNN), and logistic regression (LoR). We evaluated the performance of these models with the Brier score, the area under the ROC curve (AUC), accuracy, sensitivity (recall), specificity, precision, and F1-score. We further developed eight regression models for LoS prediction: Linear Regression (LR), including the penalized linear models Least Absolute Shrinkage and Selection Operator (LASSO), Ridge and Elastic-net regression, Support vector regression, RF regression, KNN, and eXtreme Gradient Boosting (XGBoost) regression. The model performances were measured by their mean square error, mean absolute error, and mean relative error. The dataset was randomly split into a training set (70%) and a validation set (30%).ResultsA total of 12,858 eligible patients were included in our study, of whom 60.88% had a PloS. The GB classifier best predicted PloS (accuracy 75%, AUC 75.4%, Brier score 0.181), followed by LoR classifier (accuracy 75%, AUC 75.2%, Brier score 0.182). These models also showed to be adequately calibrated. Ridge and XGBoost regressions best predicted LoS, with the smallest total prediction error. The overall prediction error is between 6 and 7 days, meaning there is a 6–7 day mean difference between actual and predicted LoS.ConclusionOur results demonstrate the potential of machine learning-based methods to predict LoS and provide valuable insights into the risks behind prolonged hospitalizations. In addition to physicians' clinical expertise, the results of these models can be utilized as input to make informed decisions, such as predicting hospitalizations and enhancing the overall performance of a public healthcare system.

Recycled aggregate concrete (RAC) contributes to mitigating the depletion of natural aggregates, alleviating the carbon footprint of concrete construction, and averting the landfilling of colossal amounts of construction and demolition waste. However, complexities in the mixture optimization of RAC due to the variability of recycled aggregates and lack of accuracy in estimating its compressive strength require novel and sophisticated techniques. This paper aims at developing state-of-the-art machine learning models to predict the RAC compressive strength and optimize its mixture design. Results show that the developed models including Gaussian processes, deep learning, and gradient boosting regression achieved robust predictive performance, with the gradient boosting regression trees yielding highest prediction accuracy. Furthermore, a particle swarm optimization coupled with gradient boosting regression trees model was developed to optimize the mixture design of RAC for various compressive strength classes. The hybrid model achieved cost-saving RAC mixture designs with lower environmental footprint for different target compressive strength classes. The model could be further harvested to achieve sustainable concrete with optimal recycled aggregate content, least cost, and least environmental footprint.

This study presents a comparative analysis of the classification performance of facial and emotion recognition systems using Machine Learning (ML) and Deep Learning (DL) algorithms. The primary objective of this work is to evaluate the applicability of emotion recognition in fields such as psychotherapy and crime analysis, using the FER-2013 dataset.The study was conducted with ML algorithms such as Support Vector Machines (SVM), Random Forest (RF), K-Nearest Neighbor (KNN), Decision Trees, and Gradient Boosting, as well as the DL algorithm, Convolutional Neural Network (CNN). Supported by the application of feature selection, preprocessing, and feature extraction techniques, the model’s performance was measured using standard metrics such as accuracy, precision, F1-score, and AUC-ROC.The experimental results showed that the highest classification accuracy (46.61%) was achieved with the CNN model. While the ML models generally offered lower accuracy, they provided advantages in terms of computational efficiency in specific scenarios.This study aims to contribute to the literature by providing a comparative analysis of ML and DL algorithms and by highlighting the effect of data preprocessing on performance. The findings set targets for future work, such as real-time system integration and hybrid model development.

Groundwater is a critical resource for sustaining agricultural, domestic, and ecological needs in the Upper Blue Nile Basin of Ethiopia, where rapid population growth and climate variability have intensified water stress. The Temecha River Catchment, part of this basin, faces recurrent droughts and declining groundwater levels, underscoring the need for effective groundwater management strategies. One promising approach is Managed Aquifer Recharge (MAR), whose success relies heavily on accurate groundwater level monitoring and forecasting. Although MAR decisions are influenced by hydrogeological and land use factors, hydraulic head data are essential for determining recharge timing and suitability. This study evaluates the performance of five machine learning models—Gradient Boosting Regression (GBR), Support Vector Machine (SVM), Random Forest (RF), Decision Tree (DT), and Linear Regression (LR)—for predicting groundwater level fluctuations in the Temecha River Catchment from 1995 to 2023. Rainfall, temperature, and evapotranspiration were used as predictive variables. Among the models, GBR showed superior performance, with a Mean Absolute Error (MAE) of 0.07 m, Nash-Sutcliffe efficiency (NSE) of 0.7934 m, Coefficient of Determination (R 2 ) of 0.7856, and Percent Bias (PBIAS) of −2.408 %. The results demonstrate GBR's effectiveness in groundwater level forecasting and support its application in data-scarce regions to inform sustainable water resource management. However, adaptation to other regions must consider local hydrogeological and climatic conditions. One sentence summary: This study explores the use of machine learning techniques in Python to predict temporal groundwater levels in the Temecha River Catchment.

With the rapid expansion of deep‐sea development activities, the performance requirements for buoyancy materials have become increasingly stringent. Carbon fiber reinforced epoxy resin composites (CFRP) are a leading candidate due to their superior mechanical properties; however, the degradation mechanisms of the epoxy matrix and CFRP under high‐pressure water absorption remain poorly understood. In this study, standard epoxy and CFRP specimens were prepared and subjected to high‐pressure water absorption tests, followed by comprehensive mechanical characterization. Pearson correlation analysis revealed a strong association between water uptake and mechanical degradation. Eight machine learning algorithms—including Ridge Regression, Decision Tree, Support Vector Regression, Random Forest Regressor, Gradient Boosting Regressor, AdaBoost Regressor, XGBoost, and Neural Networks—were employed to construct predictive models; after data preprocessing and hyperparameter optimization, Gradient Boosting and AdaBoost achieved R 2 values above 0.89 and root mean square error (RMSE) below 1 for epoxy strength prediction, while ensemble methods for CFRP all exceeded R 2 of 0.90. Model forecasts indicate that the strength degradation of epoxy and CFRP stabilizes after approximately 36 and 40 days, respectively. These findings provide a theoretical basis for the design, performance optimization, and lifespan prediction of deep‐sea buoyancy materials and introduce a robust machine learning framework for material performance forecasting.

Optimizing machine learning-based site-specific nitrogen application recommendations for canola production

While the transportation sector is one of largest economic growth drivers for many countries, the adverse impacts of transportation on air quality are also well-noted, especially in developing countries. Carbon dioxide (CO2) emissions are one of the direct results of a transportation sector powered by burning fossil-based fuels. Detailed knowledge of CO2 emissions produced by the transportation sectors in various countries is essential for these countries to revise their future energy investments and policies. In this framework, three machine learning algorithms, ordinary least squares regression (OLS), support vector machine (SVM), and gradient boosting regression (GBR), are used to forecast transportation-based CO2 emissions. Both socioeconomic factors and transportation factors are also included as features in the study. We study the top 30 CO2 emissions-producing countries, including the Tier 1 group (the top five countries, accounting for 61% of global CO2 emissions production) and the Tier 2 group (the next 25 countries, accounting for 35% of total CO2 emissions production). We evaluate our model using four-fold cross-validation and report four frequently used statistical metrics (R2, MAE, rRMSE, and MAPE). Of the three machine learning algorithms, the GBR model with features combining socioeconomic and transportation factors (GBR_ALL) has the best performance, with an R2 value of 0.9943, rRMSE of 0.1165, and MAPE of 0.1408. We also find that both transportation features and socioeconomic features are important for transportation-based CO2 emission prediction. Transportation features are more important in modeling for 30 countries, while socioeconomic features (especially GDP and population) are more important when modeling for Tier 1 and Tier 2 countries.

Artificial intelligence (AI), machine learning (ML) and deep learning (DL) along with big data (BD) management are currently viable approaches that can significantly help gas turbine components’ design and development. Optimizing microstructures of hot section components such as thermal barrier coatings (TBCs) to improve their durability has long been a challenging task in the gas turbine industry. In this paper, a literature review on ML principles and its various associated algorithms was presented first and then followed by its application to investigate thermal conductivity of TBCs. This combined approach can help better understand the physics behind thermal conductivity, and on the other hand, can also boost the design of low thermal conductivity of the TBCs system in terms of microstructure–property relationships. Several ML models and algorithms such as support vector regression (SVR), Gaussian process regression (GPR) and convolution neural network and regression algorithms were used via Python. A large volume of thermal conductivity data was compiled and extracted from the literature for TBCs using PlotDigitizer software and then used to test and validate ML models. It was found that the test data were strongly associated with five key factors as identifiers. The prediction of thermal conductivity was performed using three approaches: polynomial regression, neural network (NN) and gradient boosting regression (GBR). The results suggest that NN using the BR model and GBR have better prediction capability.