A comparative study of machine learning regression models for predicting construction duration

The evaluation of load-settlement behavior of piles is very crucial in meeting the serviceability criteria for pile analysis and design. The most reliable approach for estimating this behavior can be achieved by conducting pile load tests. However, due to the considerable expense and time requirement of such in-situ testing, the load-transfer methods have been used routinely in practice. In this paper, an alternative tree-based machine learning (ML) modeling is explored to predict the load-settlement behavior of axially loaded single piles from cone penetration test (CPT) data. Two variants of tree-based ML models, the random forest (RF) and gradient boosted tree (GBT), are developed in this study to estimate the load-settlement behavior of piles from CPT data (corrected cone tip resistance, qt, and sleeve friction, fs). A database of load-settlement curves of 64 static pile load tests and the corresponding CPT test data were compiled and used for the development of these ML models. The developed RF and GBT models are evaluated based on several statistical criteria. The load-settlement curves for six PLTs predicted using the developed RF and GBT models were compared with the measured data and the load-settlement curves predicted using the conventional load-transfer methods. The results demonstrated the great potential of tree-based ML (RF, GBT) models for predicting the load-settlement behavior of axially loaded piles from CPT data. The comparison clearly shows that the ML models outperformed the conventional load-transfer methods. Amongst the two ML models, the results show that the GBT model outperformed the RF model.

Abstract. Machine learning (ML) models are becoming a meaningful tool for modeling air pollutant concentrations. ML models are capable of learning and modeling complex nonlinear interactions between variables, and they require less computational effort than chemical transport models (CTMs). In this study, we used gradient-boosted tree (GBT) and multi-layer perceptron (MLP; neural network) algorithms to model near-surface nitrogen dioxide (NO2) and ozone (O3) concentrations over Germany at 0.1∘ spatial resolution and daily intervals. We trained the ML models using TROPOspheric Monitoring Instrument (TROPOMI) satellite column measurements combined with information on emission sources, air pollutant precursors, and meteorology as feature variables. We found that the trained GBT model for NO2 and O3 explained a major portion of the observed concentrations (R2=0.68–0.88 and RMSE=4.77–8.67 µg m−3; R2=0.74–0.92 and RMSE=8.53–13.2 µg m−3, respectively). The trained MLP model performed worse than the trained GBT model for both NO2 and O3 (R2=0.46–0.82 and R2=0.42–0.9, respectively). Our NO2 GBT model outperforms the CAMS model, a data-assimilated CTM but slightly underperforms for O3. However, our NO2 and O3 ML models require less computational effort than CTM. Therefore, we can analyze people's exposure to near-surface NO2 and O3 with significantly less effort. During the study period (30 April 2018 and 1 July 2021), it was found that around 36 % of people lived in locations where the World Health Organization (WHO) NO2 limit was exceeded for more than 25 % of the days during the study period, while 90 % of the population resided in areas where the WHO O3 limit was surpassed for over 25 % of the study days. Although metropolitan areas had high NO2 concentrations, rural areas, particularly in southern Germany, had high O3 concentrations. Furthermore, our ML models can be used to evaluate the effectiveness of mitigation policies. Near-surface NO2 and O3 concentration changes during the 2020 COVID-19 lockdown period over Germany were indeed reproduced by the GBT model, with meteorology-normalized near-surface NO2 having significantly decreased (by 23±5.3 %) and meteorology-normalized near-surface O3 having slightly increased (by 1±4.6 %) over 10 major German metropolitan areas when compared to 2019. Finally, our O3 GBT model is highly transferable to neighboring countries and locations where no measurements are available (R2=0.87–0.94), whereas our NO2 GBT model is moderately transferable (R2=0.32–0.64).

Shale caprocks, often organic-rich, play a significant role in hydrogen sequestration, but their wettability is typically altered from water-wet to H2-wet due to the organic matter, compromising their sealing performance. Silica nanofluids can restore water-wet conditions, which is vital for effective containment of H2 in geological formations. However, laboratory experiments to test the effects of nanomaterials on wettability and interfacial tension (IFT) are resource-intensive and time-consuming. To address this challenge, data-driven machine learning (ML) models offer a promising alternative, allowing for the prediction of key UHS parameters—such as advancing and receding contact angles without the need for extensive laboratory testing. We have applied multiple ML models, including gradient boosting, extreme gradient boosting (XGboost), Catboost, random forest, and extra trees to predict the wettability of organic rich shale samples treated with silica nanofluids. The ML models were trained on a comprehensive dataset generated through laboratory experiments conducted under a wide range of pressure (0.5 – 1600psi) and temperature (298-323 K) conditions, using brine (1 wt.% KCl + 2 wt.% NaCl) and varying silica nanofluid concentrations (0.05–1.0 wt.%). Advanced visualization techniques, including swarm maps, box plots, and heatmap plots, were utilized to analyze the experimental dataset thoroughly. To enhance the generalization capabilities of the ML models, we employed Bayesian optimization for hyperparameter tuning, along with k-fold cross-validation and grid-search optimization techniques. Both statistical and graphical analyses were performed to show the reliability and performance of the models. The results demonstrated that the implemented ML models, particularly the XGboost and Catboost, accurately predicted the wettability behavior under various operating conditions. The training and testing average R² scores were above 0.998 and 0.949, respectively, confirming the high accuracy of the predictions. These findings highlight the reliability of the ML algorithms in predicting the complex interaction between shale, brine, and H2. Further investigation into the feature importance revealed that nanomaterials had the greatest influence on the wettability of the ternary system. The silica nanofluid concentrations played a key role in modifying wettability, with varying impacts on the receding contact angles, emphasizing the effect of nanomaterial treatment. The model's predictions were consistent with experimental results, suggesting that geological conditions, including pressure and temperature, have a profound influence on rock mineral wettability, directly impacting H2 geo-storage capacity. This study is the first to employ data-driven ML models to predict critical UHS parameters in organic-rich shales treated with silica nanofluids. The ability of the ML models to predict UHS parameters across a wide range of conditions provides a valuable tool for optimizing UHS system designs, offering a more efficient and scalable approach compared to traditional experimental methods.

Global solar radiation prediction by ANN integrated with European Centre for medium range weather forecast fields in solar rich cities of Queensland Australia

This study focuses on using various machine learning (ML) models to evaluate the shear behaviors of ultra-high-performance concrete (UHPC) beams reinforced with glass fiber-reinforced polymer (GFRP) bars. The main objective of the study is to predict the shear strength of UHPC beams reinforced with GFRP bars using ML models. We use four different ML models: support vector machine (SVM), artificial neural network (ANN), random forest (R.F.), and extreme gradient boosting (XGBoost). The experimental database used in the study is acquired from various literature sources and comprises 54 test observations with 11 input features. These input features are likely parameters related to the composition, geometry, and properties of the UHPC beams and GFRP bars. To ensure the ML models' generalizability and scalability, random search methods are utilized to tune the hyperparameters of the algorithms. This tuning process helps improve the performance of the models when predicting the shear strength. The study uses the ACI318M-14 and Eurocode 2 standard building codes to predict the shear capacity behavior of GFRP bars-reinforced UHPC I-shaped beams. The ML models' predictions are compared to the results obtained from these building code standards. According to the findings, the XGBoost model demonstrates the highest predictive test performance among the investigated ML models. The study employs the SHAP (SHapley Additive exPlanations) analysis to assess the significance of each input parameter in the ML models' predictive capabilities. A Taylor diagram is used to statistically compare the accuracy of the ML models. This study concludes that ML models, particularly XGBoost, can effectively predict the shear capacity behavior of GFRP bars-reinforced UHPC I-shaped beams.

For the various climatic zones of India, machine learning (ML) models are created in the current work to forecast monthly-average diffuse solar radiation (DSR). The long-term solar radiation data are taken from Indian Meteorological Department (IMD), Pune, provided for 21 cities that span all of India’s climatic zones. The diffusion coefficient and diffuse fraction are the two groups of ML models with dual input parameters (sunshine ratio and clearness index) that are built and compared (each category has seven models). To create ML models, two well-known ML techniques, random forest (RF) and k-nearest neighbours (KNN), are used. The proposed ML models are compared with well-known models that are found in the literature. The ML models are ranked according to their overall and within predictive power using the Global Performance Indicator (GPI). It is discovered that KNN models generally outperform RF models. The results reveal that in diffusion coefficient models perform well than diffuse fraction models. Moreover, functional form 2 is the best followed by form 6. The ML models created here can be effectively used to accurately forecast DSR in various climates.

PurposeThis study explores the pioneering approach of utilising machine learning (ML) models and integrating social media data for predicting tourist arrivals in Sri Lanka.Design/methodology/approachTwo sets of experiments are performed in this research. First, the predictive accuracy of three ML models, support vector regression (SVR), random forest (RF) and artificial neural network (ANN), is compared against the seasonal autoregressive integrated moving average (SARIMA) model using historical tourist arrivals as features. Subsequently, the impact of incorporating social media data from TripAdvisor and Google Trends as additional features is investigated.FindingsThe findings reveal that the ML models generally outperform the SARIMA model, particularly from 2019 to 2021, when several unexpected events occurred in Sri Lanka. When integrating social media data, the RF model performs significantly better during most years, whereas the SVR model does not exhibit significant improvement. Although adding social media data to the ANN model does not yield superior forecasts, it exhibits proficiency in capturing data trends.Practical implicationsThe findings offer substantial implications for the industry's growth and resilience, allowing stakeholders to make accurate data-driven decisions to navigate the unpredictable dynamics of Sri Lanka's tourism sector.Originality/valueThis study presents the first exploration of ML models and the integration of social media data for forecasting Sri Lankan tourist arrivals, contributing to the advancement of research in this domain.

Machine Learning Can Accurately Predict Overnight Stay, Readmission, and 30-Day Complications Following Anterior Cruciate Ligament Reconstruction

Propofol is a widely used sedative-hypnotic agent for critically-ill patients requiring invasive mechanical ventilation (IMV). Despite its clinical benefits, propofol is associated with increased risks of hypertriglyceridemia. Early identification of patients at risk for propofol-associated hypertriglyceridemia is crucial for optimizing sedation strategies and preventing adverse outcomes. Machine learning (ML) models offer a promising approach for predicting individualized patient risks of propofol-associated hypertriglyceridemia. We propose the development of a ML model aimed at predicting the risk of propofol-associated hypertriglyceridemia in ICU patients receiving IMV. The study will utilize retrospective data from four Mayo Clinic sites. Nested cross-validation (CV) will be employed, with a 10-fold inner CV loop for model tuning and selection as well as an outer loop using leave-one-site-out CV for external validation. Feature selection will be conducted using Boruta and LASSO-penalized logistic regression. Data preprocessing steps include missing data imputation, feature scaling, and dimensionality reduction techniques. Six ML algorithms will be tuned and evaluated. Bayesian optimization will be used for hyperparameter selection. Global model explainability will be assessed using permutation importance, and local model explainability will be assessed using SHapley Additive exPlanations (SHAP). The proposed ML model aims to provide a reliable and interpretable tool for clinicians to predict the risk of propofol-associated hypertriglyceridemia in ICU patients. The final model will be deployed in a web-based clinical risk calculator. The model development process and performance measures obtained during nested cross-validation will be described in a study publication to be disseminated in a peer-reviewed journal. The proposed study has received ethics approval from the Mayo Clinic Institutional Review Board (IRB #23-007416). Robust external validation using a nested cross-validation (CV) framework will help assess the generalizability of models produced from the modeling pipeline across different hospital settings.A diverse set of machine learning (ML) algorithms and advanced hyperparameter tuning techniques will be employed to identify the most optimal model configuration.Integration of feature explainability will enhance the clinical applicability of the ML models by providing transparency in predictions, which can improve clinician trust and encourage adoption.Reliance on retrospective data may introduce biases due to inconsistent or erroneous data collection, and the computational intensity of the validation approach may limit replication and future model expansion in resource-constrained settings.

The ongoing Russia–Ukraine conflict has exacerbated the global crisis of natural gas supply, particularly in Europe. During the winter season, major importers of liquefied natural gas (LNG), such as South Korea and Japan, were directly affected by fluctuating spot LNG prices. This study aimed to use machine learning (ML) to predict the Japan Korea Marker (JKM), a spot LNG price index, to reduce price fluctuation risks for LNG importers such as the Korean Gas Corporation (KOGAS). Hence, price prediction models were developed based on long short-term memory (LSTM), artificial neural network (ANN), and support vector machine (SVM) algorithms, which were used for time series data prediction. Eighty-seven variables were collected for JKM prediction, of which eight were selected for modeling. Four scenarios (scenarios A, B, C, and D) were devised and tested to analyze the effect of each variable on the performance of the models. Among the eight variables, JKM, national balancing point (NBP), and Brent price indexes demonstrated the largest effects on the performance of the ML models. In contrast, the variable of LNG import volume in China had the least effect. The LSTM model showed a mean absolute error (MAE) of 0.195, making it the best-performing algorithm. However, the LSTM model demonstrated a decreased in performance of at least 57% during the COVID-19 period, which raises concerns regarding the reliability of the test results obtained during that time. The study compared the ML models’ prediction performances with those of the traditional statistical model, autoregressive integrated moving averages (ARIMA), to verify their effectiveness. The comparison results showed that the LSTM model’s performance deviated by an MAE of 15–22%, which can be attributed to the constraints of the small dataset size and conceptual structural differences between the ML and ARIMA models. However, if a sufficiently large dataset can be secured for training, the ML model is expected to perform better than the ARIMA. Additionally, separate tests were conducted to predict the trends of JKM fluctuations and comprehensively validate the practicality of the ML models. Based on the test results, LSTM model, identified as the optimal ML algorithm, achieved a performance of 53% during the regular period and 57% d during the abnormal period (i.e., COVID-19). Subject matter experts agreed that the performance of the ML models could be improved through additional studies, ultimately reducing the risk of price fluctuations when purchasing spot LNG.

Couplings connect the spinning shafts of driving and driven machines in the industry. A coupling guard encloses the coupling to protect personnel from the high-speed rotating coupling. The American Petroleum Institute API publishes standards that restrict the overheating of the coupling guards due to windage caused by the spinning shaft. Based on the most recent version of API 671, the peak temperature for the coupling guard should not exceed 60 °C. This paper proposes a machine learning (ML) model and an empirical formula to predict the maximum guard temperature and power loss. The ML models use a database obtained from simulated computational fluid dynamics (CFD) cases for different coupling guards under various conditions. Also, the paper provides validation for the CFD models with experimental tests for different cases. The proposed ML model uses eight different input parameters to predict temperature and power loss. The model shows an accurate prediction for a varied number of CFD cases. The performance of the generated model has been verified with the experimental results. Also, an empirical formula has been created using the same database from CFD results. The results show that the ML model has better prediction accuracy than the empirical formula for predicting peak temperature and power loss for all cases.

The sea surface temperature (SST) is an environmental indicator closely related to climate, weather, and atmospheric events worldwide. Its forecasting is essential for supporting the decision of governments and environmental organizations. Literature has shown that single machine learning (ML) models are generally more accurate than traditional statistical models for SST time series modeling. However, the parameters tuning of these ML models is a challenging task, mainly when complex phenomena, such as SST forecasting, are addressed. Issues related to misspecification, overfitting, or underfitting of the ML models can lead to underperforming forecasts. This work proposes using hybrid systems (HS) that combine (ML) models using residual forecasting as an alternative to enhance the performance of SST forecasting. In this context, two types of combinations are evaluated using two ML models: support vector regression (SVR) and long short-term memory (LSTM). The experimental evaluation was performed on three datasets from different regions of the Atlantic Ocean using three well-known measures: mean square error (MSE), mean absolute percentage error (MAPE), and mean absolute error (MAE). The best HS based on SVR improved the MSE value for each analyzed series by 82.26%, 98.93%, and 65.03% compared to its respective single model. The HS employing the LSTM improved 92.15%, 98.69%, and 32.41% concerning the single LSTM model. Compared to literature approaches, at least one version of HS attained higher accuracy than statistical and ML models in all study cases. In particular, the nonlinear combination of the ML models obtained the best performance among the proposed HS versions.

Stream temperature (Ts) is an important water quality parameter that affects ecosystem health and human water use for beneficial purposes. Accurate Ts predictions at different spatial and temporal scales can inform water management decisions that account for the effects of changing climate and extreme events. In particular, widespread predictions of Ts in unmonitored stream reaches can enable decision makers to be responsive to changes caused by unforeseen disturbances. In this study, we demonstrate the use of classical machine learning (ML) models, support vector regression and gradient boosted trees (XGBoost), for monthly Ts predictions in 78 pristine and human-impacted catchments of the Mid-Atlantic and Pacific Northwest hydrologic regions spanning different geologies, climate, and land use. The ML models were trained using long-term monitoring data from 1980–2020 for three scenarios: (1) temporal predictions at a single site, (2) temporal predictions for multiple sites within a region, and (3) spatiotemporal predictions in unmonitored basins (PUB). In the first two scenarios, the ML models predicted Ts with median root mean squared errors (RMSE) of 0.69–0.84 °C and 0.92–1.02 °C across different model types for the temporal predictions at single and multiple sites respectively. For the PUB scenario, we used a bootstrap aggregation approach using models trained with different subsets of data, for which an ensemble XGBoost implementation outperformed all other modeling configurations (median RMSE 0.62 °C).The ML models improved median monthly Ts estimates compared to baseline statistical multi-linear regression models by 15–48% depending on the site and scenario. Air temperature was found to be the primary driver of monthly Ts for all sites, with secondary influence of month of the year (seasonality) and solar radiation, while discharge was a significant predictor at only 10 sites. The predictive performance of the ML models was robust to configuration changes in model setup and inputs, but was influenced by the distance to the nearest dam with RMSE <1 °C at sites situated greater than 16 and 44 km from a dam for the temporal single site and regional scenarios, and over 1.4 km from a dam for the PUB scenario. Our results show that classical ML models with solely meteorological inputs can be used for spatial and temporal predictions of monthly Ts in pristine and managed basins with reasonable (<1 °C) accuracy for most locations.

This study highlights drought forecasting for understanding the semi-arid area in India, where drought phenomena play vital role in the irrigation, drinking water supplies, and sustaining the ecological with economic balance for every nation. Therefore, drought forecasting is important for the future drought planning based on the machine learning (ML) models. Hence, The Standardized Precipitation Index (SPI) at 3- and 6-month periods have been selected and used for future drought forecasting scenarios in area. The combinations of ten inputs SPI-1- and SPI-10 were used for predicting modeling for SPI-3 and SPI-6 timescales, that modeling developed based on the historical SPI datasets from 1989 to 2019 years. The SPI-3 and SPI-6 maximum and minimum values are shown SPI-3 (2.03 and -5.522) and SPI-6 (1.94 and -6.93). The SPI is a popular method for estimating the drought analysis and has been used everywhere at global level. The developed models have been compared with each other, with the best combination of input variables selected using subset regression models and sensitivity studies. After that, the active input parameters were used for forecasting of SPI-3 and SPI-6 values to understanding of drought in semi-arid area. The finest input variables combination have been used in the Ml models and established the novel five models such as robust linear regression, bagged trees, boosted trees, support vector regression (SVM-Linear), and Matern Gaussian Process Regression (Matern GPR) models. Such kind of models first time has been applied for the forecasting of future drought conditions. Whole models were fine and improved modeling by using hyperparameters tuning, bagging, and boosting models. Entire ML models’ accuracy was compared using different statistical metrics. Compared with five ML models accuracy, we have found that the Matern GPR model better accuracy than other ML models. The best model accuracy is R2 = 0.95 and 0.93, RMSE, MSE, MAE, MARE, and NSE values, respectively, for predicting SPI-3 and SPI-6 values in the area. Therefore, the Matern GPR model was identified as the finest ML algorithm for predicting SPI-3 and SPI-6 associated with other algorithms. This research demonstrates the Matern GPR model's efficacy in predicting multiscale SPI-3 and SPI-6 under climate variations. It can be helpful in soil and water resource conservation planning and management and understanding droughts in the entire basin areas of the country India.

• ML models for predicting SCM concretes compressive strength developed • Multiple datasets devised from a global dataset were used for training • Multiple ML models used in conjunction with multiple datasets for best results • Accurate prediction results obtained for all SCM concrete types studied • Prediction uncertainty bounds established by conformal prediction analysis Supplementary cementitious materials (SCM) used as partial replacements of cement can reduce carbon emissions and construction costs. The amount and types of SCM used directly influence the compressive strength of concrete, which is a crucial indicator of concrete performance. However, predicting compressive strength is challenging for SCM-based concretes due to their complex compositions, especially when multiple types of SCM are used. This study used five machine learning (ML) models, namely artificial neural network (ANN), random forest (RF), extra trees (ET), extreme gradient boosting (XGB), and adaptive boosting (AdaBoost) to predict the 28-day compressive strength of SCM concretes incorporating fly ash (FA), silica fume (SF), and ground granulated blast furnace slag (GGBFS). Firstly, a global dataset comprising 1456 data points was organized into seven individual and six combined datasets for training of the five ML models. After this, the performances of the ML models’ predictions using individual and combined datasets were compared using root mean squared error (RMSE) and coefficient of determination (R 2 ) to identify the optimal ML model-dataset configurations for each type of SCM-based concrete. It is found that in general, XGB models developed using the combined datasets demonstrated superior prediction accuracy, achieving an R² value exceeding 0.8. Both Shapley Additive Explanation (SHAP) and partial dependence plots (PDPs) were employed to illustrate the effects of input variables on compressive strength. Finally, uncertainty quantification using the Jackknife method combined with a normalized nonconformity measure was employed to provide accurate uncertainty bounds for compressive strength predictions.