Bayesian Hyperparameter Optimization and Ensemble Learning for Machine Learning Models on Software Effort Estimation

An effective method for finding the relevant number (window size) and the elapsed time (window age) of recently completed projects has proven elusive in software effort estimation. Although these two parameters significantly affect the prediction accuracy, there is no effective method to stratify and sample chronological projects to improve prediction performance of software effort estimation models. Exemplary projects (Bellwether) representing the training set have been empirically validated to improve the prediction accuracy in the domain of software defect prediction. However, the concept of Bellwether and its effect have not been empirically proven in software effort estimation as a method of selecting exemplary/relevant projects with defined window size and age. In view of this, we introduce a novel method for selecting relevant and recently completed projects referred to as Bellwether moving window for improving the software effort prediction accuracy. We first sort and cluster a pool of N projects and apply statistical stratification based on Markov chain modeling to select the Bellwether moving window. We evaluate the proposed approach using the baseline Automatically Transformed Linear Model on the ISBSG dataset. Results show that (1) Bellwether effect exist in software effort estimation dataset, (2) the Bellwether moving window with a window size of 82 to 84 projects and window age of 1.5 to 2 years resulted in an improved prediction accuracy than the traditional approach.

GA-based method for feature selection and parameters optimization for machine learning regression applied to software effort estimation

Prenatal prediction of CD34+ adequacy supports cord-blood banking by reducing expenses on low-yield units and reserving capacity for clinically promising grafts. We have developed and evaluated a prenatal machine learning model that predicts whether an umbilical cord blood (UCB) unit will meet a clinically supported adequacy threshold (≥1.5×105 CD34+ cells/kg recipient) before collection for single-unit grafts. In this retrospective study, we analysed 126,406 records from the Royan Stem Cell Technology Company (RSCT; Tehran, Iran), which included routinely available maternal, neonatal, and family-history variables. A pipeline of imputation (IterativeImputer numeric; SimpleImputer+OrdinalEncoder categorical), feature selection (Extra Trees), and hyperparameter tuning using Bayesian optimisation with model training/evaluation was performed within cross-validation folds. Decision Tree (DT), K-Nearest Neighbours (KNN), Random Forest (RF), Support Vector Machine (SVM), and Multilayer Perceptron (MLP) classifiers were tuned via Bayesian optimisation. Models were ranked by the Technique for Order Preference by Similarity to Ideal Solution (TOPSIS). Majority voting (MV) ensembles were constructed from the top-k models. Model interpretability used SHapley Additive exPlanations (SHAP). The MV (top-4: RF, KNN, DT, MLP) ensemble achieved an area under the receiver operating characteristic curve (ROC-AUC)=0.808 and an area under the precision-recall curve (PR-AUC)=0.744 on the held-out test set, with an accuracy=0.757, precision=0.726, recall/sensitivity=0.804, F1=0.762, specificity=0.716, and Brier score=0.181. SHAP highlighted history of hepatitis C, birth place, hyperthyroidism, history of anaemia, oral fungus, and rheumatism among the most influential features. Prenatal prediction of UCB CD34+ adequacy using an interpretable MV ensemble is feasible and accurate to support pre-collection triage and can potentially improve banking efficiency. The resultant model offers a non-invasive tool to enhance the efficiency of cord blood banking by prioritising units with higher transplantation potential.

With the sustainable development of the construction industry, recycled aggregate (RA) has been widely used in concrete preparation to reduce the environmental impact of construction waste. Compressive strength is an essential measure of the performance of recycled aggregate concrete (RAC). In order to understand the correspondence between relevant factors and the compressive strength of recycled concrete and accurately predict the compressive strength of RAC, this paper establishes a model for predicting the compressive strength of RAC using machine learning and hyperparameter optimization techniques. RAC experimental data from published literature as the dataset, extreme gradient boosting (XGBoost), random forest (RF), K-nearest neighbour (KNN), support vector machine regression Support Vector Regression (SVR), and gradient boosted decision tree (GBDT) RAC compressive strength prediction models were developed. The models were validated and compared using correlation coefficients (R2), Root Mean Square Error (RMSE), mean absolute error (MAE), and the gap between the experimental results of the predicted outcomes. In particular, The effects of different hyperparameter optimization techniques (Grid search, Random search, Bayesian optimization-Tree-structured Parzen Estimator, Bayesian optimization- Gaussian Process Regression) on model prediction efficiency and prediction accuracy were investigated. The results show that the optimal combination of hyperparameters can be searched in the shortest time using the Bayesian optimization algorithm based on TPE (Tree-structured Parzen Estimator); the BO-TPE-GBDT RAC compressive strength prediction model has higher prediction accuracy and generalisation ability. This high-performance compressive strength prediction model provides a basis for RAC’s research and practice and a new way to predict the performance of RAC.

The methods for Software Effort Estimation are divided into two, these methods are grouped into Non Machine Learning (non-ML) and Machine Learning (ML) methods [1]. The k-NN method has the disadvantage of being unable to tolerate irrelevant features and greatly affect the accuracy of k-NN. The k-NN method is also difficult to deal with missing data problems and feature categorization problems such as features that are not relevant, weight features that are not optimal, and the same features [2]. Whereas the dataset of Software Effort Estimation still has some serious challenges such as the characteristics of the data set, which are irrelevant features and the level of influence of each feature in the estimated data of the software effort [3]. This study compared the k-NN individual method with the combination of feature selection method with k-NN to find out which method was the best. The results showed that the Forward Selection (FS) method and Median Weighted Information Gain with k-Nearest Neighbor can overcome the problem of irrelevant features so as to increase accuracy in the RMSE Software Effort Estimation dataset, which is smaller in the Albrecht dataset of 5,953 using the Median method -WIG k-NN, the Miyazaki dataset is 55,421 and Kemerer is 123,081 using the FS k-NN method. The combination of kNN with Feature Selection is proven to be able to improve the estimation results better than kNN individuals. With the FS k-NN method being the best by winning in 2 datasets Miyazaki and Kemerer.

Effort estimations are a crucial aspect of software development. The tasks should be completed before the start of any software project. Accurate estimations increase the chances of project success, and inaccurate information can lead to severe issues. This study systematically reviewed the literature on effort-estimating models from 2015-2024, identifying 69 relevant studies from various publications to compile information on various software work estimation models. This review aims to analyze the models proposed in the literature and their classification, the metrics used for accuracy measurement, the leading model that has been chiefly applied for effort estimation, and the benchmark datasets available. The study utilized 542 relevant articles on software development, cost, effort, prediction, estimation, and modelling techniques in the search strategy. After 194 selections, the authors chose 69 articles to understand ML applications in SEE comprehensively. The researchers used a scoring system to assess each study’s responses (from 0 to 5 points) to their research questions. This helped them identify credible studies with higher scores for a comprehensive review aligned with its objectives. The data extraction process identified 91% (63) of 69 studies as either highly or somewhat relevant, demonstrating a successful search strategy for analysis. The literature review on SEE indicates a growing preference for ML-based models in 59% of selected studies. 17% of the studies chosen favor hybrid models to overcome software development challenges. We qualitatively analyzed all the literature on software effort estimation using expert judgment, formal estimation techniques, ML-based techniques, and hybrid techniques. We discovered that researchers have frequently used ML-based models to estimate software effort and are currently in the lead. This study also explores the application of feature importance and selection in machine learning models for Software Effort Estimation (SEE) using popular algorithms like support Vector Machine (SVM), AdaBoost (AB), Gradient Boost (GB), and Random Forest (RF) with six benchmark datasets like CHINA, COCOMO-NASA2, COCOMO, COCOMO81, DESHARNAIS, and KITCHENHAM. We analyze the dataset descriptions and feature importance of the dataset analysis using ML models for choosing crucial play attributes in SEE.

Estimation of internal states in a Li-ion battery using BiLSTM with Bayesian hyperparameter optimization

Data driven strength and strain enhancement model for FRP confined concrete using Bayesian optimization

This study proposes an investment portfolio optimization system based on the AutoML framework, which integrates Bayesian hyperparameter optimization and machine learning techniques to achieve accurate prediction of financial market volatility and dynamic asset allocation. The system first extracts multi-scale features (including short-term, medium-term, and long-term returns, volatility, trading volume, and momentum indicators) based on historical stock price data, and uses a Bayesian optimized support vector regression (SVR) model to predict the volatility for the next 20 days; At the asset allocation level, the system dynamically adjusts the weights of the investment portfolio based on the predicted volatility, controlling the target volatility at 20% while constraining the leverage ratio within the range of 0-150%. The empirical results show that: 1) The volatility prediction model has excellent performance.

Accurately estimating software development costs is paramount for software project success. Global attention has focused on addressing inaccuracies in software cost estimating models leading to numerous approaches. Although machine learning algorithm techniques have witnessed improvement, the challenge of varying bias and variance from weaker learners persists, leading to the adoption of ensemble learning techniques for increased accuracy. This paper extends previous work on ensemble learning Software Cost Estimation (SCE) approach through the use of a Stacking Ensemble Learning Model for SCE (SELM-SCE) to improve the accuracy of software cost estimation. A combination of unsupervised and supervised machine learning methods were used on software projects. Two datasets (Desharnais and Maxwell) for software cost estimation were used for the model training and validation using cross-validation techniques. The proposed SELM-SCE model developed was subjected to two training models: K-Means and SMOTE technique to solve the problem of overfitting and poor performance caused by data imbalance. The performance of the proposed SELM-SCE model was first evaluated against three regression performance metrics: coefficient of determination (R2), root mean squared error (RMSE), and mean absolute error (MAE). The results showed that the SMOTE SELM-SCE without unsupervised machine learning improves the accuracy of the model performance. The performance of SELM-SCE was compared with existing models using prediction accuracy, precision, recall, and F1-score classification performance metrics. The results from the SELM-SCE model showed remarkable improvement when compared to the existing model

Software development involves several interrelated factors that influence development efforts and productivity. Improving the estimation techniques available to project managers will facilitate more effective time and budget control in software development. Software Effort Estimation or software cost/effort estimation can help a software development company to overcome difficulties experienced in estimating software development efforts. This study aims to compare the Machine Learning method of Linear Regression (LR), Multilayer Perceptron (MLP), Radial Basis Function (RBF), and Decision Tree Random Forest (DTRF) to calculate estimated cost/effort software. Then these five approaches will be tested on a dataset of software development projects as many as 10 dataset projects. So that it can produce new knowledge about what machine learning and non-machine learning methods are the most accurate for estimating software business. As well as knowing between the selection between using Particle Swarm Optimization (PSO) for attributes selection and without PSO, which one can increase the accuracy for software business estimation. The data mining algorithm used to calculate the most optimal software effort estimate is the Linear Regression algorithm with an average RMSE value of 1603,024 for the 10 datasets tested. Then using the PSO feature selection can increase the accuracy or reduce the RMSE average value to 1552,999. The result indicates that, compared with the original regression linear model, the accuracy or error rate of software effort estimation has increased by 3.12% by applying PSO feature selection

Background and objectiveThe increasing global prevalence of diabetes has led to a surge in complications, significantly burdening healthcare systems and affecting patient quality of life. Early prediction of these complications is critical for timely intervention, yet existing models often rely heavily on clinical indicators while underutilizing fundamental laboratory test parameters. This study aims to bridge this gap by leveraging the 12 most frequently tested laboratory indicators in diabetic patients to develop an optimized predictive model for diabetes complications.MethodsA comprehensive dataset was established through meticulous data collection from a high-volume tertiary hospital, followed by extensive data cleaning and classification. Various machine learning classifiers, including Random Forest, XGBoost, Support Vector Machine (SVM), and Multilayer Perceptron (MLP), were trained on this dataset to evaluate their predictive performance. We further introduced an ensemble learning model with Bayesian optimization to enhance accuracy and cost-efficiency. Additionally, feature importance analysis was conducted to refine the model by reducing testing costs while maintaining high predictive accuracy.ResultsOur ensemble model with Bayesian optimization demonstrated superior performance, achieving over 90% accuracy in predicting various diabetic complications, with an outstanding 98.50% accuracy and 99.76% AUC for diabetic nephropathy. Feature correlation analysis enabled a refined model that not only improved predictive accuracy but also reduced overall medical costs by 2.5% through strategic feature elimination.ConclusionsThis study makes three key contributions: (1) Development of a high-quality dataset based on fundamental laboratory indicators, (2) Creation of a highly accurate predictive model using ensemble learning and Bayesian optimization, particularly excelling in diabetic nephropathy prediction, and (3) Implementation of a cost-efficient diagnostic approach that reduces testing expenses without compromising accuracy. The proposed model provides a strong foundation for future research and practical clinical applications, demonstrating the potential of integrating machine learning with cost-conscious medical testing.

Empirical studies on software effort estimation have employed hyper-parameter tuning algorithms to improve model accuracy and stability. While these tuners can improve model performance, some might be overly complex or costly for low dimensionality datasets used in SEE. In such cases a method like random search can potentially provide similar benefits as some of existing tuners, with advantage of using low amounts of resources and being simple to implement. In this study we evaluate impact on model accuracy and stability of 12 state-of-the-art hyper-parameter tuning algorithms against random search, on 9 datasets of PROMISE repository and 4 sub-datasets from ISBSG R18 dataset. This study covers 2 traditional exhaustive tuners (grid and random searches), 6 bio-inspired algorithms, 2 heuristic tuners, and 3 model-based algorithms. The tuners are used to configure support vector regression, classification and regression trees, and ridge regression models. We aim to determine techniques and datasets for which certain tuners were 1) more effective than default hyper-parameters, 2) more effective than random search, 3) which models(s) can be considered the best for which datasets. The results of this study show that hyper-parameter tuning was effective (increased accuracy and stability) in 862 (51%) of 1,690 studied scenarios. The 12 state-of-the-art tuners were more effective than random search in 95 (6%) of 1,560 studied (non-random search) scenarios. Although not effective in every dataset, combination of flash tuning, logarithm transformation and support vector regression obtained top ranking in accuracy on highest amount (8 out of 13) of datasets. Hyperband tuned ridge regression with logarithm transformation obtained top ranking in accuracy on highest amount (10 out of 13) of datasets. We endorse use of random search as a baseline for comparison for future studies that consider hyper-parameter tuning.

Software cost estimation continues to be an area of concern for managing of software development industry. We use unsupervised learning (e.g., clustering algorithms) combined with functional link artificial neural networks for software effort prediction. The unsupervised learning (clustering) indigenously divide the input space into the required number of partitions thus eliminating the need of ad-hoc selection of number of clusters. Functional link artificial neural networks (FLANNs), on the other hand is a powerful computational model. Chebyshev polynomial has been used in the FLANN as a choice for functional expansion to exhaustively study the performance. Three real life datasets related to software cost estimation have been considered for empirical evaluation of this proposed method. The experimental results show that our method could significantly improve prediction accuracy of conventional FLANN and has the potential to become an effective method for software cost estimation.

Shape memory polymers and their composites are promising smart materials for a wide range of applications associated with smart material-based actuators. Carbon fiber-reinforced shape memory epoxy composites can enable electroactive shape recovery via Joule heating, offering precise, remote-controlled actuation. However, predicting time-dependent recovery behavior across varying current levels remains challenging due to limited experimental data. This study presents an overfitting-aware machine learning framework for predicting shape recovery angles of the shape memory epoxy composites activated by Joule heating as a function of time and applied current. Four regression algorithms: Random Forest (RF), Support Vector Regression (SVR), Regularized Polynomial, and Gaussian Process Regression (GPR) are considered for constructing the shape memory recovery angle prediction algorithm. In the models, overfitting constraints are integrated into Bayesian hyperparameter optimization using a hybrid objective function that penalizes excessive generalization gaps. Stratified fivefold cross-validation ensures balanced current distribution across folds. Results demonstrate that kernel-based methods outperform tree ensembles: SVR achieves the highest accuracy with controlled generalization (R2 = 0.944) while RF exhibits limited effectiveness (R2 = 0.842), confirming tree-based methods are less suitable for small, smooth datasets. SVR provides optimal accuracy-generalization balance for reliable engineering predictions.