Feature Selection in Cybersecurity: A Comparative Study of Machine Learning Models

This research examines the efficacy of ensemble Machine Learning (ML) models, mainly focusing on Deep Neural Networks (DNNs), in predicting the need for cardiovascular surgery, a critical aspect of clinical decision-making. It addresses key challenges such as class imbalance, which is pivotal in healthcare settings. The research involved a comprehensive comparison and evaluation of the performance of previously published ML methods against a new Deep Learning (DL) model. This comparison utilized a dataset encompassing 50,000 patient records from a large hospital between 2015-2022. The study proposes enhancing the efficacy of these models through feature selection and hyperparameter optimization, employing techniques like grid search. A novel aspect of this research was the comparison of a newly developed DNN model with existing ensemble models based on similar cardiovascular datasets. The results indicated the DNN model's superior predictive accuracy, demonstrating an Area Under the Curve (AUC) of 74%, alongside notable precision (68%) and recall (72%) for the minority class, which indicates patients requiring surgery. The model further achieved a 70% F1-Score and a balanced accuracy rate of 72%, significantly outperforming the existing ensemble models in every key performance metric. The study underscores the transformative potential of DNNs in predictive modeling for cardiovascular care and highlights the importance of integrating advanced ML techniques into clinical workflows. Future research should delve into the practical application and integration of these models.

The fashion industry faces challenges in accurately predicting demand due to inherent uncertainty, leading to suboptimal inventory and financial losses. Machine learning (ML) offers a robust solution by analyzing large and complex data, identifying non-linear patterns, and providing more accurate predictions than conventional methods that rely on limited factors. This research aims to compare and evaluate the performance of six different ML models—XGBoost, SVM, RF, GBM, KNN, and NN, considering the influence of feature engineering and various data split ratios on predicting fashion product demand. KNN and NN were included due to distinct modeling approaches and competitive capabilities in identifying local and non-linear patterns across numerical, categorical, and time series data. Techniques such as feature extraction and selection and various data split ratios (70:30, 80:20, 90:10) were used. Using Adidas sales data, the models were evaluated based on Root Mean Squared Error (RMSE) and Mean Absolute Error (MAE). The results indicate that the XGBoost-based model with feature engineering consistently outperforms the other models across all data split ratios. Particularly, XGBoost with feature engineering at a data split ratio of 90:10 achieved the best performance with an RMSE of 4.46 and an MAE of 1.51. Analyzing model performance shows that the predictive ability of ML models is influenced by the implementation of feature engineering and the selection of the data split ratio. These results demonstrate the potential of using feature-engineered XGBoost models and optimized data ratios to mitigate the risk of stockouts or overstocks, and reduce financial losses and environmental waste.

Cervical cancer remains a significant global health issue, particularly in developing countries where it is a leading cause of mortality among women. The development of machine learning-based approaches has become essential for early detection and diagnosis of cervical cancer. This research explores the optimization of classification algorithms through Correlation-Based Feature Selection (CFS) for early cervical cancer detection. A dataset consisting of 198 samples and 22 attributes from medical records was processed to reduce dimensionality. CFS was used to select the most relevant features, which were then applied to three classification algorithms: Naïve Bayes, Decision Tree, and k-Nearest Neighbor (k-NN). The results showed that CFS significantly improved classification accuracy, with Decision Tree achieving the highest accuracy of 85.89%, followed by Naïve Bayes with 83.34%, and k-NN with 82.32%. These findings demonstrate the importance of feature selection in enhancing classification performance and its potential application in the development of cervical cancer detection tools.

This report provides a comprehensive study focused on land cover classification and remote sensing image analysis using the PaviaU dataset. The report first introduces the basic characteristics and application background of the data set, and then discusses in detail the application of feature selection and dimensionality reduction techniques, especially linear discriminant analysis (LDA) and principal component analysis (PCA). In terms of model application, the report not only uses traditional machine learning models, such as support vector machines, Bayesian and KNN posterior probabilities, but also explores the performance of these models under different parameter settings. Through a series of experiments, the study found that optimizing feature selection and dimensionality reduction techniques can significantly improve the classification accuracy of the model. Finally, the report compares different models and suggests alternative improvements.

Over the past few decades, the construction industry has been suffering from project delays. It remains a recognized challenge to accurately predict the actual project’s progress, despite the vast amount of field data and scheduling methods available. In this paper, four different machine learning (ML) models including K-nearest neighbor (KNN), support vector regression (SVR), gradient boosting trees (GBT), and artificial neural network (ANN) were used for forecasting the construction duration of work areas. Field data were collected, including progress, workload, labor, weather, planned days, and location, as inputs and construction days as output for each work area to develop the models. A simple and widely used baseline model was used to confirm the accuracy and generalization capability of ML models. The comparison of the results revealed that ANN and GBT models produced significantly superior results. Additionally, the GBT model is substantially more computationally efficient. In conclusion, the GBT model could be successfully and effectively applied to improve the prediction of construction duration. The proposed ML models could be utilized as a decision support tool for construction project managers to adjust construction site resource allocation.

In recent years, the term Artificial Intelligence (AI) has gained recognition across various fields, such as healthcare, finance, agriculture, and many other sectors due to its versatility and capacity to improve outcome. Within the applications of AI, classification stands out as one of the most prevalent, aiding in optimizing decision-making and efficiently organizing data. In recent years, there has been an increase in the use of zebrafish (Danio rerio) in studies related to human dermatological diseases, such as vitiligo, which involves the autoimmune-mediated destruction of the melanocytes in the epidermal layer. Despite the current interest in studies related to this disease, no papers were found applying Machine Learning (ML) or Deep Learning (DL) models to classify the effects of the disease and its treatments. In this context, this paper uses the challenge of evaluating the effects of vitiligo in zebrafish to compare the performance of different AI approaches. The methodology employed in this paper includes image acquisition, dataset creation, preprocessing, model testing, and evaluation. The ML models applied in this study were Support Vector Machine (SVM), Random Forest (RF), K-Nearest Neighbors (KNN), and Extreme Gradient Boosting (XGBoost), while the DL models included Visual Geometry Group 16 (VGG16) and GoogLeNet. Following the evaluation, SVM and GoogLeNet achieved the best results, correctly classifying 80% and 71% of the data, respectively. Moreover, the former accurately identified all samples in the healthy and treated classes, with misclassifications occurring only within the sick class. The models performed satisfactorily in relation to the objectives of this study and the results exhibited potential for future applications in treating vitiligo in humans.

Snow depth is a critical parameter for characterizing snow dynamics and water resources, and its accurate inversion is essential for hydrological processes, climate studies, and disaster prevention in cold regions. Based on long-term daily ground meteorological observation data from the hydrological years 1961 to 2015 at two meteorological stations in Mohe and Mishan, Heilongjiang Province, China, this study integrates physical parameters of snow density and snow albedo from the ERA5-Land reanalysis data to systematically compare the performance of three machine learning and three deep learning models in retrieving daily snow depth. Four feature combination schemes were designed to evaluate the contributions of meteorological factors, lagged snow depth terms, and snow physical parameters. The results indicate that, for both machine learning and deep learning models, the first-order lagged value of snow depth is the most important variable determining prediction accuracy. In terms of model performance, machine learning methods excelled, with XGBoost performing particularly outstandingly, achieving optimal prediction accuracy and stability under the best feature combination (coefficient of determination, R2, reaching 0.989; root mean square error, RMSE, of 1.19 cm). Among deep learning methods, 1D CNN demonstrated strong local feature extraction capabilities, achieving prediction accuracy comparable to the best-performing machine learning model (R2 of 0.9878, RMSE of 1.26 cm). Notably, models specifically designed for time-series data, such as LSTM (R2 of 0.9848, RMSE of 1.41 cm), and the more complex 1D CNN-LSTM hybrid model (R2 of 0.9803, RMSE of 1.60 cm) did not show significant advantages in this study. This indicates that model complexity and predictive performance are not simply positively correlated. Through comprehensive analysis of data from both stations, this study demonstrates that a prediction framework centered on historical snow depth as the core driving factor, combined with key meteorological elements, is highly robust. Although the inclusion of ERA5-Land snow physical parameters did not significantly improve model accuracy, it provides important insights for the future development of hybrid models that integrate physical mechanisms with data-driven approaches. The findings offer an effective solution for reconstructing long-term snow depth time series and hold significant application value for simulating cryospheric hydrological processes and studying climate change.

Surrogate-assisted hydraulic fracture optimization workflow with applications for shale gas reservoir development: a comparative study of machine learning models

• Neural Network feature selection methodology for both numerical and categorical data. • Embedded feature selection provides model and feature subset in a single training. • Novel feature embedding for dealing with high-cardinality categorical features. • Validation of the methodology on several open and real industry dataset. • Comparison with several state-of-the-art feature selection methodologie. In an era of effortless data collection, the impact of machine learning — especially neural networks (NNs) — is undeniable. As datasets grow in size and complexity, efficiently handling mixed data types, including categorical and numerical features, becomes critical. Feature encoding and selection play a key role in improving NN performance, efficiency, interpretability, and generalisation. This paper presents GLEm-Net (Grouped Lasso with Embeddings Network), a novel NN-based approach that seamlessly integrates feature encoding and selection directly into the training process. GLEm-Net uses embedding layers to process categorical features with high cardinality, simplifying the model and improving generalisation. By extending the grouped Lasso regularisation to explicitly consider categorical features, GLEm-Net automatically identifies the most relevant features during training and returns them to the analyst. We evaluate GLEm-Net on open and proprietary industry datasets and compare it to state-of-the-art feature selection methodologies. Results show that GLEm-Net adapts to each dataset by allowing the NN to directly select subsets of most important features, offering on par performance with the best state-of-the-art feature selection methods, while eliminating the need for the external feature encoding and selection steps that are now incorporated in the NN training stage.

Background: Osteoporosis is a systemic skeletal disorder marked by reduced bone density and microarchitectural deterioration, leading to increased fracture risk. While the dual-energy X-ray absorptiometry (DEXA) scan is the World Health Organization (WHO)-recommended diagnostic standard, its limitations necessitate alternative methods. Emerging magnetic resonance imaging (MRI) techniques, radiomics, and machine learning promise to enhance osteoporosis diagnosis through detailed analysis of lumbar MRI apparent diffusion coefficient (ADC) maps, potentially revolutionizing early detection and treatment strategies. Objectives: In this study, we are going to evaluate the performance of machine learning (ML) models using radiomics features of lumbar MRI ADC map for osteoporosis detection, and to identify significant features and their diagnostic thresholds. Specific performance metrics such as accuracy, sensitivity, specificity, and Area Under the receiver operating characteristic (ROC) Curve (AUC) were assessed. Patients and Methods: This retrospective study employed a cross-sectional design, with a total of 140 cases, including 21 with osteoporosis. The study's inclusion criteria consisted of concurrent lumbar MRI and DEXA within a year, while exclusion criteria included infectious or neoplastic lumbar lesions, fractures, instrumentation, significant osteodegenerative changes, cases where the first four lumbar vertebrae were not included in the imaging field, and absence of diffusion-weighted imaging. Manual segmentation of lumbar vertebrae from ADC maps was performed to create a comprehensive dataset, comprising 5,580 radiomics features per case. Subsequently, the top five features selected by fast correlation-based filter (FCBF) were used to test the performance of seven Machine Learning algorithms (k-Nearest neighbors, decision tree, random forest, logistic regression, support vector machine, naive bayes, and neural network). Statistical tests and ROC curve analysis were conducted to determine the significance and thresholds of these features. Results: The study included 140 cases, with 132 females (94.3%) and 8 males (5.7%), and a mean age of 65.32 ± 8.50 years. The mean BMI was 31.43 ± 5.53 kg/m² for females and 26 ± 3.59 kg/m² for males. In terms of demographic differences, no significant age difference was found between the osteoporotic and non-osteoporotic groups (P = 0.889). However, the osteoporotic group had significantly lower mean body weight (64.90 ± 10.13 kg vs. 74.68 ± 13.94 kg, P = 0.003) and BMI (27.40 ± 4.38 kg/m² vs. 31.77 ± 5.52 kg/m², P = 0.001) compared to the non-osteoporotic group. The median interval between DEXA and lumbar MRI was 1 month (range 0.1 - 11.87 months). The Neural Network model demonstrated the highest performance with an AUC of 0.616 and a classification accuracy of 0.764 using all features. The Naive Bayes model, using the top five features selected by FCBF, showed the highest performance with an AUC of 0.913, accuracy of 0.907, sensitivity of 0.667, and specificity of 0.95. All ML models’ performance were elevated by feature selection. Independent t-tests and Mann-Whitney U tests identified 521 and 670 significant features, respectively (P < 0.05). ROC analysis revealed 58 features with AUC values above 0.70. Conclusion: This study's findings suggest that ML models, particularly the Naive Bayes algorithm, can effectively use lumbar ADC map radiomics to diagnose osteoporosis. These findings could enhance early detection and treatment strategies, potentially improving patient outcomes and reducing the burden of osteoporotic fractures. This study also established threshold values for significant features.

A comparative study of machine learning models for predicting the state of reactive mixing

Identification of high-risk genes and classification of acute myocardial infarction patients utilizing deep learning in a restricted cohort.

A machine-learning-based model for predicting the effective stiffness of precast concrete columns

Autism spectrum disorder (ASD) is a multifaceted neurodevelopmental condition that affects millions of individuals globally. This study leverages the publicly available UCI Machine Learning ASD Screening Dataset, which contains behavioral questionnaire responses from children aged 4-11 years, to develop and compare multiple machine learning (ML) models for early ASD diagnosis. This study applies SHAP (SHapley Additive exPlanations) to interpret and analyze the feature importance of various attributes in the dataset. Advanced ML models like Extreme Gradient Boosting (XGBoost), Random Forest (RF), Extra Trees Classifier (ETC), Support Vector Machines (SVM), and Multilayer Perceptrons (MLPs) were implemented to evaluate and compare their performance in predicting early ASD. The developed ASD model underwent a rigorous evaluation through comprehensive performance metrics, including accuracy, precision, recall, F1-score, specificity, and Matthews correlation coefficient. Ranking-based metrics, such as area under the receiver operating characteristic curve (AUROC), provided additional insights into classification robustness, while 10-fold cross-validation ensured reliability and generalizability. Results indicate that XGBoost achieved the highest diagnostic performance (accuracy = 97.8%, AUROC = 0.99), followed closely by MLP, with RF and ETC offering balanced performance and interpretability. SHAP analysis identified key behavioral features influencing model predictions, enabling explainable AI-driven insights. These findings suggest that ML-based tools can support clinicians in identifying ASD earlier, allowing for timelier interventions in disability services, particularly in underserved populations. While results are promising, the use of secondary data and the lack of clinical validation remain limitations, highlighting the need for real-world pilot testing. These findings underscore the transformative potential of ML techniques in enhancing ASD diagnostics. Despite their promise, the results are constrained by reliance on secondary data and the absence of clinical validation, underscoring the need for real-world pilot testing. Nonetheless, incorporating these models into clinical practice could accelerate early detection, facilitate personalized interventions, and optimize resource allocation for improved ASD management outcomes.

ObjectiveCompare the performance of the Multivariable logistic regression (LR) model based on traditional statistical methods and the Random Forest (RF) model in machine learning for predicting clinically relevant postoperative pancreatic fistula (CR-POPF) after pancreatoduodenectomy (PD).BackgroundCR-POPF is a common and severe complication following PD. Traditional statistical models are widely used to predict it, but the rise of machine learning has garnered attention for its potential in predictive medicine. Comparing the performance of traditional statistical methods and machine learning models provides insight into the optimal approach for CR-POPF prediction.MethodsClinical data from patients undergoing PD were collected. CR-POPF prediction models were developed using Multivariable LR and RF, and their predictive performance was compared using Calibration curves, ROC curves and DCA curves.ResultsIn the calibration curve analysis, the Multivariable LR model shows better calibration than the RF. The Multivariable LR model achieved an AUC of 0.96, while the RF model achieved an AUC of 0.90, indicating superior predictive accuracy of the Multivariable LR model. Decision curve analysis demonstrated that the Multivariable LR model provided higher net benefit across most threshold ranges than the RF model.ConclusionThe Multivariable LR model outperformed the RF model in predicting CR-POPF after PD and can be considered the preferred method for CR-POPF risk assessment.