A Hybrid Ensemble-Based Intrusion Detection System for IoT Using Feature Optimization and Class Balancing

This study investigates the use of machine learning (ML) models combined with a Synthetic Minority Over-sampling Technique (SMOTE) and its variants to predict perioperative pressure injuries (PIs) in an imbalanced dataset. PIs are a significant healthcare problem, often leading to prolonged hospitalization and increased medical costs. Conventional risk assessment scales are limited in their ability to predict PIs accurately, prompting the exploration of ML techniques to address this challenge.We utilized data from 7,292 patients admitted to a tertiary care hospital in Shanghai between May 2017 and July 2023, with a final dataset of 2,972 patients, including 158 with PIs. Seven ML algorithms-Support Vector Machine (SVM), Logistic Regression (LR), Random Forest (RF), Extreme Gradient Boosting (XGBoost), Extra Trees (ET), K-Nearest Neighbors (KNN), and Decision Trees (DT)-were used in conjunction with SMOTE, SMOTE+ENN, Borderline-SMOTE, ADASYN, and GAN to balance the dataset and improve model performance.Results revealed significant improvements in model performance when SMOTE and its variants were used. For instance, the XGBoost model hadan AUC of 0.996 with SMOTE, compared to 0.800 on raw data. SMOTE+ENN and Borderline-SMOTE further enhanced the models' ability to identify minority classes. External validation indicatedthat XGBoost, RF, and ET exhibited the highest stability and accuracy, with XGBoost having an AUC of 0.977. SHAP analysis revealed that factors such as anesthesia grade, age, and serum albumin levels significantly influenced model predictions.In conclusion, integrating SMOTE with ML algorithms effectively addressed a data imbalance and improved the prediction of perioperative PIs. Future work should focus on refining SMOTE techniques and exploring their application to larger, multi-center datasets to enhance the generalizability of these findings, and especially for diseaseswith a lowincidence.

Parkinson's disease (PD) presents a growing global health challenge, with early detection being crucial for effective management and treatment. This study seeks to develop an innovative machine learning (ML) framework for the early detection of PD by integrating advanced techniques for data preprocessing, dimensionality reduction, feature selection, and ensemble classification, aiming to significantly improve detection accuracy and timeliness. The research employs a robust ML pipeline, beginning with data preprocessing using mean imputation, standardization, min-max scaling, and SMOTE (Synthetic Minority Over-sampling Technique) to handle imbalanced data. Dimensionality reduction is achieved through Principal Component Analysis (PCA), while feature selection is performed using SelectKBest coupled with the ANOVA F-test to identify the most relevant features. Four ensemble methods—Random Forest, Gradient Boosting, XGBoost, and Support Vector Machine (SVM)—are evaluated for classification. Among the classifiers tested, the Gradient Boosting model stands out with an impressive accuracy of 0.9487, demonstrating its superior performance in PD detection. Integrating multiple preprocessing, dimensionality reduction, and feature selection techniques proves essential in optimizing model performance, highlighting the importance of a multifaceted approach in handling complex datasets. This research introduces a comprehensive ML framework that combines multiple advanced techniques in a streamlined process, significantly improving the early detection of Parkinson's disease. Ensemble methods, combined with strategic feature selection and data balancing techniques, offer a novel approach that could be applied to other neurodegenerative disorders, expanding its potential impact beyond PD detection.

Breast cancer remains a leading cause of mortality among women worldwide, with early and accurate diagnosis being critical for effective treatment. Gene expressionprofiling has emerged as a powerful tool for understanding the molecular mechanisms of cancer and for developing predictive models. However, the high dimensionality and class imbalance inherent in gene expression data pose significant challenges for developing robust predictive models.This study aims to develop and evaluate a predictive model for classifying breast cancer subtypes using high-dimensional gene expression data, addressing the challenges of class imbalance. The objective is to improve the accuracy and reliability of breast cancer subtype prediction to facilitate better diagnostic and treatment strategies.A comprehensive dataset of breast cancer gene expression profiles was utilized, comprising numerous gene expression levels across multiple samples. To address class imbalance, resampling techniques such as Synthetic Minority Over-sampling Technique (SMOTE) and Random Under-sampling, were employed. The machine learning algorithms employed included Support Vector Machines (SVM), Random Forests, and Neural Networks. The algorithms were trained and evaluated using cross-validation to identify the most effective model. The performance of these models was assessed based on metrics such as accuracy, precision, recall, and F1-score. The results indicate that the use of SMOTE in combination with SVM provided the most balanced and accurate predictions, with an F1-score significantly higher than models without resampling. The Random Forest algorithm also showed promising results, particularly in handling the high-dimensionality aspect of the data.The study demonstrates that addressing class imbalance through advanced resampling techniques can significantly enhance the predictive accuracy of models trained on high-dimensional gene expression data. The findings underscore the potential of machine learning models, particularly SVMs and Random Forests, in improving breast cancer subtype classification.

<p>Soil properties could be assessed with reflectance spectroscopy (soil spectroscopy, SS) in vis-NIR region (400-2500 nm) through absorption features found in soil spectra. A high spectral resolution (up to 1 nm) drives to high dimensional and multicollinear data. This issue is usually addressed prior to modelling with feature extraction methods such as Principal Component Analysis, or embedded methods such as Partial Least Squares Regression (PLSR). Feature Selection (FS) wrapper methods are promising dimensionality reduction approaches barely used in SS. The objective of this study was two-fold: i) evaluate the performance of FS wrapper methods built from Random Forest (RF) algorithm to predict soil organic matter (SOM), clay and carbonates using laboratory spectroscopy, ii) test the performance of FS methods for dimensionality reduction in SS. The reflectance of 100 soil samples from Sierra de las Nieves National Park (Spain), was measured under laboratory conditions using an ASD FieldSpec Pro JR. A spectral preprocessing method, Continumm Removal (CR), was applied to raw spectra. The RF wrapper considered two different feature searching approaches: Sequential Forward Selection (SFS) and Sequential Flotant Forward Selection (SFFS). The performance of RF with FS (RF-FS) was compared to that of Partial Least Squares Regression (PLSR) and RF (without FS). Models were evaluated with R-squared, root mean squared error (RMSE) and ratio of prediction to deviation (RPD).</p><p>RF-FS models outperformed PLSR and RF models for the three SAP. RF-FS best models had a RPD of 2.19 for SOM, 1.64 for carbonates and 1.52 for clay, whereas PLSR models had RPD values of 1.59, 1.22 and 1.3, and RF 1.38, 1.23 and 1.23 for SOM, carbonates, and clay, respectively. Therefore, FS was useful in obtaining models with improved accuracy by reducing redundant features and avoiding multicollinearity (Hughes effect). The application of FS wrapper methods reduced the number of features in the RF-FS models to less than 1% of the starting features. Features were selected across all spectra from SOM and clay, and around 900, 1900 and 2350 nm for carbonates. This research, thus, shows an alternative to different feature extraction approaches for modelling soil properties based on FS methods and machine learning.</p>

Unbalanced datasets are a common problem in supervised machine learning. It leads to a deeper understanding of the majority of classes in machine learning. Therefore, the machine learning model is more effective at recognizing the majority classes than the minority classes. Naturally, imbalanced data, such as disease data and data networking, has emerged in real life. DDOS is one of the network intrusions found to happen more often than R2L. There is an imbalance in the composition of network attacks in Intrusion Detection System (IDS) public datasets such as NSL-KDD and UNSW-NB15. Besides, researchers propose many techniques to transform it into balanced data by duplicating the minority class and producing synthetic data. Synthetic Minority Oversampling Technique (SMOTE) and Adaptive Synthetic (ADASYN) algorithms duplicate the data and construct synthetic data for the minority classes. Meanwhile, machine learning algorithms can capture the labeled data's pattern by considering the input features. Unfortunately, not all the input features have an equal impact on the output (predicted class or value). Some features are interrelated and misleading. Therefore, the important features should be selected to produce a good model. In this research, we implement the recursive feature elimination (RFE) technique to select important features from the available dataset. According to the experiment, SMOTE provides a better synthetic dataset than ADASYN for the UNSW-B15 dataset with a high level of imbalance. RFE feature selection slightly reduces the model's accuracy but improves the training speed. Then, the Decision Tree classifier consistently achieves a better recognition rate than Random Forest and KNN.

Purpose:There has been burgeoning interest in applying machine learning methods for predicting radiotherapy outcomes. However, the imbalanced ratio of a large number of variables to a limited sample size in radiation oncology constitutes a major challenge. Therefore, dimensionality reduction methods can be a key to success. The study investigates and contrasts the application of traditional machine learning methods and deep learning approaches for outcome modeling in radiotherapy. In particular, new joint architectures based on variational autoencoder (VAE) for dimensionality reduction are presented and their application is demonstrated for the prediction of lung radiation pneumonitis (RP) from a large-scale heterogeneous dataset.Methods:A large-scale heterogeneous dataset containing a pool of 230 variables including clinical factors (e.g., dose, KPS, stage) and biomarkers (e.g., single nucleotide polymorphisms (SNPs), cytokines and micro-RNAs) in a population of 106 non-small cell lung cancer (NSCLC) patients who received radiotherapy was used for modeling RP. 22 patients had grade 2 or higher RP. Four methods were investigated, including feature selection (case A) and feature extraction (case B) with traditional machine learning methods, a VAE-MLP joint architecture (case C) with deep learning and lastly, the combination of feature selection and joint-architecture (case D). For feature selection, Random forest (RF), Support Vector Machine (SVM) and multi-layer perceptron (MLP) were implemented to select relevant features. Specifically, each method was run for multiple times to rank features within several cross-validated (CV) resampled sets. A collection of ranking lists were then aggregated by top 5% and Kemeny graph methods to identify the final ranking for prediction. A synthetic minority oversampling technique (SMOTE) was applied to correct for class imbalance during this process. For deep learning, a VAE-MLP joint architecture where a VAE aimed for dimensionality reduction and an MLP aimed for classification was developed. In this architecture, reconstruction loss and prediction loss were combined into a single loss function to realize simultaneous training and weights were assigned to different classes to mitigate class imbalance. To evaluate the prediction performance and conduct comparisons, the area under receiver operating characteristic (ROC) curves (AUCs) were performed for nested CVs for both handcrafted feature selections and the deep learning approach. The significance of differences in AUCs was assessed using the DeLong test of U-statistics.Results:An MLP-based method using weight pruning (WP) feature selection yielded the best performance among the different hand-crafted feature selection methods (case A), reaching an AUC of 0.804 (95% CI: 0.761–0.823) with 29 top features. A VAE-MLP joint architecture (case C) achieved a comparable but slightly lower AUC of 0.781 (95% CI: 0.737–0.808) with the size of latent dimension being 2. The combination of handcrafted features (case A) and latent representation (case D) achieved a significant AUC improvement of 0.831 (95% CI: 0.805–0.863) with 22 features (p-value = 0.000642 compared with handcrafted features only (Case A) and p-value=0.000453 compared to VAE alone (Case C)) with an MLP classifier.Conclusion:The potential for combination of traditional machine learning methods and deep learning VAE techniques has been demonstrated for dealing with limited datasets in modeling radiotherapy toxicities. Specifically, latent variables from a VAE-MLP joint architecture are able to complement handcrafted features for the prediction of RP and improve prediction over either method alone.

Water quality assessment plays a crucial role in various aspects, including human health, environmental impact, agricultural productivity, and industrial processes. Machine learning (ML) algorithms offer the ability to automate water quality evaluation and allow for effective and rapid assessment of parameters associated with water quality. This article proposes an ML-based classification model for water quality prediction. The model was tested with 14 ML algorithms and considers 20 features that represent various substances present in water samples and their concentrations. The dataset used in the study comprises 7,996 samples, and the model development involves several stages, including data preprocessing, Yeo–Johnson transformation for data normalization, principal component analysis (PCA) for feature selection, and the application of the synthetic minority over-sampling technique (SMOTE) to address class imbalance. Performance metrics, such as accuracy, precision, recall, and F1 score, are provided for each algorithm with and without SMOTE. LightGBM, XGBoost, CatBoost, and Random Forest were identified as the best-performing algorithms. XGBoost achieved the highest accuracy of 96.31% without SMOTE and had a precision of 0.933. The application of SMOTE enhanced the performance of CatBoost. These findings provide valuable insights for ML-based water quality assessment, aiding researchers and professionals in decision-making and management.

Prediction of the stage of cancer plays an important role in planning the course of treatment and has been largely reliant on imaging tools which do not capture molecular events that cause cancer progression. Gene-expression data-based analyses are able to identify these events, allowing RNA-sequence and microarray cancer data to be used for cancer analyses. Breast cancer is the most common cancer worldwide, and is classified into four stages - stages 1, 2, 3, and 4 [2]. While machine learning models have previously been explored to perform stage classification with limited success, multi-class stage classification has not had significant progress. There is a need for improved multi-class classification models, such as by investigating deep learning models. Gene-expression-based cancer data is characterised by the small size of available datasets, class imbalance, and high dimensionality. Class balancing methods must be applied to the dataset. Since all the genes are not necessary for stage prediction, retaining only the necessary genes can improve classification accuracy. The breast cancer samples are to be classified into 4 classes of stages 1 to 4. Invasive ductal carcinoma breast cancer samples are obtained from The Cancer Genome Atlas (TCGA) and Molecular Taxonomy of Breast Cancer International Consortium (METABRIC) datasets and combined. Two class balancing techniques are explored, synthetic minority oversampling technique (SMOTE) and SMOTE followed by random undersampling. A hybrid feature selection pipeline is proposed, with three pipelines explored involving combinations of filter and embedded feature selection methods: Pipeline 1 - minimum-redundancy maximum-relevancy (mRMR) and correlation feature selection (CFS), Pipeline 2 - mRMR, mutual information (MI) and CFS, and Pipeline 3 - mRMR and support vector machine-recursive feature elimination (SVM-RFE). The classification is done using deep learning models, namely deep neural network, convolutional neural network, recurrent neural network, a modified deep neural network, and an AutoKeras generated model. Classification performance post class-balancing and various feature selection techniques show marked improvement over classification prior to feature selection. The best multiclass classification was found to be by a deep neural network post SMOTE and random undersampling, and feature selection using mRMR and recursive feature elimination, with a Cohen-Kappa score of 0.303 and a classification accuracy of 53.1%. For binary classification into early and late-stage cancer, the best performance is obtained by a modified deep neural network (DNN) post SMOTE and random undersampling, and feature selection using mRMR and recursive feature elimination, with an accuracy of 81.0% and a Cohen-Kappa score (CKS) of 0.280. This pipeline also showed improved multiclass classification performance on neuroblastoma cancer data, with a best area under the receiver operating characteristic (auROC) curve score of 0.872, as compared to 0.71 obtained in previous work, an improvement of 22.81%. The results and analysis reveal that feature selection techniques play a vital role in gene-expression data-based classification, and the proposed hybrid feature selection pipeline improves classification performance. Multi-class classification is possible using deep learning models, though further improvement particularly in late-stage classification is necessary and should be explored further.

Diagnosing a parasitic disease is a very difficult job in clinical practice. In this study, we constructed a machine learning model for diagnosis prediction using patient information. First, we diagnosed whether a patient has a parasitic disease. Next, we predicted the proper diagnosis method among the six types of diagnostic terms (biopsy, endoscopy, microscopy, molecular, radiology, and serology) if the patient has a parasitic disease. To make the datasets, we extracted patient information from PubMed abstracts from 1956 to 2019. We then used two datasets: the prediction for parasite-infected patient dataset (N = 8748) and the prediction for diagnosis method dataset (N = 3780). We then compared four machine learning models: support vector machine, random forest, multi-layered perceptron, and gradient boosting. To solve the data imbalance problem, the synthetic minority over-sampling technique and TomekLinks were used. In the parasite-infected patient dataset, the random forest, random forest with synthetic minority over-sampling technique, gradient boosting, gradient boosting with synthetic minority over-sampling technique, and gradient boosting with TomekLinks demonstrated the best performances (AUC: 79%). In predicting the diagnosis method dataset, gradient boosting with synthetic minority over-sampling technique was the best model (AUC: 87%). For the class prediction, gradient boosting demonstrated the best performances in biopsy (AUC: 88%). In endoscopy (AUC: 94%), molecular (AUC: 90%), and radiology (AUC: 88%), gradient boosting with synthetic minority over-sampling technique demonstrated the best performance. Random forest demonstrated the best performances in microscopy (AUC: 82%) and serology (AUC: 85%). We calculated feature importance using gradient boosting; age was the highest feature importance. In conclusion, this study demonstrated that gradient boosting with synthetic minority over-sampling technique can predict a parasitic disease and serve as a promising diagnosis tool for binary classification and multi-classification schemes.

There are a large number of high-dimensional data in the Internet of Brain Things, and the data is reduced from the high-dimensional to low-dimensional to maintain the similarity between the data, thereby effectively ensuring the operating speed of the Internet of Brain Things. In the traditional method, the distributed dimensionality reduction reconstruction algorithm of the High dimensional data has poor dimensionality reduction effect and serious data distortion after reconstruction. A method for dimensionality reconstruction of high-dimensional data in the Internet of Brain Things is proposed. By using the algorithm of linear discriminant analysis, the projection matrix of high dimensional data is constructed to solve it. According to the solution results, using improved implicit variable model to establish a high dimensional large data dimensionality reduction model for the Internet of Brain Things. The fitness value of data after dimensionality reduction is calculated by quantum immune clonal algorithm, and the optimal individual and optimal solution are determined. The data reconfiguration is realized through the optimal solution marshalling. Experimental results show that the proposed algorithm can effectively improve the dimensionality reduction of High dimensional data in the Internet of Brain Things. After reconstruction, the reconstructed data retain accurate data information, the reliability of reconstructed data is high, and the computational complexity is not high, the need for small storage space, and the advantages of strong promotion ability.

Predicting survival outcomes in critical accidents has been a focal point in machine learning research. This study addresses several limitations of existing methods, including insufficient management of data imbalance, lack of emphasis on hyperparameter tuning, and proneness to overfitting. Many existing models struggle to generalize effectively on imbalanced datasets or depend on default hyperparameter settings, resulting in biased predictions. By integrating Principal Component Analysis (PCA), hyperparameter optimization, and resampling methods, as well as combining Edited Nearest Neighbors (ENN) with the Synthetic Minority Oversampling Technique (SMOTE), the model significantly improves predictive accuracy and model generalization. An ensemble model combining seven machine learning algorithms—Logistic Regression, Support Vector Machine, KNN, Random Forest, XGBoost, LightGBM, and CatBoost—was applied to predict survival outcomes. Stochastic Weighted Averaging (SWA) was applied to mitigate overfitting and enhance generalization. The accuracy increased from 91.97% to 94.89% after SWA was applied in this specific scenario. The combination of PCA-based dimensionality reduction, hyperparameter tuning, and resampling techniques (ENN + SMOTE) ensured the model handled data imbalance and optimized predictive accuracy. The final model demonstrated excellent performance, with Area Under the Curve (AUC) and Average Precision (AP) values both reaching 0.98, indicating high accuracy and precision. These improvements were validated using the Titanic dataset in a binary classification problem of predicting passenger survival. The results emphasize that ensemble learning, enhanced by SWA, offers a powerful framework for handling imbalanced and complex datasets, providing significant advancements in predictive modeling accuracy. This study provides insights into how machine learning techniques can be effectively combined to solve classification challenges in real-world scenarios.

Landslides are a significant natural disaster threat in Indonesia, including Samarinda City. This study aims to evaluate the classification accuracy of landslides using the Naïve Bayes algorithm, combined with the Synthetic Minority Oversampling Technique (SMOTE) for class imbalance and Particle Swarm Optimization (PSO) for feature Selection and parameter optimization. Data were obtained from BPBD and BMKG Samarinda for 2022-2024, evaluated using 10-Fold Cross Validation and Confusion Matrix. The results show that SMOTE did not always improve model accuracy. Naïve Bayes without SMOTE achieved an average accuracy of 87.18%, while with SMOTE, it decreased to 70.55%. This indicates that SMOTE’s impact depends on the dataset. PSO, used for feature Selection and parameter optimization, gave mixed results. Naïve Bayes + PSO Feature Selection achieved an average accuracy of 68.25%, lower than the base model with SMOTE. Naïve Bayes + PSO Optimization achieved the best accuracy at 71.03%, though the improvement was minimal. The combination of Naïve Bayes + PSO Feature Selection + PSO Optimization had the lowest accuracy at 68.08%. This study emphasizes that the use of PSO for feature Selection and optimization needs careful consideration, especially when combined with SMOTE, as its effect on model performance is not always optimal.

Chronic kidney disease (CKD) remains a major global health concern, often progressing silently until advanced stages. Early detection is therefore critical to improving outcomes. This study focuses on identifying the most important predisposing risk factors for CKD and optimizing prediction performance using a stacking machine learning ensemble. Two datasets were analyzed: the UCI CKD dataset and a synthetically simulated CKD dataset designed to mirror real-world variability. A leakage-safe preprocessing pipeline was implemented, including median and mode imputation for missing values, Z-score capping for outliers, min–max normalization, and class balancing through the Synthetic Minority Over-sampling Technique (SMOTE). Feature selection was performed using six complementary approaches: Logistic Regression (LR), Recursive Feature Elimination (RFE), Random Forest (RF), Mutual Information (MI), Chi-Square (χ²), and Principal Component Analysis (PCA) with a majority-vote strategy used to identify features consistently recognized as predictive. For the UCI CKD dataset, the common risk factors were serum creatinine, hemoglobin, packed cell volume, red blood cell count, specific gravity, albumin, sugar, hypertension, diabetes mellitus, and appetite. For the simulated CKD dataset, key predictors included blood glucose random, blood urea, serum creatinine, potassium, hemoglobin, packed cell volume, specific gravity, albumin, red blood cells, pus cell clumps, appetite, pedal edema, and anemia. Using these selected features, the stacking ensemble achieved 100.0% accuracy on the UCI CKD dataset and 96.7% accuracy on the simulated dataset, both with negligible misclassification rates. Bootstrap confidence intervals confirmed the robustness of these results. The findings highlight that combining systematic feature selection with stacking significantly improves predictive accuracy while maintaining interpretability. This integrated framework offers a reliable tool for early CKD detection and can support clinical decision-making in real-world healthcare settings. Future work will focus on expanding validation across multi-site data and developing a clinical decision support interface. IJSS, Vol. 25(2), November, 2025, pp 1-32

At this stage, the high dimension and large variety of network data have increased the difficulty of intrusion detection. In this paper, we discuss the advantages and disadvantages of the MDBoost algorithm. Subsequently to optimize it, we add a slack variable in the objective function, so that the algorithm can effectively prevent over fitting, and the accuracy of the prediction is also improved. Then, we propose a model, which uses the MDBoost-2 algorithm to generate a strong classifier, and we use this model for intrusion detection. Finally, we use the CUP KDD 1999 data set to carry out the experiment. The results show that the new approach outperforms MDBoost and other well-known methods.

This article is aimed at the low accuracy of student weariness prediction in education and the poor prediction effect of traditional prediction models. It was established the SMOTE (Synthetic Minority Oversampling Technique) algorithm and random forest prediction models. This study puts forward to useing the SMOTE oversampling method to balance the data set and then use the random forest algorithm to train the classifier. By comparing the common single classifier with the ensemble learning classifier, it was found that the SMOTE and Random forest method performed more prominently, and the reasons for the increase in the AUC value after using the SMOTE method were analyzed. Using Massive Open Online Course (Mooc) synthesis student’s datasets, which mainly include the length of class, whether the mouse has moved, whether there is a job submitted, whether there is participating in discussions and completing the accuracy of the assignments. It is proved that this method can significantly improve the classification effect of classifiers, so teachers can choose appropriate teaching and teaching interventions to improve student’s learning outcomes.