Anomaly-Based Intrusion Detection: Feature Selection and Normalization Influence to the Machine Learning Models Accuracy

The evolution of the Internet of Things (IoT) has empowered modern industries with the capability to implement large-scale IoT ecosystems, such as the Industrial Internet of Things (IIoT). The IIoT is vulnerable to a diverse range of cyberattacks that can be exploited by intruders and cause substantial reputational and financial harm to organizations. To preserve the confidentiality, integrity, and availability of IIoT networks, an anomaly-based intrusion detection system (IDS) can be used to provide secure, reliable, and efficient IIoT ecosystems. In this paper, we propose an anomaly-based IDS for IIoT networks as an effective security solution to efficiently and effectively overcome several IIoT cyberattacks. The proposed anomaly-based IDS is divided into three phases: pre-processing, feature selection, and classification. In the pre-processing phase, data cleaning and normalization are performed. In the feature selection phase, the candidates’ feature vectors are computed using two feature reduction techniques, minimum redundancy maximum relevance and neighborhood components analysis. For the final step, the modeling phase, the following classifiers are used to perform the classification: support vector machine, decision tree, k-nearest neighbors, and linear discriminant analysis. The proposed work uses a new data-driven IIoT data set called X-IIoTID. The experimental evaluation demonstrates our proposed model achieved a high accuracy rate of 99.58%, a sensitivity rate of 99.59%, a specificity rate of 99.58%, and a low false positive rate of 0.4%.

Anomaly-based intrusion detection systems identify the computer network behavior which deviates from the statistical model of typical network behavior. Binary classifiers based on supervised machine learning are very accurate at classifying network data into two categories: normal traffic and anomalous activity. Most problems with supervised learning are related to the large amount of data required to train the classifiers. Feature selection can be used to reduce datasets. The goal of feature selection is to select a subset of relevant input features to optimize the evaluation and improve performance of a given classifier. Feature scaling normalizes all features to the same range, preventing the large size of features from affecting classification models or other features. The most commonly used supervised machine learning models, including decision trees, support vector machine, k-nearest neighbors, weighted k-nearest neighbors and feedforward neural network, can all be improved by using feature selection and feature scaling. This paper introduces a new feature scaling technique based on a hyperbolic tangent function and damping strategy of the Levenberg–Marquardt algorithm.

Text mining is a special case of data mining which explore unstructured or semi-structured text documents, to establish valuable patterns and rules that indicate trends and significant features about specific topics. Text mining has been in pattern recognition, predictive studies, sentiment analysis and statistical theories in many areas of research, medicine, financial analysis, social life analysis, and business intelligence. Text mining uses concept of natural language processing and machine learning. Machine learning algorithms have been used and reported to give great results, but their performance of machine learning algorithms is affected by factors such as dataset domain, number of classes, length of the corpus, and feature selection techniques used. Redundant attribute affects the performance of the classification algorithm, but this can be reduced by using different feature selection techniques and dimensionality reduction techniques. Feature selection is a data preprocessing step that chooses a subset of input variable while eliminating features with little or no predictive information. Feature selection techniques are Information gain, Term Frequency, Term Frequency-Inverse document frequency, Mutual Information, and Chi-Square, which can use a filters, wrappers, or embedded approaches. To get the most value from machine learning, pairing the best algorithms with the right tools and processes is necessary. Little research has been done on the effect of feature selection techniques on classification accuracy for pairing of these algorithms with the best feature selection techniques for optimal results. In this research, a text classification experiment was conducted using incident management dataset, where incidents were classified into their resolver groups. Support vector machine (SVM), K-Nearest Neighbors (KNN), Naïve Bayes (NB) and Decision tree (DT) machine learning algorithms were examined. Filtering approach was used on the feature selection techniques, with different ranking indices applied for optimal feature set and classification accuracy results analyzed. The classification accuracy results obtained using TF were, 88% for SVM, 70% for NB, 79% for Decision tree, and KNN had 55%, while Boolean registered 90%, 83%, 82% and 75%, for SVM, NB, DT, and KNN respectively. TF-IDF, had 91%, 83%, 76%, and 56% for SVM, NB, DT, and KNN respectively. The results showed that algorithm performance is affected by feature selection technique applied. SVM performed best, followed by DT, KNN and finally NB. In conclusion, presence of noisy data leads to poor learning performance and increases the computational time. The classifiers performed differently depending on the feature selection technique applied. For optimal results, the classifier that performed best together with the feature selection technique with the best feature subset should be applied for all types of data for accurate classification performance. Keywords: Text Classification, Supervised Machine Learning, Feature Selection DOI : 10.7176/JIEA/9-3-06 Publication date :May 31 st 2019

An intrusion detection system (IDS) is an important protection instrument for detecting complex network attacks. Various machine learning (ML) or deep learning (DL) algorithms have been proposed for implementing anomaly-based IDS (AIDS). Our review of the AIDS literature identifies some issues in related work, including the randomness of the selected algorithms, parameters, and testing criteria, the application of old datasets, or shallow analyses and validation of the results. This paper comprehensively reviews previous studies on AIDS by using a set of criteria with different datasets and types of attacks to set benchmarking outcomes that can reveal the suitable AIDS algorithms, parameters, and testing criteria. Specifically, this paper applies 10 popular supervised and unsupervised ML algorithms for identifying effective and efficient ML-AIDS of networks and computers. These supervised ML algorithms include the artificial neural network (ANN), decision tree (DT), k-nearest neighbor (k-NN), naive Bayes (NB), random forest (RF), support vector machine (SVM), and convolutional neural network (CNN) algorithms, whereas the unsupervised ML algorithms include the expectation-maximization (EM), k-means, and self-organizing maps (SOM) algorithms. Several models of these algorithms are introduced, and the turning and training parameters of each algorithm are examined to achieve an optimal classifier evaluation. Unlike previous studies, this study evaluates the performance of AIDS by measuring the true positive and negative rates, accuracy, precision, recall, and F-Score of 31 ML-AIDS models. The training and testing time for ML-AIDS models are also considered in measuring their performance efficiency given that time complexity is an important factor in AIDSs. The ML-AIDS models are tested by using a recent and highly unbalanced multiclass CICIDS2017 dataset that involves real-world network attacks. In general, the k-NN-AIDS, DT-AIDS, and NB-AIDS models obtain the best results and show a greater capability in detecting web attacks compared with other models that demonstrate irregular and inferior results.

Heart disease is still one of the top causes of death globally, demanding early and effective prediction tools. Traditional predictive models frequently have limitations because they rely on single‐source statistics, which may not represent the entire scenario of a patient's complex health profile. Feature selection is an essential preprocessing step in machine learning that improves model efficiency by eliminating irrelevant or redundant features. This study evaluates the impact of feature selection on six classification modelsNaïve Bayes, Logistic Regression, Decision Tree, K‐Nearest Neighbors (KNN), Support Vector Machine (SVM), and Random Forestacross four datasets. Performance was assessed using accuracy, precision, recall, F1‐score, and area under the curve (AUC) before and after feature selection. The results indicate that feature selection had varying effects across different models and datasets. In Dataset‐01, Random Forest maintained its strong performance (98.70% accuracy) even after feature selection, while SVM and KNN saw noticeable drops, with accuracy decreasing from 88.96% to 79.87% and 87.01% to 81.17% respectively. For Dataset‐02, SVM showed improvement, increasing from 80.00% to 85.00%, while Naïve Bayes and KNN also improved, reaching 82.50% and 82.50% accuracy, respectively. In Dataset‐03, Logistic Regression and KNN experienced declines, with accuracy dropping from 88.76% to 83.71% and 91.57% to 87.08%, respectively, while Random Forest fell from 97.75% to 91.57%. Dataset‐04 showed mixed results, with Logistic Regression improving from 86.67% to 91.11%, but KNN and Random Forest dropping from 97.78% to 84.44% and 91.11% to 82.22%, respectively. These findings suggest that while feature selection enhances performance for some models, it can negatively impact others, particularly those reliant on a larger feature set. Future research should explore adaptive feature selection techniques to optimize model performance across diverse datasets.

Cardiovascular disease remains the most significant contributor to global mortality, highlighting the importance of early and precise risk assessment within preventive healthcare frameworks. Alongside the rapid growth of clinical data availability, machine learning approaches have increasingly been adopted to assist medical decision-making, particularly for interpreting complex and high-dimensional health information. This research investigates the predictive capability of six supervised machine learning models in determining the likelihood of cardiovascular disease incidence: Logistic Regression, Support Vector Machine, k-Nearest Neighbors, Decision Tree, Random Forest, and Gradient Boosting. The Cleveland Heart Disease dataset from the UCI Machine Learning Repository served as the study's foundation. It includes 303 patient samples with a total of 76 recorded attributes. From this dataset, 14 clinically significant variables frequently reported in previous studies were selected for analysis. Considering the relatively small dataset size and the possibility of redundant or low-impact features, a feature selection approach was implemented to improve model robustness, minimize overfitting, and enhance interpretability. The data preparation process involved cleaning, normalization, feature selection, and division into datasets for testing and training. Metrics like accuracy, precision, recall, and F1-score were used to evaluate the model. The results of the experiment show that Random Forest and Logistic Regression models produced the highest predictive performance, followed by k-Nearest Neighbours and Support Vector Machine. These results indicate that supervised machine learning techniques, when supported by appropriate feature selection methods, are effective as decision-support tools for the early detection of cardiovascular disease.

Feature selection is typically employed before or in conjunction with classification algorithms to reduce the feature dimensionality and improve the classification performance, as well as reduce processing time. While particular approaches have been developed for feature selection, such as filter and wrapper approaches, some algorithms perform feature selection through their learning strategy. In this paper, we are investigating the effect of the implicit feature selection of the PRISM algorithm, which is rule-based, when compared with the wrapper feature selection approach employing four popular algorithms: decision trees, na'ive bayes, k-nearest neighbors and support vector machine. Moreover, we investigate the performance of the algorithms on target classes, i.e. where the aim is to identify one or more phenomena and distinguish them from their absence (i.e. non-target classes), such as when identifying benign and malign cancer (two target classes) vs. non-cancer (the non-target class).

Objectives: (1) To apply various traditional classification tools, (2) To check effectiveness of the classifiers to the Parkinson Dataset (3) To use boosting classification tools and (4) Compare performance of all used classification tools and find the best accuracy classifier algorithm. Thus, the main aim of the study is to discriminate healthy people from those with PD. Methods: The methodology of this study is categorised into three stages:(1) Preprocessing and feature selection; (2) Application of classifiers; (3) Comparative study. We have used secondary dataset of voice recordings originally collected by University of Oxford by Max Little. In first step, the voice data of PD patients is collected for analysis. Then the collected data is normalized using min-max normalization followed by feature extraction. Thus, uses classification Data Mining Techniques viz., KNN, Logistic Regression, Decision Tree, SVM, Random Forest and boosting algorithm etc. to predict whether the person is healthy or has Parkinson’s disease. Finally, comparative analysis is made based on the accuracy provided by different data mining models. Findings: Results of our study reveals that GB algorithm is more accurate as compared with other models. It gives the highest accuracy, so that we recommend this algorithm to deal similar kind of studies in the future. These models are very useful in better and exact medical diagnosis and decision making. It is also found that, proposed methods are fully computerized and produce enhanced performance hence can be recommended for similar studies. Here, it is observed that Gradient Boost algorithm provide the best accuracy (100% for training and 92.02% for testing, 98.46% overall). Novelty: We have used boosting classification model for the classification of Parkinson’s disease. Our proposed method is one such good example giving faster and more accurate results for the classification of Parkinson’s disease patients with excellent accuracy. We have also compared the results with other existing approaches like linear discriminant analysis, support vector machine, K-nearest neighbour, decision tree, classification and regression trees, random forest, linear regression, logistic regression and Naive Bayes, but our proposed techniques were superior to existing studies in which gradient boost algorithm yielded an accuracy of 98.46%, so our method can be used as an effective means of computer-aided diagnosis of PD, and has important practical value. Keywords: Data Mining, Parkinson's Disease, Classification, Boosting Algorithms, Feature Selection

Exploring sex classification from earprints - A comparison of supervised machine learning algorithms and conventional linear discriminant analysis.

Data driven simulations for accurately predicting thermodynamic properties of H2 during geological storage

This project focuses on using machine learning to predict a patient’s risk of developing diabetes based on a test. This database, compiled by the National Institute of Diabetes and Digestive and Kidney Diseases, contains health indicators for Pima Indian patients. The project involves several key steps: initial data, selecting a feature, selecting a model, updating the hyperparameter, and deploying it via the Flask web application. In the data preprocessing phase, feature scaling and normalization are used to standardize the dataset, while missing values and outliers are handled to ensure data integrity. Feature selection uses correlation matrix and recursive feature elimination (RFE) to reduce dimensionality and improve model efficiency. To ensure the model is optimized for latent data, the dataset is split into two parts: 66% for training and 34% for testing. Various machine learning algorithms are evaluated, including logistic regression, naive Bayes, K-nearest neighbours, decision trees, and support vector classifiers. Logistic regression was selected as the final model due to its accuracy on the test data (80.53%). The model uses grid search for hyperparameter tuning to improve its performance. The training model is embedded in the Flask web application, allowing users to access health metrics and get real-time estimates of blood pressure. The system is designed to be user-friendly and scalable, providing a practical tool for early diagnosis of diabetes. All methods ensure that the model is accurate, reliable, and capable of making real-world predictions. Keywords: prediction, diabetes, glucoses, insulin, machine learning, logistic regression, naive bayes, k- nearest neighbours, decision tree, support vector classifier.

Intrusion Detection Systems (IDS) play a vital role in securing Internet of Things (IoT) networks against cyber-attacks. Previous work used a binary classification approach to detect only Denial-of-Service (DoS) attacks. This paper presents an improved multiclass IDS that identifies multiple attack types. The proposed approach uses simulated IoTID20-based traffic datasets to classify Normal, Denial-of-Service, Mirai botnet, Man-in-the-Middle (MITM), and Scan activity attacks. The proposed approach uses a combination of feature selection methods, including correlation-based filtering and a genetic algorithm, followed by machine learning classifiers such as Decision Tree (DT), Random Forest (RF), K-Nearest Neighbors (KNN), and Support Vector Machine (SVM). Experimental results show a significant improvement in classification performance. The RF classifier with Genetic Algorithm (GA)-based feature selection achieved the highest accuracy (96.5%), followed closely by DT and SVM. Therefore, based on our theoretical and experimental comparisons, the proposed approach could be a practical step toward deploying more robust, realistic IDS models in IoT environments.

In recent years, COVID-19 has been regarded as the most dangerous pandemic for several countries. On various social media platforms, such as Twitter, Facebook, and Instagram, a variety of rumours, hypes, and news are published. This might have a detrimental impact on people’s life. As a result, social media platforms have always had a difficult time authenticating this fake information. Different machine learning (ML) and deep learning (DL) classifiers were used in this work to categorize the continuing impacts of tweets and forecast their after-effects. Support vector machine (SVM), random forest (RF), decision tree (DT), and k-nearest neighbor (KNN) were used for classification, while AdaBoost and convolutional neural network (CNN) were utilized for future effects. The tweets dataset from Kaggle was used to train the SVM, RF, KNN, and DT models, which were then assessed on multiple evaluation criteria such as accuracy, precision, recall, and F1-score, using a 70 : 30 ratio. The CNN and AdaBoost, on the other hand, have been taught to detect the mean square error, root mean square error, and mean absolute error. With 0.74 and 0.73 percent score out of 1, respectively, RF and SVM exhibit the best accuracy in impact when classifying the outcomes on the obtained dataset. In terms of a regression problem, CNN beat the ADA Regressor across the board.

Feature selection is referred to the process of obtaining a subset from an original feature set according to certain feature selection criterion, which selects the relevant features of the dataset. It plays a role in compressing the data processing scale, where the redundant and irrelevant features are removed. Feature selection techniques show that more information is not always good in machine learning applications. Apply different algorithms for the data at hand and with baseline classification performance values we can select a final feature selection algorithm. In this paper, we propose a hybrid classification model, which has correlation based filter feature selection algorithm and Machine learning as classifiers. The objective of this study is to select relevant features and analyze the outperform machine learning algorithms in order to train our model, predict and compare their classification performance. In this method, features are ordered according to their Absolute correlation value with respect to the class attribute. Then top K Features are selected from ordered list of features to form a reduced dataset. This proposed classifier model is applied to our smart meter datasets. To measure the performance of these selected features; seven benchmark classifier are used; Random Forest (RF), Logistic Regression (LR), k-Nearest Neighbor (kNN), Naive Bayes (NB), Decision Tree (DT), Linear Discriminant Analysis (LDA) and Support Vector Machine (SVM). This paper then analyzes the performance of all classifiers with feature selection in term of accuracy, sensitivity, F-Measure, Specificity, Precision, and MCC. From our experiment, we found that Random Forest classifier performed higher than other used classifiers.

BackgroundAcute kidney injury (AKI) is the most common and serious complication of sepsis, accompanied by high mortality and disease burden. The early prediction of AKI is critical for timely intervention and ultimately improves prognosis. This study aims to establish and validate predictive models based on novel machine learning (ML) algorithms for AKI in critically ill patients with sepsis.MethodsData of patients with sepsis were extracted from the Medical Information Mart for Intensive Care III (MIMIC- III) database. Feature selection was performed using a Boruta algorithm. ML algorithms such as logistic regression (LR), k-nearest neighbors (KNN), support vector machine (SVM), decision tree, random forest, Extreme Gradient Boosting (XGBoost), and artificial neural network (ANN) were applied for model construction by utilizing tenfold cross-validation. The performances of these models were assessed in terms of discrimination, calibration, and clinical application. Moreover, the discrimination of ML-based models was compared with those of Sequential Organ Failure Assessment (SOFA) and the customized Simplified Acute Physiology Score (SAPS) II model.ResultsA total of 3176 critically ill patients with sepsis were included for analysis, of which 2397 cases (75.5%) developed AKI during hospitalization. A total of 36 variables were selected for model construction. The models of LR, KNN, SVM, decision tree, random forest, ANN, XGBoost, SOFA and SAPS II score were established and obtained area under the receiver operating characteristic curves of 0.7365, 0.6637, 0.7353, 0.7492, 0.7787, 0.7547, 0.821, 0.6457 and 0.7015, respectively. The XGBoost model had the best predictive performance in terms of discrimination, calibration, and clinical application among all models.ConclusionThe ML models can be reliable tools for predicting AKI in septic patients. The XGBoost model has the best predictive performance, which can be used to assist clinicians in identifying high-risk patients and implementing early interventions to reduce mortality.