Ant Colony Optimization Combining with Mutual Information for Feature Selection in Support Vector Machines

Global navigation satellite systems (GNSS) are extensively utilized for military and civilian applications. Unfortunately, because of the signal weakness, GNSS is susceptible to interference, fading, and jamming, which reduces the position accuracy. Therefore, it would be beneficial to have a simple and highly accurate model for detecting the jamming signals to improve the GNSS receiver accuracy. In this paper, we propose a hybrid deep learning (DL) model for predicting jamming signals. Initially, we utilize a feature selection algorithm that combines mutual information (MI) with the minimal redundancy maximum relevance (mRMR) technique to identify the most crucial features. Subsequently, the model undergoes training using a soft attention-double-layer bidirectional long short-term memory (A-DBiLSTM) model. This particular model has shown outstanding performance in comparison to other DL models when applied to datasets collected from both kinematic and static jamming scenarios. To assess the effectiveness and efficiency of the proposed MI feature selection algorithm, we evaluate its performance through the calculation of confusion matrices and conducting numerical simulations. The simulation results of the A-DBiLSTM model demonstrate higher accuracy, precision, recall, and F1Score\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\mathrm {F1_{Score}}$$\\end{document} of 98.82%\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$98.82\\%$$\\end{document}, 98.4%\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$98.4\\%$$\\end{document}, 98.68%\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$98.68\\%$$\\end{document}, and 98.36%\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$98.36\\%$$\\end{document}, respectively. By employing the MI feature selection algorithm, dimensionality reduction is achieved. Moreover, the MI feature selection algorithm reduces 19%\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$19\\%$$\\end{document} of the learning time with almost the same accuracy.

Feature selection is one of the important research areas in pattern recognition. The aim of feature selection is to select those of informative features to improve the classifier's performance. In this paper, we propose a novel multi-objective algorithm based on mutual information for feature selection, called multi-objective mutual information (MOMI). The proposed method identifies a set of features with minimal redundancy and maximum relevancy with the target class. Several experiments are performed to evaluate the performance of MOMI compared to that of well-known and state-of-the-art feature selection methods over five benchmark datasets. The results show that in most cases MOMI achieves better classification performance than others.

Analyzing EEG (electroencephalograph) signal is one of the most important methods of mental fatigue classification. Previous studies mainly describe the EEG features they use, however, the process of selecting features has not been synthetically researched. This study proposes the weighted-normalized mutual information feature selection (W-NMIFS) method to select features. W-NMIFS criterion optimizes the estimation of redundancy information in mutual information algorithm. Considering general features which are used in EEG analysis, several important ones are selected by W-NMIFS criterion. In order to guarantee the rigor of the experiment, the source datasets are collected from 10 volunteers using 10–20 EEG system. 3 different mental tasks are designed to introduce mental fatigue in low, medium and high levels. LDA (Linear discriminant analysis) classifier is used to assess mental fatigue and compare different feature selection algorithms. The results of mental fatigue classification show that the chosen features using W-NMIFS criterion are suitable for LDA algorithm. The classification precision is improved compared with mRMR (minimal-redundancy-maximal-relevance), NMIFS (normalized mutual information feature selection) and N-NMIFS (novel normalized mutual information feature selection) algorithm.

An unsupervised feature selection algorithm based on ant colony optimization

In this paper we analyse the strengths and weaknesses of the mainly used feature selection methods in text categorization when they are applied to the spam problem domain. Several experiments with different feature selection methods and content-based filtering techniques are carried out and discussed. Information Gain, χ2-text, Mutual Information and Document Frequency feature selection methods have been analysed in conjunction with Naive Bayes, boosting trees, Support Vector Machines and ECUE models in different scenarios. From the experiments carried out the underlying ideas behind feature selection methods are identified and applied for improving the feature selection process of SpamHunting, a novel anti-spam filtering software able to accurate classify suspicious e-mails.

In healthcare, the Internet of Things (IoT) is used to remotely monitor patients and provide real-time diagnoses, which is referred to as the Internet of Medical Things (IoMT). This integration poses a risk from cybersecurity threats that can harm patient data and well-being. Hackers can manipulate biometric data from biosensors or disrupt the IoMT system, which is a major concern. To address this issue, intrusion detection systems (IDS) have been proposed, particularly using deep learning algorithms. However, developing IDS for IoMT is challenging due to high data dimensionality leading to model overfitting and degraded detection accuracy. Feature selection has been proposed to prevent overfitting, but the existing methods assume that feature redundancy increases linearly with the size of the selected features. Such an assumption does not hold, as the amount of information a feature carries about the attack pattern varies from feature to feature, especially when dealing with early patterns, due to data sparsity that makes it difficult to perceive the common characteristics of selected features. This negatively affects the ability of the mutual information feature selection (MIFS) goal function to estimate the redundancy coefficient accurately. To overcome this issue, this paper proposes an enhanced feature selection technique called Logistic Redundancy Coefficient Gradual Upweighting MIFS (LRGU-MIFS) that evaluates candidate features individually instead of comparing them with common characteristics of the already-selected features. Unlike the existing feature selection techniques, LRGU calculates the redundancy score of a feature using the logistic function. It increases the redundancy value based on the logistic curve, which reflects the nonlinearity of the relationship of the mutual information between features in the selected set. Then, the LRGU was incorporated into the goal function of MIFS as a redundancy coefficient. The experimental evaluation shows that the proposed LRGU was able to identify a compact set of significant features that outperformed those selected by the existing techniques. The proposed technique overcomes the challenge of perceiving common characteristics in cases of insufficient attack patterns and outperforms existing techniques in identifying significant features.

Feature selection plays a vital role in promoting classifier performance. However, current methods cannot effectively distinguish the complex interaction in the selected features. To further remove these hidden negative interactions, we propose a Genetic Algorithm (GA)-like Dynamic Probability (GADP) method with mutual information method (MI), which has a two-layer structure. The first layer applies MRMR to obtain a primary feature subset. Based on the former candidate features, the GADP, as the second layer, can mine more supportive features. Instead of the commonly used approaches that they frequently focus on improving the operators of GA to enhance the search ability and lowering the convergence time, we improve GA by boldly abandoning the operators and employing the dynamic probability that relies on the performance of each chromosome to determine feature selection in the new generation. The dynamic probability mechanism significantly reduces the parameter number in GA and makes it easy to use. As each gene's probability is independent, the chromosome variety in GADP is more notable than in traditional GA, which ensures GADP has a wider search space and can effectively and accurately select relevant features. To verify the superiority of our method, we evaluate it under multiple conditions on 15 datasets. The results demonstrate the outperformance of the proposed method. Generally, it has the best accuracy. Further, we also compare the proposed model to the state-of-the-art heuristic methods like Harris Hawk Optimization (HHO), Salp Swarm Algorithm (SSA), and White Shark Optimizer (WSO). Our model still owns advantages over them.

Mutual Information (MI) is a good selector of relevance between input and output feature and have been used as a measure for ranking features in several feature selection methods. Theses methods cannot estimate optimal feature subsets by themselves, but depend on user defined performance. In this paper, we propose estimation of optimal feature subsets by using rough sets to determine candidate feature subset which receives from MI feature selector. The experiment shows that we can correct nonlinear problems and problems in situation of two or more combined features are dominant features, maintain an improve classification accuracy.KeywordsMutual InformationRough setsNonlinear ProblemsMLPC4.5

Objectives: To develop a machine learning-based model to select the most important features from a high-dimensional dataset to classify the patterns at high accuracy and reduce their dimensionality. Methods: The proposed feature selection method (FSIFC) forms and combines feature clusters incrementally and produces feature subsets each time. The method uses K-means clustering and Mutual Information (MI) to refine the feature selection process iteratively. Initially, two clusters of features are formed using K-means clustering (K=2) by taking features as the basis of clustering instead of taking the patterns (a traditional way). From these two clusters, the features with the highest MI value in each cluster are kept in a feature subset. Classification accuracies (CA) of the feature subset are calculated using three classifiers namely Support Vector Machines (SVM), Random Forest (RF), and k-nearest Neighbor (knn). The process is repeated by incrementing the value of K i.e. number of clusters; until a maximum user-defined value of K is reached. The best value of CA obtained from these trials is recorded and the corresponding feature set is finally accepted. Findings: The proposed method is demonstrated using ten datasets and the results are compared with the existing published results using three classifiers to determine the method's performance. The ten datasets are classified with average CAs of 92.72%, 93.13%, and 91.5%, using the SVM, RF, and K-NN classifiers respectively. The proposed method selects a maximum of thirty features from the datasets. In terms of selecting the most effective and the smallest feature sets, the proposed method outperforms eight other feature selection methods considering CAs. Novelty: The proposed model applies feature reduction using combined feature clustering and filter methods in an incremental way. This provides an improved selection of relevant features while removing those which are irrelevant at different trials. Keywords: Feature selection, High-dimensional datasets, K-means algorithm, Mutual information, Machine learning

Most traditional IP networks face serious security and management challenges due to their rapid increase in complexity. SDN resolves these issues by the separation of control and data planes, hence enabling programmability for centralized management with flexibility. On the other hand, its centralized architecture makes SDN very prone to DDoS attacks, hence necessitating the use of advanced and efficient IDSs. This study focuses on improving IDS performance in SDN environments through the integration of deep learning techniques and novel feature selection methods. This study presents an Enhanced Maximum Relevance Minimum Redundancy (EMRMR) approach that incorporates a Mutual Information Feature Selection (MIFS) strategy and a new Contextual Redundancy Coefficient Upweighting (CRCU) strategy to optimize feature selection for early attack detection. Experiments on the inSDN dataset showed that EMRMR achieved better precision, recall, F1-score, and accuracy compared to the state-of-the-art approaches, especially when fewer features are selected. These results highlight the efficiency of the proposed EMRMR approach in the selection of relevant features with minimal computational overhead, which enhances the real-time capability for IDS in SDN environments.

Feature selection is an important task which can significantly affect the performance of a pattern recognition system. In this paper, we present a feature selection algorithm based on Ant Colony Optimization (ACO) for human activity recognition. The algorithm incorporates the classification performance into the state transition rule, and uses the pheromone of the ant colony to find the optimal feature set, then selects feature set with small size and high classification accuracy. We perform experiments on human activity dataset using four classifiers, Naive Bayes, Random Forest, K-Nearest Neighbor and Support Vector Machine(NB, RF, KNN and SVM). The results show that the proposed method can improve the recognition accuracy, and it is comparable to other feature selection methods.

Ant-TD: Ant colony optimization plus temporal difference reinforcement learning for multi-label feature selection

Feature selection is selecting a subset of optimal features. Feature selection is being used in high dimensional data reduction and it is being used in several applications like medical, image processing, text mining, etc. Several methods were introduced for unsupervised feature selection. Among those methods some are based on filter approach and some are based on wrapper approach. In the existing work, unsupervised feature selection methods using Genetic Algorithm, Particle Swarm Optimization with Relative Reduct, Quick Reduct and Ant Colony Optimization have been introduced. These methods yield better performance for unsupervised feature selection. In this paper we proposed a novel method to select subset of features from unlabeled data using binary bat algorithm with sum of squared error as the fitness function. The proposed method is then tested with various classification algorithms like decision tree, multilayer perceptron, support vector machine and clustering quality measures like sum of squared error. The results show that our proposed method gives more accuracy when compared with other optimization algorithm.

Ultrasound imaging is most suitable method for early detection of prostate cancer. It is very difficult to distinguish benign and malignant in the early stage of cancer. This is reflected in the high percentage of unnecessary biopsies that are performed and many deaths caused by late detection or misdiagnosis. A computer based classification system can provide a second opinion to the radiologists. Generally objects are described in terms of a set of measurable features in pattern recognition. Feature selection is a process of selecting the most wanted or dominating features set from the original features set in order to reduce the cost of data visualization and increasing classification efficiency and accuracy. The Region of Interest (ROI) is identified from the Transrectal Ultrasound (TRUS) images using DBSCAN clustering with morphological operators. Then the statistical texture features are extracted from the ROIs. Rough Set based Quick Reduct (QR) and Evolutionary based Ant Colony Optimization (ACO) is studied. In this paper, Hybridization of Rough Set based QR and ACO is proposed for dimensionality reduction. The selected features may have the best discriminatory power for classifying prostate cancer based on TRUS images. Support Vector Machine (SVM) is tailored for evaluation of the proposed feature selection methods through classification. Then, the comparative analysis is performed among these methods. Experimental results show that the proposed method QR-ACO produces significant results. Number of features selected using QR-ACO algorithm is minimal, and is successful and has high detection accuracy.

This book chapter presents a short-term load, locational marginal price, and optimal topology-forecasting procedure for a distribution system that utilizes fossil-fueled distributed generation units. An intelligent hybrid model is proposed for forecasting of price and load based on mutual information and Kalman-Kohonen feature selection. The proposed method uses an adaptive neuro-fuzzy inference system. Further, the optimal topology of the system is forecasted based on the historical data and co-active neuro-fuzzy inference system. The feature selection determines the most proper inputs among a huge historical data. The proposed feature selection is based on Kalman-Kohonen model for load forecasting and adaptive neuro-fuzzy inference system model for price forecasting. The obtained results for a distribution system confirmed the model’s effective performance.