A GA-like Dynamic Probability Method with Mutual Information for Feature Selection

Feature selection is a required preprocess stage in most of the data mining tasks. This paper presents an improved Harris hawks optimization (HHO) to find high-quality solutions for global optimization and feature selection tasks. This method is an efficient optimizer inspired by the behaviors of Harris' hawks, which try to catch the rabbits. In some cases, the original version tends to stagnate to the local optimum solutions. Hence, a novel HHO called IHHO is proposed by embedding the salp swarm algorithm (SSA) into the original HHO to improve the search ability of the optimizer and expand the application fields. The update stage in the HHO optimizer, which is performed to update each hawk, is divided into three phases: adjusting population based on SSA to generate SSA-based population, generating hybrid individuals according to SSA-based individual and HHO-based individual, and updating search agent in the light of greedy selection and HHO’s mechanisms. A large group of experiments on many functions is carried out to investigate the efficacy of the proposed optimizer. Based on the overall results, the proposed IHHO can provide a faster convergence speed and maintain a better balance between exploration and exploitation. Moreover, according to the proposed continuous IHHO, a more stable binary IHHO is also constructed as a wrapper-based feature selection (FS) approach. We compare the resulting binary IHHO with other FS methods using well-known benchmark datasets provided by UCI. The experimental results reveal that the proposed IHHO has better accuracy rates over other compared wrapper FS methods. Overall research and analysis confirm the improvement in IHHO because of the suitable exploration capability of SSA.

An efficient hybrid sine-cosine Harris hawks optimization for low and high-dimensional feature selection

With the rapid development of the Internet of Things (IoT), the curse of dimensionality becomes increasingly common. Feature selection (FS) is to eliminate irrelevant and redundant features in the datasets. Particle swarm optimization (PSO) is an efficient metaheuristic algorithm that has been successfully applied to obtain the optimal feature subset with essential information in an acceptable time. However, it is easy to fall into the local optima when dealing with high-dimensional datasets due to constant parameter values and insufficient population diversity. In the paper, an FS method is proposed by utilizing adaptive PSO with leadership learning (APSOLL). An adaptive updating strategy for parameters is used to replace the constant parameters, and the leadership learning strategy is utilized to provide valid population diversity. Experimental results on 10 UCI datasets show that APSOLL has better exploration and exploitation capabilities through comparison with PSO, grey wolf optimizer (GWO), Harris hawks optimization (HHO), flower pollination algorithm (FPA), salp swarm algorithm (SSA), linear PSO (LPSO), and hybrid PSO and differential evolution (HPSO-DE). Moreover, less than 8% of features in the original datasets are selected on average, and the feature subsets are more effective in most cases compared to those generated by 6 traditional FS methods (analysis of variance (ANOVA), Chi-Squared (CHI2), Pearson, Spearman, Kendall, and Mutual Information (MI)).

Global navigation satellite systems (GNSS) are extensively utilized for military and civilian applications. Unfortunately, because of the signal weakness, GNSS is susceptible to interference, fading, and jamming, which reduces the position accuracy. Therefore, it would be beneficial to have a simple and highly accurate model for detecting the jamming signals to improve the GNSS receiver accuracy. In this paper, we propose a hybrid deep learning (DL) model for predicting jamming signals. Initially, we utilize a feature selection algorithm that combines mutual information (MI) with the minimal redundancy maximum relevance (mRMR) technique to identify the most crucial features. Subsequently, the model undergoes training using a soft attention-double-layer bidirectional long short-term memory (A-DBiLSTM) model. This particular model has shown outstanding performance in comparison to other DL models when applied to datasets collected from both kinematic and static jamming scenarios. To assess the effectiveness and efficiency of the proposed MI feature selection algorithm, we evaluate its performance through the calculation of confusion matrices and conducting numerical simulations. The simulation results of the A-DBiLSTM model demonstrate higher accuracy, precision, recall, and F1Score\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\mathrm {F1_{Score}}$$\\end{document} of 98.82%\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$98.82\\%$$\\end{document}, 98.4%\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$98.4\\%$$\\end{document}, 98.68%\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$98.68\\%$$\\end{document}, and 98.36%\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$98.36\\%$$\\end{document}, respectively. By employing the MI feature selection algorithm, dimensionality reduction is achieved. Moreover, the MI feature selection algorithm reduces 19%\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$19\\%$$\\end{document} of the learning time with almost the same accuracy.

In healthcare, the Internet of Things (IoT) is used to remotely monitor patients and provide real-time diagnoses, which is referred to as the Internet of Medical Things (IoMT). This integration poses a risk from cybersecurity threats that can harm patient data and well-being. Hackers can manipulate biometric data from biosensors or disrupt the IoMT system, which is a major concern. To address this issue, intrusion detection systems (IDS) have been proposed, particularly using deep learning algorithms. However, developing IDS for IoMT is challenging due to high data dimensionality leading to model overfitting and degraded detection accuracy. Feature selection has been proposed to prevent overfitting, but the existing methods assume that feature redundancy increases linearly with the size of the selected features. Such an assumption does not hold, as the amount of information a feature carries about the attack pattern varies from feature to feature, especially when dealing with early patterns, due to data sparsity that makes it difficult to perceive the common characteristics of selected features. This negatively affects the ability of the mutual information feature selection (MIFS) goal function to estimate the redundancy coefficient accurately. To overcome this issue, this paper proposes an enhanced feature selection technique called Logistic Redundancy Coefficient Gradual Upweighting MIFS (LRGU-MIFS) that evaluates candidate features individually instead of comparing them with common characteristics of the already-selected features. Unlike the existing feature selection techniques, LRGU calculates the redundancy score of a feature using the logistic function. It increases the redundancy value based on the logistic curve, which reflects the nonlinearity of the relationship of the mutual information between features in the selected set. Then, the LRGU was incorporated into the goal function of MIFS as a redundancy coefficient. The experimental evaluation shows that the proposed LRGU was able to identify a compact set of significant features that outperformed those selected by the existing techniques. The proposed technique overcomes the challenge of perceiving common characteristics in cases of insufficient attack patterns and outperforms existing techniques in identifying significant features.

Recycling tasks are the most effective method for reducing waste generation, protecting the environment, and boosting the overall national economy. The productivity and effectiveness of the recycling process are strongly dependent on the cleanliness and precision of processed primary sources. However, recycling operations are often labor intensive, and computer vision and deep learning (DL) techniques aid in automatically detecting and classifying trash types during recycling chores. Due to the dimensional challenge posed by pre-trained CNN networks, the scientific community has developed numerous techniques inspired by biology, swarm intelligence theory, physics, and mathematical rules. This research applies a new meta-heuristic algorithm called the artificial hummingbird algorithm (AHA) to solving the waste classification problem based on feature selection. However, the performance of the AHA is barely satisfactory; it may be stuck in optimal local regions or have a slow convergence. To overcome these limitations, this paper develops two improved versions of the AHA called the AHA-ROBL and the AHA-OBL. These two versions enhance the exploitation stage by using random opposition-based learning (ROBL) and opposition-based learning (OBL) to prevent local optima and accelerate the convergence. The main purpose of this paper is to apply the AHA-ROBL and AHA-OBL to select the relevant deep features provided by two pre-trained models of CNN (VGG19 & ResNet20) to recognize a waste classification. The TrashNet dataset is used to verify the performance of the two proposed approaches (the AHA-ROBL and AHA-OBL). The effectiveness of the suggested methods (the AHA-ROBL and AHA-OBL) is compared with that of 12 modern and competitive optimizers, namely the artificial hummingbird algorithm (AHA), Harris hawks optimizer (HHO), Salp swarm algorithm (SSA), aquila optimizer (AO), Henry gas solubility optimizer (HGSO), particle swarm optimizer (PSO), grey wolf optimizer (GWO), Archimedes optimization algorithm (AOA), manta ray foraging optimizer (MRFO), sine cosine algorithm (SCA), marine predators algorithm (MPA), and rescue optimization algorithm (SAR). A fair evaluation of the proposed algorithms’ performance is achieved using the same dataset. The performance analysis of the two proposed algorithms is applied in terms of different measures. The experimental results confirm the two proposed algorithms’ superiority over other comparative algorithms. The AHA-ROBL and AHA-OBL produce the optimal number of selected features with the highest degree of precision.

Feature selection is one of the important research areas in pattern recognition. The aim of feature selection is to select those of informative features to improve the classifier's performance. In this paper, we propose a novel multi-objective algorithm based on mutual information for feature selection, called multi-objective mutual information (MOMI). The proposed method identifies a set of features with minimal redundancy and maximum relevancy with the target class. Several experiments are performed to evaluate the performance of MOMI compared to that of well-known and state-of-the-art feature selection methods over five benchmark datasets. The results show that in most cases MOMI achieves better classification performance than others.

Improved Salp Swarm Algorithm based on opposition based learning and novel local search algorithm for feature selection

Analyzing EEG (electroencephalograph) signal is one of the most important methods of mental fatigue classification. Previous studies mainly describe the EEG features they use, however, the process of selecting features has not been synthetically researched. This study proposes the weighted-normalized mutual information feature selection (W-NMIFS) method to select features. W-NMIFS criterion optimizes the estimation of redundancy information in mutual information algorithm. Considering general features which are used in EEG analysis, several important ones are selected by W-NMIFS criterion. In order to guarantee the rigor of the experiment, the source datasets are collected from 10 volunteers using 10–20 EEG system. 3 different mental tasks are designed to introduce mental fatigue in low, medium and high levels. LDA (Linear discriminant analysis) classifier is used to assess mental fatigue and compare different feature selection algorithms. The results of mental fatigue classification show that the chosen features using W-NMIFS criterion are suitable for LDA algorithm. The classification precision is improved compared with mRMR (minimal-redundancy-maximal-relevance), NMIFS (normalized mutual information feature selection) and N-NMIFS (novel normalized mutual information feature selection) algorithm.

Feature selection is the most important step in the design of a breast cancer diagnosis system. The basic objective of the proposed methodology is to reduce the size of the feature space to improve the performance of the classification system. In this article, a hybrid teaching–learning based optimization (TLBO) with a Salp swarm algorithm (SSA) is presented to select the features with an artificial neural network as a fitness evaluator. The features selected by TLBO-SSA are evaluated using an adaptive neuro-fuzzy inference system. The performance of the proposed methodology is tested over 651 mammograms. The experimental results show that TLBO-SSA appears to be the best when compared with the basic TLBO algorithm. TLBO-SSA archived an accuracy of 98.46% with 98.81% sensitivity, 98.08% specificity, 0.9852 F-score, 0.9692 Cohen’s kappa coefficient, and area under curve AZ = 0.997 ± 0.001. Again the robustness of the proposed TLBO-SSA method is tested using a benchmark dataset obtained from the UCI repository. The result obtained by TLBO-SSA is compared with the Genetic Algorithm. The results show that TLBO-SSA is better than the Genetic Algorithm.

Mutual Information (MI) is a good selector of relevance between input and output feature and have been used as a measure for ranking features in several feature selection methods. Theses methods cannot estimate optimal feature subsets by themselves, but depend on user defined performance. In this paper, we propose estimation of optimal feature subsets by using rough sets to determine candidate feature subset which receives from MI feature selector. The experiment shows that we can correct nonlinear problems and problems in situation of two or more combined features are dominant features, maintain an improve classification accuracy.KeywordsMutual InformationRough setsNonlinear ProblemsMLPC4.5

An effective feature selection scheme is proposed, which utilizes the combination of wrapper and filter: ant colony optimization (ACO) and mutual information (MI). By examining the modeling based on SVMs at the Australian Bureau of Meteorology, the simulation of three different methods of feature selection shows that the proposed method can reduce the dimensionality of inputs, speed up the training of the network and get better performance.KeywordsSupport Vector MachineFeature SelectionMutual InformationCandidate InputPheromone IntensityThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

In this paper we analyse the strengths and weaknesses of the mainly used feature selection methods in text categorization when they are applied to the spam problem domain. Several experiments with different feature selection methods and content-based filtering techniques are carried out and discussed. Information Gain, χ2-text, Mutual Information and Document Frequency feature selection methods have been analysed in conjunction with Naive Bayes, boosting trees, Support Vector Machines and ECUE models in different scenarios. From the experiments carried out the underlying ideas behind feature selection methods are identified and applied for improving the feature selection process of SpamHunting, a novel anti-spam filtering software able to accurate classify suspicious e-mails.

This chapter presents a comparison of the performance of integral (I) and proportional-integral-derivative (PID) controllers in frequency stabilization or load frequency control (LFC) of two-area interconnected power system considering generation rate constraints (GRCs) with Doubly fed induction generator (DFIG)-based wind energy. Two mathematically models are identified for investigations. Power system model 1 is two-area interconnected power system which contains two identical non-reheat thermal plants without DFIG participation. Whereas, power system model 2 contains two identical non-reheat thermal plants with dynamic participation of DFIG at both areas. Moreover, Harris Hawks Optimizer (HHO), Salp Swarm Algorithm (SSA), and Sine Cosine Algorithm (SCA) are applied to find the optimal values of the controller settings mentioned above. The effectiveness of the proposed controllers, which are optimally designed by several optimization techniques (i.e., HHO, SSA, and SCA) is tested and verified through an interconnected power system comprises two identical non-reheat thermal power plants with/without DFIG participation. Time-domain simulation results of the studied power system with all mentioned optimization techniques are carried out using Matlab/Simulink® software to validate the robustness of the proposed controllers.

A considerable number of intermittent renewable resources like photovoltaic generation and wind energy when integrated into the conventional grid technology causes serious issues in the power systems like frequency instability. So an intelligent controller is desired for steady and reliable operation of the electric grid. This paper deals with the frequency control of hybrid power system (HPS) employing a sine cosine adopted Harris' hawks optimization (SCaHHO) technique. The proposed technique aims at improving the performance of the original Harris' hawks optimization (HHO) algorithm by incorporating a sine and cosine function in the calculation of escaping energy and chances of escaping of prey respectively to emphasize the exploitation process of hawks. The effectiveness of the SCaHHO technique is validated with the novel HHO technique as well as moth flame optimization, sine cosine algorithm, grey wolf optimization, salp swarm algorithm and gravitational search algorithm using 23 standard test functions. A non-parametric statistical investigation is also carried out to verify the effectiveness of the suggested technique. Later the SCaHHO technique is utilized to tune a newly proposed adaptive fuzzy proportional integral derivative controller (AFPID) for frequency control of HPS. The suggested SCaHHO-tuned AFPID controller achieves an improved control action by moderating the frequency fluctuations as compared to the PID controller. Hardware-in-loop validation of the proposed load frequency control scheme is also carried out using OPAL-RT to measure its fidelity with the numerical simulation results.