Bi-stage feature selection for crop mapping using grey wolf metaheuristic optimization

EEG-based mild depressive detection using feature selection methods and classifiers

Feature selection is an important part of data mining, image recognition and other fields. The efficiency and accuracy of classification algorithm can be improved by selecting the best feature subset. The classical feature selection technology has some limitations, and heuristic optimization algorithm for feature selection is an alternative method to solve these limitations and find the optimal solution. In this paper, we proposed a Multi Strategy Grey Wolf Optimizer algorithm (MSGWO) based on random guidance, local search and subgroup cooperation strategies for feature selection, which solves the problem that the traditional grey wolf optimizer algorithm (GWO) is easy to fall into local optimization with a single search strategy. Among them, the random guidance strategy can make full use of the random characteristics to enhance the global search ability of the population, and the local search strategy makes grey wolf individuals make full use of the search space around the current best solution, and the subgroup cooperation strategy is very important to balance the global search and local search of the algorithm in the iterative process. MSGWO algorithm cooperates with each other in three strategies to update the location of grey wolf individuals, and enhances the global and local search ability of grey wolf individuals. Experimental results show that MSGWO can quickly find the optimal feature combination and effectively improve the performance of the classification model.

<p>Classification aims to identify a class label of an instance according to the information from its characteristics or features. Unfortunately, many classification problems have a large feature set containing irrelevant and redundant features, which reduce the classification performance. In order to address the above problem, feature selection is proposed to select a small subset of relevant features. There are three main types of feature selection methods, i.e. wrapper, embedded and filter approaches. Wrappers use a classification algorithm to evaluate candidate feature subsets. In embedded approaches, the selection process is embedded in the training process of a classification algorithm. Different from the other two approaches, filters do not involve any classification algorithm during the selection process. Feature selection is an important process but it is not an easy task due to its large search space and complex feature interactions. Because of the potential global search ability, Evolutionary Computation (EC), especially Particle Swarm Optimization (PSO), has been widely and successfully applied to feature selection. However, there is potential to improve the effectiveness and efficiency of EC-based feature selection. The overall goal of this thesis is to investigate and improve the capability of EC for feature selection to select small feature subsets while maintaining or even improving the classification performance compared to using all features. Different aspects of feature selection are considered in this thesis such as the number of objectives (single-objective/multi-objective), the fitness function (filter/wrapper), and the searching mechanism. This thesis introduces a new fitness function based on mutual information which is calculated by an estimation approach instead of the traditional counting approach. Results show that the estimation approach works well on both continuous and discrete data. More importantly, mutual information calculated by the estimation approach can capture feature interactions better than the traditional counting approach. This thesis develops a novel binary PSO algorithm, which is the first work to redefine some core concepts of PSO such as velocity and momentum to suit the characteristics of binary search spaces. Experimental results show that the proposed binary PSO algorithm evolve better solutions than other binary EC algorithms when the search spaces are large and complex. Specifically, on feature selection, the proposed binary PSO algorithm can select smaller feature subsets with similar or better classification accuracies, especially when there are a large number of features. This thesis proposes surrogate models for wrapper-based feature selection. The surrogate models use surrogate training sets which are subsets of informative instances selected from the training set. Experimental results show that the proposed surrogate models assist PSO to reduce the computational cost while maintaining or even improving the classification performance compared to using only the original training set. The thesis develops the first wrapper-based multi-objective feature selection algorithm using MOEA/D. A new decomposition strategy using multiple reference points for MOEA/D is designed, which can deal with different characteristics of multi-objective feature selection such as highly discontinuous Pareto fronts and complex relationships between objectives. The experimental results show that the proposed algorithm can evolve more diverse non-dominated sets than other multi-objective algorithms. This thesis introduces the first PSO-based feature selection algorithm for transfer learning. In the proposed algorithm, the fitness function uses classification performance to reduce the differences between domains while maintaining the discriminative ability on the target domain. The experimental results show that the proposed algorithm can select feature subsets which achieve better classification performance than four state-of-the-art feature-based transfer learning algorithms.</p>

High-feature dimensions is one of the obvious challenges facing large-scale hierarchical classification problem. One promising way to overcome such problem is reducing the features size by removing irrelevant, noise and redundant features. Filter-based feature selection methods are commonly used to construct smaller subset of most relevant features in either adaptive or global approach at each non leaf node within the hierarchy. Both the Feature selection approaches do not only reduce time and space complexity but also improve classification accuracy of the learning models. In this study, the effectiveness of the main approaches for constructing relevant features subset, that is adaptive and global approaches are analyzed using the two filter-based methods suitable for text classification. Experiment evaluation conducted on 20NG, IPC, and DMOZ-small datasets shows that implementing feature selection adaptively has more capability of selecting most relevant features, thus, achieving the best classification accuracy. Moreover, further evaluation shown integrating feature selection with LSHC records improve performance when compared to without feature selection.

Image segmentation is a popular application area of machine learning. In this context, each target region drawn from an image is defined as a class towards recognition of instances that belong to this region (class). In order to train classifiers that recognize the target region to which an instance belongs, it is important to extract and select features relevant to the region. In traditional machine learning, all features extracted from different regions are simply used together to form a single feature set for training classifiers, and feature selection is usually designed to evaluate the capability of each feature or feature subset in discriminating one class from other classes. However, it is possible that some features are only relevant to one class but irrelevant to all the other classes. From this point of view, it is necessary to undertake feature selection for each specific class, i.e, a relevant feature subset is selected for each specific class. In this paper, we propose the so-called multi-task feature selection approach for identifying features relevant to each target region towards effective image segmentation. This way of feature selection requires to transform a multi-class classification task into $n$ binary classification tasks, where $n$ is the number of classes. In particular, the Prism algorithm is used to produce a set of rules for class specific feature selection and the K nearest neighbour algorithm is used for training a classifier on a feature subset selected for each class. The experimental results show that the multi-task feature selection approach leads to an significant improvement of classification performance comparing with traditional feature selection approaches.

An effective bearing fault diagnosis model based on machine learning is proposed in this study. The model can separate into three stages: feature extraction, feature selection, and classification. In the stage of feature extraction, multiresolution analysis (MRA) and fast Fourier transform (FFT) are applied to extract the features from the raw signal measured from the rotating machine. In the second stage, a powerful feature selection method is proposed and utilized in the stage of feature selection. The new feature selection method is based on grey wolf optimization (GWO) and heap-based optimizer (HBO) with new strategies combined. Finally, support vector machine (SVM) and linear discriminant analysis (LDA) are used as the classifier independently. To verify the capability of the proposed model, four different datasets are applied to test the model in this study, respectively University of California Irvine (UCI) benchmark dataset, bearing dataset, Case Western Reserve University (CWRU) benchmark dataset, and Machinery Failure Prevention Technology (MFPT) benchmark dataset. The proposed method is compared with the existing methods and can certify the robustness with the experiment results.

In machine learning, Case Based Reasoning is a prominent technique for harvesting knowledge from past experiences. The past experiences are represented in the form of a repository of cases having a set of features. But each feature may not have the equal relevancy in describing a case. Measuring the relevancy of each feature is always a prime issue. A subset of relevant features describes a case with adequate accuracy. An appropriate subset of relevant features should be selected for improving the performance of the system and to reduce dimensionality. In case based domain, feature selection is a process of selecting an appropriate subset of relevant features. There are various real domains which are inherently case based and features are expressed in terms of linguistic variables. To assign a numerical weight to each linguistic feature, a lot of feature subset selection algorithms have been proposed. But the weighting values are usually determined using subjective judgement or a trial and error basis.

Based on benchmarking various methods and analyzing multiple real scRNA-seq datasets, a computational platform/workflow and a set of tips for best practices are developed for analyzing causal interactions or relationships in single cells.

With the rapid development of the Internet of Things (IoT), the curse of dimensionality becomes increasingly common. Feature selection (FS) is to eliminate irrelevant and redundant features in the datasets. Particle swarm optimization (PSO) is an efficient metaheuristic algorithm that has been successfully applied to obtain the optimal feature subset with essential information in an acceptable time. However, it is easy to fall into the local optima when dealing with high-dimensional datasets due to constant parameter values and insufficient population diversity. In the paper, an FS method is proposed by utilizing adaptive PSO with leadership learning (APSOLL). An adaptive updating strategy for parameters is used to replace the constant parameters, and the leadership learning strategy is utilized to provide valid population diversity. Experimental results on 10 UCI datasets show that APSOLL has better exploration and exploitation capabilities through comparison with PSO, grey wolf optimizer (GWO), Harris hawks optimization (HHO), flower pollination algorithm (FPA), salp swarm algorithm (SSA), linear PSO (LPSO), and hybrid PSO and differential evolution (HPSO-DE). Moreover, less than 8% of features in the original datasets are selected on average, and the feature subsets are more effective in most cases compared to those generated by 6 traditional FS methods (analysis of variance (ANOVA), Chi-Squared (CHI2), Pearson, Spearman, Kendall, and Mutual Information (MI)).

Feature selection (FS) is a very important pre-processing technique in machine learning and data mining. It aims to select a small subset of relevant and informative features from the original feature space that may contain many irrelevant, redundant and noisy features. Feature selection usually leads to better performance, interpretability, and lower computational cost. In the literature, FS methods are categorized into three main approaches: Filters, Wrappers, and Embedded. In this paper we introduce a new feature selection method called graph feature selection (GFS). The main steps of GFS are the following: first, we create a weighted graph where each node corresponds to each feature and the weight between two nodes is computed using a matrix of individual and pairwise score of a Decision tree classifier. Second, at each iteration, we split the graph into two random partitions having the same number of nodes, then we keep moving the worst node from one partition to another until the global modularity is converged. Third, from the final best partition, we select the best ranked features according to a new proposed variable importance criterion. The results of GFS are compared to three well-known feature selection algorithms using nine benchmarking datasets. The proposed method shows its ability and effectiveness at identifying the most informative feature subset.

In recent years, detecting credit card fraud transactions has been a difficult task due to the high dimensions and imbalanced datasets. Selecting a subset of important features from a high-dimensional dataset has proven to be the most prominent approach for solving high-dimensional dataset issues, and the selection of features is critical for improving classification performance, such as the fraud transaction identification process. To contribute to the field, this paper proposes a novel feature selection (FS) approach based on a metaheuristic algorithm called Rock Hyrax Swarm Optimization Feature Selection (RHSOFS), inspired by the actions of rock hyrax swarms in nature, and implements supervised machine learning techniques to improve credit card fraud transaction identification approaches. This approach is used to select a subset of optimal relevant features from a high-dimensional dataset. In a comparative efficiency analysis, RHSOFS is compared with Differential Evolutionary Feature Selection (DEFS), Genetic Algorithm Feature Selection (GAFS), Particle Swarm Optimization Feature Selection (PSOFS), and Ant Colony Optimization Feature Selection (ACOFS) in a comparative efficiency analysis. The proposed RHSOFS outperforms existing approaches, such as DEFS, GAFS, PSOFS, and ACOFS, according to the experimental results. Various statistical tests have been used to validate the statistical significance of the proposed model.

Optimization methods are considered as one of the highly developed areas in Artificial Intelligence (AI). The success of the Particle Swarm Optimization (PSO) and Genetic Algorithms (GA) has encouraged researchers to develop other methods that can obtain better performance outcomes and to be more responding to the modern needs. The Grey Wolf Optimization (GWO), and the Krill Herd (KH) are some of those methods that showed a great success in different applications in the last few years. In this paper, we propose a comparative study of using different optimization methods including KH and GWO in order to solve the problem of document feature selection for the classification problem. These methods are used to model the feature selection problem as a typical optimization method. Due to the complexity and the non-linearity of this kind of problems, it becomes necessary to use some advanced techniques to make the judgement of which features subset that is optimal to enhance the performance of classification of text documents. The test results showed the superiority of GWO over the other counterparts using the specified evaluation measures.

Intrusion Detection Systems (IDS) play a vital role in securing Internet of Things (IoT) networks against cyber-attacks. Previous work used a binary classification approach to detect only Denial-of-Service (DoS) attacks. This paper presents an improved multiclass IDS that identifies multiple attack types. The proposed approach uses simulated IoTID20-based traffic datasets to classify Normal, Denial-of-Service, Mirai botnet, Man-in-the-Middle (MITM), and Scan activity attacks. The proposed approach uses a combination of feature selection methods, including correlation-based filtering and a genetic algorithm, followed by machine learning classifiers such as Decision Tree (DT), Random Forest (RF), K-Nearest Neighbors (KNN), and Support Vector Machine (SVM). Experimental results show a significant improvement in classification performance. The RF classifier with Genetic Algorithm (GA)-based feature selection achieved the highest accuracy (96.5%), followed closely by DT and SVM. Therefore, based on our theoretical and experimental comparisons, the proposed approach could be a practical step toward deploying more robust, realistic IDS models in IoT environments.

There are two fundamentally different approaches for feature selection: wrapper and filter. It is also possible to combine them, obtaining hybrid approaches. This paper describes a hybrid method for selecting relevant features in clustering problems. The proposed approach is based on the combination of the widely known k-means algorithm and a Bayesian filter, which is based on the Markov Blanket concept. Since the number of clusters and the subset of relevant features are usually inter-related, we propose a method that iterates between clustering (assuming that the number of clusters is not known a priori) and filtering. Experiments in a number of datasets show that the proposed approach allows selecting features that provide good partitions.

Optimised autoencoder-based ensemble deep learning approaches for cyber-physical event classification utilizing synchrophasor PMU data