A simple and efficient filter feature selection method via document-term matrix unitization

The feature selection (FS) plays a important role in identification of the significant genes in bioinformatics and related fields. Additionally, it is frequently necessary step to avoid over-fitting and to reduce complexity and computational time. Wang et al [1] proposed new two stage feature selection method achieving excellent classification performance while selecting only few relevant genes. We present new feature selection methods, based on the idea of the Wang's paper, and analyze how the particular filter FS method, used in first stage, influence overall performance. The performance is analyzed by means of the FS stability and influence on the prediction performance. Our results indicate that the stability of FS is significantly affected by the type of FS used in the first stage, but the prediction performance is not so sensitive to the choice of FS in the first stage.

<p>The feature selection method enhances machine learning performance by enhancing learning precision. Determining the optimal feature selection method for a given machine learning task involving big-dimension data is crucial. Therefore, the purpose of this study is to make a comparison of feature selection methods highlighting several filters (information gain, chi-square, ReliefF) and embedded (Lasso, Ridge) hybrid with logistic regression (LR). A sample size of n=100, 75 is chosen randomly, and the reduction features d=50, 22, and 10 are applied. The procedure for feature reduction makes use of the entire sample sizes. Each sample size's results are compared, including tests with no feature selection process. The results indicate that LR+ReliefF is the best method for mammary cancer data, whereas LR+IG is the best for prostatic cancer data, making the filter more suitable than embedded for big-dimension data. This study revealed that the sample's features and size influence the most effective method for selecting features from big-dimension data. Therefore, it provides insight into the most effective methods for particular features and sample sizes in high-dimensional data.</p>

Target spectrum based feature selection (TSFS): A new method based on chain coding for target detection problems

On the benefit of feature selection and ensemble feature selection for fuzzy k-nearest neighbor classification

Background: Feature selection techniques are important factors for improving machine learning models because they increase prediction accuracy and decrease the time to create a model. Recently, feature selection techniques have been employed on software quality prediction problems with different results and no clear indication of which techniques are frequently used.Objective: This study aims to conduct a systematic review of the application of feature selection techniques in software quality prediction and answers eight research questions.Method: The review evaluates 15 papers in 9 journals and 6 conference proceedings from 2007 to 2017 using the standard systematic literature review method.Results: The results obtained from this study reveal that the filter feature selection method was the most commonly used in the studies (60%) and RELIEF was the most employed among this method, and a limited number of studies employed an ensemble method. Several studies used public datasets available in the PROMISE software project repository (60%). Most studies focused on software defect prediction (classification problem) using area under curve (AUC) as a primary evaluation measure, whereas only two studies focused on software maintainability prediction (regression problem) using mean magnitude of relative error (MMRE) as a primary evaluation measure. All selected studies performed k-fold cross-validation to evaluate model accuracy. Individual prediction models were mostly employed and ensemble models appeared only in three studies. Naive Bayes was the most investigated among individual models, whereas Random forest was the most investigated among ensemble models.Conclusion: Feature selection techniques used by selected primary studies have a positive impact on the performance of the prediction models. Further, both ensemble feature selection method and ensemble models have the ability for increasing prediction accuracy over single methods or individual models and have reported improvement in the prediction accuracy; however, the application of these techniques in software quality prediction is still limited.

An efficient wavelet-based feature selection (FS) method is proposed in this paper for subject recognition using ground reaction force measurements. Our approach relies on a local fuzzy evaluation measure with respect to patterns that reveal the adequacy of data coverage for each feature. Furthermore, FS is driven by a fuzzy complementary criterion (FuzCoC) which assures that those features are iteratively introduced, providing the maximum additional contribution with regard to the information content given by the previously selected features. On the basis of the principles of FuzCoC, we develop two novel techniques. At Stage 1, wavelet packet (WP) decomposition of gaits is accomplished to obtain a set of discriminating frequency sub-bands. A computationally simple FS method is then applied at Stage 2, providing a compact set of powerful and complementary features, from WP coefficients. The quality of our approach is validated via comparative analysis against existing methods on gait recognition.

In this paper, PCA based modeling of geometric structure of the face for automatic face detection is presented. The method improves the face detection rate and limits the search space. Skin Color Modeling (SCM) is one of the best face detection techniques for image and video. However, feature selection is very important for even better template matching performance in terms of detection rate and time. This paper presents an efficient feature extraction and selection method based on geometric structure of the facial image boundary and interior. To model the geometric structure of face, Principle Component Analysis (PCA) and canny edge detection are used. Fusion of PCA based geometric modeling and SCM method provides higher face detection accuracy and improves time complexity. Both models provide filtering of image in term of pixel values to get the face location that are very fast and efficient for large image databases.

Network traffic in cloud computing is characterized by a large volume of data, with potential high dimensionality and high levels of redundancy. “Big data” not only retard the execution process of intrusion detection systems (IDS) but they can also lead to unsatisfactory classification accuracy. The efficient correlation-based feature selection (ECOFS) approach proposed in this paper, can handle linearly and nonlinearly dependent data and it can eliminate redundant and irrelevant features. Its effectiveness has been evaluated through its employment in an intrusion detection system. A Libsvm-IDS has been built to operate using the features selected by the proposed ECOFS algorithm. The performance of the hybrid Libsvm-IDS + ECOFS approach has been evaluated using two well-known intrusion detection evaluation datasets, namely the KDD Cup 99 and the NSL-KDD. The evaluation results show that our ECOFS algorithm selects the smallest number of features, resulting in the lowest computational cost for the Libsvm-IDS, without any performance compromise. In fact, our algorithm has achieved higher accuracy compared with two well established methods.

Subspace distance is an invaluable tool exploited in a wide range of feature selection methods. The power of subspace distance is that it can identify a representative subspace, including a group of features that can efficiently approximate the space of original features. On the other hand, employing intrinsic statistical information of data can play a significant role in a feature selection process. Nevertheless, most of the existing feature selection methods founded on the subspace distance are limited in properly fulfilling this objective. To pursue this void, we propose a framework that takes a subspace distance into account which is called “Variance–Covariance subspace distance”. The approach gains advantages from the correlation of information included in the features of data, thus determines all the feature subsets whose corresponding Variance–Covariance matrix has the minimum norm property. Consequently, a novel, yet efficient unsupervised feature selection framework is introduced based on the Variance–Covariance distance to handle both the dimensionality reduction and subspace learning tasks. The proposed framework has the ability to exclude those features that have the least variance from the original feature set. Moreover, an efficient update algorithm is provided along with its associated convergence analysis to solve the optimization side of the proposed approach. An extensive number of experiments on nine benchmark datasets are also conducted to assess the performance of our method from which the results demonstrate its superiority over a variety of state-of-the-art unsupervised feature selection methods. The source code is available at https://github.com/SaeedKarami/VCSDFS.

Stock market prediction is a very noisy problem and the use of any additional information to increase accuracy is necessary. In this paper, for the stock daily return prediction problem, the set of features is expanded to include indicators not only for the stock to be predicted itself but also a set of other stocks and currencies. Afterwards, different feature selection and classification methods are utilized for prediction. The daily close returns of the 3 most traded stocks (GARAN, THYAO, and ISCTR) in Borsa Istanbul (BIST) are predicted using indicators computed on those stocks, indicators for all the other stocks listed in the BIST100 index, and indicators on the dollar-gold prices. Twenty-five different indicators on daily stock prices are computed to form feature vectors for each trading day. These feature vectors are assigned class labels according to the daily close returns. Expanding the feature space with BIST100 stocks features results in a high dimensional feature space, with possibly noisy or irrelevant features. Therefore, feature selection methods are utilized to select the most informative features. In order to determine relevance scores of features, fast filter-based methods, gain ratio and relief, are used. Experiments are performed based on individual stock features, dollar-gold features (DG), BIST100 stock features (BIST100), and a combination of BIST100 and DG with and without feature selection. Using the gain ratio feature selection with a gradient boosting machine (GBM), the movements of GARAN stock were predicted with an accuracy of 0.599 and an F-measure of 0.614. For THYAO, the relief feature selection with the GBM gave an accuracy of 0.558, and for ISCTR, the gain ratio feature selection with logistic regression achieved an accuracy of 0.581. It was found that using BIST100 stock features boosts classification results for all stocks in terms of accuracy.

Designing a brain computer interface (BCI) system one can choose from a variety of features that may be useful for classifying brain activity during a mental task. For the special case of classifying electroencephalogram (EEG) signals we propose the usage of the state of the art feature selection algorithms Recursive Feature Elimination and Zero-Norm Optimization which are based on the training of support vector machines (SVM). These algorithms can provide more accurate solutions than standard filter methods for feature selection. We adapt the methods for the purpose of selecting EEG channels. For a motor imagery paradigm we show that the number of used channels can be reduced significantly without increasing the classification error. The resulting best channels agree well with the expected underlying cortical activity patterns during the mental tasks. Furthermore we show how time dependent task specific information can be visualized.

The analysis of medical data is not an easy task for health care systems since a comprehensive medical examination is performed with hundreds of parameters. These data come with considerable amount of irrelevant and redundant features. But only a subset of these features are useful for prediction, feature selection plays a significant part in to identify critical features for building prediction models. However, a number of existing filter feature selection methods can cope with redundancy and irrelevance by using single correlation measure, and ignore the interaction between features as well. In this paper, the idea of using multiple correlation measures is adopted for filter feature selection, additionally, the used method is also take feature interaction into account. Two correlation measures that converted to the standard form of 0–1 are fused together in the used algorithm, and the complementary correlation between candidate feature and selected feature can be distinguished by introducing an item. According to four common classifiers and a medical examination dataset, the experiments can be developed in order to demonstrate the validity of the method. Experimental results can confirm that the method is superior to four representative filter feature selection methods.

With the rapid development of artificial intelligence in recent years, the research on image processing, text mining, and genome informatics has gradually deepened, and the mining of large-scale databases has begun to receive more and more attention. The objects of data mining have also become more complex, and the data dimensions of mining objects have become higher and higher. Compared with the ultra-high data dimensions, the number of samples available for analysis is too small, resulting in the production of high-dimensional small sample data. High-dimensional small sample data will bring serious dimensional disasters to the mining process. Through feature selection, redundancy and noise features in high-dimensional small sample data can be effectively eliminated, avoiding dimensional disasters and improving the actual efficiency of mining algorithms. However, the existing feature selection methods emphasize the classification or clustering performance of the feature selection results and ignore the stability of the feature selection results, which will lead to unstable feature selection results, and it is difficult to obtain real and understandable features. Based on the traditional feature selection method, this paper proposes an ensemble feature selection method, Random Bits Forest Recursive Clustering Eliminate (RBF-RCE) feature selection method, combined with multiple sets of basic classifiers to carry out parallel learning and screen out the best feature classification results, optimizes the classification performance of traditional feature selection methods, and can also improve the stability of feature selection. Then, this paper analyzes the reasons for the instability of feature selection and introduces a feature selection stability measurement method, the Intersection Measurement (IM), to evaluate whether the feature selection process is stable. The effectiveness of the proposed method is verified by experiments on several groups of high-dimensional small sample data sets.

Intrusion detection is a significant technique for detecting various attacks in Wireless Mesh Networks (WMNs). Intrusion Detection Systems (IDS) powered by machine learning algorithms play a vital role in enhancing security by actively monitoring data traffic in real-time. The performance of attack detection in IDS is reduced by the existence of noisy and unwanted features in the network data. Hence, extracting Informative Features (IF) is essential for enhancing the effectiveness of the IDS. The proposed approach optimally selects IF from the network dataset, which facilitates in accurately detecting intrusions. Existing methods lack in efficient feature selection, leading to higher computational complexity and lower detection performance. This research aims to enhance IDS accuracy by leveraging the eXtreme Gradient Boosting (XGBoost) algorithm to classify the intrusions in the mesh network of smart grid and the optimal feature selection carried out with the help of Golden Eagle Optimization Algorithm (GEOA). This novel hybrid IDS model enhances the attack detection in WMNs. The proposed IDS’s performance is improved by using a feature selection method based on the GEOA, and then obtained results are compared against evolutionary algorithms-based features selection method. The proposed model of IDS using XGBoost algorithm is tested on a smart grid dataset, and its performance is compared with existing literature works.

New feature selection methods based on opposition-based learning and self-adaptive cohort intelligence for predicting patient no-shows