A hybrid discriminant embedding with feature selection: application to image categorization

The main goal of discriminant embedding is to extract features that can be compact and informative representations of the original set of features. This paper introduces a hybrid scheme for linear feature extraction for supervised multiclass classification. We introduce a unifying criterion that is able to retain the advantages of robust sparse LDA and Interclass sparsity. Thus, the estimated transformation includes two types of discrimination which are the inter-class sparsity and robust Linear Discriminant Analysis with feature selection. In order to optimize the proposed objective function, we deploy an iterative alternating minimization scheme for estimating the linear transformation and the orthogonal matrix. The introduced scheme is generic in the sense that it can be used for combining and tuning many other linear embedding methods. In the lights of the experiments conducted on six image datasets including faces, objects, and digits, the proposed scheme was able to outperform competing methods in most of the cases.

Feature extraction methods have been successfully applied to many real-world applications. The classical Linear Discriminant Analysis (LDA) and its variants are widely used as feature extraction methods. Although they have been used for different classification tasks, these methods have some shortcomings. The main one is that the projection axes obtained are not informative about the relevance of original features. In this paper, we propose a linear embedding method that merges two interesting properties: Robust LDA and inter-class sparsity. Furthermore, the targeted projection transformation focuses on the most discriminant original features, The proposed method is called Robust Discriminant Analysis with Feature Selection and Inter-class Sparsity (RDA FSIS). Two kinds of sparsity are explicitly included in the proposed model. The first kind is obtained by imposing the `2;1 constraint on the projection matrix in order to perform feature ranking. The second kind is obtained by imposing the inter-class sparsity constraint used for getting a common sparsity structure in each class. Comprehensive experiments on five real-world image datasets demonstrate the effectiveness and advantages of our framework over existing linear methods.

A modified genetic algorithm and weighted principal component analysis based feature selection and extraction strategy in agriculture

Feature selection is referred to the process of obtaining a subset from an original feature set according to certain feature selection criterion, which selects the relevant features of the dataset. It plays a role in compressing the data processing scale, where the redundant and irrelevant features are removed. Feature selection techniques show that more information is not always good in machine learning applications. Apply different algorithms for the data at hand and with baseline classification performance values we can select a final feature selection algorithm. In this paper, we propose a hybrid classification model, which has correlation based filter feature selection algorithm and Machine learning as classifiers. The objective of this study is to select relevant features and analyze the outperform machine learning algorithms in order to train our model, predict and compare their classification performance. In this method, features are ordered according to their Absolute correlation value with respect to the class attribute. Then top K Features are selected from ordered list of features to form a reduced dataset. This proposed classifier model is applied to our smart meter datasets. To measure the performance of these selected features; seven benchmark classifier are used; Random Forest (RF), Logistic Regression (LR), k-Nearest Neighbor (kNN), Naive Bayes (NB), Decision Tree (DT), Linear Discriminant Analysis (LDA) and Support Vector Machine (SVM). This paper then analyzes the performance of all classifiers with feature selection in term of accuracy, sensitivity, F-Measure, Specificity, Precision, and MCC. From our experiment, we found that Random Forest classifier performed higher than other used classifiers.

Feature engineering is treated as a crucial step in any intelligent system pipeline and practitioners often spend 70–80% of their time in this phase before modeling. Especially, in machine learning (ML), the performance of the classification systems deteriorates as irrelevant features are added, even when the features presented contain enough information about the problem. ML researchers often use feature selection and feature extraction techniques individually to filter out the redundant information from the data, but would not get desired results or performance due to choosing a single method only. In this article, we take an attempt to combine both feature selection and feature extraction methods that give an improvement in the prediction performance than using a single feature selection or feature extraction method. In the application of the proposed method, we consider the Indian liver disease patient dataset to classify liver disease using machine learning classifiers. In addition, ensemble classifiers also take into account reproducing and robustifying classification accuracy. The performance is evaluated and compared with different metrics such as accuracy, precision, recall, and F1-score. Using the proposed method, the KNN classifier can diagnosis 76.03% with a precision of 76.69%, recall 96.23%, and 85.36% F1-score which is much better than the individual feature selection and feature extraction approach. The proposed method can also be extended to medical datasets other than liver disease so that these types of deadly diseases can be correctly diagnosed for the betterment of humanity.

Traditional speech recognition methods based on static features of a word to justify whether the word is correctly recognized or not, which neglected the information carried by its contexts and the surrounding environment.In this paper,a 14.1% word error rate (WER) speech recognizer (SR) is used as the baseline system,and 10-dimension static features achived 24.9% decline of Classification Error Rate (CER). Context features and dynamic features are extracted in relation to the static features.The total 42-dimension features get an better CER of 7.4% than static features.But not all these features have a positive impact on the classification.Too many features not only take redundant information,but also make the classification process time-consuming.To solve this problem,feature extraction which can extract prime information from original features and feature selection method which can select effective features from the original feature set are proposed in this paper.The experimental results show that context features and dynamic features are effective features for classification,and the features can be considerably compressed through feature extraction and feature selection.

Feature selection is a satisfactory optimization problem. Most feature selection methods did not consider the cost of feature extraction and the automatic decision of feature subset dimension. So a novel approach called satisfactory feature selection method (SFSM) was proposed. SFSM integrated feature extraction with feature selection and considered classification performance of feature samples, the dimension of feature subset and the complexity of feature extraction simultaneously. Experimental results show that SFSM selects more satisfying feature subset than sequential forward selection using distance criterion (SFSDC) and the method presented by Tiejun Lu (GADC). Also, SFSM achieves higher accurate recognition rate than original feature set, SFSDC and GADC, which verifies the validity of the proposed method.

With the continuous development of Internet technology, data gradually present a complicated and high-dimensional trend. These high-dimensional data have a large number of redundant features and irrelevant features, which bring great challenges to the existing machine learning algorithms. Feature selection is one of the important research topics in the fields of machine learning, pattern recognition and data mining, and it is also an important means in the data preprocessing stage. Feature selection is to look for the optimal feature subset from the original feature set, which would improve the classification accuracy and reduce the machine learning time. The traditional feature selection algorithm tends to ignore the kind of feature which has a weak distinguishing capacity as a monomer, whereas the feature group’s distinguishing capacity is strong. Therefore, a new dynamic interaction feature selection (DIFS) algorithm is proposed in this paper. Initially, under the theoretical framework of interactive information, it redefines the relevance, irrelevance and redundancy of the features. Secondly, it offers the computational formulas for calculating interactive information. Finally, under the eleven data sets of UCI and three different classifiers, namely, KNN, SVM and C4.5, the DIFS algorithm increases the classification accuracy of the FullSet by 3.2848% and averagely decreases the number of features selected by 15.137. Hence, the DIFS algorithm can not only identify the relevance feature effectively, but also identify the irrelevant and redundant features. Moreover, it can effectively improve the classification accuracy of the data sets and reduce the feature dimensions of the data sets.

Feature selection and construction are important pre-processing techniques in machine learning and data mining. They may allow not only dimensionality reduction but also classifier accuracy and efficiency improvement. Feature selection aims at selecting relevant features from the original feature set, which could be less informative to achieve good performance. Feature construction may work well as it creates new highlevel features, but these features do not have the same degree of importance, which makes the use of weighted-features construction a very challenging topic. In this paper, we propose a bi-level evolutionary approach for efficient feature selection and simultaneous feature construction and feature weighting, called Bi-level Weighted-Features Construction (BWFC). The basic idea of our BWFC is to exploit the bi-level model for performing feature selection and weighted-features construction with the aim of finding an optimal subset of features combinations. Our approach has been assessed on six high-dimensional datasets and compared against three existing approaches, using three different classifiers for accuracy evaluation. Experimental results show that our proposed algorithm gives competitive and better results with respect to the state-of-the-art algorithms.

In many fields such as data mining, machine learning, pattern recognition and signal processing, data sets containing huge number of features are often involved. Feature selection is an essential data preprocessing technique for such high-dimensional data classification tasks. Traditional dimensionality reduction approach falls into two categories: Feature Extraction (FE) and Feature Selection (FS). Principal component analysis is an unsupervised linear FE method for projecting high-dimensional data into a low-dimensional space with minimum loss of information. It discovers the directions of maximal variances in the data. The Rough set approach to feature selection is used to discover the data dependencies and reduction in the number of attributes contained in a data set using the data alone, requiring no additional information. For selecting discriminative features from principal components, the Rough set theory can be applied jointly with PCA, which guarantees that the selected principal components will be the most adequate for classification. We call this method Rough PCA. The proposed method is successfully applied for choosing the principal features and then applying the Upper and Lower Approximations to find the reduced set of features from a gene expressiondata.

An effective bearing fault diagnosis model based on machine learning is proposed in this study. The model can separate into three stages: feature extraction, feature selection, and classification. In the stage of feature extraction, multiresolution analysis (MRA) and fast Fourier transform (FFT) are applied to extract the features from the raw signal measured from the rotating machine. In the second stage, a powerful feature selection method is proposed and utilized in the stage of feature selection. The new feature selection method is based on grey wolf optimization (GWO) and heap-based optimizer (HBO) with new strategies combined. Finally, support vector machine (SVM) and linear discriminant analysis (LDA) are used as the classifier independently. To verify the capability of the proposed model, four different datasets are applied to test the model in this study, respectively University of California Irvine (UCI) benchmark dataset, bearing dataset, Case Western Reserve University (CWRU) benchmark dataset, and Machinery Failure Prevention Technology (MFPT) benchmark dataset. The proposed method is compared with the existing methods and can certify the robustness with the experiment results.

The electromyography (EMG) signals play a very important role in the muscular activity of the human body. Muscular activities can be measured and analyzed through various movements such as writing, walking, running, hand movements. This paper is focused on hand movements of the amputee people (people with prosthetic hands). Classification of prosthetic hand movements is a research problem that can be solved by classifying the EMG signal generated during movements. The EMG signal has many redundant and irrelevant features that need to be removed. Feature selection (FS) is a technique that is used to select the important features. Therefore, this paper proposes beta artificial bee colony (BetaABC) and binary BetaABC (BBABC) to select prominent features in EMG pattern recognition to increase classification performance and to reduce classifier complexity. BABC is the enhanced version of the ABC inspired from Gaussian ABC (GABC). Initially, discrete wavelet transform (DWT) was applied to decompose and extract the features from EMG signal. After that, BBABC is applied to select the optimal features from the original feature set. The NinaPro dataset (DB3) is used for the experiment and is divided into two sections: (i) Comparative results of proposed BetaABC with ABC and GABC on 10 benchmark functions; (ii) comparative results of proposed BBABC-based feature selection with binary ABC (BABC) and binary GABC (BGABC) on 11 amputee subjects. The obtained results prove that the proposed method is superior to ABC and GABC.

Feature selection plays a central role in data analysis and is also a crucial step in machine learning, data mining and pattern recognition. Feature selection algorithm focuses mainly on the design of a criterion function and the selection of a search strategy. In this paper, a novel feature selection approach (NFSA) based on quantum genetic algorithm (QGA) and a good evaluation criterion is proposed to select the optimal feature subset from a large number of features extracted from radar emitter signals (RESs). The criterion function is given firstly. Then, detailed algorithm of QGA is described and its performances are analyzed. Finally, the best feature subset is selected from the original feature set (OFS) composed of 16 features of RESs. Experimental results show that the proposed approach reduces greatly the dimensions of OFS and heightens accurate recognition rate of RESs, which indicates that NFSA is feasible and effective.

Stroke is a disease that occurs in the brain and can cause both vocal and global brain dysfunction. Stroke research mainly aims to predict risk and mortality. Machine learning can be used to diagnose and predict diseases in the healthcare field, especially in stroke prediction. However, collecting medical record data to predict a disease usually makes much noise because not all variables are important and relevant to the prediction process. In this case, dimensionality reduction is essential to remove noisy (i.e., irrelevant) and redundant features. This study aims to predict stroke using Recursive Feature Elimination as feature selection, Principal Component Analysis as feature extraction, and a combination of Recursive Feature Elimination and Principal Component Analysis. The dataset used in this research is stroke prediction from Kaggle. The research methodology consists of pre-processing, SMOTE, 10-fold Cross-Validation, feature selection, feature extraction, and machine learning, which includes SVM, Random Forest, Naive Bayes, and Linear Discriminant Analysis. From the results obtained, the SVM and Random Forest get the highest accuracy value of 0.8775 and 0.9511 without using PCA and RFE, Naive Bayes gets the highest value of 0.7685 when going through PCA with selection of 20 features followed by RFE feature selection with selection of 5 features, and LDA gets the highest accuracy with 20 features from feature selection and continued feature extraction with a value of 0. 7963. It can be concluded in this study that SVM and Random Forest get the highest accuracy value without PCA and RFE techniques, while Naive Bayes and LDA show better performance using a combination of PCA and RFE techniques. The implication of this research is to know the effect of RFE and PCA on machine learning to improve stroke prediction.

Fault diagnosis methods based on machine learning have made great progress for rotating machinery. The main steps of the machine learning process involve feature extraction, selection, and classification. Feature selection improves classification accuracy and reduces diagnosis time by selecting the better features. Due to the difficulty of traditional feature selection methods to rank the feature importance of each class, the best subset of features could hardly be obtained. Therefore, this article proposes a new feature selection method to address the shortcomings of the above traditional methods, called Feature Ranking based on Optimal Class Distance Ratio (FROCDR), which can choose the optimal features between every two classes of samples to obtain feature ranking that is conducive to classification. In order to comprehensively extract the fault information in the signal, the multiscale analysis and the variational mode decomposition (VMD) method are applied to process the vibration signals under different scales and frequency bands, and the processed signals are visualized by symmetrized dot pattern (SDP). In addition, features are extracted from the obtained SDP images, and the proposed FROCDR method is used to select the best subset of features. The final diagnosis task is accomplished by a random forest (RF) classifier. Experimental cases of bearing and gear data show that the proposed method has higher diagnostic accuracy and stability.