Unsupervised Optimal Discriminant Vector Based Feature Selection Method

Algebraic feature extraction for image recognition based on an optimal discriminant criterion

A new feature selection method based on a validity index of feature subset

A novel algebraic feature extraction method for image recognition is presented. For the training image samples, a set of optimal discriminant projection vectors are calculated according to a generalized Fisher criterion function. On the basis of the optimal discriminant projection vector, the algebraic feature vectors of an image can be extracted by projecting the image onto all optimal discriminant projection vectors. Experimental results shows that the algebraic features extracted by the presented method have good recognition performance.© (1993) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.

Fault diagnosis methods based on machine learning have made great progress for rotating machinery. The main steps of the machine learning process involve feature extraction, selection, and classification. Feature selection improves classification accuracy and reduces diagnosis time by selecting the better features. Due to the difficulty of traditional feature selection methods to rank the feature importance of each class, the best subset of features could hardly be obtained. Therefore, this article proposes a new feature selection method to address the shortcomings of the above traditional methods, called Feature Ranking based on Optimal Class Distance Ratio (FROCDR), which can choose the optimal features between every two classes of samples to obtain feature ranking that is conducive to classification. In order to comprehensively extract the fault information in the signal, the multiscale analysis and the variational mode decomposition (VMD) method are applied to process the vibration signals under different scales and frequency bands, and the processed signals are visualized by symmetrized dot pattern (SDP). In addition, features are extracted from the obtained SDP images, and the proposed FROCDR method is used to select the best subset of features. The final diagnosis task is accomplished by a random forest (RF) classifier. Experimental cases of bearing and gear data show that the proposed method has higher diagnostic accuracy and stability.

A simple and efficient filter feature selection method via document-term matrix unitization

Subspace distance is an invaluable tool exploited in a wide range of feature selection methods. The power of subspace distance is that it can identify a representative subspace, including a group of features that can efficiently approximate the space of original features. On the other hand, employing intrinsic statistical information of data can play a significant role in a feature selection process. Nevertheless, most of the existing feature selection methods founded on the subspace distance are limited in properly fulfilling this objective. To pursue this void, we propose a framework that takes a subspace distance into account which is called “Variance–Covariance subspace distance”. The approach gains advantages from the correlation of information included in the features of data, thus determines all the feature subsets whose corresponding Variance–Covariance matrix has the minimum norm property. Consequently, a novel, yet efficient unsupervised feature selection framework is introduced based on the Variance–Covariance distance to handle both the dimensionality reduction and subspace learning tasks. The proposed framework has the ability to exclude those features that have the least variance from the original feature set. Moreover, an efficient update algorithm is provided along with its associated convergence analysis to solve the optimization side of the proposed approach. An extensive number of experiments on nine benchmark datasets are also conducted to assess the performance of our method from which the results demonstrate its superiority over a variety of state-of-the-art unsupervised feature selection methods. The source code is available at https://github.com/SaeedKarami/VCSDFS.

An unsupervised feature selection algorithm based on ant colony optimization

Conventional model predictive current control (MPCC) applies only one vector during one control period, which produces large torque and flux ripples and high current harmonics in permanent magnet synchronous motor (PMSM) drives. Recently MPCC with duty cycle control has been proposed to improve the steady state performance by applying one non-zero vector and one null vector during one control period. However, the prior method requires lots of calculations and predictions to find the optimal voltage vectors and calculate their respective duration. Different from prior enumeration-based MPCC, this paper proposes an efficient two-vector MPCC by applying two arbitrary voltage vectors during one control period. The reference voltage vector is firstly calculated based on the principle of deadbeat current control. Two optimal vectors and their duration are then obtained in a very efficient way, which does not require the calculation of current slopes in prior MPCC methods. The proposed method is compared to the state-of-the-art predictive control methods, including conventional MPCC, MPCC with duty cycle control, deadbeat control with space vector modulation (SVM) and modulated model predictive control (M2PC). Both simulation and experimental results prove that the proposed method achieves better steady state performance than conventional MPCC with or without duty cycle and the dynamic response is not degraded. Under the condition of insufficient dc bus voltage, the proposed method outperforms deadbeat control and M2PC by presenting even higher speed range and less torque ripples.

Hyperspectral images (HSIs) generally contain a large amount of spectral bands (features), and the redundant information in them will cause the Hughes phenomenon in the classification process. And feature extraction and feature selection are the two main existing methods to effectively reduce the redundancy of spectral information in the field of HSIs classification. Compared with feature extraction methods, feature selection methods can preserve most of the features of the original HSIs data without losing their valuable details. However, most existing feature selection methods based on single scene (domain) perform poorly in some scenes (domains) with insufficient labeled samples. Therefore, how to adopt an efficient feature selection method to select the optimal feature subsets of source scene and target scene and use the sample information of source scene to assist in the classification of target scene so as to improve the classification accuracy of images in the target scene as much as possible is still very challenging. In order to solve the above problem, this paper proposes a new cross-scene algorithm: Improved Ant Colony Optimization Algorithm-Based Cross-Scene Feature Selection Algorithm (IMACO-CSFS). In order to obtain more accurate feature subsets of the two scenes, IMACO-CSFS proposes a priority sorting-based ant colony strategy to make the subsequent search process focus on the global optimal solution (optimal feature subset) found in the previous iteration. In addition, in order to further accelerate the convergence speed of the global optimal solution, an ant colony strategy based on elite ants is proposed in IMACO-CSFS to more efficiently obtain the optimal feature subsets of the two scenes for training the classifier. Furthermore, this paper simultaneously considers overall classification accuracies of the optimal feature subsets for both scenes and dynamically adjusts their scale to ensure the consistency of the selected features between the two scenes, attenuating the effect of spectral shift and achieving the higher image classification accuracy in the target scene. Experimental results on three cross-scene HSI data pairs demonstrate that IMACO-CSFS is superior in cross-scene feature selection and cross-scene HSIs classification.

Computer-aided diagnosis has the potential of increasing diagnostic accuracy by providing a second reading to radiologists. In many computerized schemes, numerous features can be extracted to describe suspect image regions. A subset of these features is then employed in a data classifier to determine whether the suspect region is abnormal or normal. Different subsets of features will, in general, result in different classification performances. A feature selection method is often used to determine an "optimal" subset of features to use with a particular classifier. A classifier performance measure (such as the area under the receiver operating characteristic curve) must be incorporated into this feature selection process. With limited datasets, however, there is a distribution in the classifier performance measure for a given classifier and subset of features. In this paper, we investigate the variation in the selected subset of "optimal" features as compared with the true optimal subset of features caused by this distribution of classifier performance. We consider examples in which the probability that the optimal subset of features is selected can be analytically computed. We show the dependence of this probability on the dataset sample size, the total number of features from which to select, the number of features selected, and the performance of the true optimal subset. Once a subset of features has been selected, the parameters of the data classifier must be determined. We show that, with limited datasets and/or a large number of features from which to choose, bias is introduced if the classifier parameters are determined using the same data that were employed to select the "optimal" subset of features.

An efficient wavelet-based feature selection (FS) method is proposed in this paper for subject recognition using ground reaction force measurements. Our approach relies on a local fuzzy evaluation measure with respect to patterns that reveal the adequacy of data coverage for each feature. Furthermore, FS is driven by a fuzzy complementary criterion (FuzCoC) which assures that those features are iteratively introduced, providing the maximum additional contribution with regard to the information content given by the previously selected features. On the basis of the principles of FuzCoC, we develop two novel techniques. At Stage 1, wavelet packet (WP) decomposition of gaits is accomplished to obtain a set of discriminating frequency sub-bands. A computationally simple FS method is then applied at Stage 2, providing a compact set of powerful and complementary features, from WP coefficients. The quality of our approach is validated via comparative analysis against existing methods on gait recognition.

Multi-filter semi-supervised transformer model for fault diagnosis

In network security, intrusion detection plays an important role. Feature subsets obtained by different feature selection methods will lead to different accuracy of intrusion detection. Using individual feature selection method can be unstable in different intrusion detection scenarios. In this paper, the idea of ensemble is applied to feature selection to adjust feature subsets. Feature selection is converted into a two-category problem, and odd number of feature selection methods is used for voting method to decide whether a feature is required or discarded. In actual operation, mean decrease impurity, random forest classifier, stability selection, recursive feature elimination and chi-square test are used. Feature subsets obtained from them will be adjusted by our proposed method to get ensemble feature subsets. To test the performance, support vector machine, decision tree, knn and multi-layer perception are used to observe and compare the classification accuracy with ensemble feature subsets. Three intrusion detection data sets, including kddcup99, cidds-001 and unsw_nb15 are used in our experiments. The best result is reflected on cidds-001 with a 99.40% classification accuracy. The investigation shows that our method has a certain improvement on classification accuracy of intrusion detection.

In many real-world unsupervised learning applications, given data with balanced distribution, that is, there are an approximately equal number of instances in each class, we often need to construct a model to reveal such balance. However, in many data, especially the high-dimensional ones, the data in the original feature space often do not present such balance due to the redundant and noisy features. To tackle this problem, we apply an unsupervised spectral feature selection method to select some informative features, which can better reveal the balanced structure of data. Although spectral feature selection is one of the most popular unsupervised feature selection methods and has been widely studied, none of the existing spectral feature selection methods consider the balance property of data. To address this issue, in this article, we propose a novel balanced spectral feature selection (BSFS) method, which not only selects the discriminative features but also picks those to reveal the balanced structure of data. To the best of our knowledge, this is the first spectral feature selection method considering balance structure of data. By introducing a balanced regularization term, we integrate the balanced spectral clustering and feature selection into a unified framework seamlessly. At last, the experiments on benchmark datasets show that the proposed one outperforms the conventional feature selection methods in both clustering performance and balance, which demonstrates the effectiveness and efficiency of the proposed method.

Feature selection has received attention recently in the field of object detection. A vehicle detection method using feature selection is presented in this work. An efficient feature subset is selected using feature selection methods and each feature subset is evaluated by computing the average error rate in different classification methods. The feature selection methods used in this work are the logistic regression, least absolute shrinkage and selection operator (LASSO) and the random forest (RF) methods. The proposed method is evaluated using actual data, showing good performance.