Unsupervised Feature Selection for Multi-cluster Data via Smooth Distributed Score

In many data analysis tasks, one is often confronted with very high dimensional data. Feature selection techniques are designed to find the relevant feature subset of the original features which can facilitate clustering, classification and retrieval. In this paper, we consider the feature selection problem in unsupervised learning scenario, which is particularly difficult due to the absence of class labels that would guide the search for relevant information. The feature selection problem is essentially a combinatorial optimization problem which is computationally expensive. Traditional unsupervised feature selection methods address this issue by selecting the top ranked features based on certain scores computed independently for each feature. These approaches neglect the possible correlation between different features and thus can not produce an optimal feature subset. Inspired from the recent developments on manifold learning and L1-regularized models for subset selection, we propose in this paper a new approach, called Multi-Cluster Feature Selection (MCFS), for unsupervised feature selection. Specifically, we select those features such that the multi-cluster structure of the data can be best preserved. The corresponding optimization problem can be efficiently solved since it only involves a sparse eigen-problem and a L1-regularized least squares problem. Extensive experimental results over various real-life data sets have demonstrated the superiority of the proposed algorithm.

Text clustering involves data that are of very high dimension. Feature selection techniques find subsets of relevant features from the original feature space that help in efficient and effective clustering. Selection of relevant features merely on ranking scores without considering correlation interferes with the clustering performance. An efficient feature selection technique should be capable of preserving the multi-cluster structure of the data. The purpose of the present work is to demonstrate that feature selection techniques which take into consideration the correlation among features in multi-cluster scenario show better clustering results than those techniques that simply rank features independent of each other. This paper compares two feature selection techniques in this regard viz. the traditional Tf-Idf and the Multi-Cluster Feature Selection (MCFS) technique. The experimental results over the TDT2 and Reuters-21,578 datasets show the superior clustering results of MCFS over traditional Tf-Idf.

Credit as a part of our consumptive life has helped a lot of people. As a financial product, it is used widely along with the growth of economic and financial services. Therefore, credit is very risky so that motivating the financial institution to use a system called credit scoring to make a decision about acceptance. However, the conventional credit scores is calculated from the user’s financial history in financial institution. This method causes people without any financial history or account, such as students, being unnoticed by the system, then their credit proposal is declined. This phenomenon insists the researchers thinking about a new credit scoring system that facilitates people from different economic background. Knowing that people nowadays will spend any money and time on their smartphone, we make a hypothesis about how smartphone usage behavior can be the answer. Then, the survey is conducted to 90 respondents from low to high economic background to model their credit limit. This paper shows that smartphone usage has some insights that can be computed through Multi-Cluster Feature Selection (MCFS). The selected features are Brand of phone, Frequency of changing the phone, Occupation, Data usage for game, Cost for phone, Data usage for social media, Reason of changing the phone, Money for data, Protect personal interest, Age, Spend last money, and Pay for you.

The vibration signals collected by the sensor often have non-stationary and non-linear characteristics owing to the complexity of working environment of rolling bearing, so it is difficult to obtain useful and stable vibration information for diagnosis. Empirical Wavelet Transform (EWT) can effectively decompose non-stationary and nonlinear signals, but it is not suitable for signal analysis of bearing with a complicated spectrum. In this paper, an improved EWT (IEWT) method is proposed by developing a new segmentation approach. Meanwhile, the IEWT is compared with empirical mode decomposition (EMD) and EWT to verify the superiority of IEWT in decomposition accuracy. By combining with the refined composite multiscale dispersion entropy (RCMDE), which is a powerful nonlinear tool for irregularity measurement of vibration signals, a new diagnosis method based on IEWT, RCMDE, multi-cluster feature selection and support vector machine is proposed. Then the method is applied to analysis of bearing in this paper and the results show that the new method has higher identifying rate and better performance than that of the methods of RCMDE combining with EMD or EWT. Also, the superiority of RCMDE to dispersion entropy and multiscale dispersion entropy is investigated, together with the superiority of MCFS for feature selection.

Depression or Major Depressive Disorder (MDD) is a mental illness which negatively affects how a person thinks, acts or feels. MDD has become a major disease affecting millions of people presently. The diagnosis of depression is questionnaire based and is not based on any objective criteria. In this paper, feature extracted from EEG signal are used for the diagnosis of depression. Alpha, alpha1, alpha2, beta, delta and theta power and theta asymmetry was used as feature. Alpha1, alpha2 along with theta asymmetry was also used as a feature. Multi-Cluster Feature Selection (MCFS) was used for feature selection when feature combination was used. The classifiers used were Support Vector Machine (SVM), Logistic Regression (LR), Naïve-Bayesian (NB) and Decision Tree (DT). Alpha2 showed higher classification accuracy than alpha1 and alpha power in all applied classifier. From t-test it was found that there was a significant difference in the theta power of left and right hemisphere of normal subjects, but there was no significant difference in depression patients. Average theta asymmetry in normal subjects is higher than MDD patients but the difference in theta asymmetry in normal subjects and MDD patients is not significant. The combination of alpha2 and theta asymmetry showed the highest classification accuracy of 88.33% in SVM.

Application of improved bubble entropy and machine learning in the adaptive diagnosis of rotating machinery faults

Accurate and efficient diagnosis in a short time period is an important part of brain magnetic resonance imaging (MRI) classification. In this paper, we use multi-cluster feature selection (MCFS) method to select efficient features from the primary features for brain MRI classification. The primary features are obtained from a three-level two-dimensional discrete wavelet transform (2D DWT). The selected features are then applied to the K-nearest neighbor (KNN) classifier. We classify the MRI as normal or one of the seven different diseases. The results demonstrate that the proposed method achieves higher accuracy than the other methods in distinguishing different types of disease.

Differentiation between benign and malignant nodules is a problem encountered by radiologists when visualizing computed tomography (CT) scans. Adenocarcinomas and granulomas have a characteristic spiculated appearance and may be fluorodeoxyglucose avid, making them difficult to distinguish for human readers. In this retrospective study, we aimed to evaluate whether a combination of radiomic texture and shape features from noncontrast CT scans can enable discrimination between granulomas and adenocarcinomas. Our study is composed of CT scans of 195 patients from two institutions, one cohort for training ([Formula: see text]) and the other ([Formula: see text]) for independent validation. A set of 645 three-dimensional texture and 24 shape features were extracted from CT scans in the training cohort. Feature selection was employed to identify the most informative features using this set. The top ranked features were also assessed in terms of their stability and reproducibility across the training and testing cohorts and between scans of different slice thickness. Three different classifiers were constructed using the top ranked features identified from the training set. These classifiers were then validated on the test set and the best classifier (support vector machine) yielded an area under the receiver operating characteristic curve of 77.8%.

Feature selection is commonly used to reduce dimensionality of datasets with tens or hundreds of thousands of features. A major problem of text categorization is the high dimensionality of the feature space; therefore, feature selection is the most important step in text categorization. This paper presents a novel feature selection algorithm that is based on ant colony optimization. Ant colony optimization algorithm is inspired by observation on real ants in their search for the shortest paths to food sources. Proposed algorithm is easily implemented and because of use of a simple classifier in that, its computational complexity is very low. The performance of proposed algorithm is compared to the performance of information gain and CHI algorithms on the task of feature selection in Reuters-21578 dataset. Simulation results on Reuters-21578 dataset show the superiority of the proposed algorithm.

DroidRL: Feature selection for android malware detection with reinforcement learning

Feature selection is the process of finding the set of inputs to a machine learning algorithm that will yield the best performance. Developing a way to solve this problem automatically would make current machine learning methods much more useful. Previous efforts to automate feature selection rely on expensive meta-learning or are applicable only when labeled training data is available. This paper presents a novel method called FS-NEAT which extends the NEAT neuroevolution method to automatically determine an appropriate set of inputs for the networks it evolves. By learning the network's inputs, topology, and weights simultaneously, FS-NEAT addresses the feature selection problem without relying on meta-learning or labeled data. Initial experiments in an autonomous car racing simulation demonstrate that FS-NEAT can learn better and faster than regular NEAT. In addition, the networks it evolves are smaller and require fewer inputs. Furthermore, FS-NEAT's performance remains robust even as the feature selection task it faces is made increasingly difficult.

An important task in knowledge discovery is feature selection. This paper describes a practical approach to feature subset selection proposed as part of a hybrid rough sets/neural network framework for knowledge discovery for decision support. In this framework neural networks and rough sets are combined and used cooperatively during the system life cycle. The reason for combining rough sets with neural networks in the proposed framework is twofold. Firstly, rough sets based systems provide domain knowledge expressed in the form of If-then rules as well as tools for data analysis. Secondly, rough sets are used in this framework in the task of feature selection for neural network models. This paper examines the feature selection aspect of the framework. An empirical study that tested the approach on artificial datasets and real-world datasets was carried out. Experimental results indicate that the proposed approach can improve the performance of neural network models. The framework was also applied in the development of a real-world decision support system. The experience with this application has shown that the approach can support the users in the task of feature selection.

The “Curse of Dimensionality” induced by the rapid development of information science might have a negative impact when dealing with big datasets, and it also makes the problems of symmetry and asymmetry increasingly prominent. Feature selection (FS) can eliminate irrelevant information in big data and improve accuracy. As a recently proposed algorithm, the Sparrow Search Algorithm (SSA) shows its advantages in the FS tasks because of its superior performance. However, SSA is more subject to the population’s poor diversity and falls into a local optimum. Regarding this issue, we propose a variant of the SSA called the Tent Lévy Flying Sparrow Search Algorithm (TFSSA) to select the best subset of features in the wrapper-based method for classification purposes. After the performance results are evaluated on the CEC2020 test suite, TFSSA is used to select the best feature combination to maximize classification accuracy and simultaneously minimize the number of selected features. To evaluate the proposed TFSSA, we have conducted experiments on twenty-one datasets from the UCI repository to compare with nine algorithms in the literature. Nine metrics are used to evaluate and compare these algorithms’ performance properly. Furthermore, the method is also used on the coronavirus disease (COVID-19) dataset, and its classification accuracy and the average number of feature selections are 93.47% and 2.1, respectively, reaching the best. The experimental results and comparison in all datasets demonstrate the effectiveness of our new algorithm, TFSSA, compared with other wrapper-based algorithms.

Feature selection is a common method for improving classification performance. Selecting features for high-dimensional data is challenging due to the large search space. Traditional feature ranking methods that search for top-ranked features cannot remove redundant and irrelevant features and may also ignore interrelated features. Evolutionary computation (EC) techniques are widely used in feature selection due to their global search capability. However, EC can easily fall into local optima when dealing with feature selection for high-dimensional applications. The top-ranked features are more likely to construct effective feature subsets and help EC reduce the search space. This paper proposes a feature selection method based on Genetic Programming (GP) with hybrid feature ranking criterion called GPHC, which combines multiple feature ranking methods into the GP structure using a novel GP representation to search for effective feature subsets. Experiments on eight high-dimensional datasets show that GPHC achieves significantly better classification performance compared to five feature ranking methods. Further comparisons between GPHC and other evolutionary algorithms demonstrate that GPHC has advantages in terms of classification performance, the number of features, and convergence speed.

Densely connected sub-Gaussian linear structural equation model learning via ℓ1- and ℓ2-regularized regressions