Cost-sensitive feature selection based on adaptive neighborhood granularity with multi-level confidence

Distance metric learning-based multi-granularity neighborhood rough sets for attribute reduction

Neighborhood rough sets are an effective model for handling numerical and categorical data entangled with vagueness, imprecision, or uncertainty. However, existing neighborhood rough set models and their feature selection methods treat each sample equally, whereas different types of samples inherently play different roles in constructing neighborhood granules and evaluating the goodness of features. In this study, the sample weight information is first introduced into neighborhood rough sets, and a novel weighted neighborhood rough set model is consequently constructed. Then, considering the lack of sample weight information in practical data, a margin-based weight optimization function is designed, based on which a gradient descent algorithm is provided to adaptively learn sample weights through maximizing sample margins. Finally, an average granule margin measure is put forward for feature selection, and a forward-adding heuristic algorithm is developed to generate an optimal feature subset. The proposed method constructs the weighted neighborhood rough sets using sample weights for the first time and is able to yield compact feature subsets with a large margin. Extensive experiments and statistical analysis on UCI datasets show that the proposed method achieves highly competitive performance in terms of feature reduction rate and classification accuracy when compared with other state-of-the-art methods.

Incremental feature selection for dynamic hybrid data using neighborhood rough set

Neighborhood margin rough set: Self-tuning neighborhood threshold

Multi-label feature selection, which is an efficient and effective pre-processing step in machine learning and data mining, can select a feature subset that contains more contributions for multi-label classification while improving the performance of the classifiers. In real-world applications, an instance may be associated with multiple related labels with different relative importances, and the process of obtaining different features usually requires different costs, containing money, and time, etc. However, most existing works with regard to multi-label feature selection do not take into consideration the above two critical issues simultaneously. Therefore, in this paper, we exploit the idea of neighborhood granularity to enhance the traditional logical labels into label distribution forms to excavate the deeper supervised information hidden in multi-label data, and further consider the effect of the test cost under three different distributions, simultaneously. Motivated by these issues, a novel test cost multi-label feature selection algorithm with label enhancement and neighborhood granularity is designed. Moreover, the proposed algorithm is tested upon ten publicly available benchmark multi-label datasets with six widely-used metrics from two different aspects. Finally, two groups of experimental results demonstrate that the proposed algorithm achieves the satisfactory and superior performance over other four state-of-the-art comparing algorithms, and it is effective for improving the learning performance and decreasing the total test costs of the selected feature subset.

Neighborhood rough set theory is an important method for dealing with uncertainty, fuzziness and undefined objects, and neighborhood rough set theory based on binary relation is studied by scholars a lot. However, most of the current research is used to deal with equivalent binary relations or other binary relations with stringent condition, which has been unable to meet the rapid development of data. This paper proposes a new multi-granulation rough set that can deal with general binary relations based on the C <inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">j</inf> neighborhoods, and verifies its related properties. At the same −time, it has richer meanings through weighted methods. Theoretical analysis and practical examples show that this rough set has better ability to process data. Finally, a weighted attribute reduction algorithm is designed based on the significant function under the general binary relation. The experimental results show that this rough set can handle complex data well. The research in this paper develops the theory of classical rough sets, and provides a theoretical basis for the knowledge acquisition of information systems under the general binary relation.

A novel approach to attribute reduction based on weighted neighborhood rough sets

Considering the fact that the early fault of the rolling-element bearing is difficult to identify correctly, a neighborhood rough set (NRS) and ensemble empirical mode decomposition (EEMD) fault diagnosis model of rolling-element bearing combination is come up with. First of all, the original signal of the vibration is decomposed by EEMD to obtain a number of IMF components, and extracted time-features from the first 3 IMF components in the time domain to form the original features. Then, NRS is used to reduce the attributes of original features, eliminate the redundant information, and put the sensitive reduced feature attributes into the SVM classifier for fault identification. This model is applied to typical rolling-element bearing fault diagnosis experiments, which shows that the NRS is used to select a large number of features containing abundant fault information from the original features. The method not only reduces the complexity of the classification algorithm, but also enhances the accuracy of the fault identification by 5%, which provides a new approach of analysis model for rolling-element bearing fault diagnosis.

Variable radius neighborhood rough sets and attribute reduction

Extracting knowledge from hybrid data, comprising both categorical and numerical data, poses significant challenges due to the inherent difficulty in preserving information and practical meanings during the conversion process. To address this challenge, hybrid data processing methods, combining complementary rough sets, have emerged as a promising approach for handling uncertainty. However, selecting an appropriate model and effectively utilizing it in data mining requires a thorough qualitative and quantitative comparison of existing hybrid data processing models. This research aims to contribute to the analysis of hybrid data processing models based on neighborhood rough sets by investigating the inherent relationships among these models. We propose a generic neighborhood rough set-based hybrid model specifically designed for processing hybrid data, thereby enhancing the efficacy of the data mining process without resorting to discretization and avoiding information loss or practical meaning degradation in datasets. The proposed scheme dynamically adapts the threshold value for the neighborhood approximation space according to the characteristics of the given datasets, ensuring optimal performance without sacrificing accuracy. To evaluate the effectiveness of the proposed scheme, we develop a testbed tailored for Parkinson’s patients, a domain where hybrid data processing is particularly relevant. The experimental results demonstrate that the proposed scheme consistently outperforms existing schemes in adaptively handling both numerical and categorical data, achieving an impressive accuracy of 95% on the Parkinson’s dataset. Overall, this research contributes to advancing hybrid data processing techniques by providing a robust and adaptive solution that addresses the challenges associated with handling hybrid data, particularly in the context of Parkinson’s disease analysis.

Classification with test costs and background knowledge

Optimal cost-sensitive granularization based on rough sets for variable costs

Decision-theoretic rough sets (DTRS) can be seen as a kind of misclassification cost-sensitive learning model. In DTRS, attribute reduction is the process of minimizing misclassification costs. However in parctice, data are not free, and there are test costs to obtain feature values of objects. Hence, the process of attribute reduction should help minimizing both of misclassification costs and test costs. In this paper, the minimal test cost attribute reduct (MTCAR) problem is defined in DTRS. The objective of attribute reduction is to minimize misclassification costs and test costs. A genetic algorithm (GA) is used to solve this problem. Experiments on UCI data sets are performed to validate the effectiveness of GA to solve MTCAR problem.KeywordsDecision-theoretic rough setsmisclassification coststest costsattribute reductcost-sensitive

Feature selection is beneficial for improving the performance of general machine learning tasks by extracting an informative subset from the high-dimensional features. Conventional feature selection methods usually ignore the class imbalance problem, thus the selected features will be biased towards the majority class. Considering that F-measure is a more reasonable performance measure than accuracy for imbalanced data, this paper presents an effective feature selection algorithm that explores the class imbalance issue by optimizing F-measures. Since F-measure optimization can be decomposed into a series of cost-sensitive classification problems, we investigate the cost-sensitive feature selection by generating and assigning different costs to each class with rigorous theory guidance. After solving a series of cost-sensitive feature selection problems, features corresponding to the best F-measure will be selected. In this way, the selected features will fully represent the properties of all classes. Experimental results on popular benchmarks and challenging real-world data sets demonstrate the significance of cost-sensitive feature selection for the imbalanced data setting and validate the effectiveness of the proposed method.

Feature selection is one of the widely used pre-processing techniques to deal with large data sets. In this context, rough set theory has been successfully implemented for feature selection of discrete data set but in case of continuous data set it requires discretization, which may cause information loss. Fuzzy rough set theory approaches have also been used successfully to resolve this issue as it can handle continuous data directly. Moreover, almost all feature selection techniques are used to handle homogeneous data set. In this article, the center of attraction is on heterogeneous feature subset reduction. A novel intuitionistic fuzzy neighborhood models have been proposed by combining intuitionistic fuzzy sets and neighborhood rough set models by taking an appropriate pair of lower and upper approximations and generalize it for feature selection, supported with theory and its validation. An appropriate algorithm along with application to a data set has been added.