A comparison between fuzzy-ID3 and OFFSS-based fuzzy-ID3

Mangroves are an important source of blue carbon that grow in coastal areas. The study of mangrove species distribution is the basis of carbon storage research. In this study, we explored the potential of combining optical (Gaofen-1, Sentinel-2, and Landsat-9) and fully polarized synthetic aperture radar data from different periods (Gaofen-3) to distinguish mangrove species in the Fucheng town of Leizhou, Guangdong Province. The Gaofen-1 data were fused with Sentinel-2 and Landsat-9 satellite data, respectively. The new data after fusion had both high spatial and spectral resolution. The backscattering coefficient and polarization decomposition parameters of the fully polarized SAR data which could characterize the canopy structure of mangroves were extracted. Ten different feature combinations were designed by combining the two types of data. The extremely randomized trees algorithm (ERT) was used to classify the species, and the optimal feature subset was selected by the feature selection algorithm on the basis of the ERT, and the importance of the features was sorted. Studies show the following: (1) When controlling a single variable, the higher the spatial resolution of the multi-spectral data, the higher the interspecific classification accuracy. (2) The coupled Sentinel-2 and Landsat-9 data with a 2 m resolution will have higher classification accuracy than a single data source. (3) The selected feature subset contains all types of features in the optical data and the polarization decomposition features of the SAR data from different periods: multi-spectral band > texture feature > polarization decomposition parameter > vegetation index. Among the optimized feature combinations, the classification accuracy of mangrove species was the highest, the overall classification accuracy was 90.13%, and Kappa was 0.84, indicating that multi-source and SAR data from different periods coupling could improve the discrimination of mangrove species. (4) The ERT classification algorithm is suitable for the study of mangrove species classification, and the classification accuracy of extremely random trees in this paper is higher than that of random forest (RF), K-nearest neighbor (KNN), and Bayesian (Bayes). The results can provide technical guidance and data support for mangrove species monitoring based on multi-source satellite data.

Fault detection in metallic structures requires a detailed and discriminative feature pool creation mechanism to develop an effective condition monitoring system. Traditional fault detection methods incorporate handcrafted features either from the time, frequency or time-frequency domains. To explore the salient information provided by the acoustic emission (AE) signals, a hybrid of feature pool creation and an optimal features subset selection mechanism is proposed for crack detection in a spherical tank. The optimal hybrid feature pool creation process is composed of two major parts: (1) extraction of statistical features from time and frequency domains, as well as extraction of traditional features associated with the AE signals; and (2) genetic algorithm (GA)-based optimal features subset selection. The optimal features subset is then provided to the k-nearest neighbor (k-NN) classifier to distinguish between normal (NC) and crack conditions (CC). Experimental results show that the proposed approach yields an average 99.8% accuracy for heath state classification. To validate the effectiveness of the proposed approach, it is compared to conventional non-linear dimensionality reduction techniques, as well as those without feature selection schemes. Experimental results show that the proposed approach outperforms conventional non-linear dimensionality reduction techniques, achieving at least 2.55% higher classification accuracy.

Jeffries-Matusita (JM) distance, a transformation of the Bhattacharyya distance, is a widely used measure of the spectral separability distance between the two class density functions and is generally used as a class separability measure. It can be considered to have good potential to be used for evaluation of the effectiveness of a feature in discriminating two classes. The capability of JM distance as a ranking based feature selection technique for binary classification problems has been verified in some research works as well as in our earlier work. It was found by our simulation experiments with benchmark data sets that JM distance works equally well compared to other popular feature ranking methods based on mutual information, information gain or Relief. Extension of JM distance measure for feature ranking in multiclass problems has also been reported in the literature. But all of them are basically rank based approaches which deliver the ranking of the features and do not automatically produce the final optimal feature subset. In this work, a novel heuristic approach for finding out the optimum feature subset from JM distance based ranked feature lists for multiclass problems have been developed without explicitly using any specific search technique. The proposed approach integrates the extension of JM measure for multiclass problems and the selection of the final optimal feature subset in a unified process. The performance of the proposed algorithm has been evaluated by simulation experiments with benchmark data sets in comparison with two other previously developed multiclass JM distance measures (weighted average JM distance and another multiclass extension equivalent to Bhattacharyya bound) and some other popular filter based feature ranking algorithms. It is found that the proposed algorithm performs better in terms of classification accuracy, F-measure, AUC with a reduced set of features and computational cost.

Predicting creep life of CrMo pressure vessel steel using machine learning models with optimal feature subset selection

A computational intelligence technique for the effective diagnosis of diabetic patients using principal component analysis (PCA) and modified fuzzy SLIQ decision tree approach

Electrocardiogram (ECG) classification is a critical task for early detection of cardiac abnormalities.However, standard machine learning classifiers such as Logistic Regression (LR), K-Nearest Neighbors (KNN), Support Vector Machine (SVM), and Random Forest (RF) face challenges when handling high-dimensional data.These classifiers often struggle to model complex relationships among features, leading to reduced classification performance.Existing metaheuristic-based feature selection methods typically search across the entire feature space, which increases computational complexity and often fails to isolate only the most informative features.This paper introduces a novel hybrid feature selection approach based on a two-phase framework combining Topsoe Distance (TD) and Particle Swarm Optimization (PSO).In the first phase, TD is used to eliminate irrelevant and redundant features, thereby reducing the feature space from 4000 to 1600 and preserving only the most relevant and harmonized features.In the second phase, PSO is applied to the reduced space, enabling efficient optimization and feature subset selection.Experimental results using the MIT-BIH Arrhythmia dataset demonstrate that the proposed TD-PSO method significantly improves classification accuracy compared to using PSO alone.Specifically, LR accuracy improved from 87% to 95%, KNN from 92% to 97%, SVM from 92% to 96%, and RF from 93% to 99%.These results confirm that narrowing the search space before optimization allows classifiers to perform more effectively while reducing computational cost.The proposed TD-PSO framework provides a robust and scalable solution for high-dimensional ECG classification and holds strong potential for broader applications in biomedical data analysis.

Computer-aided diagnosis has the potential of increasing diagnostic accuracy by providing a second reading to radiologists. In many computerized schemes, numerous features can be extracted to describe suspect image regions. A subset of these features is then employed in a data classifier to determine whether the suspect region is abnormal or normal. Different subsets of features will, in general, result in different classification performances. A feature selection method is often used to determine an "optimal" subset of features to use with a particular classifier. A classifier performance measure (such as the area under the receiver operating characteristic curve) must be incorporated into this feature selection process. With limited datasets, however, there is a distribution in the classifier performance measure for a given classifier and subset of features. In this paper, we investigate the variation in the selected subset of "optimal" features as compared with the true optimal subset of features caused by this distribution of classifier performance. We consider examples in which the probability that the optimal subset of features is selected can be analytically computed. We show the dependence of this probability on the dataset sample size, the total number of features from which to select, the number of features selected, and the performance of the true optimal subset. Once a subset of features has been selected, the parameters of the data classifier must be determined. We show that, with limited datasets and/or a large number of features from which to choose, bias is introduced if the classifier parameters are determined using the same data that were employed to select the "optimal" subset of features.

One of the most tools of data mining techniques is decision trees for both learning and reasoning from the crisp dataset. In a case of fuzzy dataset, the fuzzy decision tree must be established to extracted fuzzy rules. The paper illustrates an approach to establish fuzzy type-I decision tree from fuzzy type-II dataset using the ambiguity measure in fuzzy type-II form.

Multi-filter semi-supervised transformer model for fault diagnosis

<p>Classification aims to identify a class label of an instance according to the information from its characteristics or features. Unfortunately, many classification problems have a large feature set containing irrelevant and redundant features, which reduce the classification performance. In order to address the above problem, feature selection is proposed to select a small subset of relevant features. There are three main types of feature selection methods, i.e. wrapper, embedded and filter approaches. Wrappers use a classification algorithm to evaluate candidate feature subsets. In embedded approaches, the selection process is embedded in the training process of a classification algorithm. Different from the other two approaches, filters do not involve any classification algorithm during the selection process. Feature selection is an important process but it is not an easy task due to its large search space and complex feature interactions. Because of the potential global search ability, Evolutionary Computation (EC), especially Particle Swarm Optimization (PSO), has been widely and successfully applied to feature selection. However, there is potential to improve the effectiveness and efficiency of EC-based feature selection. The overall goal of this thesis is to investigate and improve the capability of EC for feature selection to select small feature subsets while maintaining or even improving the classification performance compared to using all features. Different aspects of feature selection are considered in this thesis such as the number of objectives (single-objective/multi-objective), the fitness function (filter/wrapper), and the searching mechanism. This thesis introduces a new fitness function based on mutual information which is calculated by an estimation approach instead of the traditional counting approach. Results show that the estimation approach works well on both continuous and discrete data. More importantly, mutual information calculated by the estimation approach can capture feature interactions better than the traditional counting approach. This thesis develops a novel binary PSO algorithm, which is the first work to redefine some core concepts of PSO such as velocity and momentum to suit the characteristics of binary search spaces. Experimental results show that the proposed binary PSO algorithm evolve better solutions than other binary EC algorithms when the search spaces are large and complex. Specifically, on feature selection, the proposed binary PSO algorithm can select smaller feature subsets with similar or better classification accuracies, especially when there are a large number of features. This thesis proposes surrogate models for wrapper-based feature selection. The surrogate models use surrogate training sets which are subsets of informative instances selected from the training set. Experimental results show that the proposed surrogate models assist PSO to reduce the computational cost while maintaining or even improving the classification performance compared to using only the original training set. The thesis develops the first wrapper-based multi-objective feature selection algorithm using MOEA/D. A new decomposition strategy using multiple reference points for MOEA/D is designed, which can deal with different characteristics of multi-objective feature selection such as highly discontinuous Pareto fronts and complex relationships between objectives. The experimental results show that the proposed algorithm can evolve more diverse non-dominated sets than other multi-objective algorithms. This thesis introduces the first PSO-based feature selection algorithm for transfer learning. In the proposed algorithm, the fitness function uses classification performance to reduce the differences between domains while maintaining the discriminative ability on the target domain. The experimental results show that the proposed algorithm can select feature subsets which achieve better classification performance than four state-of-the-art feature-based transfer learning algorithms.</p>

Medical errors are considered as the leading cause of death and injury. Breast cancer becomes one of the leading causes of death among women, not only in the Philippines but worldwide. In this paper, data mining was used to predict the stage of breast cancer using a hybrid of fuzzy logic and decision tree. This aims to help experts to make decisions rather than replacing them. The result will only give an expert a recommendation, but the final decision is still on the hands of the experts. Feature selection was used to determine the best attribute in the dataset from Surveillance Epidemiology and End Results (SEER). The data set consists of incidence from 1975 to 2016, but the study limits the analysis from 2010 to 2016. Different cleaning and preprocessing of data are conducted. After thorough preprocessing of data, six (6) attributes are selected, and one (1) target class. Performance comparison shows that the fuzzy decision tree achieved a higher accuracy of 99.96%, sensitivity of 99.26% and specificity of 99.98% than the decision tree classification technique. The simulation result shows a correctly classified instance of 165,124, which is equivalent to 99.97% and only 351 incorrect classified instances or 0.21%. Thus, a fuzzy decision tree is more robust than the traditional decision tree classifier for predicting the stage of breast cancer.

Feature selection is the process of choosing, or removing, features to obtain the most informative feature subset of minimal size. Such subsets are used to improve performance of machine learning algorithms and enable human understanding of the results. Approaches to feature selection in literature exploit several optimization algorithms. Multi-objective methods also have been proposed, minimizing at the same time the number of features and the error. While most approaches assess error resorting to the average of a stochastic K-fold cross-validation, comparing averages might be misleading. In this paper, we show how feature subsets with different average error might in fact be non-separable when compared using a statistical test. Following this idea, clusters of non-separable optimal feature subsets are identified. The performance in feature selection can thus be evaluated by verifying how many of these optimal feature subsets an algorithm is able to identify. We thus propose a multi-objective optimization approach to feature selection, EvoFS, with the objectives to i. minimize feature subset size, ii. minimize test error on a 10-fold cross-validation using a specific classifier, iii. maximize the analysis of variance value of the lowest-performing feature in the set. Experiments on classification datasets whose feature subsets can be exhaustively evaluated show that our approach is able to always find the best feature subsets. Further experiments on a high-dimensional classification dataset, that cannot be exhaustively analyzed, show that our approach is able to find more optimal feature subsets than state-of-the-art feature selection algorithms.

Hyperspectral images (HSIs) generally contain a large amount of spectral bands (features), and the redundant information in them will cause the Hughes phenomenon in the classification process. And feature extraction and feature selection are the two main existing methods to effectively reduce the redundancy of spectral information in the field of HSIs classification. Compared with feature extraction methods, feature selection methods can preserve most of the features of the original HSIs data without losing their valuable details. However, most existing feature selection methods based on single scene (domain) perform poorly in some scenes (domains) with insufficient labeled samples. Therefore, how to adopt an efficient feature selection method to select the optimal feature subsets of source scene and target scene and use the sample information of source scene to assist in the classification of target scene so as to improve the classification accuracy of images in the target scene as much as possible is still very challenging. In order to solve the above problem, this paper proposes a new cross-scene algorithm: Improved Ant Colony Optimization Algorithm-Based Cross-Scene Feature Selection Algorithm (IMACO-CSFS). In order to obtain more accurate feature subsets of the two scenes, IMACO-CSFS proposes a priority sorting-based ant colony strategy to make the subsequent search process focus on the global optimal solution (optimal feature subset) found in the previous iteration. In addition, in order to further accelerate the convergence speed of the global optimal solution, an ant colony strategy based on elite ants is proposed in IMACO-CSFS to more efficiently obtain the optimal feature subsets of the two scenes for training the classifier. Furthermore, this paper simultaneously considers overall classification accuracies of the optimal feature subsets for both scenes and dynamically adjusts their scale to ensure the consistency of the selected features between the two scenes, attenuating the effect of spectral shift and achieving the higher image classification accuracy in the target scene. Experimental results on three cross-scene HSI data pairs demonstrate that IMACO-CSFS is superior in cross-scene feature selection and cross-scene HSIs classification.

In the field of hand gesture recognition based on electromyography, the selection of features is still a matter of research because the accuracy of feature evaluation techniques depends on the distribution underling a feature space. Currently, there are a lot of kinds of features, so a selection method is required in order to find an optimal feature subset. The selection of features can be done based on either classifier methods or statistical measurement indexes. On one hand, the use of classifier methods takes considerable time, consequently, they are computationally expensive. On the other hand, the use of a statistical index allows to select features based on properties, and they are less expensive in terms of time and computational costs. In this context, this study analyzed different statistical indexes to discover optimal feature subsets and predicts which one may have the best performance in terms of classification and recognition accuracy. For this purpose, four indexes were analyzed: the sum of RES (separation and compactness index), the sum of Fuzzy Entropies, and both of these indexes divided by the mutual information of feature subset. These index values were calculated for 10 different feature subsets obtained from nineteen time-domain features which have been investigated by many researchers for extracting useful information from an electromyography signal. The feature subsets were implemented in four classifiers (Artificial Neural Network, k-nearest Neighbors Algorithm, Support Vector Machine, and decision trees) to find the best statistical index which tries to quantify the suitableness of the feature space. Each tested index in this work was analyzed in terms of classification and recognition accuracy for each machine learning algorithm.KeywordsHand gesture recognitionElectromyographyFeature evaluationFuzzy entropyMutual informationMyo armband sensor

Classification of omics data suffers from the high error rate due to their high dimensional and small sample size characteristics. To overcome the problem, this paper proposes an ensemble feature selection for omics data classification based on constrained niching binary particle swarm optimization (PSO). Particularly, optimal feature subsets in terms of best classification accuracy are identified by the binary PSO. The proposed method introduces constraint on the particle encoding to constrain the number of selected features, and niching technique from multimodal optimization is imposed to enable the algorithm to obtain multiple diverse feature subsets in a single run. Afterward, multiple base classifiers built on the obtained feature subsets are combined into a stronger classifier which is applied to classify the omics data. Experimental results on real-world omics datasets demonstrate that the proposed feature selection method can stably select compact feature subsets and obtain promising classification performance.