Information Theory Based Feature Selection for Multi-Relational Naïve Bayesian Classifier

Feature selection is the preliminary step in machine learning and data mining. It identifies the most important and relevant features within a dataset by eliminating the redundant or irrelevant features. The substantial benefits may include an improved performance in terms of high prediction accuracy, reduced computational complexity, and simply interpretable underlying models. In this paper, we present a novel framework to investigate and understand the importance of Monte Carlo tree search (MCTS) in feature selection for very high-dimensional datasets. We construct a binary feature selection tree where each node represents one of the two feature states: a feature is selected or not. The search starts with an empty root node reflecting that no feature is selected. Then, the search tree is expanded by adding nodes in an incremental fashion through MCTS-based simulations. Following tree and default policy, every iteration generates an initial feature subset, where a filter is used to select the top k features forming the candidate feature subset. The classification accuracy is used as the goodness or reward of the candidate feature subset and propagated backward up to the root node following the active path. Finally, the candidate subset with highest reward is selected as the best feature subset. Experiments are performed on 30 real-world datasets, including 14 very high-dimensional microarray datasets, and results are also compared with state-of-the-art methods in the literature, which proves the efficacy, validity, and significance of the proposed method.

Feature selection is a preprocessing step that plays a crucial role in the domain of machine learning and data mining. Feature selection methods have been shown to be effective in removing redundant and irrelevant features, improving the learning algorithm’s prediction performance. Among the various methods of feature selection based on redundancy, the fast correlation-based filter (FCBF) is one of the most effective. In this paper, we developed a novel extension of FCBF, called resampling FCBF (RFCBF) that combines resampling technique to improve classification accuracy. We performed comprehensive experiments to compare the RFCBF with other state-of-the-art feature selection methods using three competitive classifiers (K-nearest neighbor, support vector machine, and logistic regression) on 12 publicly available datasets. The experimental results show that the RFCBF algorithm yields significantly better results than previous state-of-the-art methods in terms of classification accuracy and runtime.

The performance of supervised learning-based seganalysis depends on the choice of both classifier and features which represent the image. Features extracted from images may contain irrelevant and redundant features which makes them inefficient for machine learning. Relevant features not only decrease the processing time to train a classifier but also provide better generalisation. Linear discriminant classifier which is commonly used for classification may not be able to classify in better way non-linearly separable data. Recently, exponential discriminant analysis, a variant of linear discriminant analysis (LDA), is proposed which transforms the scatter matrices to a new space by distance diffusion mapping. This provides exponential discriminant analysis (EDA) much more discriminant power to classify non-linearly separable data and helps in improving classification accuracy in comparison to LDA. In this paper, the performance of EDA in conjunction with feature selection methods has been investigated. For feature selection, Kullback divergence, Chernoff distance measures and linear regression measures are used to determine relevant features from higher-order statistics of images. The performance is evaluated in terms classification error and computation time. Experimental results show that exponential discriminate analysis in conjunction with linear regression significantly performs better in terms of both classification error and compilation time of training classifier. Defence Science Journal, 2012, 62(1), pp.19 -24 , DOI:http://dx.doi.org/10.14429/dsj. 62.1437

Background: Dimension disaster is often associated with feature extraction. The extracted features may contain more redundant feature information, which leads to the limitation of computing ability and overfitting problems. Objective: Feature selection is an important strategy to overcome the problems from dimension disaster. In most machine learning tasks, features determine the upper limit of the model performance. Therefore, more and more feature selection methods should be developed to optimize redundant features. Methods: In this paper, we introduce a new technique to optimize sequence features based on the binomial distribution (BD). Firstly, the principle of the binomial distribution algorithm is introduced in detail. Then, the proposed algorithm is compared with other commonly used feature selection methods on three different types of datasets by using a Random Forest classifier with the same parameters. Results: The results confirm that BD has a promising improvement in feature selection and classification accuracy. Conclusion: Finally, we provide the source code and executable program package (http: //lingroup. cn/server/BDselect/), by which users can easily perform our algorithm in their researches.

In the coupling state of rotor unbalance fault and bearing defect fault for rotor system, the signals contain multiple fault components, and the fault diagnosis of the rotor system needs to contain comprehensive multidimensional feature quantities. However, irrelevant feature information in the multi-dimensional feature quantities increases the complexity of classification calculation and affects the efficiency and accuracy of diagnosis. In order to eliminate redundant and irrelevant features in the feature information, and achieve the goal of fewer diagnostic features and good diagnostic results, this paper proposes an adaptive feature selection based on the maximum information coefficient FF-FC-MIC (Feature-to-Feature and Feature-to-Category Maximum Information Coefficient) method. Firstly, the sparse representation algorithm is used to reconstruct the original signal to improve the signal-to-noise ratio, and the multi-dimensional feature quantity of the reconstructed signal is calculated; Secondly, calculate the correlation between features and features through MIC to obtain a feature set of weak correlation between features; thirdly, use MIC to calculate the correlation between features and signal categories to obtain a feature set with strong correlation between features and signals; Finally, the FF-FC-MIC feature selection method is used for feature adaptive selection and input into SVM to complete fault diagnosis. The method is analyzed by simulation signals and the real experiment signals. The results show that the method can effectively remove redundant and disclosed features in the coupling fault, reducing the characteristic dimension to reduce the fault classification time, and improve classification accuracy. Different experimental cases and various feature selection comparison methods further verify the accuracy and applicability of the proposed method.

Feature selection is a dimensionality reduction method known as a main step in data mining and machine learning. The aim of feature selection is to remove redundant and unrelated features. In recent years several feature selection methods based on graph theory and social networking techniques have been proposed .In this study, a feature selection approach based on multi objective PSO algorithm and social network techniques is presented. In the proposed method, Fisher score, node centrality and edge centrality are used to construct the fitness function in order to present a multi objective particle swarm optimization (PSO) approach. The proposed method run over a variety of datasets and the results are compared with the well-known filter-based feature selection methods. The results show that the proposed method is effective and the performance of the proposed method better than other methods in some cases.

Deep learning has significantly advanced image classification by leveraging hierarchical feature representations. A key factor in enhancing classification accuracy is feature concatenation, which integrates diverse feature sets to provide a richer representation of input data. However, this fusion strategy has inherent limitations, including increased computational complexity, susceptibility to redundant or irrelevant features, and challenges in optimally weighting different feature contributions. To address these challenges, this paper presents a pseudo-multiview learning method that dynamically combines different views at the evidence level using a belief-based model known as subjective logic. This approach adaptively assigns confidence levels to each view, ensuring more effective integration of complementary information while mitigating the impact of noisy or less relevant features. Experimental evaluations of datasets from various domains demonstrate that the proposed method enhances classification accuracy and robustness compared with conventional classification techniques.

With the rapid development of artificial intelligence in recent years, the research on image processing, text mining, and genome informatics has gradually deepened, and the mining of large-scale databases has begun to receive more and more attention. The objects of data mining have also become more complex, and the data dimensions of mining objects have become higher and higher. Compared with the ultra-high data dimensions, the number of samples available for analysis is too small, resulting in the production of high-dimensional small sample data. High-dimensional small sample data will bring serious dimensional disasters to the mining process. Through feature selection, redundancy and noise features in high-dimensional small sample data can be effectively eliminated, avoiding dimensional disasters and improving the actual efficiency of mining algorithms. However, the existing feature selection methods emphasize the classification or clustering performance of the feature selection results and ignore the stability of the feature selection results, which will lead to unstable feature selection results, and it is difficult to obtain real and understandable features. Based on the traditional feature selection method, this paper proposes an ensemble feature selection method, Random Bits Forest Recursive Clustering Eliminate (RBF-RCE) feature selection method, combined with multiple sets of basic classifiers to carry out parallel learning and screen out the best feature classification results, optimizes the classification performance of traditional feature selection methods, and can also improve the stability of feature selection. Then, this paper analyzes the reasons for the instability of feature selection and introduces a feature selection stability measurement method, the Intersection Measurement (IM), to evaluate whether the feature selection process is stable. The effectiveness of the proposed method is verified by experiments on several groups of high-dimensional small sample data sets.

Microarray technology makes it easier for scientists to rapidly measure thousands of gene’s expression levels. By analyzing these data, we can find the altered genes, thereby facilitating easy diagnosis and classification of the genetic-related diseases. However, predicting and identifying cancer types is a great challenge in the medical field. Gene expression microarray contains information that can help in this regard, but microarray data have high dimensionality problem which means a large number of genes or features and a small number of samples, also there are redundant and irrelevant features that increase the challenge of microarray analysis. This study reviewed recent studies about methods, algorithms, and limitations of feature selection for microarray gene expression classification. This study compared and focused on four aspects for each related study: datasets, feature selection methods, classifiers, and accuracy results. Feature selection methods are considered as a pre-processing step which plays a vital role in the effectiveness of a classification. This paper showed that applying filter methods such as t-Test, Pearson’s Correlation Coefficient (PCC), and Bhattacharyya distance eliminate irrelevant features that help to increase classification performance and accuracy. Therefore, applying wrapper or embedded methods such as Genetic Algorithm (GA) without applying filter methods in advance could affect the effectiveness of a classification negatively.

With the continuous development of Internet technology, data gradually present a complicated and high-dimensional trend. These high-dimensional data have a large number of redundant features and irrelevant features, which bring great challenges to the existing machine learning algorithms. Feature selection is one of the important research topics in the fields of machine learning, pattern recognition and data mining, and it is also an important means in the data preprocessing stage. Feature selection is to look for the optimal feature subset from the original feature set, which would improve the classification accuracy and reduce the machine learning time. The traditional feature selection algorithm tends to ignore the kind of feature which has a weak distinguishing capacity as a monomer, whereas the feature group’s distinguishing capacity is strong. Therefore, a new dynamic interaction feature selection (DIFS) algorithm is proposed in this paper. Initially, under the theoretical framework of interactive information, it redefines the relevance, irrelevance and redundancy of the features. Secondly, it offers the computational formulas for calculating interactive information. Finally, under the eleven data sets of UCI and three different classifiers, namely, KNN, SVM and C4.5, the DIFS algorithm increases the classification accuracy of the FullSet by 3.2848% and averagely decreases the number of features selected by 15.137. Hence, the DIFS algorithm can not only identify the relevance feature effectively, but also identify the irrelevant and redundant features. Moreover, it can effectively improve the classification accuracy of the data sets and reduce the feature dimensions of the data sets.

Deep learning model for seabed sediment classification based on fuzzy ranking feature optimization

Nowadays, many disciplines have to deal with big datasets that additionally involve a high number of features. Feature selection methods aim at eliminating noisy, redundant, or irrelevant features that may deteriorate the classification performance. However, traditional methods lack enough scalability to cope with datasets of millions of instances and extract successful results in a delimited time. This paper presents a feature selection algorithm based on evolutionary computation that uses the MapReduce paradigm to obtain subsets of features from big datasets. The algorithm decomposes the original dataset in blocks of instances to learn from them in the map phase; then, the reduce phase merges the obtained partial results into a final vector of feature weights, which allows a flexible application of the feature selection procedure using a threshold to determine the selected subset of features. The feature selection method is evaluated by using three well-known classifiers (SVM, Logistic Regression, and Naive Bayes) implemented within the Spark framework to address big data problems. In the experiments, datasets up to 67 millions of instances and up to 2000 attributes have been managed, showing that this is a suitable framework to perform evolutionary feature selection, improving both the classification accuracy and its runtime when dealing with big data problems.

The whole-brain functional connectivity (FC) pattern obtained from resting-state functional magnetic resonance imaging data are commonly applied to study neuropsychiatric conditions such as autism spectrum disorder (ASD) by using different machine learning models. Recent studies indicate that both hyper- and hypo- aberrant ASD-associated FCs were widely distributed throughout the entire brain rather than only in some specific brain regions. Deep neural networks (DNN) with multiple hidden layers have shown the ability to systematically extract lower-to-higher level information from high dimensional data across a series of neural hidden layers, significantly improving classification accuracy for such data. In this study, a DNN with a novel feature selection method (DNN-FS) is developed for the high dimensional whole-brain resting-state FC pattern classification of ASD patients vs. typical development (TD) controls. The feature selection method is able to help the DNN generate low dimensional high-quality representations of the whole-brain FC patterns by selecting features with high discriminating power from multiple trained sparse auto-encoders. For the comparison, a DNN without the feature selection method (DNN-woFS) is developed, and both of them are tested with different architectures (i.e., with different numbers of hidden layers/nodes). Results show that the best classification accuracy of 86.36% is generated by the DNN-FS approach with 3 hidden layers and 150 hidden nodes (3/150). Remarkably, DNN-FS outperforms DNN-woFS for all architectures studied. The most significant accuracy improvement was 9.09% with the 3/150 architecture. The method also outperforms other feature selection methods, e.g., two sample t-test and elastic net. In addition to improving the classification accuracy, a Fisher's score-based biomarker identification method based on the DNN is also developed, and used to identify 32 FCs related to ASD. These FCs come from or cross different pre-defined brain networks including the default-mode, cingulo-opercular, frontal-parietal, and cerebellum. Thirteen of them are statically significant between ASD and TD groups (two sample t-test p < 0.05) while 19 of them are not. The relationship between the statically significant FCs and the corresponding ASD behavior symptoms is discussed based on the literature and clinician's expert knowledge. Meanwhile, the potential reason of obtaining 19 FCs which are not statistically significant is also provided.

Biological datasets, such as gene expression data, often suffer from high dimensionality, containing numerous irrelevant or redundant features that can lead to overfitting and increased computational complexity. Effective feature selection is essential for reducing dimensionality, enhancing model performance, and improving interpretability. While deep neural networks, such as autoencoders, have shown promise in feature selection, their performance often diminishes when confronted with noisy data. To address these challenges, we propose a novel feature selection method that leverages multigranularity fuzzy autoencoders (FAEs). This approach integrates fuzzy theory with autoencoder models to effectively manage noise and outliers in data. The FAE method introduces a feature selection layer that approximates discrete feature selection using continuous probability distributions. To further enhance the discriminative power of the selected features, we incorporate a coarse-grained loss function designed to exploit clustering structures. In addition, intuitionistic fuzzy weights are applied to account for uncertainty by computing membership and nonmembership degrees for each sample, thereby mitigating the impact of noise and outliers. Test results validate the effectiveness of our approach, demonstrating significant improvements over existing feature selection techniques across 20 public datasets and a real-world schizophrenia dataset. These findings highlight the potential of our method to enhance classification accuracy and robustness, particularly in the context of schizophrenia research.

Feature extraction is viewed as an important preprocessing step for pattern recognition, machine learning and data mining. Neighborhood rough set (NRS) based feature extracting algorithm is able to delete most of the redundant and irrelevant features, which avoid the step of data discretization and hence decreased the information lost in preprocess. In this paper, we firstly introduce the basic definitions and operations of NRS, and propose a palmprint recognition method based on NRS. The neighborhood model is used to reduce the attributes and extract the recognition features. Experimental results on PolyU palmprint database demonstrate that the proposed method is effective and feasible for palmprint recognition.