Differential prioritization in feature selection and classifier aggregation for multiclass microarray datasets

The large number of genes in microarray data makes feature selection techniques more crucial than ever. From various ranking-based filter procedures to classifier-based wrapper techniques, many studies have devised their own flavor of feature selection techniques. Only a handful of the studies delved into the effect of redundancy in the predictor set on classification accuracy, and even fewer on the effect of varying the importance between relevance and redundancy. We present a filter-based feature selection technique which incorporates the three elements of relevance, redundancy and differential prioritization. With the aid of differential prioritization, our feature selection technique is capable of achieving better accuracies than those of previous studies, while using fewer genes in the predictor set. At the same time, the pitfalls of over-optimistic estimates of accuracy are avoided through the use of a more realistic evaluation procedure than the internal leave-one-out-cross-validation.KeywordsMolecular classificationmicroarray data analysisfeature selection

BackgroundAntimicrobial peptides (AMPs) are short peptides with diverse biological activities and playing a crucial role in various biological processes. Due to the widespread misuse of traditional antibiotics and the increasing resistance of microorganisms to these drugs, AMPs have emerged as a promising alternative. Consequently, the identification of AMPs has garnered significant research interest. Numerous computational methods based on machine learning algorithms have been developed to facilitate AMP recognition. However, some existing AMPs recognition models only focus on binary classification tasks or only identify the functional activity of a limited number of AMPs categories in multi-class classification tasks. To address this limitation, this study proposes a two-stage AMPs recognition model, iAMP-SeE.MethodsThe iAMP-SeE model extracts features from protein sequences using ESM2, employs a Convolutional Neural Network (CNN) module to capture local patterns from ESM features and utilizes a Bidirectional Long Short-Term Memory (BiLSTM) network to capture long-term dependencies. Furthermore, it incorporates Squeeze-and-Excitation (SE) and Efficient Channel Attention (ECA) mechanisms, which focus on global and local channel relationships, respectively. These two attention mechanisms are complementary, as they enhance features across various dimensions and granularities while simultaneously suppressing irrelevant or redundant features, thereby boosting the model’s performance. Additionally, to address the issue of imbalanced datasets, the Synthetic Minority Over-sampling Technique (SMOTE) is incorporated into the multi-classification task. This method balances the number of AMP categories and ensures that minority classes are not overlooked during model training.ResultsEvaluation across a range of classification thresholds demonstrated the stability of the model’s performance metrics in both binary and multi-class tasks. Furthermore, comparative experiments with existing AMP recognition models confirmed the superior performance of iAMP-SeE.ConclusionsRigorous experimental comparisons and ablation studies demonstrate the effectiveness of iAMP-SeE for both binary and multi-class AMP classification tasks. The source code is publicly available at: https://github.com/cqw0715/iAMP-SeE.git.

For many researchers working on feature selection techniques, finding an appropriate feature from the microarray dataset has turned into a bottleneck. Researchers often create feature selection approaches and algorithms with the goal of improving accuracy in microarray datasets. The main goal of this study is to present a variety of contemporary feature selection techniques, such as Filter, Wrapper, and Embedded methods proposed for microarray datasets to work on multi-class classification problems and different approaches to enhance the performance of learning algorithms, to address the imbalance issue in the data set, and to support research efforts on microarray dataset. This study is based on Critical Review Questions (CRQ) constructed using feature election methods described in the review methodology and applied to a microarray dataset. We discussed the analysed findings and future prospects of feature selection strategies for multi-class classification issues using microarray datasets, as well as prospective ways to speed up computing environment

Prediction of the stage of cancer plays an important role in planning the course of treatment and has been largely reliant on imaging tools which do not capture molecular events that cause cancer progression. Gene-expression data-based analyses are able to identify these events, allowing RNA-sequence and microarray cancer data to be used for cancer analyses. Breast cancer is the most common cancer worldwide, and is classified into four stages - stages 1, 2, 3, and 4 [2]. While machine learning models have previously been explored to perform stage classification with limited success, multi-class stage classification has not had significant progress. There is a need for improved multi-class classification models, such as by investigating deep learning models. Gene-expression-based cancer data is characterised by the small size of available datasets, class imbalance, and high dimensionality. Class balancing methods must be applied to the dataset. Since all the genes are not necessary for stage prediction, retaining only the necessary genes can improve classification accuracy. The breast cancer samples are to be classified into 4 classes of stages 1 to 4. Invasive ductal carcinoma breast cancer samples are obtained from The Cancer Genome Atlas (TCGA) and Molecular Taxonomy of Breast Cancer International Consortium (METABRIC) datasets and combined. Two class balancing techniques are explored, synthetic minority oversampling technique (SMOTE) and SMOTE followed by random undersampling. A hybrid feature selection pipeline is proposed, with three pipelines explored involving combinations of filter and embedded feature selection methods: Pipeline 1 - minimum-redundancy maximum-relevancy (mRMR) and correlation feature selection (CFS), Pipeline 2 - mRMR, mutual information (MI) and CFS, and Pipeline 3 - mRMR and support vector machine-recursive feature elimination (SVM-RFE). The classification is done using deep learning models, namely deep neural network, convolutional neural network, recurrent neural network, a modified deep neural network, and an AutoKeras generated model. Classification performance post class-balancing and various feature selection techniques show marked improvement over classification prior to feature selection. The best multiclass classification was found to be by a deep neural network post SMOTE and random undersampling, and feature selection using mRMR and recursive feature elimination, with a Cohen-Kappa score of 0.303 and a classification accuracy of 53.1%. For binary classification into early and late-stage cancer, the best performance is obtained by a modified deep neural network (DNN) post SMOTE and random undersampling, and feature selection using mRMR and recursive feature elimination, with an accuracy of 81.0% and a Cohen-Kappa score (CKS) of 0.280. This pipeline also showed improved multiclass classification performance on neuroblastoma cancer data, with a best area under the receiver operating characteristic (auROC) curve score of 0.872, as compared to 0.71 obtained in previous work, an improvement of 22.81%. The results and analysis reveal that feature selection techniques play a vital role in gene-expression data-based classification, and the proposed hybrid feature selection pipeline improves classification performance. Multi-class classification is possible using deep learning models, though further improvement particularly in late-stage classification is necessary and should be explored further.

In this work, we propose a novel Feature Selection framework, called Sparse-Modeling Based Approach for Class Specific Feature Selection (SMBA-CSFS), that simultaneously exploits the idea of Sparse Modeling and Class-Specific Feature Selection. Feature selection plays a key role in several fields (e.g., computational biology), making it possible to treat models with fewer variables which, in turn, are easier to explain, by providing valuable insights on the importance of their role, and might speed the experimental validation up. Unfortunately, also corroborated by the no free lunch theorems, none of the approaches in literature is the most apt to detect the optimal feature subset for building a final model, thus it still represents a challenge. The proposed feature selection procedure conceives a two steps approach: (a) a sparse modeling-based learning technique is first used to find the best subset of features, for each class of a training set; (b) the discovered feature subsets are then fed to a class-specific feature selection scheme, in order to assess the effectiveness of the selected features in classification tasks. To this end, an ensemble of classifiers is built, where each classifier is trained on its own feature subset discovered in the previous phase, and a proper decision rule is adopted to compute the ensemble responses. In order to evaluate the performance of the proposed method, extensive experiments have been performed on publicly available datasets, in particular belonging to the computational biology field where feature selection is indispensable: the acute lymphoblastic leukemia and acute myeloid leukemia, the human carcinomas, the human lung carcinomas, the diffuse large B-cell lymphoma, and the malignant glioma. SMBA-CSFS is able to identify/retrieve the most representative features that maximize the classification accuracy. With top 20 and 80 features, SMBA-CSFS exhibits a promising performance when compared to its competitors from literature, on all considered datasets, especially those with a higher number of features. Experiments show that the proposed approach might outperform the state-of-the-art methods when the number of features is high. For this reason, the introduced approach proposes itself for selection and classification of data with a large number of features and classes.

In many real-world applications, it is common to have uneven number of examples among multiple classes. The data imbalance, however, usually complicates the learning process, especially for the minority classes, and results in deteriorated performance. Boosting methods were proposed to handle the imbalance problem. These methods need elongated training time and require diversity among the classifiers of the ensemble to achieve improved performance. Additionally, extending the boosting method to handle multi-class data sets is not straightforward. Examples of applications that suffer from imbalanced multi-class data can be found in face recognition, where tens of classes exist, and in capsule endoscopy, which suffers massive imbalance between the classes. This dissertation introduces RegBoost, a new boosting framework to address the imbalanced, multi-class problems. This method applies a weighted stratified sampling technique and incorporates a regularization term that accommodates multi-class data sets and automatically determines the error bound of each base classifier. The regularization parameter penalizes the classifier when it misclassifies instances that were correctly classified in the previous iteration. The parameter additionally reduces the bias towards majority classes. Experiments are conducted using 12 diverse data sets with moderate to high imbalance ratios. The results demonstrate superior performance of the proposed method compared to several state-of-the-art algorithms for imbalanced, multi-class classification problems. More importantly, the sensitivity improvement of the minority classes using RegBoost is accompanied with the improvement of the overall accuracy for all classes. With unpredictability regularization, a diverse group of classifiers are created and the maximum accuracy improvement reaches above 24%. Using stratified undersampling, RegBoost exhibits the best efficiency. The reduction in computational cost is significant reaching above 50%. As the volume of training data increase, the gain of efficiency with the proposed method becomes more significant.

In microarray experiments, the sample size is considerably smaller than that of the feature size, thereby imposing the curse of dimensionality problem. To resolve this issue, evolutionary algorithms are often utilized. In this paper, a novel framework for feature selection and classification of the microarray data is presented. Initially, a statistical filter, namely ANOVA, is used to select the relevant genes (features) from the original set of genes. Then, an evolutionary wrapper-based approach utilizing the principles of enhanced Jaya (EJaya) algorithm and forest optimization algorithm (FOA) is proposed to find the optimal set of genes from the previously selected genes. The main objective of using EJaya is to tune the two important parameters, namely local seeding changes and global seeding changes of FOA. During the selection of the optimal set of genes, support vector machine is employed as a classifier to classify the microarray data. To perform a comprehensive experimental study, the proposed method is tested on both binary-class and multi-class microarray datasets. From the extensive result analysis, it has been observed that the proposed technique achieves better classification accuracy with considerably less number of features than that of the benchmark schemes.

Many evolutionary computation (EC) methods have been used to solve feature selection problems and they perform well on most small-scale feature selection problems. However, as the dimensionality of feature selection problems increases, the solution space increases exponentially. Meanwhile, there are more irrelevant features than relevant features in datasets, which leads to many local optima in the huge solution space. Therefore, the existing EC methods still suffer from the problem of stagnation in local optima on large-scale feature selection problems. Furthermore, large-scale feature selection problems with different datasets may have different properties. Thus, it may be of low performance to solve different large-scale feature selection problems with an existing EC method that has only one candidate solution generation strategy (CSGS). In addition, it is time-consuming to find a suitable EC method and corresponding suitable parameter values for a given large-scale feature selection problem if we want to solve it effectively and efficiently. In this article, we propose a self-adaptive particle swarm optimization (SaPSO) algorithm for feature selection, particularly for large-scale feature selection. First, an encoding scheme for the feature selection problem is employed in the SaPSO. Second, three important issues related to self-adaptive algorithms are investigated. After that, the SaPSO algorithm with a typical self-adaptive mechanism is proposed. The experimental results on 12 datasets show that the solution size obtained by the SaPSO algorithm is smaller than its EC counterparts on all datasets. The SaPSO algorithm performs better than its non-EC and EC counterparts in terms of classification accuracy not only on most training sets but also on most test sets. Furthermore, as the dimensionality of the feature selection problem increases, the advantages of SaPSO become more prominent. This highlights that the SaPSO algorithm is suitable for solving feature selection problems, particularly large-scale feature selection problems.

Strabismus, or eye misalignment, is a common condition affecting individuals of all ages. Early detection and accurate classification are essential for proper treatment and avoiding long-term complications. This research presents a new deep-learning-based approach for automatically identifying and classifying strabismus from facial images. The proposed methodology leverages Convolutional Neural Networks (CNNs) to achieve high accuracy in both binary (strabismus vs. normal) and multi-class (eight-class deviation angle for esotropia and exotropia) classification tasks. The dataset for binary classification consisted of 4,257 facial images, including 1,599 normal cases and 2,658 strabismus cases, while the multi-class classification involved 480 strabismic and 142 non-strabismic images. These images were labeled based on ophthalmologist measurements using the Alternate Prism Cover Test (APCT) or the Modified Krimsky Test (MK). Five-fold cross-validation was employed, and performance was evaluated using sensitivity, accuracy, F1-score, and recall metrics. The proposed deep learning model achieved an accuracy of 86.38% for binary classification and 92.7% for multi-class classification. These results demonstrate the potential of our approach to assist healthcare professionals in early strabismus detection and treatment planning, ultimately improving patient outcomes.

With the advancement of industrial digitization, utilizing large datasets for model training to boost performance is a pivotal technical approach for industry progress. However, raw training datasets often contain abundant redundant features, which increase model training's computational cost and impair generalization ability. To tackle this, this study proposes the bionic ABCCOA algorithm, an enhanced version of the bionic Coati Optimization Algorithm (COA), to improve redundant feature elimination in datasets. To address the bionic COA's inadequate global search performance in feature selection (FS) problems, leading to lower classification accuracy, an adaptive search strategy is introduced. This strategy combines individual learning capability and the learnability of disparities, enhancing global exploration. For the imbalance between the exploration and exploitation phases in the bionic COA algorithm when solving FS problems, which often traps it in suboptimal feature subsets, a balancing factor is proposed. By integrating phase control and dynamic adjustability, a good balance between the two phases is achieved, reducing the likelihood of getting stuck in suboptimal subsets. Additionally, to counter the bionic COA's insufficient local exploitation performance in FS problems, increasing classification error rates, a centroid guidance strategy is presented. By combining population centroid guidance and fractional-order historical memory, the algorithm lowers the classification error rate of feature subsets and speeds up convergence. The bionic ABCCOA algorithm was tested on the CEC2020 test functions and engineering problem, achieving an over 90% optimization success rate and faster convergence, confirming its efficiency. Applied to 27 FS problems, it outperformed comparative algorithms in best, average, and worst fitness function values, classification accuracy, feature subset size, and running time, proving it an efficient and robust FS algorithm.

In classification, feature selection (FS) removes irrelevant and redundant features to improve the classification performance. Most of the existing FS approaches are designed specifically for single-label classification, wherein each instance is associated to a single class label. However, many real-world problems have more than one class label for each instance, also known as multi-label classification (MLC). Label-to-label interactions in MLC information presents a new challenge for FS, whereas single-label FS methods typically only consider feature-to-feature (redundancy) and feature-to-label (relevancy) information. Sparsity-based FS is an efficient and effective method for MLC which has demonstrated its ability in considering label-to-label interactions. However, existing sparsity-based approaches can be trapped at local optima due to their gradient-based search. In addition, they cannot consider different trade-off between the two objectives of FS: to minimise the number of features and to maximise the classification performance. Evolutionary multi-objective optimisation (EMO) can address the above issues, but EMO has not been applied to achieve sparsity-based FS for MLC. This paper proposes a novel sparsity-based representation, objective function, and local search method to achieve sparsity-based EMO FS for MLC. The experimental results indicate improved efficiency and effectiveness in the MLC performance in comparison to existing state-of-the-art multi-label feature selection methods.

In multi-label text classification, the central aim is to associate an array of descriptive labels for a better understanding of the text. There are three main challenges in doing multi-label text classification (i) a large number of text (input) features, (ii) the underlying implicit relationship between input features and output labels, and (iii) an implicit inter-label dependency. In traditional approaches to multi-label classification, these problems are not being addressed collectively. A feature selection strategy that inherently uses local features to discriminate a class and similarly global features that can distinctly separate classes can be very effective for multi-label classification. In this research, we perform a feature selection and ranking strategy based on local and global features. A Naïve Bayes classifier is being used using a combination of these two -feature sets, it is compared with the baseline implemented with the term frequency-inverse document frequency (TF-IDF). A series of experiments have been carried out on standard multi-label text datasets, using evaluation metrics like Hamming loss, Subset Accuracy and Micro/Macro F1 scores, and encouraging results are obtained.

A feature selection method for multimodal multispectral LiDAR sensing

This paper presents a novel procedure to apply in a sequential way two data preparation techniques from a different nature such as data cleansing and feature selection. For the former we have experienced with a partial removal of outliers via inter-quartile range whereas for the latter we have chosen relevant attributes with two widespread feature subset selectors like CFS (Correlation-based Feature Selection) and CNS (Consistency-based Feature Selection), which are founded on correlation and consistency measures, respectively. Empirical results on seven difficult binary and multi-class data sets, that is, with a test error rate of at least a 10%, according to accuracy, with C4.5 or 1-nearest neighbour classifiers without any kind of prior data pre-processing are outlined. Non-parametric statistical tests assert that the meeting of the aforementioned two data preparation strategies using a correlation measure for feature selection with C4.5 algorithm is significant better, measured with roc measure, than the single application of the data cleansing approach. Last but not least, a weak and not very powerful learner like PART achieved promising results with the new proposal based on a consistency measure and is able to compete with the best configuration of C4.5. To sum up, bearing in mind the new approach, for roc measure PART classifier with a consistency metric behaves slightly better than C4.5 and a correlation measure.

FAMCDM: A fusion approach of MCDM methods to rank multiclass classification algorithms