Towards a Memetic Feature Selection Paradigm [Application Notes

In this paper, the bat-inspired algorithm (BA) is tolerated to gene selection for cancer classification using microarray datasets. Microarray data consists of irrelevant, redundant, and noisy genes. Gene selection problem is tackled by determining the most informative genes taken from microarray data to accurately diagnose the cancer disease. Gene selection problem is widely solved by optimisation algorithms. BA is a recent swarm-based algorithm, which imitates the echolocation system of bat individuals. It has been successfully applied to several optimisation problems. Gene selection is tackled by combining two stages, namely, filter stage, which uses Minimum Redundancy Maximum Relevancy (MRMR) method; and wrapper stage, which uses BA and SVM. To test the accuracy performance of the proposed method, ten microarray datasets were used. For comparative evaluation, the proposed method was compared with popular gene selection methods. The proposed method achieves comparable results of some datasets and produced new results for one dataset.

BackgroundMicroarray technology is widely used in cancer diagnosis. Successfully identifying gene biomarkers will significantly help to classify different cancer types and improve the prediction accuracy. The regularization approach is one of the effective methods for gene selection in microarray data, which generally contain a large number of genes and have a small number of samples. In recent years, various approaches have been developed for gene selection of microarray data. Generally, they are divided into three categories: filter, wrapper and embedded methods. Regularization methods are an important embedded technique and perform both continuous shrinkage and automatic gene selection simultaneously. Recently, there is growing interest in applying the regularization techniques in gene selection. The popular regularization technique is Lasso (L1), and many L1 type regularization terms have been proposed in the recent years. Theoretically, the Lq type regularization with the lower value of q would lead to better solutions with more sparsity. Moreover, the L1/2 regularization can be taken as a representative of Lq (0 <q < 1) regularizations and has been demonstrated many attractive properties.ResultsIn this work, we investigate a sparse logistic regression with the L1/2 penalty for gene selection in cancer classification problems, and propose a coordinate descent algorithm with a new univariate half thresholding operator to solve the L1/2 penalized logistic regression. Experimental results on artificial and microarray data demonstrate the effectiveness of our proposed approach compared with other regularization methods. Especially, for 4 publicly available gene expression datasets, the L1/2 regularization method achieved its success using only about 2 to 14 predictors (genes), compared to about 6 to 38 genes for ordinary L1 and elastic net regularization approaches.ConclusionsFrom our evaluations, it is clear that the sparse logistic regression with the L1/2 penalty achieves higher classification accuracy than those of ordinary L1 and elastic net regularization approaches, while fewer but informative genes are selected. This is an important consideration for screening and diagnostic applications, where the goal is often to develop an accurate test using as few features as possible in order to control cost. Therefore, the sparse logistic regression with the L1/2 penalty is effective technique for gene selection in real classification problems.

Cancer remains a leading cause of death worldwide; the World Health Organization (WHO) reports that there have been nearly 10 million cancer-related deaths in recent years, with breast cancer affecting over 2.1 million women annually on a global scale, posing significant challenges for early detection and diagnosis. Gene selection, using DNA microarray data, is crucial for reducing the presence of less informative genes and ensuring the selection of genes relevant to disease diagnosis. Cancer classification involves identifying the type of cancer and determining the extent of tumor growth and spread. This research focuses on improving gene selection for cancer classification using the XGBoost classifier, an efficient open-source implementation of the gradient-boosted trees algorithm. The primary goal is to enhance the performance of gene selection, enabling timely and appropriate treatments for cancer patients, as early detection is vital for ensuring a full recovery. Additionally, this research aims to reduce the time and expense associated with gene selection for cancer classification while increasing classification accuracy. The proposed method achieved an accuracy of approximately 93%, with precision, recall, and F1-score values of 93%, 87%, and 90%, respectively. The study highlights the potential of the XGBoost classifier in optimizing gene selection and improving diagnostic processes. Future work will focus on enhancing the accuracy of gene selection for cancer classification and reducing the number of irrelevant genes before proceeding to subsequent processes. This approach holds promises for streamlining the diagnostic process, improving patient outcomes, and offering significant benefits in timely cancer treatment.

A new hybrid filter–wrapper feature selection method for clustering based on ranking

Design of a differentiable L-1 norm for pattern recognition and machine learning

In recent years, gene selection for cancer classification based on the expression of a small number of gene biomarkers has been the subject of much research in genetics and molecular biology. The successful identification of gene biomarkers will help in the classification of different types of cancer and improve the prediction accuracy. Recently, regularized logistic regression using the L 1 regularization has been successfully applied in high-dimensional cancer classification to tackle both the estimation of gene coefficients and the simultaneous performance of gene selection. However, the L 1 has a biased gene selection and dose not have the oracle property. To address these problems, we investigate L 1 / 2 regularized logistic regression for gene selection in cancer classification. Experimental results on three DNA microarray datasets demonstrate that our proposed method outperforms other commonly used sparse methods ( L 1 and L E N ) in terms of classification performance.

Feature selection is a commonly addressed problem in classification. In gene expression-based cancer classification, a large number of genes in conjunction with a small number of samples makes the gene selection problem more important but also more challenging. Support vector machine as a popular classification algorithm, has been successfully used in SVM-RFE method for gene selection. This paper proposes a variant of SVM-RFE to do gene selection for cancer classification with expression data. Multiple support vector machine classifiers from a leave-one-out procedure are used to compute the feature ranking scores. The numerical experiments also show the good and stable performance of the proposed method.

Objective:Cancer diagnosis is one of the most vital emerging clinical applications of microarray data. Due to the high dimensionality, gene selection is an important step for improving expression data classification performance. There is therefore a need for effective methods to select informative genes for prediction and diagnosis of cancer. The main objective of this research was to derive a heuristic approach to select highly informative genes.Methods:A metaheuristic approach with a Genetic Algorithm with Levy Flight (GA-LV) was applied for classification of cancer genes in microarrays. The experimental results were analyzed with five major cancer gene expression benchmark datasets.Result:GA-LV proved superior to GA and statistical approaches, with 100% accuracy for the dataset for Leukemia, Lung and Lymphoma. For Prostate and Colon datasets the GA-LV was 99.5% and 99.2% accurate, respectively.Conclusion:The experimental results show that the proposed approach is suitable for effective gene selection with all benchmark datasets, removing irrelevant and redundant genes to improve classification accuracy.

In today’s digital environment, the pattern is everywhere. It is present in many aspects of our daily life. Algorithms can be used to detect or mathematically observe a pattern physically. Vector feature values represent a pattern in the digital world. With the advent of artificial intelligence (AI) techniques in the recent era, there are so many machine learning (ML) and deep learning (DL) models that have been developed. ML is the branch of AI, which can perform tasks like data analysis, analytical model building, and classification without being explicitly programmed. DL is the subset of ML in AI. In DL, mostly artificial neural networks (ANNs) are utilized. It has the capability of work based upon unsupervised learning from the data that is unlabeled and unstructured. Using these DL and ML models, extract the meaningful features from the given image or video is known as pattern recognition (PR). PR is used in many engineering applications such as computer vision, natural language processing, character recognition, robotics, speech recognition, and so on. It is also used in many medical image processing applications and telemedicine. The present work discussed the pattern recognition problem and its various stages in detail. In addition to that, the application of deep learning and machine learning in pattern recognition is also explained briefly.

BackgroundSuccessfully modeling high-dimensional data involving thousands of variables is challenging. This is especially true for gene expression profiling experiments, given the large number of genes involved and the small number of samples available. Random Forests (RF) is a popular and widely used approach to feature selection for such "small n, large p problems." However, Random Forests suffers from instability, especially in the presence of noisy and/or unbalanced inputs.ResultsWe present RKNN-FS, an innovative feature selection procedure for "small n, large p problems." RKNN-FS is based on Random KNN (RKNN), a novel generalization of traditional nearest-neighbor modeling. RKNN consists of an ensemble of base k-nearest neighbor models, each constructed from a random subset of the input variables. To rank the importance of the variables, we define a criterion on the RKNN framework, using the notion of support. A two-stage backward model selection method is then developed based on this criterion. Empirical results on microarray data sets with thousands of variables and relatively few samples show that RKNN-FS is an effective feature selection approach for high-dimensional data. RKNN is similar to Random Forests in terms of classification accuracy without feature selection. However, RKNN provides much better classification accuracy than RF when each method incorporates a feature-selection step. Our results show that RKNN is significantly more stable and more robust than Random Forests for feature selection when the input data are noisy and/or unbalanced. Further, RKNN-FS is much faster than the Random Forests feature selection method (RF-FS), especially for large scale problems, involving thousands of variables and multiple classes.ConclusionsGiven the superiority of Random KNN in classification performance when compared with Random Forests, RKNN-FS's simplicity and ease of implementation, and its superiority in speed and stability, we propose RKNN-FS as a faster and more stable alternative to Random Forests in classification problems involving feature selection for high-dimensional datasets.

Low-rank learning for feature selection in multi-label classification

In recent years, the exponential growth of both structured and unstructured data has highlighted the importance of efficient knowledge discovery techniques. Among these, image mining and data mining play pivotal roles in extracting meaningful patterns and insights from different forms of information. At the same time, data mining focuses on structured datasets such as databases and transaction records, while image mining deals with unstructured visual content, requiring advanced feature extraction, pattern recognition, and machine learning methods. This paper presents a comparative analysis of image mining and data mining techniques, emphasising their methodologies, applications, and challenges. The comparison explores key dimensions, including data preprocessing, feature selection, algorithmic approaches, and application domains such as healthcare, security, business intelligence, and multimedia. Experimental results show clustering accuracy of 87% for data mining and image classification accuracy of 92% for image mining, highlighting the effectiveness of specialised approaches. The study highlights both the similarities and differences in knowledge discovery processes, demonstrating how integrating image mining and data mining can enhance decision-making in diverse fields. The findings provide a comprehensive understanding of their complementary roles, offering valuable insights for researchers and practitioners aiming to develop hybrid approaches to knowledge discovery.

Feature selection plays an important role in the fields of pattern recognition, data mining and machine learning. Rough set method is one of effective methods for feature selection, which can preserve the meaning of the features. Presently ant colony optimization (ACO) has been successfully applied to rough set-based feature selection, however, it has the limitations of many control parameters, premature convergence and stagnation. In this paper, we propose a novel discrete artificial bee colony algorithm to overcome these drawbacks. In our algorithm, a search mechanism centered with feature core is introduced to avoid blindly searching, in which the values of corresponding bits of these features in feature core are unchanged during the iterative process. Meanwhile, Tabu search is adopted to avoid falling into local optima for every employed bee who can not select the solution of the Tabu list in each generation. In this work, our algorithm is applied to eight datasets from UCI database and the results produced by our algorithm and some rough set-based feature selection algorithms have been compared. The results show that our algorithm outperforms the other algorithms.

Gene selection and classification using Taguchi chaotic binary particle swarm optimization

With the continuous development of Internet technology, data gradually present a complicated and high-dimensional trend. These high-dimensional data have a large number of redundant features and irrelevant features, which bring great challenges to the existing machine learning algorithms. Feature selection is one of the important research topics in the fields of machine learning, pattern recognition and data mining, and it is also an important means in the data preprocessing stage. Feature selection is to look for the optimal feature subset from the original feature set, which would improve the classification accuracy and reduce the machine learning time. The traditional feature selection algorithm tends to ignore the kind of feature which has a weak distinguishing capacity as a monomer, whereas the feature group’s distinguishing capacity is strong. Therefore, a new dynamic interaction feature selection (DIFS) algorithm is proposed in this paper. Initially, under the theoretical framework of interactive information, it redefines the relevance, irrelevance and redundancy of the features. Secondly, it offers the computational formulas for calculating interactive information. Finally, under the eleven data sets of UCI and three different classifiers, namely, KNN, SVM and C4.5, the DIFS algorithm increases the classification accuracy of the FullSet by 3.2848% and averagely decreases the number of features selected by 15.137. Hence, the DIFS algorithm can not only identify the relevance feature effectively, but also identify the irrelevant and redundant features. Moreover, it can effectively improve the classification accuracy of the data sets and reduce the feature dimensions of the data sets.