A class-specific metaheuristic technique for explainable relevant feature selection

Introduction: The early and accurate detection of COVID-19 remains a life-threatening challenge in medical analysis. Machine learning is used for predicting disease outcomes based on clinical parameters. This analysis proposes a comparative analysis of feature selection method and classification techniques to enhance COVID-19 detection accuracy using blood biomarkers. We used a pensourse dataset of 1,724 cases, including 35 features. To improve the model performance data preprocessing process included outlier handling, normalization, and transformation techniques to improve model performance. To identify the relevant features, we employed the three-feature selection methods Chi-Square test, Pearson correlation coefficient, and Random Forest. The model prediction accuracy was enhanced using a stacking ensemble classication techniques. The machine learning based classification models effectively predicted COVID-19 infectious disease using blood biomarkers with optimized feature selection techniques. Objectives: To enhance the accuracy of COVID-19 prediction using machine learning techniques by applying feature selection and classification techniques on blood biomarkers. Methods: The comparative analysis utilized a publicly available dataset containing 1,724 cases with 35 attributes. Data preprocessing involved outlier handling, normalization, and transformation techniques. Employed Chi-Square test, Pearson correlation coefficient, and Random Forest feature selection techniques. Stacking ensemble classification algorithm was utilized for the better performance of a model. Results: The classification models demonstrated efficiency in predicting COVID-19 using blood biomarkers. Optimized feature selection significantly improved predictive accuracy, highlighting the importance of selecting relevant features for model performance enhancement. Conclusions: This study showcases the potential of ML-driven approaches for COVID-19 detection, emphasizing the role of feature selection in improving classification accuracy. The findings contribute to the advancement of diagnostic tools, offering a data-driven solution for rapid and reliable COVID-19 screening.

Prediction of the stage of cancer plays an important role in planning the course of treatment and has been largely reliant on imaging tools which do not capture molecular events that cause cancer progression. Gene-expression data-based analyses are able to identify these events, allowing RNA-sequence and microarray cancer data to be used for cancer analyses. Breast cancer is the most common cancer worldwide, and is classified into four stages - stages 1, 2, 3, and 4 [2]. While machine learning models have previously been explored to perform stage classification with limited success, multi-class stage classification has not had significant progress. There is a need for improved multi-class classification models, such as by investigating deep learning models. Gene-expression-based cancer data is characterised by the small size of available datasets, class imbalance, and high dimensionality. Class balancing methods must be applied to the dataset. Since all the genes are not necessary for stage prediction, retaining only the necessary genes can improve classification accuracy. The breast cancer samples are to be classified into 4 classes of stages 1 to 4. Invasive ductal carcinoma breast cancer samples are obtained from The Cancer Genome Atlas (TCGA) and Molecular Taxonomy of Breast Cancer International Consortium (METABRIC) datasets and combined. Two class balancing techniques are explored, synthetic minority oversampling technique (SMOTE) and SMOTE followed by random undersampling. A hybrid feature selection pipeline is proposed, with three pipelines explored involving combinations of filter and embedded feature selection methods: Pipeline 1 - minimum-redundancy maximum-relevancy (mRMR) and correlation feature selection (CFS), Pipeline 2 - mRMR, mutual information (MI) and CFS, and Pipeline 3 - mRMR and support vector machine-recursive feature elimination (SVM-RFE). The classification is done using deep learning models, namely deep neural network, convolutional neural network, recurrent neural network, a modified deep neural network, and an AutoKeras generated model. Classification performance post class-balancing and various feature selection techniques show marked improvement over classification prior to feature selection. The best multiclass classification was found to be by a deep neural network post SMOTE and random undersampling, and feature selection using mRMR and recursive feature elimination, with a Cohen-Kappa score of 0.303 and a classification accuracy of 53.1%. For binary classification into early and late-stage cancer, the best performance is obtained by a modified deep neural network (DNN) post SMOTE and random undersampling, and feature selection using mRMR and recursive feature elimination, with an accuracy of 81.0% and a Cohen-Kappa score (CKS) of 0.280. This pipeline also showed improved multiclass classification performance on neuroblastoma cancer data, with a best area under the receiver operating characteristic (auROC) curve score of 0.872, as compared to 0.71 obtained in previous work, an improvement of 22.81%. The results and analysis reveal that feature selection techniques play a vital role in gene-expression data-based classification, and the proposed hybrid feature selection pipeline improves classification performance. Multi-class classification is possible using deep learning models, though further improvement particularly in late-stage classification is necessary and should be explored further.

In this work, we propose a novel Feature Selection framework, called Sparse-Modeling Based Approach for Class Specific Feature Selection (SMBA-CSFS), that simultaneously exploits the idea of Sparse Modeling and Class-Specific Feature Selection. Feature selection plays a key role in several fields (e.g., computational biology), making it possible to treat models with fewer variables which, in turn, are easier to explain, by providing valuable insights on the importance of their role, and might speed the experimental validation up. Unfortunately, also corroborated by the no free lunch theorems, none of the approaches in literature is the most apt to detect the optimal feature subset for building a final model, thus it still represents a challenge. The proposed feature selection procedure conceives a two steps approach: (a) a sparse modeling-based learning technique is first used to find the best subset of features, for each class of a training set; (b) the discovered feature subsets are then fed to a class-specific feature selection scheme, in order to assess the effectiveness of the selected features in classification tasks. To this end, an ensemble of classifiers is built, where each classifier is trained on its own feature subset discovered in the previous phase, and a proper decision rule is adopted to compute the ensemble responses. In order to evaluate the performance of the proposed method, extensive experiments have been performed on publicly available datasets, in particular belonging to the computational biology field where feature selection is indispensable: the acute lymphoblastic leukemia and acute myeloid leukemia, the human carcinomas, the human lung carcinomas, the diffuse large B-cell lymphoma, and the malignant glioma. SMBA-CSFS is able to identify/retrieve the most representative features that maximize the classification accuracy. With top 20 and 80 features, SMBA-CSFS exhibits a promising performance when compared to its competitors from literature, on all considered datasets, especially those with a higher number of features. Experiments show that the proposed approach might outperform the state-of-the-art methods when the number of features is high. For this reason, the introduced approach proposes itself for selection and classification of data with a large number of features and classes.

High-dimensional datasets can lead to overfitting and computationally expensive model building on machine learning. This study uses a dimensionality reduction technique, namely feature selection techniques, to overcome these problems. Five feature selection methods were used, i.e., Chi-Square (CS), Information Gain (IG), Genetic Algorithm (GA), Particle Swarm Optimization (PSO), and Least Absolute Shrinkage and Selection Operator (LASSO), and three classifier methods viz. Naïve Bayes, Extreme Gradient Boosting (XGB), and RF Classifier. The dataset used is the Heart Attack Analysis & Prediction Dataset. In this study, three scenarios of the best feature selection were carried out, namely: 1. selection of the best feature using a specific feature selection, 2. the intersection of selection of the best feature from the same category, 3. the intersection of selection of the best feature from the five proposed feature selection methods. The performance model is measured using accuracy, precision, recall, f1-score, AUC, and training time. This study reveals that feature selection is very effective in improving the performance of prediction models. Based on the experiment results, the best feature selection is CS and IG in the Filter Category with the XGB model. The best feature selected improved the performance of accuracy, precision, recall, f1-score, and AUC, i.e., 1.7%, 1%, 2.3%, 1.6%, and 0.2%, respectively. Meanwhile, training time requirements decreased by 23.5%. Feature selection with specific techniques performs better than feature selection by selecting the best features from the same category feature selection technique or various other feature selection methods.

Classification performance is adversely impacted by noisy data .Selecting features relevant to the problem is thus a critical step in classification and difficult to achieve accurate solution, especially when applied to a large data set. In this article, we propose a novel filter-based floating search technique for feature selection to select an optimal set of features for classification purposes. A genetic algorithm is utilized to increase the quality of features selected at each iteration. A criterion function is applied to choose relevant and high-quality features which can improve classification accuracy. The method is evaluated using 20 standard machine learning datasets of various sizes and complexities. Experimental results with the datasets show that the proposed method is effective and performs well in comparison with previously reported techniques.

Introduction: This paper presents a novel approach for classifying different types of steel rods using a combination of pre-trained convolutional neural networks (CNNs) and fuzzy logic systems. We utilize VGG16, ResNet50, and InceptionV3 models to extract features from grayscale images of steel rods, which are then classified using Support Vector Machines (SVM). Furthermore, we incorporate a fuzzy logic system to improve the final classification accuracy by combining the individual accuracies of each CNN model. Experimental results demonstrate the effectiveness of our approach in achieving high classification accuracy compared to traditional methods. Objectives: This research presents a novel multi-model approach for steel rod classification utilizing three distinct fuzzy logic systems: Fuzzy Recurrent Neural Network (RNN), Fuzzy Convolutional Neural Network (CNN), and Fuzzy Neural Network. The proposed system addresses the critical challenge of accurate defect detection and classification in steel manufacturing processes. By combining these complementary approaches, we achieve enhanced feature extraction capabilities and improved classification accuracy. Experimental results demonstrate that our multi-model system achieves a classification accuracy of 97.8%, surpassing single-model approaches by an average of 8.2%. The system successfully identifies surface defects, structural anomalies, and dimensional variations with high precision, making it suitable for real-time industrial applications. Methods: This paper introduces a novel approach for the feature extraction and classification of steel rods by integrating multiple models and utilizing fuzzy logic systems. The methodology leverages pre-trained convolutional neural networks (CNNs) for extracting comprehensive feature representations from steel rod images. These extracted features are then combined and integrated using fuzzy logic to enhance classification accuracy and reliability. By addressing the limitations of traditional manual inspection methods and single-model approaches, this integrated multi-model system offers a robust solution for real-time industrial implementation. The proposed system aims to improve the efficiency and effectiveness of quality control processes in steel rod manufacturing, ensuring consistent and accurate classification even under varying conditions and complexities. This innovative approach paves the way for more advanced and adaptable automated inspection systems in the industry. Results: The paper aims to develop an integrated multi-model system that combines three distinct fuzzy logic approaches to enhance feature extraction capabilities through complementary methodologies. By improving the classification accuracy and reliability, the system addresses the limitations of existing methods. The goal is to create a robust and adaptable system that is suitable for real-time industrial implementation for steel rod analysis and classification. This comprehensive approach leverages the strengths of multiple models and fuzzy logic techniques, ensuring accurate and efficient classification processes, even under complex and varying conditions. The proposed multi model system is designed to meet the stringent quality control requirements of industrial applications, ultimately improving productivity and product quality through reliable automated inspection and classification for steel rod with accuracy of 97.80%, precision of 97.30% and recall 97.50%. Conclusions: This paper presents a novel approach for the classification of steel rods using a combination of pre-trained CNN models and fuzzy logic systems. The experimental results demonstrate the effectiveness of our method in achieving high classification accuracy. This research demonstrates the effectiveness of a multi-model fuzzy logic approach for steel rod classification. The system achieves significant improvements over single-model approaches while maintaining real-time processing capabilities. Our approach provides a foundation for further research and development in automated industrial image classification.

Text mining is a special case of data mining which explore unstructured or semi-structured text documents, to establish valuable patterns and rules that indicate trends and significant features about specific topics. Text mining has been in pattern recognition, predictive studies, sentiment analysis and statistical theories in many areas of research, medicine, financial analysis, social life analysis, and business intelligence. Text mining uses concept of natural language processing and machine learning. Machine learning algorithms have been used and reported to give great results, but their performance of machine learning algorithms is affected by factors such as dataset domain, number of classes, length of the corpus, and feature selection techniques used. Redundant attribute affects the performance of the classification algorithm, but this can be reduced by using different feature selection techniques and dimensionality reduction techniques. Feature selection is a data preprocessing step that chooses a subset of input variable while eliminating features with little or no predictive information. Feature selection techniques are Information gain, Term Frequency, Term Frequency-Inverse document frequency, Mutual Information, and Chi-Square, which can use a filters, wrappers, or embedded approaches. To get the most value from machine learning, pairing the best algorithms with the right tools and processes is necessary. Little research has been done on the effect of feature selection techniques on classification accuracy for pairing of these algorithms with the best feature selection techniques for optimal results. In this research, a text classification experiment was conducted using incident management dataset, where incidents were classified into their resolver groups. Support vector machine (SVM), K-Nearest Neighbors (KNN), Naïve Bayes (NB) and Decision tree (DT) machine learning algorithms were examined. Filtering approach was used on the feature selection techniques, with different ranking indices applied for optimal feature set and classification accuracy results analyzed. The classification accuracy results obtained using TF were, 88% for SVM, 70% for NB, 79% for Decision tree, and KNN had 55%, while Boolean registered 90%, 83%, 82% and 75%, for SVM, NB, DT, and KNN respectively. TF-IDF, had 91%, 83%, 76%, and 56% for SVM, NB, DT, and KNN respectively. The results showed that algorithm performance is affected by feature selection technique applied. SVM performed best, followed by DT, KNN and finally NB. In conclusion, presence of noisy data leads to poor learning performance and increases the computational time. The classifiers performed differently depending on the feature selection technique applied. For optimal results, the classifier that performed best together with the feature selection technique with the best feature subset should be applied for all types of data for accurate classification performance. Keywords: Text Classification, Supervised Machine Learning, Feature Selection DOI : 10.7176/JIEA/9-3-06 Publication date :May 31 st 2019

A novel binary greater cane rat algorithm for feature selection

One of the important practical applications of computer vision techniques is computer-aided diagnosis (CAD) in medical imaging. It has been shown that CAD can improve the accuracy of breast cancer detection and characterization by radiologists on mammograms. In this chapter, we discuss an important step — feature selection — in classifier design for CAD algorithms. Feature selection reduces the dimensionality of an available feature space and is therefore often used to prevent over-parameterization of a classifier. Many feature selection techniques have been proposed in the literature. We will illustrate the usefulness of genetic algorithms (GAs) for feature selection by comparing GA with a commonly used sequential selection method. A brief introduction to the GA is given and several examples using GA feature selection for the characterization of mammographic lesions are discussed. The examples illustrate the design of a fitness function for optimizing classification accuracy in terms of the receiver operating characteristics of the classifier, the dependence of GA performance on its evolution parameters, and the design of a fitness function tailored to a specific classification task.

Feature selection is the preliminary step in machine learning and data mining. It identifies the most important and relevant features within a dataset by eliminating the redundant or irrelevant features. The substantial benefits may include an improved performance in terms of high prediction accuracy, reduced computational complexity, and simply interpretable underlying models. In this paper, we present a novel framework to investigate and understand the importance of Monte Carlo tree search (MCTS) in feature selection for very high-dimensional datasets. We construct a binary feature selection tree where each node represents one of the two feature states: a feature is selected or not. The search starts with an empty root node reflecting that no feature is selected. Then, the search tree is expanded by adding nodes in an incremental fashion through MCTS-based simulations. Following tree and default policy, every iteration generates an initial feature subset, where a filter is used to select the top k features forming the candidate feature subset. The classification accuracy is used as the goodness or reward of the candidate feature subset and propagated backward up to the root node following the active path. Finally, the candidate subset with highest reward is selected as the best feature subset. Experiments are performed on 30 real-world datasets, including 14 very high-dimensional microarray datasets, and results are also compared with state-of-the-art methods in the literature, which proves the efficacy, validity, and significance of the proposed method.

High-dimensional datasets (HDDs) pose significant challenges in feature selection due to their complex nature. While metaheuristic algorithms like the Whale Optimisation Algorithm (WOA) have shown promise in addressing these challenges, stability issues are least addressed. Through an extensive literature review, it was identified that stability is manifested in the equilibrium of exploration and exploitation. Hence, this study introduced the Hierarchical Whale Optimisation Algorithm (HiWOA), an approach designed to enhance the WOA’s stability and performance in HDD feature selection tasks. The HiWOA incorporates a two-phase strategy comprising a nonlinear control parameter based on the arcsine function and a hierarchical position-update mechanism adapted from the Grey Wolf Optimiser. The proposed HiWOA was evaluated through 23 benchmark optimisation functions and feature selection experiments on 11 medical HDDs. The results indicate that the HiWOA outperformed the WOA and a modified variant (mWOA) in terms of a better fitness value, a more balanced exploration-exploitation ratio, and improved classification accuracy with fewer selected features. These findings demonstrate the HiWOA’s effectiveness in enhancing stability, making it a robust solution for high-dimensional optimisation and feature selection.

Today, video data, as a powerful multimedia component, is accompanied by some problems with increasing usage in communication, health, education, and social media in particular. Classification and detection of concepts in video data by automatic methods are some of these challenging problems. In this study, we propose a video classification system, which incorporates deep convolutional neural networks (CNNs) by leveraging feature selection and data fusion techniques to improve the accuracy of the classification. Principal Component Analysis (PCA) as a feature selection method and Discriminant Correlation Analysis (DCA) technique, which incorporates class associations into the correlation analysis of feature sets for data fusion, are applied to the problem at the feature level. Support Vector Machines (SVMs) have been trained with new feature vectors obtained from different deep convolutional neural networks by feature selection and data fusion methods. The proposed method is tested for 38 concepts on TRECVID 2013 SIN video task dataset and the results are evaluated. Our results show that the classification accuracy is improved by 4% with an accuracy of 50.27% when the proposed data fusion and feature selection techniques are used.

This paper is concerned with the selection of process variables and time lags for fault detection. For this purpose, a feature selection technique known as Successive Projections Algorithm (SPA) is employed with Linear Discriminant Analysis (LDA) classifiers to discriminate between normal operating conditions and faults. SPA was originally designed to minimize multicollinearity among the selected features, which is a known cause of generalization problems for LDA. In the present work, a modification to the basic SPA formulation is proposed to place larger emphasis on the selection of relevant features for the classification task. The proposed SPA-LDA methodology is illustrated in a case study involving the Tennessee Eastman benchmark process. For comparison, a genetic algorithm (GA) for feature selection is also employed. In this study, the pre-selection of process variables was found to improve the accuracy of the resulting classifiers. In practice, such a pre-selection would have the extra advantage of reducing the number of sensors required to detect a given fault. Moreover, the proposed modification in SPA-LDA resulted in an improvement of average classification accuracy from 88 % to 96 %. This result was similar to that obtained by GA-LDA (97 %). However, the SPA-LDA classifiers were found to be less sensitive to measurement noise.

Schizophrenia is a serious psychiatric illness which needs early and accurate diagnosis. Difference in activation patterns of schizophrenia patients and healthy subjects can be identified with the help of functional magnetic resonance imaging (fMRI). However, manual diagnosis using fMRI depends on subjective observation and may be erroneous. This has motivated the pattern recognition and machine learning research community to develop a reliable and efficient decision model for classification of schizophrenia patients and healthy subjects. However, high dimensionality and low availability of fMRI data leads to the curse-of-dimensionality problem which may degrade the performance of decision model. In the present research work, a combination of feature extraction and feature selection techniques is utilised to obtain a reduced set of relevant and non-redundant features for classification of schizophrenia patients and healthy subjects. Features are extracted from pre-processed fMRI data using a general linear model approach. Next Fisher's discriminant ratio, a univariate method, is employed for feature selection i.e. To determine a subset of discriminative features. Further, minimum Redundancy Maximum Relevance (mRMR) feature selection, a multivariate method, is employed to obtain a set of relevant and non-redundant features which are used for learning a decision model using support vector machine. Two balanced and well-age matched datasets of auditory oddball task derived from a publicly available multisite FBIRN dataset are used for experiments. First dataset consists of fMRI scans of 34 schizophrenia patients and 34 healthy subjects acquired through 1.5 Tesla scanners while second dataset consists of 25 schizophrenia patients and 25 healthy subjects acquired through 3 Tesla scanners. The performance is evaluated in terms of sensitivity, specificity and classification accuracy, and compared with two well-known existing approaches for classification of schizophrenia patients and healthy subjects using fMRI. Experimental results demonstrate that the proposed model outperforms the existing approaches in terms of sensitivity, specificity and classification accuracy. The proposed approach achieves classification accuracy of 88.2% and 78.0% for 1.5 Tesla and 3 Tesla datasets respectively. In addition, the brain regions containing the discriminative features are identified which may be potential biomarkers for diagnosis of schizophrenia using fMRI.

Software effort estimation is a vital process in the software industry for successfully administering 5Ds of the software development life cycle (SDLC). The 5Ds stand for demand, development, direction, deployment, and designated cost of the software. Software development effort estimation (SDEE) is an effort prediction mechanism to calculate the effort for the development of the software product in order to minimize the challenges in the software field. Academics and practitioners are striving to identify which machine learning estimation technique yields more accurate results based on evaluation metrics, datasets, and other pertinent aspects. The feature selection techniques impact accuracy by selecting the main and relevant features in the dataset and eliminating the redundant and irrelevant features in the dataset. To achieve accurate estimations, the paper utilizes feature selection algorithms, along with various machine learning techniques, which predict the desired effort and the performance of the model has been measured in terms of prediction accuracy, value, relative error, and mean absolute error. The datasets China and Maxwell are trained with the relevant features by applying feature selection algorithms, and estimation techniques are applied to predict the effort. The performance is compared with the regression models and feature selection techniques utilized by many authors previously. The result of the proposed methodology significantly gives the best performance with the combination of feature selection and estimation models than all regression models when applied alone, to both datasets. From the results, it is perceptible that random forest is performing well with the feature selection techniques and obtains the highest prediction accuracy of 99.33% with the China and 89.47% with the Maxwell datasets.