Drug-Protein Interactions Prediction Models Using Feature Selection and Classification Techniques.

Prediction of the stage of cancer plays an important role in planning the course of treatment and has been largely reliant on imaging tools which do not capture molecular events that cause cancer progression. Gene-expression data-based analyses are able to identify these events, allowing RNA-sequence and microarray cancer data to be used for cancer analyses. Breast cancer is the most common cancer worldwide, and is classified into four stages - stages 1, 2, 3, and 4 [2]. While machine learning models have previously been explored to perform stage classification with limited success, multi-class stage classification has not had significant progress. There is a need for improved multi-class classification models, such as by investigating deep learning models. Gene-expression-based cancer data is characterised by the small size of available datasets, class imbalance, and high dimensionality. Class balancing methods must be applied to the dataset. Since all the genes are not necessary for stage prediction, retaining only the necessary genes can improve classification accuracy. The breast cancer samples are to be classified into 4 classes of stages 1 to 4. Invasive ductal carcinoma breast cancer samples are obtained from The Cancer Genome Atlas (TCGA) and Molecular Taxonomy of Breast Cancer International Consortium (METABRIC) datasets and combined. Two class balancing techniques are explored, synthetic minority oversampling technique (SMOTE) and SMOTE followed by random undersampling. A hybrid feature selection pipeline is proposed, with three pipelines explored involving combinations of filter and embedded feature selection methods: Pipeline 1 - minimum-redundancy maximum-relevancy (mRMR) and correlation feature selection (CFS), Pipeline 2 - mRMR, mutual information (MI) and CFS, and Pipeline 3 - mRMR and support vector machine-recursive feature elimination (SVM-RFE). The classification is done using deep learning models, namely deep neural network, convolutional neural network, recurrent neural network, a modified deep neural network, and an AutoKeras generated model. Classification performance post class-balancing and various feature selection techniques show marked improvement over classification prior to feature selection. The best multiclass classification was found to be by a deep neural network post SMOTE and random undersampling, and feature selection using mRMR and recursive feature elimination, with a Cohen-Kappa score of 0.303 and a classification accuracy of 53.1%. For binary classification into early and late-stage cancer, the best performance is obtained by a modified deep neural network (DNN) post SMOTE and random undersampling, and feature selection using mRMR and recursive feature elimination, with an accuracy of 81.0% and a Cohen-Kappa score (CKS) of 0.280. This pipeline also showed improved multiclass classification performance on neuroblastoma cancer data, with a best area under the receiver operating characteristic (auROC) curve score of 0.872, as compared to 0.71 obtained in previous work, an improvement of 22.81%. The results and analysis reveal that feature selection techniques play a vital role in gene-expression data-based classification, and the proposed hybrid feature selection pipeline improves classification performance. Multi-class classification is possible using deep learning models, though further improvement particularly in late-stage classification is necessary and should be explored further.

With the rapid development of artificial intelligence in recent years, the research on image processing, text mining, and genome informatics has gradually deepened, and the mining of large-scale databases has begun to receive more and more attention. The objects of data mining have also become more complex, and the data dimensions of mining objects have become higher and higher. Compared with the ultra-high data dimensions, the number of samples available for analysis is too small, resulting in the production of high-dimensional small sample data. High-dimensional small sample data will bring serious dimensional disasters to the mining process. Through feature selection, redundancy and noise features in high-dimensional small sample data can be effectively eliminated, avoiding dimensional disasters and improving the actual efficiency of mining algorithms. However, the existing feature selection methods emphasize the classification or clustering performance of the feature selection results and ignore the stability of the feature selection results, which will lead to unstable feature selection results, and it is difficult to obtain real and understandable features. Based on the traditional feature selection method, this paper proposes an ensemble feature selection method, Random Bits Forest Recursive Clustering Eliminate (RBF-RCE) feature selection method, combined with multiple sets of basic classifiers to carry out parallel learning and screen out the best feature classification results, optimizes the classification performance of traditional feature selection methods, and can also improve the stability of feature selection. Then, this paper analyzes the reasons for the instability of feature selection and introduces a feature selection stability measurement method, the Intersection Measurement (IM), to evaluate whether the feature selection process is stable. The effectiveness of the proposed method is verified by experiments on several groups of high-dimensional small sample data sets.

With the development of technology, large databases become more accessible thanks to automation systems that automatically keep data and allow the use of large databases in many areas. Machine learning approaches, a sub-branch of artificial intelligence, are used in making decisions about the process by analyzing the data stored in databases and converting them into information. In this paper, the body production process of the surgical (medical) mask is analyzed. As it is known, surgical masks have become a part of our lives by becoming widespread all over the world with the COVID-19 pandemic. In the surgical mask body production process, using the real data of the production factors, first of all, filtering feature selection methods and analyzes were made and the feature selection method to be used was determined. With the specified feature selection method, the factors affecting the product quality are determined. Secondly, machine learning methods were used to determine the values and value ranges of factors (features) in the production of defect-free products. The performances of the machine learning models established in the second stage were increased by feature selection analysis. In the study, together with the parameter optimizations made to machine learning algorithms, it was seen that the best algorithm to estimate the defective product rate was the Ibk algorithm with 92.3% accuracy, 91.9% F measurement and 93% AUC value. Finally, in line with the decision rules revealed in the study, it was observed that the fabric types used for the upper/middle/lower layers that make up the body part in the mask body production process greatly affect the rates of defective or defect-free products. If the rod apparatus around the nose belongs to class k, it has been determined that many masks are defective. Improvement suggestions were presented according to the application results.

BackgroundThe growing demand for healthcare services challenges patient flow management in health systems. Alternative Level of Care (ALC) patients who no longer need acute care yet face discharge barriers contribute to prolonged stays and hospital overcrowding. Predicting these patients at admission allows for better resource planning, reducing bottlenecks, and improving flow. This study addresses three objectives: identifying likely ALC patients, key predictive features, and preparing guidelines for early ALC identification at admission.MethodsData from Nova Scotia Health (2015-2022) covering patient demographics, diagnoses, and clinical information was extracted. Data preparation involved managing outliers, feature engineering, handling missing values, transforming categorical variables, and standardizing. Data imbalance was addressed using class weights, random oversampling, and the Synthetic Minority Over-Sampling Technique (SMOTE). Three ML classifiers, Random Forest (RF), Artificial Neural Network (ANN), and eXtreme Gradient Boosting (XGB), were tested to classify patients as ALC or not. Also, to ensure accurate ALC prediction at admission, only features available at that time were used in a separate model iteration.ResultsModel performance was assessed using recall, F1-Score, and AUC metrics. The XGB model with SMOTE achieved the highest performance, with a recall of 0.95 and an AUC of 0.97, excelling in identifying ALC patients. The next best models were XGB with random oversampling and ANN with class weights. When limited to admission-only features, the XGB with SMOTE still performed well, achieving a recall of 0.91 and an AUC of 0.94, demonstrating its effectiveness in early ALC prediction. Additionally, the analysis identified diagnosis 1, patient age, and entry code as the top three predictors of ALC status.ConclusionsThe results demonstrate the potential of ML models to predict ALC status at admission. The findings support real-time decision-making to improve patient flow and reduce hospital overcrowding. The ALC guideline groups patients first by diagnosis, then by age, and finally by entry code, categorizing prediction outcomes into three probability ranges: below 30%, 30-70%, and above 70%. This framework assesses whether ALC status can be accurately predicted at admission or during the patient’s stay before discharge.

Background: Feature selection (FS), a crucial preprocessing step in machine learning, greatly reduces the dimension of data and improves model performance. This paper focuses on selecting features for medical data classification. Methods: In this work, a new form of ensemble FS method called weighted rank difference ensemble (WRD-Ensemble) has been put forth. It combines three FS methods to produce a stable and diverse subset of features. The three base FS approaches are Pearson’s correlation coefficient (PCC), reliefF, and gain ratio (GR). These three FS approaches produce three distinct lists of features, and then they order each feature by importance or weight. The final subset of features in this study is chosen using the average weight of each feature and the rank difference of a feature across three ranked lists. Using the average weight and rank difference of each feature, unstable and less significant features are eliminated from the feature space. The WRD-Ensemble method is applied to three medical datasets: chronic kidney disease (CKD), lung cancer, and heart disease. These data samples are classified using logistic regression (LR). Results: The experimental results show that compared to the base FS methods and other ensemble FS methods, the proposed WRD-Ensemble method leads to obtaining the highest accuracy value of 98.97% for CKD, 93.24% for lung cancer, and 83.84% for heart disease. Conclusion: The results indicate that the proposed WRD-Ensemble method can potentially improve the accuracy of disease diagnosis models, contributing to advances in clinical decision-making.

Background: Cholangiocarcinoma is a molecularly heterogenous cancer arising from the biliary epithelium. Individual omic techniques have identified clinically relevant, targetable subtypes but most tumors have no targetable alterations based on exome or transcriptomic evaluation. Integrated multiomics approaches incorporating transcriptomic, proteomic, and phosphoproteomic characterization provide deeper understanding and identify non-mutated, activated pathways that can be targeted. To optimize the potential clinical utility of these approaches, minimal classifier features must be defined. Methods: Approximately 60,000 RNA, protein, and phosphoprotein features were extracted from 35 cholangiocarcinomas treated at the primary US study site. 206 patients treated and profiled at an institution in China were also included. Using the Multiomics Factor Analysis clustering method, patients were sorted into 3 distinct molecular profiles. Important molecular features were selected using competing feature selection (FS) methods: Boruta, RreliefF, variable selection using random forests (RF), recursive factor elimination (RFE), median decrease accuracy in RF, and Concrete Autoencoders. Machine learning classification prediction models including Extreme Gradient Boosting, gradient-boosted trees, multinomial logit, and support vector machine (SVM) were tested on each FS. Each combination of FS and predictive model was tested with 10 replicates of 10-fold cross-validation of a randomly-selected balanced test set and an oversampled, balanced training subset. Accuracy, F1 score, AUC, precision, and recall were assessed for each FS and predictive model combination. A custom function of physical assay cost and classification accuracy was optimized to find the best assay. KEGG pathway overrepresentation analysis was used to analyze feature subsets. Results: A consistent set of 50 important RNA and protein features was identified, including previously reported prognostic factors, like SpryD4 protein expression, as well as previously unreported pathway surrogates, such as Slc2A2 or Pipox protein. Pathway overrepresentation analysis found insulin signaling and resistance and amino acid biosynthesis/metabolism pathways were overrepresented in the discriminative feature set. The gradient-boosted trees algorithm combined with the RFE FS method had the best performance, with a multiclass macro average (±sd) cross-validation accuracy 89% (±6%), F1 score 89% (±8%), AUC 0.98 (±0.02), precision 90% (±10%), recall 89% (±6%). Conclusions: Feature selection methods and predictive machine learning models work synchronously to develop accurate molecular subtype assays based on just 50 protein and RNA levels. This strategy has the potential to make molecular subtyping for cholangiocarcinoma and other molecularly complex cancers fast and widely available. Citation Format: Ellen Larson, Erik Jessen, Dong-Gi Mun, Jennifer Tomlinson, Amro Abdelrahman, Danielle Carlson, Hojjat Salehinejad, Rory Smoot. Feature selection and machine learning strategies optimize an affordable molecular assay for cholangiocarcinoma subtype [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2025; Part 1 (Regular Abstracts); 2025 Apr 25-30; Chicago, IL. Philadelphia (PA): AACR; Cancer Res 2025;85(8_Suppl_1):Abstract nr 1107.

The incidence of diabetes, a chronic disease, is increasing worldwide, especially in low- and middle-income countries. To reduce complications and improve patient outcomes, early and accurate prediction is critical. Using two benchmark datasets, this test demonstrates an ensemble-based machine learning framework for diabetes prediction. Two ensemble strategies were evaluated using the Diabetes Prediction dataset and the Indian Diabetes Pima dataset: a sequential ensemble combining XGBoost, gradient boosting, and AdaBoost, and a parallel ensemble using a smooth voting classifier that encompassed logistic regression, decision tree, and K-Nearest Neighbors. forward feature selection strategies were used to find the most relevant predictors, improving model performance and generalizability. 70% of the data was used for training, 15% for validation, and 15% for testing. According to the experimental results, the sequential ensemble performed better on the Indian Pima dataset, achieving a training accuracy of 98.95%, a validation accuracy of 97.59%, and an F1 accuracy of 97.77%. This performance was better than the parallel ensemble, which achieved an F1 score of 96.62%, a validation accuracy of 96.38%, and a training accuracy of 98.16%. Overall, the sequential model outperformed both datasets, with the diabetes prediction dataset showing better performance than the parallel model. These results demonstrate how feature selection methods and boosting-based ensemble models can work together to create accurate and reliable medical prediction systems.

Accurately classifying medical data is critical for improving diagnostic prediction system and identifying threptic targets for treatments. Analysing gene expression data has a major challenge in extracting disease-related genes from the large number of genes output from next generation sequencing technology. Therefore, eliminating irrelevant and redundant genes is a major step to process data for prediction. Our objective is to predict more accurately the presence of cancer disease in a sample cell from the gene expression.In this paper, we create a function called Classification Technique as Feature Selection (CTFS) as a new feature selection (FS) method to extract a subset (small number) of genes from classified big number of genes expression to improve cancer prediction result. The enrolled classification techniques in CTFS function for selection are K-Nearest Neighbors (K-NN) and Extreme Gradient Boosting (XGBoosting) optimized by Bayesian Parameter Tuning (BPT). The feature selection methods used to investigate the performance of CTFS function are Univariate Feature Selection (UFS) and Feature Importance (FI). The classification stage is carried out after the feature selection stage using three machine learning (ML) algorithms, Naïve Bayes (NB), Linear Support Vector Machine (LSVM), and Random Forest (RF). Results shows that, using XGBoosting optimized by BPT for FS outperforms FI method in terms of increasing the prediction accuracies along with minimum number of features but with higher running time. The performance of K-NN in FS outperforms all other FS methods in terms of accuracies providing an accuracy that is up to 100% when applied with LSVM on simulation dataset.

Software defect prediction models are used for predicting high risk software components. Feature selection has significant impact on the prediction performance of the software defect prediction models since redundant and unimportant features make the prediction model more difficult to learn. Ensemble feature selection has recently emerged as a new methodology for enhancing feature selection performance. This paper proposes a new multi-criteria-decision-making (MCDM) based ensemble feature selection (EFS) method. This new method is termed as MCDM-EFS. The proposed method, MCDM-EFS, first generates the decision matrix signifying the feature’s importance score with respect to various existing feature selection methods. Next, the decision matrix is used as the input to well-known MCDM method TOPSIS for assigning a final rank to each feature. The proposed approach is validated by an experimental study for predicting software defects using two classifiers K-nearest neighbor (KNN) and naïve bayes (NB) over five open-source datasets. The predictive performance of the proposed approach is compared with existing feature selection algorithms. Two evaluation metrics – nMCC and G-measure are used to compare predictive performance. The experimental results show that the MCDM-EFS significantly improves the predictive performance of software defect prediction models against other feature selection methods in terms of nMCC as well as G-measure.

QSAR analysis of diaryl COX-2 inhibitors: Comparison of feature selection and train-test data selection methods

An evidential reasoning rule based feature selection for improving trauma outcome prediction

Feature selection has been an important issue in machine learning and data mining, and is unavoidable when confronting with high‐dimensional data. With the advent of multilabel (ML) datasets and their vast applications, feature selection methods have been developed for dimensionality reduction and improvement of the classification performance. In this work, we provide a comprehensive review of the existing multilabel feature selection (ML‐FS) methods, and categorize these methods based on different perspectives. As feature selection and data classification are closely related to each other, we provide a review on ML learning algorithms as well. Also, to facilitate research in this field, a section is provided for setup and benchmarking that presents evaluation measures, standard datasets, and existing software for ML data. At the end of this survey, we discuss some challenges and open problems in this field that can be pursued by researchers in future. WIREs Data Mining Knowl Discov 2018, 8:e1240. doi: 10.1002/widm.1240This article is categorized under: Technologies > Data Preprocessing

In the field of fault diagnosis, machine learning is highly valued for its broad applicability and efficiency. Feature extraction and feature selection are key steps in the application of machine learning, and the performance of fault diagnosis methods relies heavily on the effective execution of these two steps. For this reason, this paper aims to enhance the performance of fault diagnosis methods by improving these two aspects. Firstly, to address the non-linearity and non-stationarity of rotating machinery vibration signals under variable operation conditions, this paper proposes an improved rapid refined composite multiscale sample entropy (IR2CMSE) feature extraction method. In addition, this paper decomposes the vibration signals with improved complete ensemble empirical mode decomposition with adaptive noise (ICEEMDAN) and extracts the sensitive intrinsic modal functions’ (IMFs') IR2CMSE values (SI-IR2CMSE) as the initial feature vector, which more accurately reveals the intrinsic time-scale characteristics of the vibration signals. Secondly, to address the problem of over-reliance on sample labels in most feature selection methods, this paper proposes a semi-supervised Gaussian mixing model with sparse regularization (SSGMM-SR) feature selection model. The model does not require complete fault labels and can automatically identify important features. Finally, validation with two rotating machinery fault datasets shows that the method proposed in this study exhibits high diagnostic accuracy and stability across multiple classifiers.

High-dimensional datasets can lead to overfitting and computationally expensive model building on machine learning. This study uses a dimensionality reduction technique, namely feature selection techniques, to overcome these problems. Five feature selection methods were used, i.e., Chi-Square (CS), Information Gain (IG), Genetic Algorithm (GA), Particle Swarm Optimization (PSO), and Least Absolute Shrinkage and Selection Operator (LASSO), and three classifier methods viz. Naïve Bayes, Extreme Gradient Boosting (XGB), and RF Classifier. The dataset used is the Heart Attack Analysis & Prediction Dataset. In this study, three scenarios of the best feature selection were carried out, namely: 1. selection of the best feature using a specific feature selection, 2. the intersection of selection of the best feature from the same category, 3. the intersection of selection of the best feature from the five proposed feature selection methods. The performance model is measured using accuracy, precision, recall, f1-score, AUC, and training time. This study reveals that feature selection is very effective in improving the performance of prediction models. Based on the experiment results, the best feature selection is CS and IG in the Filter Category with the XGB model. The best feature selected improved the performance of accuracy, precision, recall, f1-score, and AUC, i.e., 1.7%, 1%, 2.3%, 1.6%, and 0.2%, respectively. Meanwhile, training time requirements decreased by 23.5%. Feature selection with specific techniques performs better than feature selection by selecting the best features from the same category feature selection technique or various other feature selection methods.

The paper proposes a hybrid approach to reduce dimension in text classification problems, to overcome the issue of Curse of Dimensionality. This hybrid approach is a combination of Feature Selection (FS) and Feature Extraction (FE) methods, considering different aspects of feature relevance, to effectively reduce the dimension in large text datasets. It prevents feature selection biased in favor of a particular FS method. Many FS methods like Term Variance, Document Frequency, Information Gain, Shannons Entropy measure, Mean-Median and Mean Absolute Difference, were implemented and a comparative study was made on their performance when implemented in a hybrid system. The features selected by the individual FS methods are merged using three approaches, namely, Union, Intersection and Modified Union. The sub lists of features further undergo Feature Extraction by PCA, and the reduced feature sub list is clustered with k-means. Finally, the sentiment-score of the individual clusters are calculated using SentiWordNet database which gives the polarity of the data. The experiments were conducted on the benchmark datasets namely Reuters-21,578 and Classic4. The performance evaluation of the system made using the measures like precision, recall, f-score and accuracy shows that the proposed method has improved performance compared to its competitive methods.