A Comparative Study: Application of Principal Component Analysis and Recursive Feature Elimination in Machine Learning for Stroke Prediction

Brain–computer interface (BCI) technology uses electrophysiological (EEG) signals to detect user intent. Research on BCI has seen rapid advancement, with researchers proposing and implementing several signal processing and machine learning approaches for use in different contexts. BCI technology is also used in neuromarketing to study the brain’s responses to marketing stimuli. This study sought to detect two preference states (like and dislike) in EEG neuromarketing data using the proposed EEG-based consumer preference recognition system. This study investigated the role of feature selection in BCI to improve the accuracy of preference detection for neuromarketing. Several feature selection methods were used for benchmark testing in multiple BCI studies. Four feature selection approaches, namely, principal component analysis (PCA), minimum redundancy maximum relevance (mRMR), recursive feature elimination (RFE), and ReliefF, were used with five different classifiers: deep neural network (DNN), support vector machine (SVM), k-nearest neighbors (KNN), linear discriminant analysis (LDA), and random forest (RF). The four approaches were compared to evaluate the importance of feature selection. Moreover, the performance of classification algorithms was evaluated before and after feature selection. It was found that feature selection for EEG signals improves the performance of all classifiers.

Early diagnosis and treatment is very essential in monitoring Brain tumor using MRI images. Convolutional Neural Networks (CNN) and Machine Learning (ML) classifiers have been widely used but there is not much work on how feature selection techniques would affect the performance of the CNN. Secondly, there is a need for investigation concerning small dataset adaptability and ML-CNN comparisons. To improve the classification accuracy, we integrate Univariate, Recursive Feature Elimination (RFE), Recursive Feature Elimination with Cross Validation (RFECV) with CNN in this study. Preprocessing, feature extraction & selection was carried out on the dataset consisting of 253 MRI images and they are classified using CNN and ML models (Logistic Regression, Decision Tree, Random Forest, Naïve Bayes). With the results 96%, CNN with Univariate Feature Selection performed better than ML classifiers, and other selection techniques. The results demonstrate that feature selection is necessary to get the best performance out of CNN models operating on small datasets. Future studies should be based on different deep learning architectures to improve classification and application in other datasets.

Blood pressure abnormality detection and Interpretation utilizing Explainable Artificial Intelligence

We develop and validate a radiomics model based on multiparametric magnetic resonance imaging (MRI) in the classification of the pulmonary lesion and identify optimal machine learning methods. This retrospective analysis included 201 patients (143 malignancies, 58 benign lesions). Radiomics features were extracted from multiparametric MRI, including T2-weighted imaging (T2WI), T1-weighted imaging (TIWI), and apparent diffusion coefficient (ADC) map. Three feature selection methods, including recursive feature elimination (RFE), t test, and least absolute shrinkage and selection operator (LASSO), and three classification methods, including linear discriminate analysis (LDA), support vector machine (SVM), and random forest (RF) were used to distinguish benign and malignant pulmonary lesions. Performance was compared by AUC, sensitivity, accuracy, precision, and specificity. Analysis of performance differences in three randomly drawn cross-validation sets verified the stability of the results. For most single MR sequences or combinations of multiple MR sequences, RFE feature selection method with SVM classifier had the best performance, followed by RFE with RF. The radiomics model based on multiple sequences showed a higher diagnostic accuracy than single sequence for every machine learning method. Using RFE with SVM, the joint model of T1WI, T2WI, and ADC showed the highest performance with AUC = 0.88 ± 0.02 (sensitivity 83%; accuracy 82%; precision 91%; specificity 79%) in test set. Quantitative radiomics features based on multiparametric MRI have good performance in differentiating lung malignancies and benign lesions. The machine learning method of RFE with SVM is superior to the combination of other feature selection and classifier methods. • Radiomics approach has the potential to distinguish between benign and malignant pulmonary lesions. • Radiomics model based on multiparametric MRI has better performance than single-sequence models. • The machine learning methods RFE with SVM perform best in the current cohort.

<p>Soil properties could be assessed with reflectance spectroscopy (soil spectroscopy, SS) in vis-NIR region (400-2500 nm) through absorption features found in soil spectra. A high spectral resolution (up to 1 nm) drives to high dimensional and multicollinear data. This issue is usually addressed prior to modelling with feature extraction methods such as Principal Component Analysis, or embedded methods such as Partial Least Squares Regression (PLSR). Feature Selection (FS) wrapper methods are promising dimensionality reduction approaches barely used in SS. The objective of this study was two-fold: i) evaluate the performance of FS wrapper methods built from Random Forest (RF) algorithm to predict soil organic matter (SOM), clay and carbonates using laboratory spectroscopy, ii) test the performance of FS methods for dimensionality reduction in SS. The reflectance of 100 soil samples from Sierra de las Nieves National Park (Spain), was measured under laboratory conditions using an ASD FieldSpec Pro JR. A spectral preprocessing method, Continumm Removal (CR), was applied to raw spectra. The RF wrapper considered two different feature searching approaches: Sequential Forward Selection (SFS) and Sequential Flotant Forward Selection (SFFS). The performance of RF with FS (RF-FS) was compared to that of Partial Least Squares Regression (PLSR) and RF (without FS). Models were evaluated with R-squared, root mean squared error (RMSE) and ratio of prediction to deviation (RPD).</p><p>RF-FS models outperformed PLSR and RF models for the three SAP. RF-FS best models had a RPD of 2.19 for SOM, 1.64 for carbonates and 1.52 for clay, whereas PLSR models had RPD values of 1.59, 1.22 and 1.3, and RF 1.38, 1.23 and 1.23 for SOM, carbonates, and clay, respectively. Therefore, FS was useful in obtaining models with improved accuracy by reducing redundant features and avoiding multicollinearity (Hughes effect). The application of FS wrapper methods reduced the number of features in the RF-FS models to less than 1% of the starting features. Features were selected across all spectra from SOM and clay, and around 900, 1900 and 2350 nm for carbonates. This research, thus, shows an alternative to different feature extraction approaches for modelling soil properties based on FS methods and machine learning.</p>

The exponential rise of internet platforms, notably YouTube, has resulted in a massive volume of user-generated material, including video comments. Understanding audience input and enhancing user experience require analyzing and forecasting the mood of YouTube comments. This work provides a comprehensive method for YouTube comment prediction and sentiment classification that combines feature selection using Recursive Feature Elimination (RFE), Elastic Net Random Forest with Logistic Regression (RF with LR), and Principal Component Analysis (PCA). The first stage is to choose the most informative features from a given dataset using RFE, a common approach for doing so. RFE aids in the elimination of unnecessary or redundant features, resulting in enhanced model performance and decreased computing complexity. The Elastic Net Random Forest with Logistic Regression (RF with LR) technique is then used to construct a robust sentiment classification model. Elastic Net regularization combines the advantages of both L1 (Lasso) and L2 (Ridge) regularization, allowing for improved feature selection and management of multicollinearity concerns. By integrating many decision trees, the Random Forest ensemble approach improves the model's predictive potential even more. We employ Principal Component Analysis (PCA) to increase the classification model's effectiveness and solve possible difficulties created by high-dimensional data. PCA decreases the dataset's complexity while retaining its fundamental qualities, resulting in a more manageable and efficient feature space for classification. Finally, we compare the performance of three prominent classifiers on the preprocessed dataset: Linear Support Vector Machine (LSVM), Gaussian Naive Bayes (GNB), Logistic Regression (LR), and Decision Tree (DT). We can select the best-performing model for YouTube comment categorization by comparing these classifiers.

Background: Epilepsy is a common neurological disorder and affects approximately 70 million people worldwide. The traditional approach used by neurologists for the detection of seizure is time consuming. Aim: An automated approach is required that can assist the neurologists in seizure diagnosis in order to minimise the diagnostic time. Materials and Methods: Electroencephalogram (EEG) signals record the electrical activities of the brain and can be used effectively in seizure diagnosis. We have evaluated our system using Bonn university database. Appropriate feature selection is very important in pattern recognition problems. We used recursive feature elimination (wrapper method), for the selection of best features. In our study, we have used EEG signals to evaluate the performance of seven machine learning algorithms. Results: We obtained a classification accuracy of 99%, which is a significant improvement over the state-of-the-art methods. The classification performance across 10 folds of cross validation of Logistic regression and Adaboost was the highest with an accuracy of 99%. While using naive bayes and random forest, the Area under the ROC curve value obtained was 1. The sensitivity obtained using naive bayes classifier was 100%. Conclusion: Our approach requires minimum feature extraction that makes it an efficient approach. Recursive feature elimination was used to select the features which yield good accuracy results. It was observed that logistic regression and Adaboost performed superior with an accuracy of 99%.

Breast cancer remains one of the leading causes of cancer-related deaths in women globally and, therefore any means of diagnosing this disease, accurately and early enough is quite important in handling it. Here in this paper, author aims to compare the result of different machine learning algorithms in classification of breast cancer based on feature selection. In the present study we utilize Dataset that includes clinical and imaging data derived from breast cancer patients. Classification models are regularized with Recursive Feature Elimination (RFE) and Principal Component Analysis (PCA) in an attempt to manage and minimize the dimensionality of certain datasets. Some of the learning algorithms that are used include SVM, Random forest and the k-NN algorithms with the selected features being used to train and test the various algorithms. Classification models’ efficacy is measured by performance indicators: accuracy, sensitivity, specificity, and AUC-ROC. In our experiments, we have observed an area under the curve and reduced classification error along with increased computation time and iter – a consequence of selection or rejection of features. Moreover, we compute the discriminant features detectable to classify breast cancer well. The knowledge gained from this experience can be of significant help in the improvement of current machine learning methods aimed at diagnosing breast cancer, and may help to identify this illness at an early stage and significantly enhance people’s quality of life.

Predicting lodging severity in dry peas using UAS-mounted RGB, LIDAR, and multispectral sensors

Background: Cardiovascular diseases belong to the top three leading causes of mortality in the Philippines with 17.8 % of the total deaths. Lifestyle-related habits such as alcohol consumption, smoking, poor diet and nutrition, high sedentary behavior, overweight, and obesity have been increasingly implicated in the high rates of heart disease among Filipinos leading to a significant burden to the country's healthcare system. The objective of this study was to predict the presence of heart disease using various machine learning algorithms (support vector machine, naïve Bayes, random forest, logistic regression, decision tree, and adaptive boosting) evaluated on an anonymized publicly available cardiovascular disease dataset. Methodology: Various machine learning algorithms were applied on an anonymized publicly available cardiovascular dataset from a machine learning data repository (IEEE Dataport). A web-based application system named Heart Alert was developed based on the best machine learning model that would predict the risk of developing heart disease. An assessment of the effects of different optimization techniques as to the imputation methods (mean, median, mode, and multiple imputation by chained equations) and as to the feature selection method (recursive feature elimination) on the classification performance of the machine learning algorithms was made. All simulation experiments were implemented via Python 3.8 and its machine learning libraries (Scikit-learn, Keras, Tensorflow, Pandas, Matplotlib, Seaborn, NumPy). Results: The support vector machine without imputation and feature selection obtained the highest performance metrics (90.2% accuracy, 87.7% sensitivity, 93.6% specificity, 94.9% precision, 91.2% F1-score and an area under the receiver operating characteristic curve of 0.902) and was used to implement the heart disease prediction system (Heart Alert). Following very closely were random forest with mean or median imputation and logistic regression with mode imputation, all having no feature selection which also performed well. Conclusion: The performance of the best four machine learning models suggests that for this dataset, imputation technique for missing values may or may not be done. Likewise, recursive feature elimination for feature selection may not apply as all variables seem to be important in heart disease prediction. An early accurate diagnosis leading to prompt intervention efforts is very crucial as it improves the patient's quality of life and diminishes the risk of developing cardiac events.

This paper devoted to the optimization of surface electromyography (sEMG) based hand gesture recognition classifier model. Twenty-two features in time domain, frequency domain and time-frequency domain were extracted from sEMG of 6 muscles to classify 13 hand gestures. Correlation analysis was used to screen the features. Random forest (RF) and principal component analysis (PCA) feature selection were used to further reduce the feature dimensions. Finally, mean absolute value (MAV), root mean square (RMS), slope sign change (SSC), mean frequency (MNF), Willison amplitude (WAMP) and autoregressive coefficients (AR1), totally 6 features were selected. LDA (Linear Discriminant Analysis), SVM (Support Vector Machine), KNN (k-Nearest Neighbor), DT (Decision Tree) and NN (Neural Network), in total of five classification methods were applied. The accuracy of the classifier had no significant reduction, but the dimension of the feature set was reduced from 132 to 36, and the training time was greatly reduced. The result indicated that the combination of correlation analysis and random forest feature selection can optimize the classifier model and greatly reduce the training time of the classifier.

Alzheimer’s disease is a progressive neurodegenerative disorder for which early detection remains a significant challenge due to the complexity of clinical features and the high dimensionality of medical data. This study aims to improve the accuracy and reliability of Alzheimer’s disease detection by evaluating the performance of multiple machine learning algorithms integrated with intelligent feature selection strategies. Five classification models, Decision Tree, Naïve Bayes, Random Forest, Logistic Regression, and Deep Learning, were investigated under two experimental scenarios: without feature selection and with feature selection using Recursive Feature Elimination, Binary Particle Swarm Optimization, and Variance Threshold. Model performance was evaluated using K-fold cross-validation based on accuracy, precision, recall, and F1-score metrics. The results demonstrate that feature selection consistently enhances classification performance, particularly for conventional machine learning models such as Random Forest and Logistic Regression. Although the Deep Learning model achieves competitive accuracy, its reduced precision and F1-score indicate limitations when applied to reduced feature spaces. These findings highlight the importance of incorporating appropriate feature selection techniques to address data complexity and improve the effectiveness of early Alzheimer’s disease detection.

In conventional morphometrics, researchers often collect and analyze data using large numbers of morphometric features to study the shape variation among biological organisms. Feature selection is a fundamental tool in machine learning which is used to remove irrelevant and redundant features. Recursive feature elimination (RFE) is a popular feature selection technique that reduces data dimensionality and helps in selecting the subset of attributes based on predictor importance ranking. In this study, we perform RFE on the craniodental measurements of the Rattus rattus data to select the best feature subset for both males and females. We also performed a comparative study based on three machine learning algorithms such as Naïve Bayes, Random Forest, and Artificial Neural Network by using all features and the RFE-selected features to classify the R. rattus sample based on the age groups. Artificial Neural Network has shown to provide the best accuracy among these three predictive classification models.

Feature selection is referred to the process of obtaining a subset from an original feature set according to certain feature selection criterion, which selects the relevant features of the dataset. It plays a role in compressing the data processing scale, where the redundant and irrelevant features are removed. Feature selection techniques show that more information is not always good in machine learning applications. Apply different algorithms for the data at hand and with baseline classification performance values we can select a final feature selection algorithm. In this paper, we propose a hybrid classification model, which has correlation based filter feature selection algorithm and Machine learning as classifiers. The objective of this study is to select relevant features and analyze the outperform machine learning algorithms in order to train our model, predict and compare their classification performance. In this method, features are ordered according to their Absolute correlation value with respect to the class attribute. Then top K Features are selected from ordered list of features to form a reduced dataset. This proposed classifier model is applied to our smart meter datasets. To measure the performance of these selected features; seven benchmark classifier are used; Random Forest (RF), Logistic Regression (LR), k-Nearest Neighbor (kNN), Naive Bayes (NB), Decision Tree (DT), Linear Discriminant Analysis (LDA) and Support Vector Machine (SVM). This paper then analyzes the performance of all classifiers with feature selection in term of accuracy, sensitivity, F-Measure, Specificity, Precision, and MCC. From our experiment, we found that Random Forest classifier performed higher than other used classifiers.

Identification of hemodynamic biomarkers for bicuspid aortic valve induced aortic dilation using machine learning