An Informative Analysis of Applying Feature Reduction Methods to Supervised Machine Learning Algorithms

Feature extraction-reduction and machine learning for fault diagnosis in PV panels

Three-phase induction motors (IMs) are one of the most employed electric machines in industrial and household applications. Condition monitoring of these machines is essential to avoid unplanned maintenance and thereby enhance the availability. Artificial Intelligence (AI) technologies are emerging as an advanced tool for automating condition monitoring process to detect incipient faults at early stages. Machine Learning (ML) algorithms have been identified as a promising approach for condition monitoring of IMs and predicting maintenance to avoid failures. However, selecting the suitable ML algorithm for a given application is challenging because there is no predefined set of application-based algorithms. In addition, raw data processing and feature selection need careful attention to improve the accuracy of the results. This paper reviews supervised ML algorithms that can be used for condition monitoring of IMs and identifies their benefits and drawbacks. It then discusses how the dominant features from raw data can be selected through time domain and frequency domain analysis using the acoustic data collected from a three-phase induction motor. The study investigates classification accuracy of each ML algorithm and a procedure for selecting an algorithm based on the experimental results. Results of this study show that Support Vector Machines (SVM) algorithm outperforms other competing algorithms in condition monitoring of IMs when the dominant frequency components obtained through Fast Fourier Transform (FFT) are used as training data.

A simple and robust wetland classification approach by using optical indices, unsupervised and supervised machine learning algorithms

The application of machine-learning and Raman spectroscopy for the rapid detection of edible oils type and adulteration

After radical prostatectomy (RP), one-third of patients will experience biochemical recurrence (BCR), which is associated with subsequent metastasis and cancer-specific mortality. We employed machine learning (ML) algorithms to predict BCR after RP, and compare them with traditional regression models and nomograms. Utilizing a prospective Uro-oncology registry, 18 clinicopathological parameters of 1130 consecutive patients who underwent RP (2009-2018) were recorded, yielding over 20,000 data points for analysis. The data set was split into a 70:30 ratio for training and validation. Three ML models: Naïve Bayes (NB), random forest (RF), and support vector machine (SVM) were studied, and compared with traditional regression models and nomograms (Kattan, CAPSURE, John Hopkins [JHH]) to predict BCR at 1, 3, and 5 years. Over a median follow-up of 70.0 months, 176 (15.6%) developed BCR, at a median time of 16.0 months (interquartile range [IQR]: 11.0-26.0). Multivariate analyses demonstrated strongest association of BCR with prostate-specific antigen (PSA) (p: 0.015), positive surgical margins (p < 0.001), extraprostatic extension (p: 0.002), seminal vesicle invasion (p: 0.004), and grade group (p < 0.001). The 3 ML models demonstrated good prediction of BCR at 1, 3, and 5 years, with the area under curves (AUC) of NB at 0.894, 0.876, and 0.894, RF at 0.846, 0.875, and 0.888, and SVM at 0.835, 0.850, and 0.855, respectively. All models demonstrated (1) robust accuracy (>0.82), (2) good calibration with minimal overfitting, (3) longitudinal consistency across the three time points, and (4) inter-model validity. The ML models were comparable to traditional regression analyses (AUC: 0.797, 0.848, and 0.862) and outperformed the three nomograms: Kattan (AUC: 0.815, 0.798, and 0.799), JHH (AUC: 0.820, 0.757, and 0.750) and CAPSURE nomograms (AUC: 0.706, 0.720, and 0.749) (p < 0.001). Supervised ML algorithms can deliver accurate performances and outperform nomograms in predicting BCR after RP. This may facilitate tailored care provisions by identifying high-risk patients who will benefit from multimodal therapy.

Many lung diseases like Chronic Obstructive Pulmonary Disease (COPD) and Asthma are chronic diseases and worldwide considered as third main cause of death. COPD remains a forefront issue in global health, presenting significant challenges due to its complex nature and the imperative for precise classification to effectively guide treatment protocols. Traditional diagnostic methods, while effective, often come with limitations such as invasiveness, cost, and the requirement for specialized expertise. Recent advancements in machine learning (ML), particularly supervised ML algorithms, have shown promise in enhancing COPD classification through the analysis of clinical data, imaging, and physiological data. This review paper aims to systematically examine the current state of research on the application of supervised ML algorithms for the classification of COPD. We evaluate the effectiveness, accuracy, and clinical applicability of these algorithms by analyzing studies published in the last decade. Our review covers a range of supervised ML techniques, including K-Nearest Neighbor (KNN), Logistic Regression (LR), Support Vector Machine (SVM), and Convolutional Neural Network (CNN), Extreme Gradient Boosting (EGB), emphasizing their effectiveness by employing evaluating metrics such as accuracy, specificity, AUC scores and sensitivity. The observations of this review indicate that supervised machine learning algorithms can significantly enhance the classification and prediction of COPD stages, offering prospects for customized treatment plans and enhanced patient prognosis.

Smartphone-based apps are increasingly used to prevent relapse among those with substance use disorders (SUDs). These systems collect a wealth of data from participants, including the content of messages exchanged in peer-to-peer support forums. How individuals self-disclose and exchange social support in these forums may provide insight into their recovery course, but a manual review of a large corpus of text by human coders is inefficient. The study sought to evaluate the feasibility of applying supervised machine learning (ML) to perform large-scale content analysis of an online peer-to-peer discussion forum. Machine-coded data were also used to understand how communication styles relate to writers' substance use and well-being outcomes. Data were collected from a smartphone app that connects patients with SUDs to online peer support via a discussion forum. Overall, 268 adult patients with SUD diagnoses were recruited from 3 federally qualified health centers in the United States beginning in 2014. Two waves of survey data were collected to measure demographic characteristics and study outcomes: at baseline (before accessing the app) and after 6 months of using the app. Messages were downloaded from the peer-to-peer forum and subjected to manual content analysis. These data were used to train supervised ML algorithms using features extracted from the Linguistic Inquiry and Word Count (LIWC) system to automatically identify the types of expression relevant to peer-to-peer support. Regression analyses examined how each expression type was associated with recovery outcomes. Our manual content analysis identified 7 expression types relevant to the recovery process (emotional support, informational support, negative affect, change talk, insightful disclosure, gratitude, and universality disclosure). Over 6 months of app use, 86.2% (231/268) of participants posted on the app's support forum. Of these participants, 93.5% (216/231) posted at least 1 message in the content categories of interest, generating 10,503 messages. Supervised ML algorithms were trained on the hand-coded data, achieving F1-scores ranging from 0.57 to 0.85. Regression analyses revealed that a greater proportion of the messages giving emotional support to peers was related to reduced substance use. For self-disclosure, a greater proportion of the messages expressing universality was related to improved quality of life, whereas a greater proportion of the negative affect expressions was negatively related to quality of life and mood. This study highlights a method of natural language processing with potential to provide real-time insights into peer-to-peer communication dynamics. First, we found that our ML approach allowed for large-scale content coding while retaining moderate-to-high levels of accuracy. Second, individuals' expression styles were associated with recovery outcomes. The expression types of emotional support, universality disclosure, and negative affect were significantly related to recovery outcomes, and attending to these dynamics may be important for appropriate intervention.

Background: The merging of machine learning and deep learning approaches offers a viable route for improving breast cancer prediction in this era of advanced technology and data-driven healthcare. This study focused on the use of supervised machine learning and deep learning algorithms for breast cancer prediction using Wisconsin breast cancer dataset. Methods: The features from this dataset were computed from a digitized image of a fine needle aspirate (FNA) of a breast mass. Supervised machine learning algorithms, such as logistic regression, random forests, and support vector machines were compared with deep learning algorithms like convolutional and recurrent neural networks (RNN and CNN) for the prediction of breast cancer. The techniques combine clinical, genomic, and image-based data, that both aid in early diagnosis and individualized treatment suggestions. Results: The results of model comparison were very promising, with models obtaining excellent accuracy, sensitivity, and specificity in breast cancer diagnosis. The study used a pair plot and different classifiers with their accuracy scores on test dataset to determine which machine learning model was most suitable based on the relationship of variables represented in the plot. Support vector machine (SVM) has the highest accuracy of 98.8% and Artificial Neural Network (ANN) has lowest accuracy of 95.80% using confusion matrix and the accuracy metric. Conclusion: The study revealed that Support Vector Machine (SVM) has the highest prediction capability for Breast Cancer using Supervised Machine Learning and Deep Leaning. The new classifier performed better than the existing breast cancer prediction models in the literature.

The look for methodologies that can make inferences from externally supplied data develop broad hypotheses that are subsequently used to create forecasts concerning future events is known as supervised machine learning (SML). This study examine machine learning (ML) classification strategies, compares supervised learning algorithms, and determines foremost efficient classification algorithm based on the data set, number of instances, and variables (features). ML with the Waikato Environment for Knowledge Analysis (WEKA) tool, 7 different machine learning algorithms were considered: Decision Table, Random Forest (RF), Naive Bayes (NB), support vector machine (SVM), neural networks (Perceptron), JRip, and Decision Tree (J48). Time it takes to make a design and be concise (accuracy) are factors on the one end, and the kappa statistic and Mean Absolute Error (MAE) are factors on the other. For supervised predictive machine learning to work, machine learning algorithms must be accurate, and error-free.KeywordsMachine learningMean absolute errorWEKADeep learning

Surgical site infection (SSI) following a neurosurgical operation is a complication that impacts morbidity, mortality, and economics. Currently, machine learning (ML) algorithms are used for outcome prediction in various neurosurgical aspects. The implementation of ML algorithms to learn from medical data may help in obtaining prognostic information on diseases, especially SSIs. The purpose of this study was to compare the performance of various ML models for predicting surgical infection after neurosurgical operations. A retrospective cohort study was conducted on patients who had undergone neurosurgical operations at tertiary care hospitals between 2010 and 2017. Supervised ML algorithms, which included decision tree, naive Bayes with Laplace correction, k-nearest neighbors, and artificial neural networks, were trained and tested as binary classifiers (infection or no infection). To evaluate the ML models from the testing data set, their sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV), as well as their accuracy, receiver operating characteristic curve, and area under the receiver operating characteristic curve (AUC) were analyzed. Data were available for 1471 patients in the study period. The SSI rate was 4.6%, and the type of SSI was superficial, deep, and organ/space in 1.2%, 0.8%, and 2.6% of cases, respectively. Using the backward stepwise method, the authors determined that the significant predictors of SSI in the multivariable Cox regression analysis were postoperative CSF leakage/subgaleal collection (HR 4.24, p < 0.001) and postoperative fever (HR 1.67, p = 0.04). Compared with other ML algorithms, the naive Bayes had the highest performance with sensitivity at 63%, specificity at 87%, PPV at 29%, NPV at 96%, and AUC at 76%. The naive Bayes algorithm is highlighted as an accurate ML method for predicting SSI after neurosurgical operations because of its reasonable accuracy. Thus, it can be used to effectively predict SSI in individual neurosurgical patients. Therefore, close monitoring and allocation of treatment strategies can be informed by ML predictions in general practice.

In Mild Cognitive Impairment (MCI), identifying a high risk of conversion to Alzheimer’s Disease Dementia (AD) is a primary goal for patient management. Machine Learning (ML) algorithms are widely employed to pursue data-driven diagnostic and prognostic goals. An agreement on the stability of these algorithms –when applied to different biomarkers and other conditions– is far from being reached. In this study, we compared the different prognostic performances of three supervised ML algorithms fed with multimodal biomarkers of MCI subjects obtained from the Alzheimer’s Disease Neuroimaging Initiative (ADNI) database. Random Forest, Gradient Boosting, and eXtreme Gradient Boosting algorithms predict MCI conversion to AD. They can also be simultaneously employed –with the voting procedure– to improve predictivity. AD prediction accuracy is influenced by the nature of the data (i.e., neuropsychological test scores, cerebrospinal fluid AD-related proteins and APOE ε4, cerebral structural MRI (sMRI) data). In our study, independent of the applied ML algorithms, sMRI data showed the lowest accuracy (0.79) compared to other classes. Multimodal data were helpful in the algorithms’ performances by combining clinical and biological measures. Accordingly, using the three ML algorithms, the highest accuracy (0.90) was reached by employing neuropsychological and AD-related biomarkers. Finally, the feature selection procedure indicated that the most critical variables in the respective classes were the ADAS-Cog-13 scale, the medial temporal lobe and hippocampus atrophy, and the ratio between phosphorylated Tau and Aβ42 proteins. In conclusion, our data support the notion that using multiple ML algorithms and multimodal biomarkers helps make more accurate and solid predictions.

Despite careful patient selection, induction chemotherapy for acute myeloid leukemia (AML) is associated with a considerable risk for treatment-related mortality (5%-20%). We evaluated machine learning (ML) algorithms trained using factors available at the time of admission for AML therapy to predict death during the hospitalization. We included AML discharges with age > 17 years who received inpatient chemotherapy from State Inpatient Database from Arizona, Florida, New York, Maryland, Washington, and New Jersey for years 2008-2014. The primary objective was to predict inpatient mortality in patients undergoing chemotherapy using covariates present before initiation of chemotherapy. ML algorithms logistic regression (LR), decision tree, and random forest were compared. 29,613 hospitalizations for patients with AML were included in the analysis each with 4,177 features. The median age was 58.9 (18-101) years, 13,689 (53.7%) were male, and 20,203 (69%) were White. The mean time from admission to chemotherapy was 3 days (95% CI, 2.9 to 3.1), and 2,682 (9.1%) died during the hospitalization. Both LR and random forest models achieved an area under the curve (AUC) score of 0.78, whereas decision tree achieved an AUC of 0.70. The baseline LR model with age yielded an AUC of 0.62. To clinically balance and minimize false positives, we selected a decision threshold of 0.7 and at this threshold, 51 of our test set of 5,923 could have potentially averted treatment-related mortality. Using readily accessible variables, inpatient mortality of patients on track for chemotherapy to treat AML can be predicted through ML algorithms. The model also predicted inpatient mortality when tested on different data representations and paves the way for future research.

An intrusion detection system (IDS) is an important protection instrument for detecting complex network attacks. Various machine learning (ML) or deep learning (DL) algorithms have been proposed for implementing anomaly-based IDS (AIDS). Our review of the AIDS literature identifies some issues in related work, including the randomness of the selected algorithms, parameters, and testing criteria, the application of old datasets, or shallow analyses and validation of the results. This paper comprehensively reviews previous studies on AIDS by using a set of criteria with different datasets and types of attacks to set benchmarking outcomes that can reveal the suitable AIDS algorithms, parameters, and testing criteria. Specifically, this paper applies 10 popular supervised and unsupervised ML algorithms for identifying effective and efficient ML-AIDS of networks and computers. These supervised ML algorithms include the artificial neural network (ANN), decision tree (DT), k-nearest neighbor (k-NN), naive Bayes (NB), random forest (RF), support vector machine (SVM), and convolutional neural network (CNN) algorithms, whereas the unsupervised ML algorithms include the expectation-maximization (EM), k-means, and self-organizing maps (SOM) algorithms. Several models of these algorithms are introduced, and the turning and training parameters of each algorithm are examined to achieve an optimal classifier evaluation. Unlike previous studies, this study evaluates the performance of AIDS by measuring the true positive and negative rates, accuracy, precision, recall, and F-Score of 31 ML-AIDS models. The training and testing time for ML-AIDS models are also considered in measuring their performance efficiency given that time complexity is an important factor in AIDSs. The ML-AIDS models are tested by using a recent and highly unbalanced multiclass CICIDS2017 dataset that involves real-world network attacks. In general, the k-NN-AIDS, DT-AIDS, and NB-AIDS models obtain the best results and show a greater capability in detecting web attacks compared with other models that demonstrate irregular and inferior results.

In the last few decades, statistical methods and machine learning (ML) algorithms have become efficient in medical decision-making. Coronary artery disease (CAD) is a common type of cardiovascular disease that causes many deaths each year. In this study, two CAD datasets from different countries (TRNC and Iran) are tested to understand the classification efficiency of different supervised machine learning algorithms. The Z-Alizadeh Sani dataset contained 303 individuals (216 patient, 87 control), while the Near East University (NEU) Hospital dataset contained 475 individuals (305 patients, 170 control). This study was conducted in three stages: (1) Each dataset, as well as their merged version, was subject to review separately with a random sampling method to obtain train-test subsets. (2) The NEU Hospital dataset was assigned as the training data, while the Z-Alizadeh Sani dataset was the test data. (3) The Z-Alizadeh Sani dataset was assigned as the training data, while the NEU hospital dataset was the test data. Among all ML algorithms, the Random Forest showed successful results for its classification performance at each stage. The least successful ML method was kNN which underperformed at all pitches. Other methods, including logistic regression, have varying classification performances at every step.

This paper presents an automatic normal and abnormal heart sound classification model developed based on deep learning algorithm. MITHSDB heart sounds datasets obtained from the 2016 PhysioNet/Computing in Cardiology Challenge database were used in this research with the assumption that the electrocardiograms (ECG) were recorded simultaneously with the heart sounds (phonocardiogram, PCG). The PCG time series are segmented per heart beat, and each sub-segment is converted to form a square intensity matrix, and classified using convolutional neural network (CNN) models. This approach removes the need to provide classification features for the supervised machine learning algorithm. Instead, the features are determined automatically through training, from the time series provided. The result proves that the prediction model is able to provide reasonable and comparable classification accuracy despite simple implementation. This approach can be used for real-time classification of heart sounds in Internet of Medical Things (IoMT), e.g. remote monitoring applications of PCG signal.