A New Classification Model Using a Decision Tree Generated from Hyperplanes in Dimensional Space

Applying data mining techniques to classify patients with suspected hepatitis C virus infection

Machine Learning-Based Breast Tumor Ultrasound Radiomics for Pre-operative Prediction of Axillary Sentinel Lymph Node Metastasis Burden in Early-Stage Invasive Breast Cancer

This study explores the application of machine learning techniques, specifically classification, to improve data analysis outcomes. The primary objective is to evaluate and compare the performance of Decision Tree and Random Forest classifiers in the context of a structured dataset. Using the Elbow Method for optimal clustering alongside decision tree and random forest for classification algorithms, this research investigates the effectiveness of each method in accurately categorizing data. The study employs K-Means clustering to segment the data and Decision Trees and Random Forests for classification tasks. Dataset used in this research was obtained from Kaggle consisting of 13 attributes and 1048575 rows, all of which are numeric. The key results show that Random Forest outperforms Decision Trees in terms of classification accuracy, precision, recall, and F1 score, providing a more robust model for data classification. The performance improvement observed in Random Forest, particularly in handling complex datasets, demonstrates its superiority in generalizing across varied classes. The findings suggest that for applications requiring high accuracy and reliability, Random Forest is preferable to Decision Trees, especially when the dataset exhibits high variability. This research contributes to a deeper understanding of how different machine learning models can be applied to real-world classification problems, offering insights into the selection of the most appropriate model based on specific data characteristics.

BackgroundThe global morbidity and mortality of prostate cancer (PCa) increase sharply every year. Early diagnosis is essential; it determines survival and outcome. So, this study extracted the texture features of apparent diffusion coefficient images in multiparametric magnetic resonance imaging (mp-MRI) and built machine learning models based on radiomics texture analysis (TA) to determine its ability to distinguish benign from PCa lesions using the Prostate Imaging Reporting and Data System (PI-RADS) 4/5 score.MethodsWe enrolled 103 patients who underwent mp-MRI examinations and transrectal ultrasound and magnetic resonance fusion imaging (TRUS-MRI) targeted prostate biopsy and obtained pathological confirmation at our hospital from August 2017 to January 2020. We used ImageJ software to obtain texture feature parameters based on apparent diffusion coefficient (ADC) images, then standardized texture feature parameters, and used LASSO regression to reduce multiple feature parameters; 70% of the cases were randomly selected from the PCa group and the benign prostate hyperplasia group as the training set. The remaining 30% was used as the test set. The machine learning classification model for identifying benign and malignant prostate lesions was constructed using the feature parameters after dimensionality reduction. The clinical indicators were statistically analyzed, and we constructed a machine learning classification model based on clinical indicators of benign and malignant prostate lesions. Finally, we compared the model’s performance based on radiomics texture features and clinical indicators to identify benign and malignant prostate lesions in PI-RADS 4/5 score.ResultsThe area under the curve (AUC) of the R-logistic model test set was 0.838, higher than the R-SVM and R-AdaBoost classification models. At this time, the corresponding R-logistic classification model formula is as follow: Y_radiomics=9.396-7.464*median ADC-0.584*kurtosis+0.627*skewness+0.576*MRI lesions volume; analysis of clinical indicators shows that the corresponding C-logistic classification model formula is as follows: Y_clinical =-2.608+0.324*PSA-3.045*Fib+4.147*LDL-C, the AUC value of the model training set was 0.860, smaller than the training set R-logistic classification model AUC value of 0.936.ConclusionsRadiomics combined with the machine learning classifier model has strong classification performance in identifying benign and PCa in PI-RADS 4/5 score. Various treatments and outcomes for PCa patients can be applied clinically.

Underwriting decisions by insurance companies make a significant contribution to their profitability. Machine Learning (ML) techniques in underwriting decision making have saved time and improved operational efficiencies. A user-friendly cause-and-effect explanation of model’s predictions is useful to stakeholders, financial institutions and regulators. This research performed comparative analysis between tree-based classifiers such as Decision Tree, Random Forest and XGBoost. The study focused on enhancing risk assessment capabilities for life insurance companies using predictive analytics by classifying the insurance risk based on the historical data and propose the appropriate model to assess risk. Its purpose also included incorporating mechanisms that can aid in user friendly interpretation of ML models. Of all the models created as part of this research, the XGBoost classifier performed the best when compared to other classifiers, with an AUC value of 0.86 and F1-score above 0.56 on the validation set. The Random Forest classifier got AUC value of .84 and f1 score of .53 on the validation dataset. The results indicate the importance and advantages of tree -based models. These models i.e., XGBoost, decision tree and random forest are one of the best alternate techniques after the advent and popularity of the new age techniques in the machine learning such as neural networks, deep learning etc. The research also provides an insight on the interpretability of these conventional techniques by way of ‘SHAP’ or shapley values and ‘Feature Importance’ or ‘Variable Importance’. SHAP was used on complex models such as XGBoost and neural networks whereas Feature Importance is used in supervised learning methods such as Logistic Regression and tree- based models such as Decision Tree and Random Forest. Overall, the study was able to propose XGBoost as the most accurate model for Insurance risk classification and predictions.

BackgroundKnee osteoarthritis (OA) is the most common form of OA and a leading cause of disability worldwide. Chronic pain and functional loss secondary to knee OA put patients at risk of developing depression, which can also impair their treatment response. However, no tools exist to assist clinicians in identifying patients at risk. Machine learning (ML) predictive models may offer a solution. We investigated whether ML models could predict the development of depression in patients with knee OA and examined which features are the most predictive.ObjectiveThe primary aim of this study was to develop and test an ML model to predict depression in patients with knee OA at 2 years and to validate the models using an external data set. The secondary aim was to identify the most important predictive features used by the ML algorithms.MethodsOsteoarthritis Initiative Study (OAI) data were used for model development and external validation was performed using Multicenter Osteoarthritis Study (MOST) data. Forty-two features were selected, which denoted routinely collected demographic and clinical data such as patient demographics, past medical history, knee OA history, baseline examination findings, and patient-reported outcome measures. Six different ML classification models were trained (logistic regression, least absolute shrinkage and selection operator [LASSO], ridge regression, decision tree, random forest, and gradient boosting machine). The primary outcome was to predict depression at 2 years following study enrollment. The presence of depression was defined using the Center for Epidemiological Studies Depression Scale. Model performance was evaluated using the area under the receiver operating characteristic curve (AUC) and F1 score. The most important features were extracted from the best-performing model on external validation.ResultsA total of 5947 patients were included in this study, with 2969 in the training set, 742 in the test set, and 2236 in the external validation set. For the test set, the AUC ranged from 0.673 (95% CI 0.604-0.742) to 0.869 (95% CI 0.824-0.913), with an F1 score of 0.435 to 0.490. On external validation, the AUC varied from 0.720 (95% CI 0.685-0.755) to 0.876 (95% CI 0.853-0.899), with an F1 score of 0.456 to 0.563. LASSO modeling offered the highest predictive performance. Blood pressure, baseline depression score, knee pain and stiffness, and quality of life were the most predictive features.ConclusionsTo our knowledge, this is the first study to apply ML classification models to predict depression in patients with knee OA. Our study showed that ML models can deliver a clinically acceptable level of performance (AUC>0.7) in predicting the development of depression using routinely available demographic and clinical data. Further work is required to address the class imbalance in the training data and to evaluate the clinical utility of the models in facilitating early intervention and improved outcomes.

Background: Despite the development of disease-modifying therapy (DMT) for patients with transthyretin amyloid cardiomyopathy (ATTR-CM), the rates of mortality and disease progression are still high. The conventional models to predict prognosis (e.g., the Columbia score, the National Amyloidosis Centre [NAC] staging) predate DMT and offer limited prediction in the era of DMT. Aim: To develop a novel prognostication model in patients with ATTR-CM using plasma proteomics. Hypothesis: Plasma proteomics improves the prediction of death and disease progression in patients with ATTR-CM beyond the conventional models. Methods: In this prospective study, we conducted plasma proteomics profiling of 7,289 proteins in patients with ATTR-CM (n=303) at enrollment. The primary outcome was all-cause death, and the secondary was disease progression–a composite of death, heart transplant, heart failure hospitalization, and oral diuretics intensification. We randomly divided the cohort into training (2/3) and test sets (1/3). In the training set, we specified proteins to predict both outcomes using the Boruta algorithm. Using the specified proteins, we developed a random forest-based machine-learning (ML) model to predict each outcome in the training set. We compared the predictive ability of the ML model with the conventional models in the test set. We performed survival analyses between high- and low-risk groups defined by the ML models in the test set, while adjusting for the Columbia score. Results: During a median follow-up of 3.9 [1 st –3 rd quartile: 2.2–5.9] years, 69 patients (23%) died and 124 (41%) had disease progression, despite 243 (80%) receiving DMT. In the training set, 17 proteins were specified to predict both outcomes. In the test set, the area under the receiver-operating-characteristic curve (AUC) of the ML model was 0.90 (95% confidence interval 0.84–0.96) for all-cause death and 0.82 (0.73–0.91) for disease progression ( Image 1 ). Each ML model outperformed the conventional models in AUC, classification, and time-dependent AUC ( Images 1, 2 ). The high-risk group in the test set specified by the ML model had a higher event rate than the low-risk group in each outcome (all-cause death, adjusted hazard ratio [aHR] 6.6 [2.5–18.0], P =0.0002; disease progression, aHR 5.7 [2.6–12.4], P <0.0001; Image 3 ). Conclusion: This study first demonstrated that plasma proteomics improves the prediction of mortality and disease progression in patients with ATTR-CM in the DMT era.

This study proposes to identify potential liver patients based on the results of a liver function test performed during a health screening to search for signs of liver disease. It is critical to detect aliver patient at an early stage in order to treat them effectively. A liver function test's level of specific enzymes and proteins in the blood is evaluated to determine if a patient has liver disease. According to a review of the literature, general practitioners (GPs) rarely investigate any anomalies in liver function tests to the level indicated by national standards. The authors have useddata pre-processing in this work. The collection has 30691 records with 11 attributes. The classification model is utilized to construct an effective prediction system to aid general practitionersin identifying a liver patient using data mining. The collected results indicate that both the Naïve Bayes and C4.5 Decision Tree models give accurate predictions. However, given the C4.5 model offers more accurate predictions than theNaïve Bayes model, it can be assumed that the C4.5 model is superior for this research. Consequently, the liver patient prediction system will be developed using the rules given by the C4.5Decision Tree model in order to predict the patient class. The training set, suggested data mining with a classification model achieved 99.36% accuracy and on the testing set, 98.40%accuracy. On the training set, the enhanced accuracy relative to the current system was 29.5, while on the test set, it was 28.73. In compared to state-of-the-art models, the proposedapproach yields satisfactory outcomes. The proposed technique offers a variety of data visualization and user interface options, and this type of platform can be used as an early diagnosis tool for liver-related disorders in thehealthcare sector. This study suggests a machine learning-based technique for predicting liver disease. The framework includes a user interface via which healthcare providers can enterpatient information.

The method of using deep learning technology to realize automatic sleep staging needs a lot of data support, and its computational complexity is also high. In this paper, an automatic sleep staging method based on power spectral density (PSD) and random forest is proposed. Firstly, the PSDs of six characteristic waves (K complex wave, δ wave, θ wave, α wave, spindle wave, β wave) in electroencephalogram (EEG) signals were extracted as the classification features, and then five sleep states (W, N1, N2, N3, REM) were automatically classified by random forest classifier. The whole night sleep EEG data of healthy subjects in the Sleep-EDF database were used as experimental data. The effects of using different EEG signals (Fpz-Cz single channel, Pz-Oz single channel, Fpz-Cz + Pz-Oz dual channel), different classifiers (random forest, adaptive boost, gradient boost, Gaussian naïve Bayes, decision tree, K-nearest neighbor), and different training and test set divisions (2-fold cross-validation, 5-fold cross-validation, 10-fold cross-validation, single subject) on the classification effect were compared. The experimental results showed that the effect was the best when the input was Pz-Oz single-channel EEG signal and the random forest classifier was used, no matter how the training set and test set were transformed, the classification accuracy was above 90.79%. The overall classification accuracy, macro average F1 value, and Kappa coefficient could reach 91.94%, 73.2% and 0.845 respectively at the highest, which proved that this method was effective and not susceptible to data volume, and had good stability. Compared with the existing research, our method is more accurate and simpler, and is suitable for automation.

While a binary classifier aims to distinguish positives from negatives, a ranker orders instances from high to low expectation that the instance is positive. Most classification models in machine learning output some score of ‘positiveness’, and hence can be used as rankers. Conversely, any ranker can be turned into a classifier if we have some instance-independent means of splitting the ranking into positive and negative segments. This could be a fixed score threshold; a point obtained from fixing the slope on the ROC curve; the break-even point between true positive and true negative rates; to mention just a few possibilities.These connections between ranking and classification notwithstanding, there are considerable differences as well. Classification performance on n examples is measured by accuracy, an O(n) operation; ranking performance, on the other hand, is measured by the area under the ROC curve (AUC), an O(n logn) operation. The model with the highest AUC does not necessarily dominate all other models, and thus it is possible that another model would achieve a higher accuracy for certain operating conditions, even if its AUC is lower.However, within certain model classes good ranking performance and good classification performance are more closely related than suggested by the previous remarks. For instance, there is evidence that certain classification models, while designed to optimise accuracy, in effect optimise an AUC-based loss function [1]. It has also been known for some time that decision tree yield convex training set ROC curves by construction [2], and thus optimising training set accuracy is likely to lead to good training set AUC. In this talk I will investigate the relation between ranking and classification more closely.I will also consider the connection between ranking and probability estimation. The quality of probability estimates can be measured by, e.g., mean squared error in the probability estimates (the Brier score). However, like accuracy, this is an O(n) operation that doesn’t fully take ranking performance into account. I will show how a novel decomposition of the Brier score into calibration loss and refinement loss [3] sheds light on both ranking and probability estimation performance. While previous decompositions are approximate [4], our decomposition is an exact one based on the ROC convex hull. (The connection between the ROC convex hull and calibration was independently noted by [5]). In the case of decision trees, the analysis explains the empirical evidence that probability estimation trees produce well-calibrated probabilities [6].KeywordsProbability EstimationRanking PerformanceBrier ScoreTrue Negative RateLearn Decision TreeThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Given the demand for developing the efficient Machine Learning (ML) classification models for healthcare data, and the potentiality of Bio-Inspired Optimization (BIO) algorithms to tackle the problem of high dimensional data, we investigate the range of ML classification models trained with the optimal subset of features of PD data set for efficient PD classification. We used two BIO algorithms, Genetic Algorithm (GA) and Binary Particle Swarm Optimization (BPSO), to determine the optimal subset of features of PD data set. The data set chosen for investigation comprises 756 observations (rows or records) taken over 755 attributes (columns or dimensions or features) from 252 PD patients. We employed MaxAbsolute feature scaling method to normalize the data and one hold cross-validation method to avoid biased results. Accordingly, the data is split in to training and testing set in the ratio of 70% and 30%. Subsequently, we employed GA and BPSO algorithms separately on 11 ML classifiers (Logistic Regression (LR), linear Support Vector Machine (lSVM), radial basis function Support Vector Machine (rSVM), Gaussian Naïve Bayes (GNB), Gaussian Process Classifier (GPC), k-Nearest Neighbor (kNN), Decision Tree (DT), Random Forest (RF), Multilayer Perceptron (MLP), Ada Boost (AB) and Quadratic Discriminant Analysis (QDA)), to determine the optimal subset of features (reduction of dimensionality) contributing to the highest classification accuracy. Among all the bio-inspired ML classifiers employed: GA-inspired MLP produced the maximum dimensionality reduction of 52.32% by selecting only 359 features and delivering 85.1% of the classification accuracy; GA-inspired AB delivered the maximum classification accuracy of 90.7% producing the dimensionality reduction of 41.43% by selecting only 441 features; And, BPSO-inspired GNB produced the maximum dimensionality reduction of 47.14% by selecting 396 features and delivering the classification accuracy of 79.3%; BPSOMLP delivered the maximum classification accuracy of 89% and produced 46.48% of the dimensionality reduction by selecting only 403 features.

P68. Machine learning algorithms for predicting patient-controlled analgesia use after minimally invasive spine surgery

BackgroundThe challenges in cancer diagnosis underline the need for continued research and development of new diagnostic tools and methods. This study aims to explore an effective, noninvasive, and convenient diagnostic tool using urine based near-infrared spectroscopy (NIRS) analysis combined with machine learning algorithm.MethodsUrine samples were collected from a total of 327 participants, including 181 cancer cases and 146 healthy controls. These participants were randomly spit into train set (n = 218) and test set (n = 109). NIRS analysis (4,000 ∼10,000 cm−1) was performed for each sample in both train and test sets. Five pretreatment methods, including Savitzky-Golay (SG) smoothing, multiplicative scatter correction (MSC), baseline removal (BSL) with fitting polynomials to be used as baselines, the first derivative (DERIV1), and the second derivative (DERIV2), and combination with “scaling” and “center”, were investigated. Then partial least-squares (PLS) and linear support-vector machine (SVM) classification models were established, and prediction performance was evaluated in test set.ResultsNIRS had greatly overlapping in peaks, and PCA analysis failed in separation between cancers and healthy controls. In modeling with urine based NIRS data, PLS model showed its highest prediction accuracy of 0.780, with DERIV2, “scaling” and “center” pretreatment, while linear SVM displayed its best prediction accuracy of 0.844, with raw NIRS. With optimization in SVM, the prediction accuracy could improve to 0.862, when the top 262 features were involved as variables.DiscussionThis pilot study combining urine based NIRS analysis and machine learning is effective and convenient that might facilitate in cancer diagnosis, encouraging further evaluation with a large-size multi-center study.

Emotion recognition plays a pivotal role in advancing Human-Computer Interaction (HCI) by enabling systems to understand and respond to human emotional states. Among various modalities, Speech Emotion Recognition (SER) stands out as a non-invasive, cost-effective, and temporally efficient approach for detecting emotions. This study leverages the RAVDESS dataset, which provides high-quality audio recordings for emotion classification. The proposed methodology involves preprocessing audio signals to remove noise, extracting temporal and spectral features in both time and frequency domains, and implementing machine learning models for multi-class emotion classification. The study evaluates and compares the performance of several classification models, including Random Forest (RF), Multilayer Perceptron (MLP), Support Vector Machine (SVM), Convolutional Neural Networks (CNN), and Decision Trees (DT). Experimental results demonstrate promising accuracy levels, highlighting the potential of these machine learning techniques in SER. This research contributes to the integration of SER within Brain-Computer Interfaces (BCI) and other emotion-aware applications, paving the way for enhanced interactive systems. Keywords – Speech Emotion Recognition -Machine Learning - Random Forest -Multilayer Perceptron - Support Vector Machine - Convolutional Neural Networks - Decision Tree - RAVDESS Dataset

The classification model in machine learning has been employed to address different problems. Machine Learning classification is an effective method to realize customer churn prediction. This article provides a comparative study of machine learning from the perspective of predicting customer churn. Customer churn is one of the obstacles hindering the development of companies. Through classification approaches based on machine learning, customer churn can be predicted precisely, thus providing decision-making capabilities to these companies. Customer churn prediction is a typical classification problem and can be addressed by the use of a decision tree or random forest model. In this paper, decision tree and random forest models are employed to predict customer churn, using sales data provided by a chemical company from 2012 to 2020. We analyze the underlying risk of customer churn and the customer churn factors, including the low-priced count (LC), total amount of money (TM), and creation time (CT). The prediction results of the two models are evaluated by calculating various metrics. The experimental results indicate that the low-priced count (LC) is the most essential factor for customer churn. The results of the comparison, considering the training error and generalization error including: confusion matrix, ROC curve, AUC, precision, recall, and F1 score, reveal that the random forest model has better prediction accuracy than the decision tree model.