Enhancing Random Forest for Continuous Data Streams Using Divergence Measures to Select Decision Trees

The nature of stock market movement has always been ambiguous for investors because of various influential factors. This study aims to significantly reduce the risk of trend prediction with machine learning and deep learning algorithms. Four stock market groups, namely diversified financials, petroleum, non-metallic minerals and basic metals from Tehran stock exchange, are chosen for experimental evaluations. This study compares nine machine learning models (Decision Tree, Random Forest, Adaptive Boosting (Adaboost), eXtreme Gradient Boosting (XGBoost), Support Vector Classifier (SVC), Naïve Bayes, K-Nearest Neighbors (KNN), Logistic Regression and Artificial Neural Network (ANN)) and two powerful deep learning methods (Recurrent Neural Network (RNN) and Long short-term memory (LSTM). Ten technical indicators from ten years of historical data are our input values, and two ways are supposed for employing them. Firstly, calculating the indicators by stock trading values as continues data, and secondly converting indicators to binary data before using. Each prediction model is evaluated by three metrics based on the input ways. The evaluation results indicate that for the continues data, RNN and LSTM outperform other prediction models with a considerable difference. Also, results show that in the binary data evaluation, those deep learning methods are the best; however, the difference becomes less because of the noticeable improvement of models’ performance in the second way.

Abstract. Random Forest is a supervised learning algorithm developed from decision trees with the application of boostrap aggregating (bagging). This method grows trees from decision trees to produce a forest or the best model called the random forest model. Tree growth is done with randomly selected data with returns through the bagging process. Random forest is considered to provide better performance results for diabetes data among other supervised learning methods, because random forest and has the lowest error rate compared to other methods. Random forest is also an important technique for medical data classification, especially for diagnosing diabetics. In this study, classification was carried out using Pima Indian Diabetes data, which is an American tribe that lives in Arizona and Mexico. Classification analysis was carried out using an algorithm to see the level of accuracy in random forest classification on Pima Indian diabetes data. The results show that the accuracy value of random forest classification is 74.78%, this value is in the accuracy category at the fair classification level. In this random forest classification, there are three main variables that become importance variables, namely glucose then BMI, and age.
 Abstract. Random Forest is a supervised learning algorithm developed from decision trees with the application of boostrap aggregating (bagging). This method grows trees from decision trees to produce a forest or the best model called the random forest model. Tree growth is done with randomly selected data with returns through the bagging process. Random forest is considered to provide better performance results for diabetes data among other supervised learning methods, because random forest and has the lowest error rate compared to other methods. Random forest is also an important technique for medical data classification, especially for diagnosing diabetics. In this study, classification was carried out using Pima Indian Diabetes data, which is an American tribe that lives in Arizona and Mexico. Classification analysis was carried out using an algorithm to see the level of accuracy in random forest classification on Pima Indian diabetes data. The results show that the accuracy value of random forest classification is 74.78%, this value is in the accuracy category at the fair classification level. In this random forest classification, there are three main variables that become importance variables, namely glucose then BMI, and age.

This study explores the application of machine learning techniques, specifically classification, to improve data analysis outcomes. The primary objective is to evaluate and compare the performance of Decision Tree and Random Forest classifiers in the context of a structured dataset. Using the Elbow Method for optimal clustering alongside decision tree and random forest for classification algorithms, this research investigates the effectiveness of each method in accurately categorizing data. The study employs K-Means clustering to segment the data and Decision Trees and Random Forests for classification tasks. Dataset used in this research was obtained from Kaggle consisting of 13 attributes and 1048575 rows, all of which are numeric. The key results show that Random Forest outperforms Decision Trees in terms of classification accuracy, precision, recall, and F1 score, providing a more robust model for data classification. The performance improvement observed in Random Forest, particularly in handling complex datasets, demonstrates its superiority in generalizing across varied classes. The findings suggest that for applications requiring high accuracy and reliability, Random Forest is preferable to Decision Trees, especially when the dataset exhibits high variability. This research contributes to a deeper understanding of how different machine learning models can be applied to real-world classification problems, offering insights into the selection of the most appropriate model based on specific data characteristics.

As part of the Internet, a NodeMCU becomes a vulnerability to cyber-attack, so the intrusion detection system (IDS) against cyber-attack on the NodeMCU becomes a poignant research challenge. Several studies have applied several machine learning techniques for IDS on NodeMCU; however, there is a research opportunity to improve the performance of existing models. This research aims to increase the IDS performance on NodeMCU with ensemble voting. We hypothesize the realization of how the implementation of an IDS in NodeMCU. Then we obtain and observe a dataset from Kaggle. The dataset comprises five attacks: misconfiguration, DDoS, probe, scanning, and MiTM. Then we design the detection with ensemble voting consisting of a decision tree and a random forest. We benchmark our proposed solution with decision trees and random forest performance. This study uses several test metrics, including , , , and .The test results show that the decision tree has better Precision in predicting misconfiguration attacks and scan attacks than random forests. On the other hand, the random forest has better in predicting normal data, DDoS attacks, probe attacks, and MiTM attacks. In terms of , ensemble voting has the best performance, which is , compared to the decision tree and random forest, which are and , respectively. We conclude that by assembling a decision tree in the random forest with ensemble voting, random forest performance can improve.

Emotion recognition plays a pivotal role in advancing Human-Computer Interaction (HCI) by enabling systems to understand and respond to human emotional states. Among various modalities, Speech Emotion Recognition (SER) stands out as a non-invasive, cost-effective, and temporally efficient approach for detecting emotions. This study leverages the RAVDESS dataset, which provides high-quality audio recordings for emotion classification. The proposed methodology involves preprocessing audio signals to remove noise, extracting temporal and spectral features in both time and frequency domains, and implementing machine learning models for multi-class emotion classification. The study evaluates and compares the performance of several classification models, including Random Forest (RF), Multilayer Perceptron (MLP), Support Vector Machine (SVM), Convolutional Neural Networks (CNN), and Decision Trees (DT). Experimental results demonstrate promising accuracy levels, highlighting the potential of these machine learning techniques in SER. This research contributes to the integration of SER within Brain-Computer Interfaces (BCI) and other emotion-aware applications, paving the way for enhanced interactive systems. Keywords – Speech Emotion Recognition -Machine Learning - Random Forest -Multilayer Perceptron - Support Vector Machine - Convolutional Neural Networks - Decision Tree - RAVDESS Dataset

Random Forest algorithm eliminates the confines of a decision tree algorithm by reducing the over-fitting of datasets. It can handle the missing values in an effective manner and is more accurate than the decision trees. This research study presents a comparison of five different supervised machine learning algorithms namely Naive Bayes, Decision Tree, Multilayer Perceptron, Support Vector Machine and Random Forest for the classification of the disease dataset collected from an online repository. The Random Forest algorithm emerges to be the most efficient algorithm in classifying the diseases. Also, the research study applies different attribute selection measures to evaluate the impact of accuracy on the Random Forest classifier empirically. Among the different node splitting criteria, we conclude that the GINI index performed better on average than the Information Gain and Gain Ratio. The work is implemented using the Python 3.7 programming language.

Recently, artificial intelligence (AI) technologies have been employed to predict construction and demolition (C&D) waste generation. However, most studies have used machine learning models with continuous data input variables, applying algorithms, such as artificial neural networks, adaptive neuro-fuzzy inference systems, support vector machines, linear regression analysis, decision trees, and genetic algorithms. Therefore, machine learning algorithms may not perform as well when applied to categorical data. This article uses machine learning algorithms to predict C&D waste generation from a dataset, as a way to improve the accuracy of waste management in C&D facilities. These datasets include categorical (e.g., region, building structure, building use, wall material, and roofing material), and continuous data (particularly, gloss floor area), and a random forest (RF) algorithm was used. Results indicate that RF is an adequate machine learning algorithm for a small dataset consisting of categorical data, and even with a small dataset, an adequate prediction model can be developed. Despite the small dataset, the predictive performance according to the demolition waste (DW) type was R (Pearson’s correlation coefficient) = 0.691–0.871, R2 (coefficient of determination) = 0.554–0.800, showing stable prediction performance. High prediction performance was observed using three (for mortar), five (for other DW types), or six (for concrete) input variables. This study is significant because the proposed RF model can predict DW generation using a small amount of data. Additionally, it demonstrates the possibility of applying AI to multi-purpose DW management.

Lung cancer is one of the three deadliest diseases in the world and has rapidly developed. Based on this, researchers conducted research to predict the factors that influence lung cancer. One method to identify this is using data mining methods and classification techniques. Researchers used several popular algorithms in classification to make comparisons of the most accurate algorithms for lung cancer classification. The algorithms used include K-Nearest Neighbor, Random Forest Classifier, Logistic Regression, Linear SVM, Naïve Bayes, Decision Tree, Random Forest, Gradient Boosting, Kernel SVM, and MLPClassifier. The researcher used this algorithm because, in the research that the researcher found on the Kaggle platform, he examined the comparison of the algorithm using the breast cancer dataset. In previous studies, their researchers used SVM, which obtained an accuracy of 96.47%, Neural Networks of 97.06%, and Naïve Bayes with an accuracy of 91.18% to study breast cancer. The difference from previous research is that this study uses several existing algorithms in Machine Learning such as K-Nearest Neighbor, Random Forest Classifier, Logistic Regression, Linear SVM, Naïve Bayes, Decision Tree, Random Forest, Gradient Boosting, Kernel SVM, and MLPClassifier. In addition, this research was conducted to see whether the results of the accuracy of the algorithm that the researchers carried out using the lung cancer dataset had different results. The results of this study found that the more accurate algorithms were Random Forest and Gradient Boosting with an accuracy value of 100%, whereas in previous studies, it was the same. Still, Gradient Boosting had a higher accuracy value than Random Forest. Then, based on the data used in this study, the most influencing factors in predicting a diagnosis of lung cancer are obesity and coughing up blood. The results of this study found that the more accurate algorithms were Random Forest and Gradient Boosting with an accuracy value of 100%, whereas in previous studies, it was the same. Still, Gradient Boosting had a higher accuracy value than Random Forest. Then, based on the data used in this study, the most influencing factors in predicting a diagnosis of lung cancer are obesity and coughing up blood. The results of this study found that the more accurate algorithms were Random Forest and Gradient Boosting with an accuracy value of 100%, whereas in previous studies, it was the same. Still, Gradient Boosting had a higher accuracy value than Random Forest. Then, based on the data used in this study, the most influencing factors in predicting a diagnosis of lung cancer are obesity and coughing up blood.

Aim: The aim of the work is to evaluate the accuracy and precision in predicting heart disease using Support Vector Machine (SVM), Random forest (RF), Logistic Regression (LR), Decision Tree (DT) Classification algorithms. Materials and Methods: Classification algorithm is appealed on a heart dataset which consists of 180 records. A framework for heart disease prediction in the medical sector comparing Random forest, Logistic Regression, Decision Tree and SVM classifiers has been proposed and developed. The sample size was calculated as 55 in each group using G power 80%. Sample size was calculated using clincalc analysis, with alpha and beta values 0.05 and 0.5, 95% confidence, pretest power 80% and enrolment ratio 1. Results: The Novel Random Forest Algorithm (92.13%), Support Vector Machine (62.51%), Logistic Regression (84.89%), Decision Tree (86.25%) classifiers produce respectively. SVM, RF exists a statistically significant difference between the two groups (p=0.001,p=.004;p<0.05).LR, RF exists a statistically insignificant difference between the two groups (p=.103, P=.080;p>0.05) both with confidence interval 95%. Hence Random forest is better than SVM, RF, DT classifiers. Conclusion: The results show that the performance of RF is better when compared with SVM, LR and DT in terms of both precision and accuracy.

Decision Trees and Random Forests are leading Machine Learning Algorithms, which are used for Classification purposes. Through the course of this paper, a comparison is made of classification results of these two algorithms, for classifying data sets obtained from Kaggle's Bike Sharing System and Titanic problems. The solution methodology deployed is primarily broken into two segments. First, being Feature Engineering where the given instance variables are made noise free and two or more variables are used together to give rise to a valuable third. Secondly, the classification parameters are worked out, consisting of correctly classified instances, incorrectly classified instances, Precision and Accuracy. This process ensured that the instance variables and classification parameters were best treated before they were deployed with the two algorithms i.e. Decision Trees and Random Forests. The developed model has been validated by using Systems data and the Classification results. From the model it can safely be concluded that for all classification problems Decision Trees is handy with small data sets i.e. less number of instances and Random Forests gives better results for the same number of attributes and large data sets i.e. with greater number of instances. R language has been used to solve the problem and to present the results.

Electrocardiogram analysis using a combination of statistical, geometric, and nonlinear heart rate variability features

The rapid growth of the Internet of Things (IoT) has increased the risk of malware attacks, posing serious threats especially to micro, small, and medium enterprises (MSMEs) that often lack sufficient cybersecurity resources. This study aims to optimize Decision Tree (DT) and Random Forest (RF) classifiers using Grid Search, while addressing the class imbalance problem through the Synthetic Minority Oversampling Technique (SMOTE). The Security Attacks Malware IoT Networks dataset with five classes (Benign, Malware, DDoS, Brute Force, Scanning) was used and divided into training and testing sets with stratified 80:20 split. Experimental results show that DT achieved 67.3% accuracy with a macro F1-score of 42.9%, while RF achieved 70.7% accuracy but a very low macro F1-score of 21.4%, indicating bias toward the majority class despite balancing. Boosting methods provided stronger baselines, with XGBoost reaching 87.0% accuracy and 66.7% F1-score, while LightGBM achieved 85.6% accuracy and 64.4% F1-score. ROC curves and confusion matrices confirmed that boosting methods were more balanced in recognizing minority classes. In terms of efficiency, DT required the shortest training time (8 seconds), while LightGBM provided the best trade-off between accuracy and computational cost (26 seconds). Paired t-tests further confirmed that performance differences between DT and RF were not significant, while boosting methods significantly outperformed RF. Overall, optimizing DT and RF with Grid Search and SMOTE enhances their performance, but boosting methods remain more robust for malware detection in IoT traffic. These findings provide practical insights for MSMEs in balancing accuracy and efficiency when deploying intrusion detection systems.

It is indisputable that clinicians cannot exactly state the outcome of pregnancies through conventional knowledge and methods even as the surge in human knowledge continues. Hence, several computational techniques have been adapted for precise pregnancy outcome (PO) prediction. Obstetric datasets for PO determination exist as single label learning (SLL), multi-label learning (MLL) and multi-target (MTP) problems. There is however no single classifier recommended to optimally satisfy the needs of all the classification types. This work therefore identifies six widely used PO classifiers and investigates their performances in all three classification categories; to find the best performing classifier. Obstetric dataset exposed to input rank analysis via Principal component Analysis, produced thirteen (13) significant features for the experiment. Accuracy, F1-measure and build/test time were used as evaluation metrics. Decision tree (DT) had an average accuracy and F1 score of 89.23% and 88.23% respectively, with 1.0 average rank. Under MLL configuration, average accuracy (91.71%) and F1 score (94.28%) were highest in the random forest (RF) which had a 1.0 average test time rank. Using MTP, DT had an average accuracy of 88.80% and average F1 score of 71.13%, the multi-layered perceptron (MLP) had the best time cost with an average rank value of 2.0. From the results, RF is most optimal in terms of accuracy and average rank value, while DT is the most efficient in terms of time cost. The comparative analysis of global averages of the six base classifiers shows that RF is the most optimal algorithm with an average accuracy of 87.3% given all three data setups in the study. MLP on the other hand had an unexpectedly high time cost, making it unsuitable for similar data classifications if time is the main criterion. It is recommended that the choice of the classifier should either be RF or DT depending on the application domain and whether or not time cost is a major consideration.

Magnetocardiography (MCG) is a novel non-invasive technique that detects subtle magnetic fields generated by cardiomyocyte electrical activity, offering sensitive detection of myocardial ischemia. This study aimed to assess the ability of MCG to predict impaired myocardial perfusion using single-photon emission computed tomography (SPECT). A total of 112 patients with chest pain underwent SPECT and MCG scans, from which 65 MCG output parameters were analyzed. Using least absolute shrinkage and selection operator (LASSO) regression to screen for significant MCG variables, three machine learning models were established to detect impaired myocardial perfusion: random forest (RF), decision tree (DT), and support vector machine (SVM). The diagnostic performance was evaluated based on the sensitivity, specificity, accuracy, positive predictive value (PPV), negative predictive value (NPV), and area under the receiver operating characteristic curve (AUC). Five variables, the ratio of magnetic field amplitude at R-peak and positive T-peak (RoART+), R and T-peak magnetic field angle (RTA), maximum magnetic field angle (MAmax), maximum change in current angle (CCAmax), and change positive pole point area between the T-wave beginning and peak (CPPPATbp), were selected from 65 automatic output parameters. RTA emerged as the most critical variable in the RF, DT, and SVM models. All three models exhibited excellent diagnostic performance, with AUCs of 0.796, 0.780, and 0.804, respectively. While all models showed high sensitivity (RF = 0.870, DT = 0.826, SVM = 0.913), their specificity was comparatively lower (RF = 0.500, DT = 0.300, SVM = 0.100). Machine learning models utilizing five key MCG variables successfully predicted impaired myocardial perfusion, as confirmed by SPECT. These findings underscore the potential of MCG as a promising future screening tool for detecting impaired myocardial perfusion. ChiCTR2200066942, https://www.chictr.org.cn/showproj.html?proj=187904.

Background and aimsThe birth weight of a newborn is a crucial factor that affects their overall health and future well-being. Low birth weight (LBW) is a widespread global issue, which the World Health Organization defines as weighing less than 2,500 g. LBW can have severe negative consequences on an individual’s health, including neonatal mortality and various health concerns throughout their life. To address this problem, this study has been conducted using BDHS 2017–2018 data to uncover important aspects of LBW using a variety of machine learning (ML) approaches and to determine the best feature selection technique and best predictive ML model.MethodsTo pick out the key features, the Boruta algorithm and wrapper method were used. Logistic Regression (LR) used as traditional method and several machine learning classifiers were then used, including, DT (Decision Tree), SVM (Support Vector Machine), NB (Naïve Bayes), RF (Random Forest), XGBoost (eXtreme Gradient Boosting), and AdaBoost (Adaptive Boosting), to determine the best model for predicting LBW. The model’s performance was evaluated based on the specificity, sensitivity, accuracy, F1 score and AUC value.ResultsResult shows, Boruta algorithm identifies eleven significant features including respondent’s age, highest education level, educational attainment, wealth index, age at first birth, weight, height, BMI, age at first sexual intercourse, birth order number, and whether the child is a twin. Incorporating Boruta algorithm’s significant features, the performance of traditional LR and ML methods including DT, SVM, NB, RF, XGBoost, and AB were evaluated where LR, had a specificity, sensitivity, accuracy and F1 score of 0.85, 0.5, 85.15% and 0.915. While the ML methods DT, SVM, NB, RF, XGBoost, and AB model’s respective accuracy values were 85.35%, 85.15%, 84.54%, 81.18%, and 84.41%. Based on the specificity, sensitivity, accuracy, F1 score and AUC, RF (specificity = 0.99, sensitivity = 0.58, accuracy = 85.86%, F1 score = 0.9243, AUC = 0.549) outperformed the other methods. Both the classical (LR) and machine learning (ML) models’ performance has improved dramatically when important characteristics are extracted using the wrapper method. The LR method identified five significant features with a specificity, sensitivity, accuracy and F1 score of 0.87, 0.33, 87.12% and 0.9309. The region, whether the infant is a twin, and cesarean delivery were the three key features discovered by the DT and RF models, which were implemented using the wrapper technique. All three models had the identical F1 score of 0.9318. However, “child is twin” was recognized as a significant feature by the SVM, NB, and AB models, with an F1 score of 0.9315. Ultimately, with an F1 score of 0.9315, the XGBoost model recognized “child is twin” and “age at first sex” as relevant features. Random Forest again beat the other approaches in this instance.ConclusionsThe study reveals Wrapper method as the optimal feature selection technique. The ML method outperforms traditional methods, with Random Forest (RF) being the most effective predictive model for Low-Birth-Weight prediction. The study suggests that policymakers in Bangladesh can mitigate low birth weight newborns by considering identified risk factors.