Comparative analysis of methods for prediction continuous numerical features on big datasets

The method of random forest (RF) is a generalized advanced decision trees methods or techniques in which the clinical data space recursively portioned (usually binary split) according to the values of one or more predictor variables, such that the observations within a portion becomes more and more homogeneous. Moreover, the RF techniques come with a built-in protection against the overfitting by using a part of the data sets that each tree in the forest has not been calculated by its goodness of fit. Thus, the RF is a more reliable method extract decision-making approach for clinical research (in vivo or in vitro). In this regard, this chapter describes various aspects of random forest on real applications towards clinical and life science research; the model has been demonstrated by real data sets, and we also explored practical application of random forest for solving the real problems of clinical research. We examined how basic decision trees work, how individual decisions trees are combined to make a random forest, and ultimately discover why random forests are so good at what they do. Many illustrations and eye-catching figures accord to describe the model building. Summing of the research findings, the Random forests is advanced tool for clinical research each individual tree to create an uncorrelated forest of trees whose prediction by committee is more accurate than that of any individual tree.


 
 
 
 Abstract
 The classification task known as MNIST digit recognition involves identifying handwritten numbers into their corresponding values. Although there are numerous approaches proposed for this type of task, they typically face issues in achieving high accuracy. One method that can improve single models' performance is through ensemble learning. The goal of this study is to explore the use of various learning techniques, such as boosting and bagging, in combination with random forest models and decision trees, to improve the performance of MNIST digit recognition with regard to accuracy. We then perform evaluations on these methods using various metrics, such as recall, precision, accuracy, and F1. The findings of this study provide valuable insight into the various advantages of ensemble methods for the MNIST digit recognition task. It also highlights the need to explore these techniques in the context of machine learning. The objective of this study is to investigate the use of ensembles in improving the accuracy of MNIST digit recognition. We performed evaluations on two popular methods, namely boosting and bagging, with random forest and decision tree models. The evaluation parameters included F1 score, recall, accuracy, and precision. The results of the evaluations revealed that both boosting and bagging methods performed well in terms of their evaluation metrics. In most cases, the decision tree performed better than the random forest. However, the random forest method was able to achieve the highest accuracy, which is 99 percent. The findings of the evaluation revealed that ensembles can help improve single models' accuracy in MNIST digit recognition. On the other hand, the random forest method is a promising option for this task. The exact results of the evaluations will vary depending on the evaluation and implementation metrics. More research is needed to confirm their generalizability. The study emphasizes the value of exploring ensembles in machine learning systems, as well as the potential advantages of performing MNIST digit recognition using them.
 
 
 

Data is something important because it can be used to help make a decision or policy in an organization. One form of output from the use of data is to produce forecasts in the future. One of the organizations that need forecasting is the hospital. Data that can be used for forecasting is patient visit data. The purpose of this study is to compare several supervised learning methods in the case of forecasting outpatient visit data, by producing a model result from the experimental process of KNN, SVR, Decision Tree, Random Forest and Linear Regression methods. From the Plot Time Series data for outpatient visits, autocorrelation results using the ACF method have a significance level, namely at lag 1 = 0.797, lag 2 = 0.6, and lag 3 = 0.579. So that the formation of the dataset is found that the current data 15Yt"> has an influence on the data one month earlier 15Xt-1"> , the previous two months 15Xt-2"> , and the previous three months 15Xt-3"> . The results of the forecasting model carried out resulted that the random forest method had the best model with an evaluation value of the RMSE model of 204.43 and the MAPE value of 12%. Based on the criteria for the MAPE value, the model that has been made has a good category.

In the present study, gene expression profiles of acute myeloid leukemia (AML) samples were analyzed to identify feature genes with the capacity to predict the mutation status of FLT3/ITD. Two machine learning models, namely the support vector machine (SVM) and random forest (RF) methods, were used for classification. Four datasets were downloaded from the European Bioinformatics Institute, two of which (containing 371 samples, including 281 FLT3/ITD mutation-negative and 90 mutation-positive samples) were randomly defined as the training group, while the other two datasets (containing 488 samples, including 350 FLT3/ITD mutation-negative and 138 mutation-positive samples) were defined as the test group. Differentially expressed genes (DEGs) were identified by significance analysis of the micro-array data by using the training samples. The classification efficiency of the SCM and RF methods was evaluated using the following parameters: Sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV) and the area under the receiver operating characteristic curve. Functional enrichment analysis was performed for the feature genes with DAVID. A total of 585 DEGs were identified in the training group, of which 580 were upregulated and five were downregulated. The classification accuracy rates of the two methods for the training group, the test group and the combined group using the 585 feature genes were >90%. For the SVM and RF methods, the rates of correct determination, specificity and PPV were >90%, while the sensitivity and NPV were >80%. The SVM method produced a slightly better classification effect than the RF method. A total of 13 biological pathways were overrepresented by the feature genes, mainly involving energy metabolism, chromatin organization and translation. The feature genes identified in the present study may be used to predict the mutation status of FLT3/ITD in patients with AML.

Introduction: Coronary stenosis due to atherosclerosis restricts blood flow. Stenosis progression would lead to increased clinical risk such as heart attack. Although many risk factors were found to contribute to atherosclerosis progression, factors associated with fatigue is underemphasized. Our goal is to investigate the relationship between fatigue and stenosis progression based on in vivo intravascular ultrasound (IVUS) images and finite element models.Methods: Baseline and follow-up in vivo IVUS and angiography data were acquired from seven patients using Institutional Review Board approved protocols with informed consent obtained. Three hundred and five paired slices at baseline and follow-up were matched and used for plaque modeling and analysis. IVUS-based thin-slice models were constructed to obtain the coronary biomechanics and stress/strain amplitudes (stress/strain variations in one cardiac cycle) were used as the measurement of fatigue. The change of lumen area (DLA) from baseline to follow-up were calculated to measure stenosis progression. Nineteen morphological and biomechanical factors were extracted from 305 slices at baseline. Correlation analyses of these factors with DLA were performed. Random forest (RF) method was used to fit morphological and biomechanical factors at baseline to predict stenosis progression during follow-up.Results: Significant correlations were found between stenosis progression and maximum stress amplitude, average stress amplitude and average strain amplitude (p < 0.05). After factors selection implemented by random forest (RF) method, eight morphological and biomechanical factors were selected for classification prediction of stenosis progression. Using eight factors including fatigue, the overall classification accuracy, sensitivity and specificity of stenosis progression prediction with RF method were 83.61%, 86.25% and 80.69%, respectively.Conclusion: Fatigue correlated positively with stenosis progression. Factors associated with fatigue could contribute to better prediction for atherosclerosis progression.

Due to the complexity of data characteristics, multi-label learning in data mining has been proposed by scholars to solve the problem of information knowledge in the era of big data. In the era of big data, the complexity of the data structures makes it impossible for traditional single-label learning methods to meet the needs of technological development. Moreover, the importance of multi-label learning is gradually becoming evident. The random forest (RF) algorithm is regarded as one of the best classification algorithms. In this study, the traditional decision tree algorithm was improved, and the traditional RF method was converted into an adaptive RF (ARF) method for multi-label classification. By experiments, the effectiveness of the proposed method was verified. The RF method may not be able to classify massive data in a short time, but Hadoop, which was by Apache, is suitable for data-intensive tasks. On this basis, we modified the MapReduce programming mode to make it suitable for the proposed ARF method. This method was implemented on the cloud platform, and the time effectiveness of the parallel model was verified by experiments.

Intelligent classification of coal structure using multinomial logistic regression, random forest and fully connected neural network with multisource geophysical logging data

Fire is a natural disaster that can be caused by many different reasons. Recently, more environmentally friendly and innovative extinguishing methods have started to be tested, some of which are also used. For this purpose, a sound wave fire-extinguishing system was created and firefighting tests were performed. With the data obtained, as a result of 17,442 tests, a data set was created. In this study, five different machine learning methods were used by using the data set created. These are artificial neural network, k-nearest neighbor, random forest, stacking and deep neural network methods. Stacking method is an ensemble method created by using artificial neural network, k-nearest neighbor, random forest models together. Classification of extinction and non-extinction states of the flame was made with the models created with these methods. The accuracy of models in classification should be analyzed in detail in order to be used as a decision support system in the sound wave fire-extinguishing system. Hence, the classification processes were carried out through the 10-fold cross-validation method. As a result of these tests, the performance analysis of the models was carried out, and the results showed that the highest classification accuracy belongs to the stacking model with 97.06&#x0025;. The classification accuracy was determined 96.58&#x0025; in random forest method, 96.03&#x0025; in artificial neural network model, 94.88&#x0025; in deep neural network model and 92.62&#x0025; in k-NN model. The performance of the methods was compared by analyzing the performance metrics of machine learning methods. Thanks to the decision support system to be obtained based on the results of the analyzes, the sound wave fire-extinguishing system can be used efficiently.

In recent years, various types of terrorist attacks have occurred which have caused worldwide catastrophes. The ability to proactively detect and even predict a potential terrorist risk is critically important for government agencies to react in a timely manner. In this study, a method of geospatial statistics was used to analyse the spatio-temporal evolution of terrorist attacks on the Indochina Peninsula. The machine learning random forest (RF) method was adopted to predict the potential risk of terrorist attacks on the Indochina Peninsula on a spatial scale with 15 driving factors. The RF model performed well with AUC values of 0.839 [95% confidence interval of 0.833–0.844]. The map of the potential distribution of terrorist attack risk was obtained with a 0.05×0.05-degree (approximately 5×5 km) resolution. The results indicate that Thailand is the most dangerous area for terrorist attacks, especially southern Thailand, Bangkok and its surrounding cities. Middle Cambodia and the northern and southern parts of Myanmar are also high-risk areas. Other areas are relatively low risk. This study provides the hotspots for terrorist attacks on a more fine-grained geographical unit. Meanwhile, it shows that machine learning algorithms (e.g., RF) combined with GIS have great potential for simulating the risk of terrorist attacks.

The occurrence of corporate fraud often results in significant losses to stakeholders and society. Therefore, this study aims to construct a model to predict corporate fraud, with the goal of providing early warnings of potential fraudulent activities. The research focuses on fraudulent listed companies in Taiwan and selects matching non-fraudulent companies at a ratio of 1:2 as the research sample. To comprehensively capture the factors contributing to fraud, 53 indicators are selected from four dimensions: financial statements, corporate governance, market transactions, and the overall economy. This study further categorizes fraud methods into financial statement fraud and non-financial statement fraud (i.e., hollowing out/misappropriating assets/manipulating stock prices), and applies machine learning techniques, specifically decision tree and random forest algorithms, for prediction and analysis. The empirical results indicate that: (1) the random forest method, based on ensemble learning, achieves higher prediction accuracy than the decision tree model, and the prediction accuracy improves when fraud methods are distinguished; (2) the type I error of the random forest model is zero, meaning that if the model predicts a company as fraudulent, fraud will occur in the following year; and (3) the detailed techniques of fraud evolve structurally over time, leading to a relatively high type II error.

In recent years, forest fires have become an important issue in Central Europe. To model the probability of the occurrence of forest fires in the Lower Silesian Voivodeship of Poland, historical fire data and several types of predictors were collected or generated, including topographic, vegetation, climatic, and anthropogenic features. The main objectives of this study were to determine the importance of the predictors of forest fire occurrence and to map the probability of forest fire occurrence. The H2O driverless artificial intelligence (DAI) cloud platform was used to model forest fire probability. The gradient boosted machine (GBM) and random forest (RF) methods were applied to assess the probability of forest fire occurrence. Evaluation the importance of the variables was performed using the H2O platform permutation method. The most important variables were the presence of coniferous forest and the distance to agricultural land according to the GBM and RF methods, respectively. Model validation was conducted using receiver operating characteristic (ROC) analysis. The areas under the curve (AUCs) of the ROC plots from the GBM and RF models were 83.3% and 81.3%, respectively. Based on the results obtained, the GBM model can be recommended for the mapping of forest fire occurrence in the study area.

Liver has an ability to regenerate itself in animals, whereas the mechanism has not been fully explained. Here we used a GC/MS-based metabolomics method to profile the dynamic endogenous metabolic change in the serum of C57BL/6J mice at different times after 2/3 partial hepatectomy (PHx), and nine machine learning methods including Least Absolute Shrinkage and Selection Operator Regression (LASSO), Partial Least Squares Regression (PLS), Principal Components Regression (PCR), k-Nearest Neighbors (KNN), Support Vector Machines (SVM), Random Forest (RF), eXtreme Gradient Boosting (xgbDART), Neural Network (NNET) and Bayesian Regularized Neural Network (BRNN) were used for regression between the liver index and metabolomics data at different stages of liver regeneration. We found a tree-based random forest (RF) method that had the minimum average Mean Absolute Error (MAE), Root Mean Squared Error (RMSE) and the maximum R square (R2) and is time-saving. Furthermore, variable of importance in the project (VIP) analysis of RF method was performed and metabolites with VIP ranked top 25 were selected as the most critical metabolites contributing to the model. Ornithine, phenylalanine, 2-hydroxybutyric acid and lysine, etc. were chosen as the most .important metabolites which had strong correlations with the liver index. Further pathway analysis found Arginine biosynthesis, Pantothenate and CoA biosynthesis, Galactose metabolism, Valine, leucine and isoleucine degradation were the most influenced pathways. In summary, several amino acid metabolic pathways and glucose metabolism pathway were dynamically changed during liver regeneration. The RF method showed advantages for predicting the liver index after PHx over other machine learning methods used and a metabolic clock containing four metabolites is established to predict the liver index during liver regeneration.

Random Forest is a supervised classification method based on bagging (Bootstrap aggregating) Breiman and random selection of features. The choice of features randomly assigned to the Random Forest makes it possible that the selected feature is not necessarily informative. So it is necessary to select features in the Random Forest. The purpose of choosing this feature is to select an optimal subset of features that contain valuable information in the hope of accelerating the performance of the Random Forest method. Mainly for the execution of high-dimensional datasets such as the Parkinson, CNAE-9, and Urban Land Cover dataset. The feature selection is done using the Correlation-Based Feature Selection method, using the BestFirst method. Tests were carried out 30 times using the K-Cross Fold Validation value of 10 and dividing the dataset into 70% training and 30% testing. The experiments using the Parkinson dataset obtained a time difference of 0.27 and 0.28 seconds faster than using the Random Forest method without feature selection. Likewise, the trials in the Urban Land Cover dataset had 0.04 and 0.03 seconds, while for the CNAE-9 dataset, the difference time was 2.23 and 2.81 faster than using the Random Forest method without feature selection. These experiments showed that the Random Forest processes are faster when using the first feature selection. Likewise, the accuracy value increased in the two previous experiments, while only the CNAE-9 dataset experiment gets a lower accuracy. This research’s benefits is by first performing feature selection steps using the Correlation-Base Feature Selection method can increase the speed of performance and accuracy of the Random Forest method on high-dimensional data.

The increase in vehicle ownership and urban population puts a strain on roads and increases traffic congestion within developed and developing nations. Congestion affects an individual’s time and increases an individual’s fuel consumption and pollution output, undercutting efforts towards smart/ sustainable cities. Outdated statistical congestion predicting methods do not account for the non-linear, dynamic nature of congestion with regards to external factors such as, weather, road incidents, and infrastructure. The adoption of machine and deep learning develops the framework towards adaptable and data-driven congestion predicting methods. This study develops a framework that combines the fundamentals of machine learning algorithms, Queuing Theory, Decision Trees, Random Forests, and Deep Belief Networks (DBN) for robust and interprefig traffic congestion forecasting. The traffic flow micro-dynamics integration with Queuing Theory and the use of historical/ real-time multimodal data such as sensor feeds, GPS trajectories, and incident reports as a basis for forecasting helps in robust congestion prediction. Polynomials of moving and rest queues with defined traffic characteristics fuel flow mechanisms. The designed framework uses the DBN and Queuing Theory for robust traffic forecasting.

This article aims to provide a concise review on the state-of-the-art application of soft computing techniques to predict various parameters in tunnelling and underground excavations. Various soft computing techniques involving data mining and machine learning have found their application in the tunnelling-related problems. This article explores the application of artificial neural networks (ANNs), radial basis functions (RBFs), decision trees (DTs), random forest (RF) method, support vector machines (SVMs), nonlinear regression methods such as multi-adaptive regression splines (MARS), and hybrid intelligent models in the prediction of engineering response of tunnels and underground excavations. They help in predicting crucial parameters that decide the serviceability of tunnels and associated structures lying above the tunnel cavity. The researchers working in this domain have utilized the real-time data available from the construction projects in creating various machine learning models. It is observed that there are no proper guidelines to obtain optimal network architecture in ANN for assessing the parameters of the stated problem. RBFs and wavelet neural networks, which evolved from ANN, showed improvement in prediction accuracy. SVM and MARS methods are ornamented with improved computational efficiency and robustness of the algorithm. DT and RF methods are interpretable and computationally less expensive compared to neural networks. Hybrid intelligent models provided globally optimal solutions for nonlinear complex problems than simple neural network models. The limitations of the adopted soft computing methods are also emphasized. Overall, this article provides an intricate insight on the various soft computing techniques used by researchers to improve the performance of the machine learning models.