Evaluating Machine Learning Approaches: A Comparative Study of Random Forest and Neural Networks in Grade Classification

There is a lack of literature about the classification performance improvement effect of hyperparameter tuning to predict health expenditure per capita (HE). In this study, the effect of hyperparameter tuning on classification performances of random forest (RF) and neural network (NN) classification tasks is compared for grouping member of World Bank (WB) countries in terms of HE. Data gathered from 188 member countries of WB for the year 2019. GDP per capita, mortality, life expectancy at birth and population aged 65 years and over are used as predictors. Number of trees and neurons in hidden layer are changed from 5 to 100 for RF and NN by changing k-fold parameter from 2 to 20. The dependent HE variable is transformed into binary categories, and the categories are well balanced (%50–%50). Classification performances of learning techniques are good (AUC > 0.95). RF (AUC = 0.9609) is superior to NN (AUC = 0.9596) in terms of average AUC values generated by hyperparameter tuning.

Objectives: This study aims to predict slope stability using machine learning algorithms, considering geotechnical and geometric variables to improve the accuracy of safety factor estimation and mitigate risks associated with slope failures. Theoretical Background: Traditional limit equilibrium methods, such as Bishop's method, present limitations when dealing with complex environmental conditions. Recent literature has highlighted the potential of machine learning techniques, such as Random Forest, Neural Networks, and XGBoost, in modeling nonlinear and highly variable systems, which are typical in geotechnical problems. Method: A parametric dataset with 1,520 observations generated through GeoSlope simulations was used, containing variables such as cohesion, internal friction angle, height, slope angle, and water level. Four algorithms were implemented: Random Forest, Gradient Boosting, Artificial Neural Networks, and XGBoost. Model performance was evaluated through cross-validation, hyperparameter tuning, and metrics including R², MSE, and RMSE. Results and Discussion: The XGBoost model achieved the best performance (R² = 0.9841; RMSE = 0.1419), followed by Gradient Boosting and Neural Networks. Variable importance analysis indicated that cohesion and internal friction angle are the primary factors influencing slope stability. Research Implications: The findings reinforce the applicability of machine learning models as decision-support tools in geotechnical engineering projects, offering greater precision and efficiency compared to traditional methods. Originality/Value: This study advances the field by integrating multiple modern algorithms within a replicable modeling framework, demonstrating superior performance over previous approaches and expanding the scope for practical applications in geotechnical engineering.

In recent years, the booming automotive, home appliance, and construction industries have driven a surge in market demand for cold-rolled steel strips. In response to the “carbon neutrality” initiative, the steel industry has accelerated its green transformation. The high added-value characteristics of cold-rolled strip steel highlight the importance of quality control. However, its production process is complex, with intertwined parameters. This paper aims to establish a model that can accurately reflect the production mechanism. For Problem 1, this paper innovatively combines principal component analysis (PCA) with the Pearson correlation coefficient to precisely screen out the key parameters that have a decisive impact on hardness. For Problem 2, a composite model based on random forest and BP neural network is constructed. This model can capture the subtle changes in production data in real time and accurately predict product quality, providing strong support for production decision-making. For Problem 3, the advanced optimization tool of simulated annealing algorithm is introduced. Through the intelligent optimization of complex process parameters, improvements in production efficiency and product quality are achieved. For Problem 1, our objective focused on identifying the key parameters significantly affecting the hardness of cold-rolled steel strips. In the preliminary data processing stage, we addressed missing values and outliers through linear interpolation and moving median filtering, followed by data standardization using the Z-score method. This rigorous preprocessing established a reliable data foundation for subsequent modeling. We then employed two complementary statistical approaches for feature selection and model development: PCA and Pearson correlation analysis. PCA identified critical variables based on their contribution rates to variance, while Pearson correlation coefficients quantitatively assessed the strength of linear relationships. These mutually validating methodologies collectively established robust correlations between hardness and twelve key process parameters. The final identified significant parameters include: thickness, width, carbon content, silicon content, strip steel speed, heating furnace temperature, soaking furnace temperature, slow cooling furnace temperature, over-aging furnace temperature, rapid cooling furnace temperature, quenching temperature, and temper mill tension. For Problem 2, which requires establishing a data-driven online quality inspection model for steel strips and analyzing its performance, we first introduced the random forest algorithm to significantly enhance model effectiveness and computational efficiency. Through this approach, we extracted the most representative feature subset, revealing that carbon content and rapid cooling furnace temperature are the most critical factors in predicting steel strip product quality, followed by physical dimensions, with silicon content and other heat treatment parameters also exhibiting certain influence. Building upon this optimized feature subset, we employed a backpropagation neural network to construct the online prediction model. Through hyperparameter tuning, we optimized the neural network architecture configuration, determining that 12 hidden layers are optimal for the acceleration/deceleration phases while 9 hidden layers achieve peak performance during stable operation phases. Finally, we conducted efficiency optimization and performance evaluation of the model, with results demonstrating excellent prediction performance at 97.6733% accuracy, prediction errors within 5%, and overall satisfactory prediction outcomes. For Problem 3, which requires establishing a comprehensive and efficient solution for optimizing process parameters of steel strips, simulated annealing algorithm was adopted in this study to globally search for optimal solutions, given the difficulties in developing mechanistic models due to the complex interdependencies among control parameters. The methodology encompasses parameter initialization, key parameter identification, objective function formulation, constraint handling, result validation, and model integration. Building upon the results of Problem 1, we first identified critical parameters and construct the objective function, followed by setting practical production constraints. The simulated annealing algorithm performs global optimization while effectively addressing constraint limitations through a penalty function mechanism. The optimal solution yields: strip speed of 201 m/s, heating furnace temperature at 710°C, soaking furnace temperature at 645°C, slow cooling furnace temperature at 608°C, over-aging furnace temperature at 355°C, rapid cooling furnace temperature at 67°C, quenching temperature at 45°C, and temper mill tension at 2420 kN, achieving optimal mechanical properties under these conditions.

Dropping out of school is a serious challenge in the education system that negatively impacts individual and social development. Early identification of students at risk of dropping out of school is crucial to prevent its long-term impact. This study aims to develop and compare a model for predicting the risk of dropping out of school using a machine learning approach. The three models compared in the study were Random Forest, Gradient Boosting, and Neural Network, with data covering 1000 students and features such as socioeconomic status, academic performance, parental engagement, distance to school, and educational resources. The results of the evaluation showed that the Random Forest model performed best with an accuracy of 93%, followed by Neural Network (92%) and Gradient Boosting (90%). The feature importance analysis revealed that socioeconomic status, parental involvement, and academic achievement were the dominant factors in predicting the risk of dropping out. These findings demonstrate the potential of applying machine learning as an early warning system for more targeted interventions in improving student retention. Further research is recommended to include psychological variables and longitudinal data as well as develop information technology-based systems for real implementation in schools.

Three Machine Learning algorithms, namely Random Forest, Neural Networks and Extreme Gradient Boosting, were applied to recognize the shape and surface structure of diamond nanoparticles from powder diffraction data. The algorithms were trained to recognize three types of shapes: 1D - rods, 2D - plates and 3D superspheres and, in the case of plate-like shapes, two types of (111) surfaces, with either one or three dangling bonds per surface atom. The classifiers’ training was based on structure functions S(Q) of the nanograin models obtained by Molecular Dynamics simulations. The software tools for models building, diffraction data calculations, and the procedure of grain shape and surface classification are given. It is shown that both Random Forest, Neural Networks and Extreme Gradient Boosting classifiers recognize the shape and surface structure of nanodiamonds with a low number of misclassifications. The derived classifiers were applied to a series of experimental diffraction patterns of diamond nanoparticles with sizes from 1.2 to 3.3 nm. It is shown that ML classification algorithms reproduce very well the results obtained for those samples by real space diffraction data analysis, namely Pair Distribution Function. ML studies prove that the dominating shape of adamantane-synthesized nanodiamonds is a plate terminated by (111) surfaces with 3 dangling bonds of carbon atoms

Learning in the feed-forward random neural network: A critical review

Since the early 1990s, Random Neural Networks (RNNs) have gained importance in the Neural Networks and Queueing Networks communities. RNNs are inspired by biological neural networks and they are also an extension of open Jackson's networks in Queueing Theory. In 1993, a learning algorithm of gradient type was introduced in order to use RNNs in supervised learning tasks. This method considers only the weight connections among the neurons as adjustable parameters. All other parameters are deemed fixed during the training process. The RNN model has been successfully utilized in several types of applications such as: supervised learning problems, pattern recognition, optimization, image processing, associative memory. In this contribution we present a modification of the classic model obtained by extending the set of adjustable parameters. The modification increases the potential of the RNN model in supervised learning tasks keeping the same network topology and the same time complexity of the algorithm. We describe the new equations implementing a gradient descent learning technique for the model.

BackgroundDespite the interest in machine learning (ML) algorithms for analyzing real-world data (RWD) in healthcare, the use of ML in predicting time-to-event data, a common scenario in clinical practice, is less explored. ML models are capable of algorithmically learning from large, complex datasets and can offer advantages in predicting time-to-event data. We reviewed the recent applications of ML for survival analysis using RWD in healthcare.MethodsPUBMED and EMBASE were searched from database inception through March 2023 to identify peer-reviewed English-language studies of ML models for predicting time-to-event outcomes using the RWD. Two reviewers extracted information on the data source, patient population, survival outcome, ML algorithms, and the Area Under the Curve (AUC).ResultsOf 257 citations, 28 publications were included. Random survival forests (N = 16, 57%) and neural networks (N = 11, 39%) were the most popular ML algorithms. There was variability across AUC for these ML models (median 0.789, range 0.6–0.950). ML algorithms were predominately considered for predicting overall survival in oncology (N = 12, 43%). ML survival models were often used to predict disease prognosis or clinical events (N = 27, 96%) in the oncology, while less were used for treatment outcomes (N = 1, 4%).ConclusionsThe ML algorithms, random survival forests and neural networks, are mainly used for RWD to predict survival outcomes such as disease prognosis or clinical events in the oncology. This review shows that more opportunities remain to apply these ML algorithms to inform treatment decision-making in clinical practice. More methodological work is also needed to ensure the utility and applicability of ML models in survival outcomes.

In recent years, hydroforming has become the topic of a lot of active research. Researchers have been looking for better procedures and prediction tools to improve the quality of the product and reduce the prototyping cost. Similar to any other metal forming process, hydroforming leads to non-homogeneous plastic deformations of the workpiece. In this paper, a model is developed to predict the amount of deformation caused by hydroforming using random neural networks (RNNs). RNNs learn the behavior of a system from the provided input/output data in a manner similar to the way the human brain does. This is different from the usual connectionist neural network (NN) models which are based on simple functional analyses. Experimental data is collected and used in training as well as testing the RNNs. The RNN models have feedforward architectures and use a generalized learning algorithm in the training process. Multi-layer RNNs with as few as six neurons were used to capture the nonlinear correlations between the input and output data collected from an experimental setup. The RNN models were able to predict the center deflection, the thickness variation, as well as the deformed shape of circular plate specimens with good accuracy.

Journal Article The Random Neural Network: A Survey Get access Stelios Timotheou Stelios Timotheou * Intelligent Systems and Networks Group, Department of Electrical and Electronic Engineering, Imperial College, London SW7 2BT, UK *Corresponding author: stelios.timotheou05@imperial.ac.uk Search for other works by this author on: Oxford Academic Google Scholar The Computer Journal, Volume 53, Issue 3, March 2010, Pages 251–267, https://doi.org/10.1093/comjnl/bxp032 Published: 30 April 2009 Article history Received: 01 November 2008 Revision received: 17 February 2009 Published: 30 April 2009

BACKGROUND Cisplatin resistance remains a significant obstacle in cancer therapy, frequently driven by translesion DNA synthesis (TLS) mechanisms that utilize specialized polymerases such as human DNA polymerase η (hpol η). Although small-molecule inhibitors like PNR-7-02 have demonstrated potential to disrupt hpol η activity, current compounds often lack sufficient potency and specificity to effectively combat chemoresistance. The vastness of chemical space further limits traditional drug discovery approaches, underscoring the need for advanced computational strategies such as machine learning (ML)-enhanced Quantitative Structure-Activity Relationship (QSAR) modeling. OBJECTIVE This study aimed to develop and validate ML-augmented QSAR models to accurately predict hpol η inhibition by indole thio-barbituric acid (ITBA) analogs, with the goal of accelerating the discovery of potent and selective inhibitors to overcome cisplatin resistance. METHODS A curated library of 85 ITBA analogs with validated hpol η inhibition data was used, excluding outliers to ensure data integrity. Molecular descriptors spanning 1D to 4D were computed, resulting in 220 features. Seventeen ML algorithms—including Random Forests, XGBoost, and Neural Networks—were trained using 80% of the data for training and evaluated with 14 performance metrics. Robustness was ensured through hyperparameter optimization and 5-fold cross-validation. RESULTS Ensemble methods outperformed other algorithms, with Random Forest achieving near-perfect predictive performance (training MSE = 0.0002, R² = 0.9999; testing MSE = 0.0003, R² = 0.9998). SHAP analysis revealed that electronic properties, lipophilicity, and topological atomic distances were the most important predictors of hpol η inhibition. Linear models exhibited higher error rates, highlighting the non-linear relationship between molecular descriptors and inhibitory activity. CONCLUSIONS Integrating machine learning with QSAR modeling provides a robust framework for optimizing hpol η inhibition, offering both high predictive accuracy and biochemical interpretability. This approach accelerates the identification of potent, selective inhibitors and represents a promising strategy to overcome cisplatin resistance, thereby advancing precision oncology.

Enhancing safety protocols in train stations requires proactive approaches to foresee and reduce any mishaps. In order to maximize safety in railroad environments, this study presents a novel machine learning framework that combines neural networks with random forest techniques. This hybrid model departs from traditional single-algorithm approaches by utilizing a variety of methods to enhance accident investigation and prediction. The chapter focuses on using this hybridized method, showcasing the functions of random forests and neural networks in predicting future events and assessing accident data from the past. The strategy of incorporating neural networks (multi-layer perceptron (MLP)) and random forests into the accident analysis framework is covered in the study. It also investigates if this model can be implemented in real-time and at scale within railway station safety systems. The results of the research reveal that the hybridized machine learning approach is a promising tool for improving safety standards in railroad stations.

Critical infrastructure now faces greater vulnerabilities and a higher risk of cyberattacks as a result of the (IIoT) quick expansion. The security and dependability of industrial systems must be ensured by identifying and thwarting these threats. In this paper, we use a hybrid approach of deep learning and RNN called hybrid deep random neural network (HDRNN) to offer a novel method of identifying cyber-attacks in the IIoT.The proposed HDRNN model combines the benefits of random neural networks with deep learning to improve the detection of IIoT cyberattacks. The deep learning component makes use of deep neural networks' capacity to extract intricate features from unstructured data, while the random neural network component offers robustness and adaptability to manage changing attack patterns.Realistic threats and benchmark datasets such as UNSW-NB15and DS2OS are used in experimental evaluations. High accuracy, precision, and recall rates are attained by the model, which successfully detects a variety of assaults including infiltration, data manipulation, and denial of service.The suggested HDRNN model offers a promising approach for improving the security of IIoT systems by precisely identifying cyber-attacks in real-time. The model's hybrid nature enables enhanced detection capabilities, adaptability to changing attack patterns, and a reduction in false positives, enabling efficient threat mitigation and protecting crucial infrastructure in the IIoT context.

In recent years, neural networks have achieved remarkable progress in various fields and have also drawn much attention in applying them on scientific problems. A line of methods involving neural networks for solving partial differential equations (PDEs), such as Physics-Informed Neural Networks (PINNs) and the Deep Ritz Method (DRM), has emerged. Although these methods outperform classical numerical methods in certain cases, the optimization problems involving neural networks are typically non-convex and non-smooth, which can result in unsatisfactory solutions for PDEs. In contrast to deterministic neural networks, the hidden weights of random neural networks are sampled from some prior distribution and only the output weights participate in training. This makes training much simpler, but it remains unclear how to select the prior distribution. In this paper, we focus on Barron type functions and approximate them under Sobolev norms by random neural networks with clear prior distribution. In addition to the approximation error, we also derive bounds for the optimization and generalization errors of random neural networks for solving PDEs when the solutions are Barron type functions.

The two primary measures of land mine detection performance are the probability of detection P/sub d/ and the probability of false alarm P/sub fa/. These two measures are highly inter-dependent and must be evaluated together. The relationship between the two probabilities directly affects the overall performance of the sensor in the field. In this paper we introduce a novel false alarm non-parametric filter based on the random neural network (RNN) model and the /spl delta/-technique. The study is based on mine detection using electromagnetic induction (EMI) sensors. The minefield data are pre-processed via the /spl delta/-technique before applying it to the RNN. The RNN has a predefined structure that tries to implement a mapping close enough in some precise sense to the discrimination function between non-mine and mine patterns. Limited numbers of non-mine and mine patterns, extracted from a small calibration area for a certain minefield provided by DARPA, are used for training the RNN. We show that the RNN gives effective decisions on patterns measured on other locations using different EMI sensor. The results show that the RNN produces a probability of detection up to 100 per cent with a substantial reduction of false alarms over the /spl delta/-technique (up to 40 per cent false alarm filtering).