A Hybrid Model for Predicting Classification Dataset based on Random Forest, Support Vector Machine and Artificial Neural Network

Near infrared spectroscopy was used in discrimination of intact bovine teeth in terms of animal sex, diet, tooth type and place of origin. Discriminant Analysis (DA) models were developed and tested using a stratified random 70:30 data split to calibration and test sets. Of the discriminant techniques of Soft Independent Modelling of Class Analogy (SIMCA), Partial Least Squares – DA, Artificial Neural Networks (ANN) and Support Vector Machine (SVM), SVM and PLS-DA models performed best in most instances, with pretreatment choice impacting technique success. For test set prediction of animal sex, an accuracy of 95% was achieved using a PLS-DA model, and similar performance achieved using a SVM model. SVM, ANN and SIMCA models predicted three categories of tooth type (deciduous, permanent unerupted and erupted) with similar accuracies (89, 79 and 82%, respectively), however, permanent unerupted teeth were conflated with permanent erupted teeth. Prediction accuracy for ANN, SVM and PLS-DA models were also similar for discrimination of teeth from animals on a grain versus grass diet (PLS-DA 82%). There was considerable variation between models in prediction of geographic origin. Of the seven locations, two were discriminated with true positive rate around 70% using SVM and ANN models. Potential applications of this technology are discussed.

The rising machine learning (ML) models have become the preferred way for landslide detection based on remote sensing images, but the performance of these models in a sample-free area are rarely concerned in many studies. In this study, we used a cross-validation method (training model in one area and validation in another) to compare the model portability of trained ML models applied in an “off-site” area, as a consideration of the landslide detection ability of these models in sample-free areas. We integrate nighttime light imagery, multi-seasonal optical Landsat time-series and digital elevation data, and we employed support vector machines (SVM), artificial neural networks (ANN) and random forest (RF) models to classify the satellite imagery and identify landslides. Samples of two scenarios generated from two subareas of the Jiuzhaigou disaster-stricken region are used for the cross-application and accuracy evaluation of three ML models. The results revealed that when the trained models are applied in areas outside those in which they were developed, the landslide identification accuracy of these three models has declined. Especially for the SVM and ANN models, the accuracy is greatly reduced and there appears a seriously imbalanced user’s and producer’s accuracy. However, although the performance of the RF model is lower than that of SVM and ANN models in their local area, the RF model exhibits stable portability, and retains the original performance and achieves a satisfactory balance between overestimation and underestimation in “off-site” areas. An additional validation from a new area proved that the landslide detection performance of the RF model with stable portability is higher than that of the SVM and ANN models in “off-site” areas. The results suggest that evaluating the model portability through cross-application can be a useful way to determine the most suitable model for landslide detection in “off-site” areas with a similar geographic environment to model development areas, so as to maximize the accuracy of landslide detection based on limited samples.

Aggregate stability is a useful soil physical dynamic index of soil resistivity to surface wind and water erosion in all ecosystems, especially, in arid and semi-arid regions. Two machine learning techniques including support vector machines (SVMs) and artificial neural networks (ANNs) were used to develop predictive models for the estimation of geometric mean diameter (GMD) of soil aggregates. An empirical multiple linear regression (MLR) model was also constructed as the benchmark to compare their performances. Furthermore, the influence of feature space dimension reduction using parallel genetic algorithm (PGA) on the prediction accuracy of all investigated techniques was evaluated. The ANN model achieved greater accuracy in GMD prediction as compared to the MLR and SVM models. The obtained ERROR% value in GMD prediction using the ANN model was 6.9%, while it was 15.7 and 10.6% for the MLR and SVM models, respectively. Feature selection using PGA improved the prediction accuracy of all investigated techniques. The coefficient of determination (R2) values between the measured and the predicted GMD values using PGA-based MLR, SVM, and ANN models increased by 20.0, 12.2, and 8.8% in comparison with the proposed MLR, SVM, and ANN models. In conclusion, it appears that the PGA-based ANN model could be considered as an alternative to conventional regression models for the GMD prediction.

Objectives:Mandible osteoradionecrosis (ORN) is one of the most severe toxicities in patients with head and neck cancer (HNC) undergoing radiotherapy (RT). The existing literature focuses on the correlation of mandible ORN and clinical and dosimetric factors. This study proposes the use of machine learning (ML) methods as prediction models for mandible ORN incidence.Methods:A total of 96 patients (ORN incidence ratio of 1:1) treated between 2011 and 2015 were selected from the local HNC toxicity database. Demographic, clinical and dosimetric data (based on the mandible dose–volume histogram) were considered as model variables. Prediction accuracy (measured using a stratified fivefold nested cross-validation), sensitivity, specificity, precision and negative predictive value were used to evaluate the prediction performance of a multivariate logistic regression (LR) model, a support vector machine (SVM) model, a random forest (RF) model, an adaptive boosting (AdaBoost) model and an artificial neural network (ANN) model. The different models were compared based on their prediction accuracy and using the McNemar’s hypothesis test.Results:The ANN model (77% accuracy), closely followed by the SVM (76%), AdaBoost (75%) and LR (75%) models, showed the highest overall prediction accuracy. The RF model (71%) showed the lowest prediction accuracy. However, based on the McNemar’s test applied to all model pair combinations, no statistically significant difference between the models was found.Conclusion:Based on our results, we encourage the use of ML-based prediction models for ORN incidence as has already been done for other HNC toxicity end points.Advances in knowledge:This research opens a new path towards personalised RT for HNC using ML to predict mandible ORN incidence.

BackgroundAcute kidney injury (AKI) is the most common and serious complication of sepsis, accompanied by high mortality and disease burden. The early prediction of AKI is critical for timely intervention and ultimately improves prognosis. This study aims to establish and validate predictive models based on novel machine learning (ML) algorithms for AKI in critically ill patients with sepsis.MethodsData of patients with sepsis were extracted from the Medical Information Mart for Intensive Care III (MIMIC- III) database. Feature selection was performed using a Boruta algorithm. ML algorithms such as logistic regression (LR), k-nearest neighbors (KNN), support vector machine (SVM), decision tree, random forest, Extreme Gradient Boosting (XGBoost), and artificial neural network (ANN) were applied for model construction by utilizing tenfold cross-validation. The performances of these models were assessed in terms of discrimination, calibration, and clinical application. Moreover, the discrimination of ML-based models was compared with those of Sequential Organ Failure Assessment (SOFA) and the customized Simplified Acute Physiology Score (SAPS) II model.ResultsA total of 3176 critically ill patients with sepsis were included for analysis, of which 2397 cases (75.5%) developed AKI during hospitalization. A total of 36 variables were selected for model construction. The models of LR, KNN, SVM, decision tree, random forest, ANN, XGBoost, SOFA and SAPS II score were established and obtained area under the receiver operating characteristic curves of 0.7365, 0.6637, 0.7353, 0.7492, 0.7787, 0.7547, 0.821, 0.6457 and 0.7015, respectively. The XGBoost model had the best predictive performance in terms of discrimination, calibration, and clinical application among all models.ConclusionThe ML models can be reliable tools for predicting AKI in septic patients. The XGBoost model has the best predictive performance, which can be used to assist clinicians in identifying high-risk patients and implementing early interventions to reduce mortality.

GIS-based mineral prospectivity mapping using machine learning methods: A case study from Tongling ore district, eastern China

Air pollution in Macau has become a serious problem following the Pearl River Delta’s (PRD) rapid industrialization that began in the 1990s. With this in mind, Macau needs an air quality forecast system that accurately predicts pollutant concentration during the occurrence of pollution episodes to warn the public ahead of time. Five different state-of-the-art machine learning (ML) algorithms were applied to create predictive models to forecast PM2.5, PM10, and CO concentrations for the next 24 and 48 h, which included artificial neural networks (ANN), random forest (RF), extreme gradient boosting (XGBoost), support vector machine (SVM), and multiple linear regression (MLR), to determine the best ML algorithms for the respective pollutants and time scale. The diurnal measurements of air quality data in Macau from 2016 to 2021 were obtained for this work. The 2020 and 2021 datasets were used for model testing, while the four-year data before 2020 and 2021 were used to build and train the ML models. Results show that the ANN, RF, XGBoost, SVM, and MLR models were able to provide good performance in building up a 24-h forecast with a higher coefficient of determination (R2) and lower root mean square error (RMSE), mean absolute error (MAE), and biases (BIAS). Meanwhile, all the ML models in the 48-h forecasting performance were satisfactory enough to be accepted as a two-day continuous forecast even if the R2 value was lower than the 24-h forecast. The 48-h forecasting model could be further improved by proper feature selection based on the 24-h dataset, using the Shapley Additive Explanations (SHAP) value test and the adjusted R2 value of the 48-h forecasting model. In conclusion, the above five ML algorithms were able to successfully forecast the 24 and 48 h of pollutant concentration in Macau, with the RF and SVM models performing the best in the prediction of PM2.5 and PM10, and CO in both 24 and 48-h forecasts.

Stuck pipe incidents are one of the contributors to non-productive time (NPT), where they can result in a higher well cost. This research investigates the feasibility of applying machine learning to predict events of stuck pipes during drilling operations in petroleum fields. The predictive model aims to predict the occurrence of stuck pipes so that relevant drilling operation personnel are warned to enact a mitigation plan to prevent stuck pipes. Two machine learning methodologies were studied in this research, namely, the artificial neural network (ANN) and support vector machine (SVM). A total of 268 data sets were successfully collected through data extraction for the well drilling operation. The data also consist of the parameters with which the stuck pipes occurred during the drilling operations. These drilling parameters include information such as the properties of the drilling fluid, bottom-hole assembly (BHA) specification, state of the bore-hole and operating conditions. The R programming software was used to construct both the ANN and SVM machine learning models. The prediction performance of the machine learning models was evaluated in terms of accuracy, sensitivity and specificity. Sensitivity analysis was conducted on these two machine learning models. For the ANN, two activation functions—namely, the logistic activation function and hyperbolic tangent activation function—were tested. Additionally, all the possible combinations of network structures, from [19, 1, 1, 1, 1] to [19, 10, 10, 10, 1], were tested for each activation function. For the SVM, three kernel functions—namely, linear, Radial Basis Function (RBF) and polynomial—were tested. Apart from that, SVM hyper-parameters such as the regularization factor (C), sigma (σ) and degree (D) were used in sensitivity analysis as well. The results from the sensitivity analysis demonstrate that the best ANN model managed to achieve an 88.89% accuracy, 91.89% sensitivity and 86.36% specificity, whereas the best SVM model managed to achieve an 83.95% accuracy, 86.49% sensitivity and 81.82% specificity. Upon comparison, the ANN model is the better machine learning model in this study because its accuracy, sensitivity and specificity are consistently higher than those of the best SVM model. In conclusion, judging from the promising prediction accurateness as demonstrated in the results of this study, it is suggested that stuck pipe prediction using machine learning is indeed practical.

BackgroundLumbar drainage is widely used in the clinic; however, forecasting lumbar drainage-related meningitis (LDRM) is limited. We aimed to establish prediction models using supervised machine learning (ML) algorithms.MethodsWe utilized a cohort of 273 eligible lumbar drainage cases. Data were preprocessed and split into training and testing sets. Optimal hyper-parameters were archived by 10-fold cross-validation and grid search. The support vector machine (SVM), random forest (RF), and artificial neural network (ANN) were adopted for model training. The area under the operating characteristic curve (AUROC) and precision-recall curve (AUPRC), true positive ratio (TPR), true negative ratio (TNR), specificity, sensitivity, accuracy, and kappa coefficient were used for model evaluation. All trained models were internally validated. The importance of features was also analyzed.ResultsIn the training set, all the models had AUROC exceeding 0.8. SVM and the RF models had an AUPRC of more than 0.6, but the ANN model had an unexpectedly low AUPRC (0.380). The RF and ANN models revealed similar TPR, whereas the ANN model had a higher TNR and demonstrated better specificity, sensitivity, accuracy, and kappa efficiency. In the testing set, most performance indicators of established models decreased. However, the RF and AVM models maintained adequate AUROC (0.828 vs. 0.719) and AUPRC (0.413 vs. 0.520), and the RF model also had better TPR, specificity, sensitivity, accuracy, and kappa efficiency. Site leakage showed the most considerable mean decrease in accuracy.ConclusionsThe RF and SVM models could predict LDRM, in which the RF model owned the best performance, and site leakage was the most meaningful predictor.

Integrating near-infrared hyperspectral imaging with machine learning and feature selection: Detecting adulteration of extra-virgin olive oil with lower-grade olive oils and hazelnut oil

The elementary goal of this paper is to predict the best model for estimation of stock market. Machine Learning is a blooming field in computer science that has contributed to many predictions and analysis-based algorithm in Financial and economical field. Some of the algorithms used for predictions are Random Forest (RF), Support vector machine (SVM), Long-Short Term Memory (LSTM), Artificial Neural Networks (ANN). Random Forest is an ensemble supervised learning algorithm for classification problems with high accuracy factor. ANN has matured to a great extend over the past years. With the advent of high-performance computing ANN has assumed tremendous significance and huge application potentials in recent years. The innovation of ANN technology mimics the large interconnections and networking that exists between the nerve cells to process complex task. The paper has presented ANN and RF model for stock price estimation based on historical data and computed the future price, with comparative result analysis of their performance. Further, a candlestick model is designed of the stock to show the variation in price of stock over a stipulated period of time. Keyword: Random Forest, Candle-stick, ANN, RNN, CNN, Support Vector Machine, Deep Learning

Surrounding rock squeezing deformation is a common and prominent hazard in tunnel engineering projects, which often induces the shield jamming disaster during the TBM tunneling process. Based on the 139 groups of historical squeezing deformation cases, this study developed a hybrid PCA-IWGO-PNN model for squeezing classification. According to the influencing factors and characteristics of squeezing deformation, the strength-stress ratio, tunnel burial depth, tunnel equivalent diameter, rock mass quality index, and support stiffness were selected to establish the prediction index system of squeezing level. Because the probabilistic neural network (PNN) requires that the input variables are independent, principal component analysis (PCA) was used to preprocess the original data to eliminate the correlation between prediction indexes and achieve dimensionality reduction. The spread coefficient was the critical hyper-parameter in the PNN, and the improved gray wolf optimization (IGWO) algorithm was used to realize its efficient automatic optimization. Then, the PNN model was applied to engineering practice. Only 1 of 20 test samples was misjudged, achieving the 95% prediction accuracy. Finally, the comparison analysis with the artificial neural network (ANN) model, support vector machine (SVM) model, and random forest (RF) model was conducted. Among them, the PNN model achieved the highest prediction accuracy, followed by the artificial neural network (85%), RF (85%), and SVM (80%). In addition, the PNN model had the fastest running speed, which only consumed 5.6350 s, while the running time of ANN, SVM, and RF was 8.8340, 6.2290, and 6.9260 s, respectively. The hybrid PCA-IWGO-PNN model developed in this research provides an effective method for surrounding rock squeezing classification, and it has superiorities in both prediction accuracy and running speed.

Flooding causes several threats with outcomes which include peril to human and animal life, damage to property, and adversity to agricultural fields. Therefore, flood prediction is of prime importance for reducing loss of life and devastation to property. To model complex nature of hydrologic processes artificial neural network (ANN) tool is effectively being utilized for modelling different nonlinear relationships, and has proved to be an appropriate method for flood prediction. Present study investigated relative accuracy of radial basis function neural network (RBFNN) and support vector machine (SVM) models combined with Firefly Algorithm (FA) in predicting river flood discharge and contrasted with that of regular ANN, RBFNN and SVM models. Monthly river flow data of Silchar and Dholai stations located in Cachar district of Assam, India are utilized for the present study. For assessing model performance, coefficient of determination (R2), mean square error (MSE) and root mean square error (RMSE) were measured. Evaluation of outcomes shows that both RBF-FA (radial basis function - firefly algorithm) and SVM-FA (support vector machine - firefly algorithm) hybrid models give more precise forecasting results than RBFNN, FFBPNN (feed forward back propagation neural network) and SVM models. Yet, it can be observed that SVM-FA model give better forecasting outputs with R2 value 0.9818 than RBF-FA model. Results also reveal that simple SVM model performs marginally better than simple ANN model.

Land degradation from gully erosion poses a significant threat to the Erer watershed in Eastern Ethiopia, particularly due to agricultural activities and resource exploitation. Identifying erosion-prone areas and underlying factors using advanced machine learning algorithms (MLAs) and geospatial analysis is crucial for addressing this problem and prioritizing adaptive and mitigating strategies. However, previous studies have not leveraged machine learning (ML) and GIS-based approaches to generate susceptibility maps identifying these areas and conditioning factors, hindering sustainable watershed management solutions. This study aimed to predict gully erosion susceptibility (GES) and identify underlying areas and factors in the Erer watershed. Four ML models, namely, XGBoost, random forest (RF), support vector machine (SVM), and artificial neural network (ANN), were integrated with geospatial analysis using 22 geoenvironmental predictors and 1,200 inventory points (70% used for training and 30% for testing). Model performance and robustness were validated through the area under the curve (AUC), accuracy, precision, sensitivity, specificity, kappa coefficient, F1 score, and logarithmic loss. The relative slope position is most influential, with 100% importance in SVM and RF and 95% importance in XGBoost, while annual rainfall (AR) dominated ANN (100% importance). Notably, XGBoost demonstrated robustness and superior prediction/mapping, achieving an AUC of 0.97, 91% accuracy, 92% precision, and 81% kappa while maintaining a low logloss (0.0394). However, SVM excelled in classifying gully resistant/susceptible areas (97% sensitivity, 98% specificity, and 91% F1 score). The ANN model predicted the most areas with very high gully susceptibility (13.74%), followed by the SVM (11.69%), XGBoost (10.65%), and RF (7.85%) models, while XGBoost identified the most areas with very low susceptibility (70.19%). The ensemble technique was employed to further enhance GES modeling, and it outperformed the individual models, achieving an AUC of 0.99, 93.5% accuracy, 92.5% precision, 97.5% sensitivity, 95.4% specificity, 85.8% kappa, and 94.9% F1 score. This technique also classified the GES of the watershed as 36.48% very low, 26.51% low, 16.24% moderate, 11.55% high, and 9.22% very high. Furthermore, district-level analyses revealed the most susceptible areas, including the Babile, Fedis, Harar, and Meyumuluke districts, with high GES areas of 32.4%, 21.3%, 14.3%, and 13.6%, respectively. This study offers robust and flexible ML models with comprehensive validation metrics to enhance GES modeling and identify gully prone areas and factors, thereby supporting decision-making for sustainable watershed conservation and land degradation prevention.

The paper explores the hybrid of SVM and system of Artificial Neural Network as the finest binary classification system for calculating the diabetic nature of people in comparison to Support Vector Machine (SVM). In this research, the sets of all the parameters describing the diabetic nature of a person are taken from the laboratories. This approach is chosen so that a better learning method can be used for various problems. The testing results were found to be in accordance with the accepted results that resemble with the direct diagnosis of a physician. The results of this research shows that this hybrid SVM and Artificial Neural Network (ANN) model is more precise than the SVM model. These results of the hybrid SVM and ANN model suggest that it is very effective for the classification of Diabetic and Non Diabetic nature of a person. This paper highlights the concept of Support Vector Machines and its integration with Artificial Neural Network, the two key characteristics with one being the generalization theory of the SVM model that best describes how to select a hypothesis and functions given by Kernel that introduces the idea of non-linearity without the inclusion of the actual algorithm.