Opinion Mining with Interpretable Random Density Forests

An ensemble deep learning model for exhaust emissions prediction of heavy oil-fired boiler combustion

Child mortality is a big problem around the world, especially in low- and middle-income nations where there are big differences in health care and social conditions. This investigation seeks to create a predictive model for child mortality and pinpoint the key factors that significantly contribute to it, employing machine learning (ML) methodologies. The dataset includes various features such as parental age, maternal education, birth weight, wealth index, and access to healthcare services. Thirteen machine learning classifiers were used, categorized into four model groups: Traditional Models (Logistic Regression, K-Nearest Neighbors, Support Vector Machine, Naive Bayes), Tree-Based Models (Decision Tree, Random Forest, Extra Trees), Boosting Models (AdaBoost, Gradient Boosting, XGBoost), and Ensemble Learning Models (Soft Voting, Hard Voting, Stacking). The efficacy of each model was assessed using classification metrics, including Accuracy, Precision, Recall, and F1-Score within a 10-fold cross-validation framework to guarantee robustness. Results indicate that ensemble models, particularly AdaBoost, achieved the highest predictive accuracy, with perfect scores across all metrics (1.00). XGBoost and Stacking also demonstrated strong and consistent performance. The findings indicate that ensemble learning methods are effective in predicting child mortality and can assist policymakers and healthcare planners in identifying high-risk populations and implementing targeted interventions to reduce child mortality.

Purpose This study aims to propose a new ensemble learning model and compare its performance with other ensemble models to obtain the best model for detecting financial statement fraud during the COVID-19 pandemic. Design/methodology/approach This study uses a quantitative approach, using secondary data from financial reports, annual reports, regulatory reports and other information on the internet. It focuses on all companies listed on the Indonesia Stock Exchange from 2020 to 2023. The independent variables in this study use financial and nonfinancial variables. In contrast, the target variable for fraudulent financial reports is based on sanctions from regulators and the company’s special supervisory status. Findings This study results show that the ensemble blending model performs best in detecting financial statement fraud compared to the ensemble model that construct it. Research limitations/implications This study sets ensemble learning to default settings. Setting certain conditions can further improve the performance of ensemble learning models. Practical implications This study can broaden the insights of practitioners, academics, investors, regulators, stakeholders and corporate finance experts into detecting financial report fraud. Originality/value This study proposes a new ensemble learning model that previous studies have not discussed. This ensemble learning model performs best compared to other ensemble learning models.

Threats in the digital landscape increased as different emerging technologies progress. Despite the various implementations of different defensive measures and tools to monitor and detect traffic anomaly, complexity and cost are the main challenges for small to medium enterprise networks. This study developed a malicious traffic detection system that utilized signature-based and anomaly-based techniques through packet analysis and network flow using machine learning algorithms. The system consists of a Raspberry Pi-based packet capturing tool, website application, implored with Ensemble Learning Model to detect and classify malicious network traffic. The ensemble learning model which includes weak learners like Decision Tree, Naïve Bayes and Support Vector Machine (SVM), in which individual results are combined and boosted using XG Boost algorithm. CTU-13 and CSE-CIC-IDS2018 datasets were used for training and validating the model. The ensemble learning model achieved an accuracy of 96.42%, precision of 98.99%, recall of 93.79%, and <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\mathbf{F 1}$</tex>-score of 96.32% for the binary classification using the validation data. Moreover, the model also achieved an accuracy of 96.39%, precision of 95.82%, recall of 96.39%, and F1-score of 95.66% for the multiple classification. The ensemble machine learning model was also evaluated using generated traffic, resulting in a decline in performance: accuracy dropped to 60.94%, precision to 44.44%, recall to 30.77%, and the F1-score to 61.54%.

The Internet of Things (IoT) has emerged as a crucial expansion of Internet services, linking numerous devices that produce and exchange substantial quantities of data. Strong defense against evolving cyberattacks that jeopardize the confidentiality, integrity, and availability (CIA Triad) of IoT systems is necessary due to this interconnectedness. Therefore, by controlling network activity to stop invasions and identify odd data patterns, intrusion detection systems can offer a robust layer of security for IoT systems. With a focus on identifying less frequent Advanced Persistent Threats (APT), this study aims to detect attacks directed at the CIA Triad. We examined two popular and unbalanced data sets, TON-IoT and BoT-IoT, for anomaly detection in IoT networks. By examining the usefulness of the ensemble learning and deep learning models in anomaly detection inside the IoT network data, we were able to study the security element of the network. The ensemble learning models considered are the following. Random Forest, XGBoost, and Stacking. The considered deep learning models are as follows: BiLSTM, GRU, Autoencoder, and Self-Taught Learning (STL: AE-SVM). The performance metrics studied are as follows: Accuracy, False Alert Rate, Recall, Precision, and F1-score. Regrettably, there were instances in which anomalies could not be detected using conventional techniques. This is because labeled data is scarce, particularly for APT attacks, and attack traffic is frequently sparse and noisy. We suggested a deep self-taught learning system (STL: AE-XGB) based on Transfer Learning to detect anomalies in IoT network data to tackle these difficult issues. To enhance classification performance and capture the changing landscape of attack methods in IoT networks. Our models automatically augment annotated data. The results show that our proposed model (STL: AE-XGB) outperforms the conventional models studied and provides interesting results in binary and multiclass classification.

While the rapid development of vaccines during the COVID-19 pandemic helped save billions of lives, a significant percentage of the population reported adverse reactions after vaccination. Post-vaccination surveys were conducted to understand the possible side effects. Analyzing and understanding the side effects enables the relevant stakeholders to gain more understanding of the vaccine and allows the individual to make informed decisions about whether to receive the vaccine. This work aims to identify a robust approach that can be used to predict the possibility of potential adverse symptoms. A dataset comprising 840 participants with 15 Post-Vaccination Symptoms was considered for the study. Synthetic Minority Oversampling TEchnique (SMOTE) was used to handle the class imbalance of the dataset. A combination of SMOTE and ensemble machine learning models was used to predict the adverse reactions to COVID-19 vaccines. The ensemble Machine Learning (ML) models that are considered for this study are Random Forest, Extreme Gradient Boosting Machine (XGBoost), Light Gradient Boosting Machine, and AdaBoost. The metrics accuracy, precision, recall and Receiver Operating Characteristic-Area Under the Curve (ROC-AUC) were used to measure the performance of the models. The dataset was pre-processed to handle missing values and one-hot encoding was applied to convert categorical variables into the numerical format. Insights into the data distribution and relationships between variables were gained through exploratory data and correlation analysis, respectively. Class imbalance in the target variables was addressed using SMOTE, resulting in a significantly improved F1-score and ROC-AUC score. Among the ensemble ML models, XGBoost delivered the best performance metrics. A combined performance score was calculated by averaging the F1-score and accuracy to identify the best model. XGBoost obtained the highest performance score among the ensemble ML models, and its performance is further enhanced by performing the threshold adjustment using the maximum F1- score strategy. The findings suggest that the combination of SMOTE and ensemble learning models with threshold adjustment provides a more efficient prediction of adverse effects after COVID-19 vaccination, aiding in healthcare decision-making.

Previous studies on landslide susceptibility (LS) and hazard (LH) assessments have overlooked a crucial measure for evaluating validation effectiveness, which involves the use of post-temporal landslide inventories instead of solely relying on historical data. Therefore, this study constructed a regional LH prediction and validation framework at different timescales using the landslide data from 2002 to 2021. Firstly, based on updated landslide data and conditioning factors, the datasets for susceptibility modelling were built for various periods. Then, the performances of traditional ma chine learning, ensemble and deep learning models were compared. Finally, the multi-period rainfall data and susceptibility results were used in innovative LH forecasting. Our study results show that the ensemble learning model (random forest) has stronger generalisation ability in different time periods, with all Area Under Curve (AUC) values exceeding 0.9. The hazard modelling framework based on rainfall erosion intensity and susceptibility can effectively forecast LH, and the shortened timescale for rainfall erosion intensity can improve the accuracy of LH forecasting. At the very high hazard level, the percentage of landslides forecast increased from 25.23% to 42.86%. This study comprehensively explores spatio-temporal dynamic changes in LH, accurately identifying areas posing threats to the National Transmission Line Protection Regions (NTLPR).

Retinal disorders, including diabetic retinopathy and macular degeneration due to aging, can lead to preventable blindness in diabetics. Vision loss caused by diseases that affect the retinal fundus cannot be reversed if not diagnosed and treated on time. This paper employs deep-learned feature extraction with ensemble learning models to improve the multi-disease classification of fundus images. This research presents a novel approach to the multi-classification of fundus images, utilizing deep-learned feature extraction techniques and ensemble learning to diagnose retinal disorders and diagnosing eye illnesses involving feature extraction, classification, and preprocessing of fundus images. The study involves analysis of deep learning and implementation of image processing. The ensemble learning classifiers have used retinal photos to increase the classification accuracy. The results demonstrate improved accuracy in diagnosing retinal disorders using DL feature extraction and ensemble learning models. The study achieved an overall accuracy of 87.2%, which is a significant improvement over the previous study. The deep learning models utilized in the study, including NASNetMobile, InceptionResNetV4, VGG16, and Xception, were effective in extracting relevant features from the Fundus images. The average F1-score for Extra Tree was 99%, while for Histogram Gradient Boosting and Random Forest, it was 98.8% and 98.4%, respectively. The results show that all three algorithms are suitable for the classification task. The combination of DenseNet feature extraction technique and RF, ET, and HG classifiers outperforms other techniques and classifiers. This indicates that using DenseNet for feature extraction can effectively enhance the performance of classifiers in the task of image classification.

The effective operation of water injection pumps is vital for enhancing oil recovery in the oil and gas industry. To ensure optimal pump performance and prevent unplanned downtime, this study focused on implementing predictive maintenance strategies. We began by identifying five critical operational parameters—Seal Pressure 1, Seal Pressure 2, Vibration Data for the Drive End (VIB DE), Vibration Data for the Non-Drive End (VIB NDE), and Ampere. These parameters were monitored and analyzed to evaluate their impact on pump performance and maintenance needs. To achieve this, we applied three machine learning algorithms: Extreme Gradient Boosting (XGBoost), Light Gradient-Boosting Machine (LGBM), and Random Forest. Each algorithm was independently trained and tested on the dataset corresponding to each operational parameter. We assessed their performance using key accuracy metrics, including R squared, Mean Absolute Error (MAE), and Root Mean Square Error (RMSE). Following this, we developed an Ensemble model, combining the predictive outputs of XGBoost, LGBM, and Random Forest. The Ensemble model was then applied to the same parameters to evaluate its ability to address the limitations observed in standalone models. The results demonstrated that the Ensemble model consistently delivered superior performance, achieving lower RMSE and MAE values and higher R squared coefficients across all parameters. This study culminates in the validation of the Ensemble model as a robust and reliable approach for predictive maintenance. By leveraging the strengths of multiple algorithms, the Ensemble model offers significant improvements in accuracy and reliability, contributing to more effective maintenance systems for the oil and gas industry.

Background and Objectives: We aimed to apply the ensemble machine learning model to diagnose thyroid cartilage invasion detected in computer tomography (CT) images in laryngeal cancers and evaluate the diagnostic performance of the model. Materials and Methods: A total of 313 patients were divided into two groups: the cartilage invasion group and the no cartilage invasion group. At least four CT slices were randomly selected for each patient, resulting in a total of 1251 images used in the study. A total of 619 axial CT images from the no cartilage invasion group and 632 axial CT images from the cartilage invasion group were used in the study. We reviewed the CT images and histopathological diagnoses in all cases to determine the invasion positive- or negative-status as a ground truth. The ensemble model, comprising ResNet50 and MobileNet deep learning architectures, was applied to CT images. Results: The following were obtained by the ensemble model with the test dataset: area under the curve (AUC) 0.99, and accuracy 96.54%. This model demonstrates a very high level of performance in detecting thyroid cartilage invasion. Conclusions: The ensemble machine learning model is an effective method for detecting neoplastic infiltration of the thyroid cartilage. Moreover, it may be a valuable diagnostic tool for clinicians in assessing disease prognosis and determining appropriate treatment strategies in laryngeal cancers. In conclusion, this model could be integrated into future clinical practice in laryngology and head and neck surgery for the detection of cartilage neoplastic infiltration.

With the fast growth of mobile technology, social media has become important for people to share their thoughts and feelings. Businesses and governments can make better strategic decisions when they know what the public thinks. Because of this, sentiment analysis is an important tool for figuring out how different people’s opinions are. This article presents a deeplearning ensemble model for sentiment analysis. The ensemble model proposed consists of three deep-learning models, Gated Recurrent Unit (GRU), Long Short-Term Memory (LSTM), and Bidirectional LSTM (BiLSTM), as base classifiers. AraBERT is responsible for presenting the textual input data into representative embeddings. The stacking ensemble model then captures the long-range dependencies in the embedding for a given class. As a meta-classifier, Support Vector Machine (SVM) then combines the predictions made by the stacking deep learning model. In addition, data augmentation with AraGPT was implemented to address the imbalanced dataset issues. The experimental results demonstrate that the proposed model outperforms the state-of-the-art models with an accuracy of 88.89%, 90.88%, and 88.23% on the HARD, BRAD, and Twitter datasets, respectively.

The reduction of NOx emissions is a paramount endeavor in contemporary engineering and energy production, as these emissions are closely linked to adverse environmental and health impacts. The prediction of NOx emission from gas turbines through several integrated data-driven machine learning methods has been evaluated in study. The study compares the performance of ensemble and conventional machine learning models, demonstrating superior accuracy achieved by the ensemble models. Specifically, the Random Forest model achieved an accuracy rate of 91.68%, XGBoost yielded an accuracy of 91.54%, and CATBoost exhibited the highest accuracy at 92.76%. These findings highlight the capability of data-driven machine learning techniques in enhancing NOx emission predictions in gas turbines. The improved prediction by ensembles can be utilized in the development and implementation of more effective control and mitigation strategies in practical applications. Through the application of these advanced machine learning approaches, the gas turbine industry can play a pivotal role in minimizing its environmental impact while optimizing operational efficiency. This study also provides valuable insights into the effectiveness of ensemble machine learning models, advancing our understanding of their capabilities in addressing the critical issue of NOx emissions from gas turbines.

Radiologists often face challenges in differentiating benign from malignant sacral bone lesions due to their similar imaging characteristics. This study aimed to develop an ensemble deep learning (DL) model that can preoperatively distinguish between benign and malignant sacral tumors using noncontrast computed tomography images. Preoperative sacral CT scans from 569 patients with confirmed sacral lesions were analyzed. Data from Center 1 were utilized in model development and internal test via fivefold cross-validation, and those from Centers 2 and 3 were employed in external test. Various ensemble models combining human-readable interpretation and DL were developed. The diagnostic performance of the models and radiologists was assessed using metrics such as precision, recall, accuracy, area under the curve (AUC), F1 score, and confusion matrix. Furthermore, the clinical benefits derived from radiologists' interpretations and supported by the DL model were evaluated. The ensemble model, which integrates 3D-DenseNet121 with human interpretation, exhibited the most robust performance. The ensemble model demonstrated high performance on the internal and external test sets and achieved AUCs of 0.9139 and 0.8713, F1 scores of 0.9054 and 0.8571, precision of 0.9041 and 0.8824, recall of 0.9136 and 0.8333, and accuracy of 0.8630 and 0.8182, respectively. Across the external test cohort, all radiologists experienced improvements in AUC, accuracy, sensitivity, and specificity. Notably, junior radiologists demonstrated significant improvements compared with senior radiologists. The potential clinical application of the DL model lies in its capacity to considerably enhance the diagnostic efficiency of radiologists. This study presents the first ensemble deep learning model integrating 3D-DenseNet121 with radiologists' interpretation for preoperative differentiation of sacral tumors on noncontrast CT that improved diagnostic performance across all experience levels, particularly for junior radiologists. First artificial intelligence-radiologist ensemble for noncontrast computed tomography (NCCT)-based sacral tumor classification. Boosts all radiologists' performance, with the greatest gains for juniors, potentially reducing referrals. Enables reliable NCCT diagnosis, overcoming contrast/magnetic resonance imaging dependency in musculoskeletal oncology.

Diagnosing and treating at-risk patients for chronic kidney disease (CKD) relies heavily on accurately classifying the disease. The use of deep learning models in healthcare research is receiving much interest due to recent developments in the field. CKD has many features; however, only some features contribute weightage for the classification task. Therefore, it is required to eliminate the irrelevant feature before applying the classification task. This paper proposed a hybrid feature selection method by combining the two feature selection techniques: the Boruta and the Recursive Feature Elimination (RFE) method. The features are ranked according to their importance for CKD classification using the Boruta algorithm and refined feature set using the RFE, which recursively eliminates the least important features. The hybrid feature selection method removes the feature with a low recursive score. Later, selected features are given input to the proposed ensemble deep learning method for classification. The experimental ensemble deep learning model with feature selection is compared to Support Vector Machine (SVM), Logistic Regression (LR), and Random Forest (RF) models with and without feature selection. When feature selection is used, the ensemble model improves accuracy by 2%. Experimental results found that these features, age, pus cell clumps, bacteria, and coronary artery disease, do not contribute much to accurate classification tasks. Accuracy, precision, and recall are used to evaluate the ensemble deep learning model.

The potential of generalized deep learning models developed for crop water estimation was examined in the current study. This study was conducted in a semiarid region of India, i.e., Karnataka, with daily climatic data (maximum and minimum air temperatures, maximum and minimum relative humidity, wind speed, sunshine hours, and rainfall) of 44 years (1976–2020) for twelve locations. The Extreme Gradient Boosting (XGBoost), Gradient Boosting (GB), and Random Forest (RF) are three ensemble deep learning models that were developed using all of the climatic data from a single location (Bengaluru) from January 1976 to December 2017 and then immediately applied at eleven different locations (Ballari, Chikmaglur, Chitradurga, Devnagiri, Dharwad, Gadag, Haveri, Koppal, Mandya, Shivmoga, and Tumkuru) without the need for any local calibration. For the test period of January 2018–June 2020, the model’s capacity to estimate the numerical values of crop water requirement (Penman-Monteith (P-M) ETo values) was assessed. The developed ensemble deep learning models were evaluated using the performance criteria of mean absolute error (MAE), average absolute relative error (AARE), coefficient of correlation (r), noise to signal ratio (NS), Nash–Sutcliffe efficiency (ɳ), and weighted standard error of estimate (WSEE). The results indicated that the WSEE values of RF, GB, and XGBoost models for each location were smaller than 1 mm per day, and the model’s effectiveness varied from 96% to 99% across various locations. While all of the deep learning models performed better with respect to the P-M ETo approach, the XGBoost model was able to estimate ETo with greater accuracy than the GB and RF models. The XGBoost model’s strong performance was also indicated by the decreased noise-to-signal ratio. Thus, in this study, a generalized mathematical model for short-term ETo estimates is developed using ensemble deep learning techniques. Because of this type of model’s accuracy in calculating crop water requirements and its ability for generalization, it can be effortlessly integrated with a real-time water management system or an autonomous weather station at the regional level.