Ensemble-Based Deep Learning for Bengali Accident Text Classification

Biopsy is considered the gold standard for diagnosing brain tumors, but its invasive nature can pose risks to patients. Additionally, tissue analysis can be cumbersome and inconsistent among observers. This research aims to develop a cost-effective, non-invasive, MRI-based computer-aided diagnosis tool that can reliably, accurately and swiftly identify brain tumor grades. Our system employs ensemble deep learning (EDL) within an MRI multiclass framework that includes five datasets: two-class (C2), three-class (C3), four-class (C4), five-class (C5) and six-class (C6). The EDL utilizes a majority voting algorithm to classify brain tumors by combining seven renowned deep learning (DL) models-EfficientNet, VGG16, ResNet18, GoogleNet, ResNet50, Inception-V3 and DarkNet-and seven machine learning (ML) models, including support vector machine, K-nearest neighbour, Naïve Bayes, decision tree, linear discriminant analysis, artificial neural network and random forest. Additionally, local interpretable model-agnostic explanations (LIME) are employed as an explainable AI algorithm, providing a visual representation of the CNN's internal workings to enhance the credibility of the results. Through extensive five-fold cross-validation experiments, the DL-based majority voting algorithm outperformed the ML-based majority voting algorithm, achieving the highest average accuracies of 100 ± 0.00%, 98.55 ± 0.35%, 98.47 ± 0.63%, 95.34 ± 1.17% and 96.61 ± 0.85% for the C2, C3, C4, C5 and C6 datasets, respectively. Majority voting algorithms typically yield consistent results across different folds of the brain tumor data and enhance performance compared to any individual deep learning and machine learning models.

Machine learning and especially deep learning have garnered tremendous popularity in recent years due to their increased performance over other methods. The availability of large amount of data has aided in the progress of deep learning. Nevertheless, deep learning models are opaque and often seen as black boxes. Thus, there is an inherent need to make the models interpretable, especially so in the medical domain. In this work, we propose a locally interpretable method, which is inspired by one of the recent tools that has gained a lot of interest, called local interpretable model-agnostic explanations (LIME). LIME generates single instance level explanation by artificially generating a dataset around the instance (by randomly sampling and using perturbations) and then training a local linear interpretable model. One of the major issues in LIME is the instability in the generated explanation, which is caused due to the randomly generated dataset. Another issue in these kind of local interpretable models is the local fidelity. We propose novel modifications to LIME by employing an autoencoder, which serves as a better weighting function for the local model. We perform extensive comparisons with different datasets and show that our proposed method results in both improved stability, as well as local fidelity.

Deep learning algorithms have achieved high performance accuracy in complex domains such as image classification, face recognition sentiment analysis, text classification, and speech understanding. Due to the nested non-linear structure of deep learning algorithms, these highly successful models are usually applied in a black-box manner, i.e., no information is provided about what exactly causes them to arrive at their predictions. The effectiveness of these systems is thus limited by the machine’s current inability to explain its decisions and actions to human users. Such a lack of transparency can be a major drawback. For instance, in medical applications the development of methods for visualizing, explaining, and interpreting deep learning models has recently attracted increasing attention. Explainable Machine Learning (XAI) or Interpretable Machine Learning (IML) programs aim to create a suite of machine learning techniques that produce more explainable models while maintaining the high level of accuracy. It also enables human users to understand, appropriately trust, and effectively manage the emerging generation of artificially intelligent partners. Our work summarizes recent developments in this field, and we present three classification tasks wherein we make use of LIME (Local Interpretable Model-Agnostic Explanations) to explain predictions of deep learning models. LIME attempts to make these complex models at least partly understandable by evaluating using three classification tasks. Our first evaluation is performed on natural language processing application for tweet data classification and our second evaluation is on biomedical signal classification cancer detection. Our third evaluation is on Windows PC malware classification in a cybersecurity domain. In all three case studies, the interpreted results are presented in terms of visualization.

BackgroundOxygen saturation, a key indicator of COVID-19 severity, poses challenges, especially in cases of silent hypoxemia. Electronic health records (EHRs) often contain supplemental oxygen information within clinical narratives. Streamlining patient identification based on oxygen levels is crucial for COVID-19 research, underscoring the need for automated classifiers in discharge summaries to ease the manual review burden on physicians.MethodWe analysed text lines extracted from anonymised COVID-19 patient discharge summaries in German to perform a binary classification task, differentiating patients who received oxygen supplementation and those who did not. Various machine learning (ML) algorithms, including classical ML to deep learning (DL) models, were compared. Classifier decisions were explained using Local Interpretable Model-agnostic Explanations (LIME), which visualize the model decisions.ResultClassical ML to DL models achieved comparable performance in classification, with an F-measure varying between 0.942 and 0.955, whereas the classical ML approaches were faster. Visualisation of embedding representation of input data reveals notable variations in the encoding patterns between classic and DL encoders. Furthermore, LIME explanations provide insights into the most relevant features at token level that contribute to these observed differences.ConclusionDespite a general tendency towards deep learning, these use cases show that classical approaches yield comparable results at lower computational cost. Model prediction explanations using LIME in textual and visual layouts provided a qualitative explanation for the model performance.

Memory forensics is an essential cybersecurity tool that comprehensively examines volatile memory to detect the malicious activity of fileless malware that can bypass disk analysis. Image-based detection techniques provide a promising solution by visualizing memory data into images to be used and analyzed by image processing tools and machine learning methods. However, the effectiveness of image-based data for detection and classification requires high computational efforts. This paper investigates the efficacy of texture-based methods in detecting and classifying memory-resident or fileless malware using different image resolutions, identifying the best feature descriptors, classifiers, and resolutions that accurately classify malware into specific families and differentiate them from benign software. Moreover, this paper uses both local and global descriptors, where local descriptors include Oriented FAST and Rotated BRIEF (ORB), Scale-Invariant Feature Transform (SIFT), and Histogram of Oriented Gradients (HOG) and global descriptors include Discrete Wavelet Transform (DWT), GIST, and Gray Level Co-occurrence Matrix (GLCM). The results indicate that as image resolution increases, most feature descriptors yield more discriminative features but require higher computational efforts in terms of time and processing resources. To address this challenge, this paper proposes a novel approach that integrates Local Interpretable Model-agnostic Explanations (LIME) with deep learning models to automatically identify and crop the most important regions of memory images. The LIME’s ROI was extracted based on ResNet50 and MobileNet models’ predictions separately, the images were resized to 128 × 128, and the sampling process was performed dynamically to speed up LIME computation. The ROIs of the images are cropped to new images with sizes of (100 × 100) in two stages: the coarse stage and the fine stage. The two generated LIME-based cropped images using ResNet50 and MobileNet are fed to the lightweight neural network to evaluate the effectiveness of the LIME-based identified regions. The results demonstrate that the LIME-based MobileNet model’s prediction improves the efficiency of the model by preserving important features with a classification accuracy of 85% on multi-class classification.

The new global pandemic caused by the 2019 novel coronavirus (COVID-19), novel coronavirus pneumonia, has spread rapidly around the world, causing enormous damage to daily life, public health security, and the global economy. Early detection and treatment of COVID-19 infected patients are critical to prevent the further spread of the epidemic. However, existing detection methods are unable to rapidly detect COVID-19 patients, so infected individuals are not detected in a timely manner, which complicates the prevention and control of COVID-19 to some extent. Therefore, it is crucial to develop a rapid and practical COVID-19 detection method. In this work, we explored the application of deep learning in COVID-19 detection to develop a rapid COVID-19 detection method. Existing studies have shown that novel coronavirus pneumonia has significant radiographic performance. In this study, we analyze and select the features of chest radiographs. We propose a chest X-Ray (CXR) classification method based on the selected features and investigate the application of transfer learning in detecting pneumonia and COVID-19. Furthermore, we combine the proposed CXR classification method based on selected features with transfer learning and ensemble learning and propose an ensemble deep learning model based on transfer learning called COVID-ensemble to diagnose pneumonia and COVID-19 using chest x-ray images. The model aims to provide an accurate diagnosis for binary classification (no finding/pneumonia) and multivariate classification (COVID-19/No findings/ Pneumonia). Our proposed CXR classification method based on selection features can significantly improve the CXR classification accuracy of the CNN model. Using this method, DarkNet19 improved its binary and triple classification accuracies by 3.5% and 5.78%, respectively. In addition, the COVIDensemble achieved 91.5% accuracy in the binary classification task and 91.11% in the multi-category classification task. The experimental results demonstrate that the COVID-ensemble can quickly and accurately detect COVID-19 and pneumonia automatically through X-ray images and that the performance of this model is superior to that of several existing methods. Our proposed COVID-ensemble can not only overcome the limitations of the conventional COVID-19 detection method RT-PCR and provide convenient and fast COVID-19 detection but also automatically detect pneumonia, thereby reducing the pressure on the medical staff. Using deep learning models to automatically diagnose COVID-19 and pneumonia from X-ray images can serve as a fast and efficient screening method for COVID-19 and pneumonia.

Although AI's development is remarkable, end users do not know how the AI has come to a specific conclusion due to the black-box nature of AI algorithms like deep learning. This has given rise to the field of explainable AI (XAI) where techniques are being developed to explain AI algorithms. One such technique is called Local Interpretable Model-Agnostic Explanations (LIME). LIME is popular because it is modelagnostic and works well with text, tabular and image data. While it has some good features, there are still room for improvements towards the original LIME algorithm especially it's stability. In this work, the LIME stability is being reviewed and three different approaches were investigated for its effectiveness in stability improvement which are; 1) using high sample size for stable ordering, 2) using an averaging method to reduce region flipping; and 3) to evaluate different super-pixels segmentation algorithms in generating stable LIME outcome. The experiment results shows a definite increase in the stability of the improved LIME compared to the baseline LIME and thus the reliability of using it practically.

ObjectiveAsthma, a chronic respiratory condition characterized by airway inflammation and constriction, affects millions of individuals worldwide, resulting in high healthcare expenses and a lower quality of life. Early prediction and control of asthma risk are critical for avoiding exacerbations and improving outcomes.MethodsIn this study, we describe a comprehensive asthma prediction model that uses machine learning and deep learning techniques to estimate asthma risk based on a variety of health and environmental parameters. Recursive feature elimination and Extra Trees Classifier were used to choose features, and the synthetic minority over-sampling approach was used to balance the dataset to overcome class imbalance. Hyperparameter tuning was used to optimize performance for 12 machine learning models such as extreme gradient boosting, Random Forest, and support vector machine as well as deep learning models, including multilayer perceptrons, convolutional neural networks, recurrent neural network, and artificial neural network.ResultsAfter hyperparameter adjustment, ensemble approaches that used both hard and soft voting were evaluated. When hyperparameter adjustment was used, the soft voting ensemble that combined XGBoost and CatBoost achieved the highest accuracy (93.61%). Shapley additive explanations and local interpretable model-agnostic explanations were employed to make predictions interpretable, providing information on feature contributions and boosting clinician confidence. A Flask server and web interface were also deployed, enabling real-time user interaction where patients and medical professionals could enter data and obtain asthma risk estimations immediately.ConclusionsThis study presents an accurate and explainable asthma risk prediction framework using ensemble machine and deep learning models, achieving 93.61% accuracy with real-time clinical applicability.

Electrocardiogram signal analysis can be time-consuming, tedious, and error-prone. Therefore, automated analysis is need of time that will assist clinicians in detecting cardiac abnormalities accurately and efficiently. Recently, deep learning models have shown unprecedented progress and strong arrhythmia classification capabilities, but their deployment in the medical sector is constrained due to their “black-box” nature. This paper proposes a robust explainability method to assist in explaining the underlying decision-making process in deep neural networks (DNN) and help provide feedback on biases that would benefit in improving DNN models. To achieve these objectives, initially, a deep learning model is trained on the MIT-BIH Arrhythmia Database and their classification performance is evaluated. The classification findings are then interpreted using the post-hoc explanation methods such as the SHapley Additive exPlanations (SHAP), Local Interpretable Model-Agnostic Explanations (LIME) and Gradient-weighted Class Activation Mapping (Grad-CAM) that interpret the decision rationale. As these methods are initially proposed for image applications therefore a new masking approach is proposed to cater these post-hoc explainability methods for ECG time-series data. After evaluating these methods for ECG arrhythmia classification, several drawbacks are drawn such as they fail to locate a feature’s importance if there are multiple occurrences of the same feature in a signal and also, SHAP and LIME perform random perturbations that sometimes produce unreliable explanations. Therefore, to overcome the drawbacks associated with these post-hoc explainability methods on time-series data, a novel K-GradCam method is proposed. The proposed K-GradCam method ensemble the benefits of these gradient based and perturbation based approaches and has demonstrated advantages over SHAP, Grad-CAM and LIME in terms of interpreting the models’ decisions. To compare the the proposed technique with post-hoc explainability methods quantitatively, the confidence index of the proposed method is evaluated using dice loss. The proposed method shares 71% similarity with SHAP and 81% similarity with Grad-CAM methods; however, it shares the benefits of both methods and is computationally faster than SHAP and LIME.

Classifying Bangla news is still a big problem in processing languages with few resources, however most recent studies emphasize performance metrics instead of model explainability. This study introduces an explainable AI-based system for the multiclass classification of Bangla news articles into four categories: Sports, International, Entertainment, and National, utilizing a dataset of 11,904 articles. We conducted a comprehensive comparison of classical machine learning (ML) and deep learning (DL) methodologies. We used Ridge Classifier, SGD Classifier, Multinomial Naive Bayes, and XGBoost to help us learn how to use machines. We got the best score of 92.55% accuracy by using a hard-voting group of the four best linear models. We make deep learning models like CNN-LSTM, Simple LSTM, Stacked LSTM, and GRU. When they work together, the three best models get 95.76% right, and when they work alone, they get 95.46% right. We used LIME (Local Interpretable Model-agnostic Explanations) to make the Ridge Classifier and Multinomial Naive Bayes models more accurate and easier to understand. The results show that DL ensembles are better at finding the right answer than more common ML models. However, linear ML models are faster and easier to learn, train, and use. This change made Bangla NLP systems more open and reliable. This is a great way to stay up to date on the news and learn about what's going on in places where Bengali is spoken.

This study uses deep learning methods to explore the Internet of Things (IoT) network intrusion detection method based on the CIC-IoT-2023 dataset. This dataset contains extensive data on real-life IoT environments. Based on this, this study proposes an effective intrusion detection method. Apply seven deep learning models, including Transformer, to analyze network traffic characteristics and identify abnormal behavior and potential intrusions through binary and multivariate classifications. Compared with other papers, we not only use a Transformer model, but we also consider the model’s performance in the multi-class classification. Although the accuracy of the Transformer model used in the binary classification is lower than that of DNN and CNN + LSTM hybrid models, it achieves better results in the multi-class classification. The accuracy of binary classification of our model is 0.74% higher than that of papers that also use Transformer on TON-IOT. In the multi-class classification, our best-performing model combination is Transformer, which reaches 99.40% accuracy. Its accuracy is 3.8%, 0.65%, and 0.29% higher than the 95.60%, 98.75%, and 99.11% figures recorded in papers using the same dataset, respectively.

This study aimed to create a caries classification scheme based on cone-beam computed tomography (CBCT) and develop two deep learning models to improve caries classification accuracy. A total of 2713 axial slices were obtained from CBCT images of 204 carious teeth. Both classification models were trained and tested using the same pretrained classification networks on the dataset, including ResNet50_vd, MobileNetV3_large_ssld, and ResNet50_vd_ssld. The first model was used directly to classify the original images (direct classification model). The second model incorporated a presegmentation step for interpretation (interpretable classification model). Performance evaluation metrics including accuracy, precision, recall, and F1 score were calculated. The Local Interpretable Model-agnostic Explanations (LIME) method was employed to elucidate the decision-making process of the two models. In addition, a minimum distance between caries and pulp was introduced for determining the treatment strategies for type II carious teeth. The direct model that utilized the ResNet50_vd_ssld network achieved top accuracy, precision, recall, and F1 score of 0.700, 0.786, 0.606, and 0.616, respectively. Conversely, the interpretable model consistently yielded metrics surpassing 0.917, irrespective of the network employed. The LIME algorithm confirmed the interpretability of the classification models by identifying key image features for caries classification. Evaluation of treatment strategies for type II carious teeth revealed a significant negative correlation (p < 0.01) with the minimum distance. These results demonstrated that the CBCT-based caries classification scheme and the two classification models appeared to be acceptable tools for the diagnosis and categorization of dental caries.

COVID-19 is highly infectious and causes acute respiratory disease. Machine learning (ML) and deep learning (DL) models are vital in detecting disease from computerized chest tomography (CT) scans. The DL models outperformed the ML models. For COVID-19 detection from CT scan images, DL models are used as end-to-end models. Thus, the performance of the model is evaluated for the quality of the extracted feature and classification accuracy. There are four contributions included in this work. First, this research is motivated by studying the quality of the extracted feature from the DL by feeding these extracted to an ML model. In other words, we proposed comparing the end-to-end DL model performance against the approach of using DL for feature extraction and ML for the classification of COVID-19 CT scan images. Second, we proposed studying the effect of fusing extracted features from image descriptors, e.g., Scale-Invariant Feature Transform (SIFT), with extracted features from DL models. Third, we proposed a new Convolutional Neural Network (CNN) to be trained from scratch and then compared to the deep transfer learning on the same classification problem. Finally, we studied the performance gap between classic ML models against ensemble learning models. The proposed framework is evaluated using a CT dataset, where the obtained results are evaluated using five different metrics The obtained results revealed that using the proposed CNN model is better than using the well-known DL model for the purpose of feature extraction. Moreover, using a DL model for feature extraction and an ML model for the classification task achieved better results in comparison to using an end-to-end DL model for detecting COVID-19 CT scan images. Of note, the accuracy rate of the former method improved by using ensemble learning models instead of the classic ML models. The proposed method achieved the best accuracy rate of 99.39%.

The Internet of Things (IoT) has emerged as a crucial expansion of Internet services, linking numerous devices that produce and exchange substantial quantities of data. Strong defense against evolving cyberattacks that jeopardize the confidentiality, integrity, and availability (CIA Triad) of IoT systems is necessary due to this interconnectedness. Therefore, by controlling network activity to stop invasions and identify odd data patterns, intrusion detection systems can offer a robust layer of security for IoT systems. With a focus on identifying less frequent Advanced Persistent Threats (APT), this study aims to detect attacks directed at the CIA Triad. We examined two popular and unbalanced data sets, TON-IoT and BoT-IoT, for anomaly detection in IoT networks. By examining the usefulness of the ensemble learning and deep learning models in anomaly detection inside the IoT network data, we were able to study the security element of the network. The ensemble learning models considered are the following. Random Forest, XGBoost, and Stacking. The considered deep learning models are as follows: BiLSTM, GRU, Autoencoder, and Self-Taught Learning (STL: AE-SVM). The performance metrics studied are as follows: Accuracy, False Alert Rate, Recall, Precision, and F1-score. Regrettably, there were instances in which anomalies could not be detected using conventional techniques. This is because labeled data is scarce, particularly for APT attacks, and attack traffic is frequently sparse and noisy. We suggested a deep self-taught learning system (STL: AE-XGB) based on Transfer Learning to detect anomalies in IoT network data to tackle these difficult issues. To enhance classification performance and capture the changing landscape of attack methods in IoT networks. Our models automatically augment annotated data. The results show that our proposed model (STL: AE-XGB) outperforms the conventional models studied and provides interesting results in binary and multiclass classification.

Prediction of the stage of cancer plays an important role in planning the course of treatment and has been largely reliant on imaging tools which do not capture molecular events that cause cancer progression. Gene-expression data-based analyses are able to identify these events, allowing RNA-sequence and microarray cancer data to be used for cancer analyses. Breast cancer is the most common cancer worldwide, and is classified into four stages - stages 1, 2, 3, and 4 [2]. While machine learning models have previously been explored to perform stage classification with limited success, multi-class stage classification has not had significant progress. There is a need for improved multi-class classification models, such as by investigating deep learning models. Gene-expression-based cancer data is characterised by the small size of available datasets, class imbalance, and high dimensionality. Class balancing methods must be applied to the dataset. Since all the genes are not necessary for stage prediction, retaining only the necessary genes can improve classification accuracy. The breast cancer samples are to be classified into 4 classes of stages 1 to 4. Invasive ductal carcinoma breast cancer samples are obtained from The Cancer Genome Atlas (TCGA) and Molecular Taxonomy of Breast Cancer International Consortium (METABRIC) datasets and combined. Two class balancing techniques are explored, synthetic minority oversampling technique (SMOTE) and SMOTE followed by random undersampling. A hybrid feature selection pipeline is proposed, with three pipelines explored involving combinations of filter and embedded feature selection methods: Pipeline 1 - minimum-redundancy maximum-relevancy (mRMR) and correlation feature selection (CFS), Pipeline 2 - mRMR, mutual information (MI) and CFS, and Pipeline 3 - mRMR and support vector machine-recursive feature elimination (SVM-RFE). The classification is done using deep learning models, namely deep neural network, convolutional neural network, recurrent neural network, a modified deep neural network, and an AutoKeras generated model. Classification performance post class-balancing and various feature selection techniques show marked improvement over classification prior to feature selection. The best multiclass classification was found to be by a deep neural network post SMOTE and random undersampling, and feature selection using mRMR and recursive feature elimination, with a Cohen-Kappa score of 0.303 and a classification accuracy of 53.1%. For binary classification into early and late-stage cancer, the best performance is obtained by a modified deep neural network (DNN) post SMOTE and random undersampling, and feature selection using mRMR and recursive feature elimination, with an accuracy of 81.0% and a Cohen-Kappa score (CKS) of 0.280. This pipeline also showed improved multiclass classification performance on neuroblastoma cancer data, with a best area under the receiver operating characteristic (auROC) curve score of 0.872, as compared to 0.71 obtained in previous work, an improvement of 22.81%. The results and analysis reveal that feature selection techniques play a vital role in gene-expression data-based classification, and the proposed hybrid feature selection pipeline improves classification performance. Multi-class classification is possible using deep learning models, though further improvement particularly in late-stage classification is necessary and should be explored further.