Multi-Dimensional Software Defect Prediction using Ensemble and Deep Learning

With the development of information and technology, especially with the boom in big data, healthcare support systems are becoming much better. Patient data can be collected, retrieved, and stored in real time. These data are valuable and meaningful for monitoring, diagnosing, and further applications in data analysis and decision-making. Essentially, the data can be divided into three types, namely, statistical, image-based, and sequential data. Each type has a different method of retrieval, processing, and deployment. Additionally, the application of machine learning (ML) and deep learning (DL) in healthcare support systems is growing more rapidly than ever. Numerous high-performance architectures are proposed to optimize decision-making. As reliability and stability are the most important factors in the healthcare support system, enhancing the predicted performance and maintaining the stability of the model are always the top priority. The main idea of our study comes from ensemble techniques. Numerous studies and data science competitions show that by combining several weak models into one, ensemble models can attain outstanding performance and reliability. We propose three deep ensemble learning (DEL) approaches, each with stable and reliable performance, that are workable on the above-mentioned data types. These are deep-stacked generalization ensemble learning, gradient deep learning boosting, and deep aggregation learning. The experiment results show that our proposed approaches achieve more vigorous and reliable performance than traditional ML and DL techniques on statistical, image-based, and sequential benchmark datasets. In particular, on the Heart Disease UCI dataset, representing the statistical type, the gradient deep learning boosting approach dominates the others with accuracy, recall, F1-score, Matthews correlation coefficient, and area under the curve values of 0.87, 0.81, 0.83, 0.73, and 0.91, respectively. On the X-ray dataset, representing the image-based type, the deep aggregation learning approach shows the highest performance with values of 0.91, 0.97, 0.93, 0.80, and 0.94, respectively. On the Depresjon dataset, representing the sequence type, the deep-stacked generalization ensemble learning approach outperforms the others with values of 0.91, 0.84, 0.86, 0.8, and 0.94, respectively. Overall, we conclude that applying DL models using our proposed approaches is a promising method for the healthcare support system to enhance prediction and diagnosis performance. Furthermore, our study reveals that these approaches are flexible and easy to apply to achieve optimal performance.

Order-up-to-level inventory optimization model using time-series demand forecasting with ensemble deep learning

<p>In recent years, disasters are more frequent and enormous due to global warming. In the field of hydrology, high-precision rainfall-runoff modeling is required. Recently, deep learning has been applied to rainfall-runoff modeling and shows high accuracy. It is also expected that the accuracy will be improved by using ensemble learning for deep learning. This study tried to improve the accuracy of river flow estimation by performing ensemble learning for deep learning. Stacking was used as the ensemble learning method. For deep learning, LSTM, CNN, and MLP was used and compared. XGBoost was used as the learning device used for ensemble learning. The target area was the Tedori River basin in Ishikawa Prefecture, Japan. In deep learning, the input data were daily average precipitation and temperature. In deep learning and ensemble learning, the target data was the daily average river flow. RMSE was used as the evaluation index. As a result, the accuracy was the highest after ensemble learning when using LSTM. It shows that the selection of the learning device is important for ensemble learning.</p>

Sentiment analysis is a subfield of Natural Language Processing (NLP) which tries to process a text to extract opinions or attitudes towards topics or entities. Recently, the use of deep learning methods for sentiment analysis has received noticeable attention from researchers. Generally, different deep learning methods have shown superb performance in sentiment analysis problem. However, deep learning models are different in nature and have different strengths and limitations. For example, convolutional neural networks are useful for extracting local structures from data, while recurrent models are able to learn order dependence in sequential data. In order to combine the advantages of different deep models, in this paper we have proposed a novel approach for aspect-based sentiment analysis which utilizes deep ensemble learning. In the proposed method, we first build four deep learning models, namely CNN, LSTM, BiLSTM and GRU. Then the outputs of these models are combined using stacking ensemble approach where we have used logistic regression as meta-learner. The results of applying the proposed method on the real datasets show that our method has increased the accuracy of aspect-based prediction by 5% to 20% compared to the basic deep learning methods.

The Internet of Things (IoT) has emerged as a crucial expansion of Internet services, linking numerous devices that produce and exchange substantial quantities of data. Strong defense against evolving cyberattacks that jeopardize the confidentiality, integrity, and availability (CIA Triad) of IoT systems is necessary due to this interconnectedness. Therefore, by controlling network activity to stop invasions and identify odd data patterns, intrusion detection systems can offer a robust layer of security for IoT systems. With a focus on identifying less frequent Advanced Persistent Threats (APT), this study aims to detect attacks directed at the CIA Triad. We examined two popular and unbalanced data sets, TON-IoT and BoT-IoT, for anomaly detection in IoT networks. By examining the usefulness of the ensemble learning and deep learning models in anomaly detection inside the IoT network data, we were able to study the security element of the network. The ensemble learning models considered are the following. Random Forest, XGBoost, and Stacking. The considered deep learning models are as follows: BiLSTM, GRU, Autoencoder, and Self-Taught Learning (STL: AE-SVM). The performance metrics studied are as follows: Accuracy, False Alert Rate, Recall, Precision, and F1-score. Regrettably, there were instances in which anomalies could not be detected using conventional techniques. This is because labeled data is scarce, particularly for APT attacks, and attack traffic is frequently sparse and noisy. We suggested a deep self-taught learning system (STL: AE-XGB) based on Transfer Learning to detect anomalies in IoT network data to tackle these difficult issues. To enhance classification performance and capture the changing landscape of attack methods in IoT networks. Our models automatically augment annotated data. The results show that our proposed model (STL: AE-XGB) outperforms the conventional models studied and provides interesting results in binary and multiclass classification.

The advent of deep learning techniques has ignited interest in medical image processing. The proposed work in this paper suggests one of the edge technologies in deep learning, which is recommended, based on a Radiomics feature extraction model for the effective detection of Kaposi sarcoma, a vascular skin lesion expression that indicates the most prevalent cancer in AIDS patients. This work investigates the role and impact of medical image fusion on deep feature learning based on ensemble learning in the medical domain. The model is crafted wherein the pre-built ResNet50 (Residual network) and Visual Geometry Group (VGG16) are fine-tuned and an ensemble learning approach is applied. The pre-defined CNN was incrementally regulated to determine the appropriate standards for classification efficiency improvements. Our findings show that layer-by-layer fine-tuning can improve the performance of middle and deep layers. This work would serve the purpose of masking and classification of skin lesion images, primarily sarcoma using an ensemble approach. Our proposed assisted framework could be deployed in assisting radiologists by classifying Kaposi sarcoma as well as other related skin lesion diseases, based on the positive classification findings.

Wind power prediction (WPP) is extremely important in promoting the power grid's consumption of wind power. To improve the accuracy of WPP, a three‐stage multiensemble short‐term WPP method based on ensemble learning and deep learning is proposed in this paper. In the first stage, variational mode decomposition and wavelet transform were applied to decompose the original data sequence into different frequency bands. In the second stage, based on the decomposition sequences, the stacked denoising autoencoder (SDAE), long short‐term memory (LSTM), and bidirectional long short‐term memory (BLSTM) were used to predict the sequences; and 42 submodels were obtained. In the third stage, a support vector machine (SVM) was applied to give weight to each submodel to obtain the final ensemble prediction results. Based on three‐stage integration, a new multi‐integration model is proposed that repeats the third‐stage integration operation. A case study is presented to verify the effectiveness and superiority of the proposed three‐stage multi‐integration WPP method. The normalized root mean square error (NRMSE) decreased by 0.0343 compared with LSTM, decreased by 0.0336 compared with BLSTM, and decreased by 0.0323 compared with SDAE, which demonstrated the effectiveness of the proposed new multistage ensemble and deep learning WPP method.

PurposeCoal is a critical global energy source, and fluctuations in its price significantly impact related enterprises' profitability. This study aims to develop a robust model for predicting the coal price index to enhance coal purchase strategies for coal-consuming enterprises and provide crucial information for global carbon emission reduction.Design/methodology/approachThe proposed coal price forecasting system combines data decomposition, semi-supervised feature engineering, ensemble learning and deep learning. It addresses the challenge of merging low-resolution and high-resolution data by adaptively combining both types of data and filling in missing gaps through interpolation for internal missing data and self-supervision for initiate/terminal missing data. The system employs self-supervised learning to complete the filling of complex missing data.FindingsThe ensemble model, which combines long short-term memory, XGBoost and support vector regression, demonstrated the best prediction performance among the tested models. It exhibited superior accuracy and stability across multiple indices in two datasets, namely the Bohai-Rim steam-coal price index and coal daily settlement price.Originality/valueThe proposed coal price forecasting system stands out as it integrates data decomposition, semi-supervised feature engineering, ensemble learning and deep learning. Moreover, the system pioneers the use of self-supervised learning for filling in complex missing data, contributing to its originality and effectiveness.

Network congestion arises from factors like bandwidth misallocation and increased node density leading to issues such as reduced packet delivery ratios and energy efficiency, increased packet loss and delay, and diminished Quality of Service and Quality of Experience. This study highlights the potential of deep learning and ensemble learning for network congestion analysis, which has been less explored compared to packet-loss based, delay-based, hybrid-based, and machine learning approaches, offering opportunities for advancement through parameter tuning, data labeling, architecture simulation, and activation function experiments, despite challenges posed by the scarcity of labeled data due to the high costs, time, computational resources, and human effort required for labeling. In this paper, we investigate network congestion prediction using deep learning and observe the results individually, as well as analyze ensemble learning outcomes using majority voting, from data that we recorded and clustered using K-Means. We leverage deep learning models including BPNN, CNN, LSTM, and hybrid LSTM-CNN architectures on 12 scenarios formed out of the combination of level datasets, normalization techniques, and number of recommended clusters and the results reveal that ensemble methods, particularly those integrating LSTM and CNN models (LSTM-CNN), consistently outperform individual deep learning models, demonstrating higher accuracy and stability across diverse datasets. Besides that, it is preferably recommended to use the QoS level dataset and the combinations of 3 clusters due to the most consistent evaluation results across different configurations and normalization strategies. The ensemble learning evaluation results show consistently high performance across various metrics, with accuracy, Matthews Correlation Coefficient, and Cohen's Kappa values nearing 100%, indicates excellent predictive capability and agreement. Hamming Loss remains minimal highlighting the low misclassification rates. Notably, this study advances predictive modeling in network management, offering strategies to enhance network efficiency and reliability amidst escalating traffic demands for more sustainable network operations.

Predicting benefit from immune checkpoint inhibitors in patients with non-small-cell lung cancer by CT-based ensemble deep learning: a retrospective study

- This paper presents a comparative analysis between a traditional convolutional neural network (CNN) model and a hybrid ResNet50 + AdaBoost architecture for non-invasive cardiovascular risk prediction using retinal fundus images. The baseline model, trained on a small dataset for binary classification, achieves reasonable accuracy but lacks scalability and granularity. In contrast, the proposed model leverages deep residual learning for feature extraction and ensemble-based classification to predict five distinct cardiovascular risk levels. Evaluated on a class-balanced dataset of over 11,500 images, the hybrid model achieved a testing accuracy of 91.48% and a macro F1-score of approximately 0.93. The system also integrates data visualization and database logging to support clinical explainability. The findings demonstrate the proposed method’s superiority in risk stratification, robustness, and deployment readiness for real-world applications. Key Words: Retinal Imaging, Cardiovascular Risk Prediction, ResNet50, AdaBoost, Deep Learning, Ensemble Learning, Fundus Images, Heart Attack Classification

A novel way for classifying cervical cells using Convolutional Neural Networks (CNN) is presented in this work. Features are learned automatically in this technique. Classification accuracy and model diversity can be enhanced using ensemble of CNN. This is a multi-step process and it requires the training of various CNN models and they must selected accordingly for ensemble. So, for resolving these issues and for creating expert medical decision making systems, Convolutional Neural Network and Random forest with Boosting called CNNRFB is used in current research work. For cervical cancer Prediction, an optimal prediction model can be constructed by medical practitioners using this solution. One dataset with all relevant information about various cervical cancer is created initially. In an image, without information loss, image pre-processing is done using super resolution technique. Self-similarity based Non-Local-Mean (NLM) metric is used as a base for this Super-Resolution (SR) algorithm. The NLM weight is used as a self-similarity metric under noisy environment for computing best self-example accurately. Deep ensemble learning methods are used for performing image classification and cancer prediction. System's accuracy are decided by dataset values and deep learning training technique. Python code is used for executing and simulating all the techniques. The experimental results shows the accuracy, recall, f-measure and precision produced by proposed technique shows better results when compared with the traditional approches.

<span lang="EN-US">One type of skin cancer that is considered a malignant tumor is melanoma. Such a dangerous disease can cause a lot of death in the world. The early detection of skin lesions becomes an important task in the diagnosis of skin cancer. Recently, a machine learning paradigm emerged known as deep learning (DL) utilized for skin lesions classification. However, in some previous studies by using seven class images diagnostic of skin lesions classification based on a single DL approach with CNNs architecture does not produce a satisfying performance. The DL approach allows the development of a medical image analysis system for improving performance, such as the deep convolutional neural networks (DCNNs) method. In this study, we propose an ensemble learning approach that combines three DCNNs architectures such as Inception V3, Inception ResNet V2 and DenseNet 201 for improving the performance in terms of accuracy, sensitivity, specificity, precision, and F1-score. Seven classes of dermoscopy image categories of skin lesions are utilized with 10015 dermoscopy images from well-known the HAM10000 dataset. The proposed model produces good classification performance with 97.23% accuracy, 90.12% sensitivity, 97.73% specificity, 82.01% precision, and 85.01% F1-Score. This method gives promising results in classifying skin lesions for cancer diagnosis.</span>

Protein interactions play an essential role in studying living systems and life phenomena. A considerable amount of literature has been published on analyzing and predicting protein interactions, such as support vector machine method, homology-based method and similarity-based method, each has its pros and cons. Most existing methods for predicting protein interactions require prior domain knowledge, making it difficult to effectively extract protein features. Single method is dissatisfactory in predicting protein interactions, declaring the need for a comprehensive method that combines the advantages of various methods. On this basis, a deep ensemble learning method called EnAmDNN (Ensemble Deep Neural Networks with Attention Mechanism) is proposed to predict protein interactions which is an appropriate candidate for comprehensive learning, combining multiple models, and considering the advantages of various methods. Particularly, it encode protein sequences by the local descriptor, auto covariance, conjoint triad, pseudo amino acid composition and combine the vector representation of each protein in the protein interaction network. Then it takes advantage of the multi-layer convolutional neural networks to automatically extract protein features and construct an attention mechanism to analyze deep-seated relationships between proteins. We set up four different structures of deep learning models. In the ensemble learning model, second layer data sets are generated with five-fold cross validation from basic learners, then predict the protein interaction network by combining 16 models. Results on five independent PPI data sets demonstrate that EnAmDNN achieves superior prediction performance than other comparing methods.

The new global pandemic caused by the 2019 novel coronavirus (COVID-19), novel coronavirus pneumonia, has spread rapidly around the world, causing enormous damage to daily life, public health security, and the global economy. Early detection and treatment of COVID-19 infected patients are critical to prevent the further spread of the epidemic. However, existing detection methods are unable to rapidly detect COVID-19 patients, so infected individuals are not detected in a timely manner, which complicates the prevention and control of COVID-19 to some extent. Therefore, it is crucial to develop a rapid and practical COVID-19 detection method. In this work, we explored the application of deep learning in COVID-19 detection to develop a rapid COVID-19 detection method. Existing studies have shown that novel coronavirus pneumonia has significant radiographic performance. In this study, we analyze and select the features of chest radiographs. We propose a chest X-Ray (CXR) classification method based on the selected features and investigate the application of transfer learning in detecting pneumonia and COVID-19. Furthermore, we combine the proposed CXR classification method based on selected features with transfer learning and ensemble learning and propose an ensemble deep learning model based on transfer learning called COVID-ensemble to diagnose pneumonia and COVID-19 using chest x-ray images. The model aims to provide an accurate diagnosis for binary classification (no finding/pneumonia) and multivariate classification (COVID-19/No findings/ Pneumonia). Our proposed CXR classification method based on selection features can significantly improve the CXR classification accuracy of the CNN model. Using this method, DarkNet19 improved its binary and triple classification accuracies by 3.5% and 5.78%, respectively. In addition, the COVIDensemble achieved 91.5% accuracy in the binary classification task and 91.11% in the multi-category classification task. The experimental results demonstrate that the COVID-ensemble can quickly and accurately detect COVID-19 and pneumonia automatically through X-ray images and that the performance of this model is superior to that of several existing methods. Our proposed COVID-ensemble can not only overcome the limitations of the conventional COVID-19 detection method RT-PCR and provide convenient and fast COVID-19 detection but also automatically detect pneumonia, thereby reducing the pressure on the medical staff. Using deep learning models to automatically diagnose COVID-19 and pneumonia from X-ray images can serve as a fast and efficient screening method for COVID-19 and pneumonia.