Optimizing lipocalin sequence classification with ensemble deep learning models.

Radiologists often face challenges in differentiating benign from malignant sacral bone lesions due to their similar imaging characteristics. This study aimed to develop an ensemble deep learning (DL) model that can preoperatively distinguish between benign and malignant sacral tumors using noncontrast computed tomography images. Preoperative sacral CT scans from 569 patients with confirmed sacral lesions were analyzed. Data from Center 1 were utilized in model development and internal test via fivefold cross-validation, and those from Centers 2 and 3 were employed in external test. Various ensemble models combining human-readable interpretation and DL were developed. The diagnostic performance of the models and radiologists was assessed using metrics such as precision, recall, accuracy, area under the curve (AUC), F1 score, and confusion matrix. Furthermore, the clinical benefits derived from radiologists' interpretations and supported by the DL model were evaluated. The ensemble model, which integrates 3D-DenseNet121 with human interpretation, exhibited the most robust performance. The ensemble model demonstrated high performance on the internal and external test sets and achieved AUCs of 0.9139 and 0.8713, F1 scores of 0.9054 and 0.8571, precision of 0.9041 and 0.8824, recall of 0.9136 and 0.8333, and accuracy of 0.8630 and 0.8182, respectively. Across the external test cohort, all radiologists experienced improvements in AUC, accuracy, sensitivity, and specificity. Notably, junior radiologists demonstrated significant improvements compared with senior radiologists. The potential clinical application of the DL model lies in its capacity to considerably enhance the diagnostic efficiency of radiologists. This study presents the first ensemble deep learning model integrating 3D-DenseNet121 with radiologists' interpretation for preoperative differentiation of sacral tumors on noncontrast CT that improved diagnostic performance across all experience levels, particularly for junior radiologists. First artificial intelligence-radiologist ensemble for noncontrast computed tomography (NCCT)-based sacral tumor classification. Boosts all radiologists' performance, with the greatest gains for juniors, potentially reducing referrals. Enables reliable NCCT diagnosis, overcoming contrast/magnetic resonance imaging dependency in musculoskeletal oncology.

This paper introduces ensemble deep learning (EDL) models for accurate demodulation of Fiber Bragg Grating (FBG) sensor spectra, with input training data enhanced through the numerical square method (NSM) technique. The technique combines numerical squaring with an EDL to improve prediction accuracy in noisy and overlapping spectral conditions. The proposed system uses an FBG interrogator with sensors embedded in transparent liquid containers to detect liquid level changes via Bragg wavelength shifts. However, changes in the liquid level shift the float position, causing overlap or cross-talk between adjacent sensors. To address this, the NSM enhances signal clarity by boosting authentic FBG responses and minimizing noise. Single, double, and triple NSM operations are applied to the experimental data to sharpen the reflection peaks and enhance overall signal quality, ensuring the data are well-prepared for training the EDL model. The EDL model combines CNNs for spectral feature extraction and LSTMs and GRUs for sequential pattern recognition and efficient learning, effectively capturing both spectral and sequential features. The EDL model is trained on the dataset enhanced by the NSM technique and validated using unseen experimental data to evaluate its performance. The results confirm that NSM efficiently sharpens reflection peaks, improves signal clarity, and reduces noise in FBG spectra, while the EDL model effectively demodulates the overlapping spectra. Experimental results demonstrate that the integrated NSM-EDL approach outperforms traditional machine learning and standalone deep learning models in terms of prediction accuracy, minimal errors, and computational time. The proposed method is cost-effective, hardware-efficient, and ideal for real-time multiplexed FBG sensing applications, such as liquid level monitoring.

The potential of generalized deep learning models developed for crop water estimation was examined in the current study. This study was conducted in a semiarid region of India, i.e., Karnataka, with daily climatic data (maximum and minimum air temperatures, maximum and minimum relative humidity, wind speed, sunshine hours, and rainfall) of 44 years (1976–2020) for twelve locations. The Extreme Gradient Boosting (XGBoost), Gradient Boosting (GB), and Random Forest (RF) are three ensemble deep learning models that were developed using all of the climatic data from a single location (Bengaluru) from January 1976 to December 2017 and then immediately applied at eleven different locations (Ballari, Chikmaglur, Chitradurga, Devnagiri, Dharwad, Gadag, Haveri, Koppal, Mandya, Shivmoga, and Tumkuru) without the need for any local calibration. For the test period of January 2018–June 2020, the model’s capacity to estimate the numerical values of crop water requirement (Penman-Monteith (P-M) ETo values) was assessed. The developed ensemble deep learning models were evaluated using the performance criteria of mean absolute error (MAE), average absolute relative error (AARE), coefficient of correlation (r), noise to signal ratio (NS), Nash–Sutcliffe efficiency (ɳ), and weighted standard error of estimate (WSEE). The results indicated that the WSEE values of RF, GB, and XGBoost models for each location were smaller than 1 mm per day, and the model’s effectiveness varied from 96% to 99% across various locations. While all of the deep learning models performed better with respect to the P-M ETo approach, the XGBoost model was able to estimate ETo with greater accuracy than the GB and RF models. The XGBoost model’s strong performance was also indicated by the decreased noise-to-signal ratio. Thus, in this study, a generalized mathematical model for short-term ETo estimates is developed using ensemble deep learning techniques. Because of this type of model’s accuracy in calculating crop water requirements and its ability for generalization, it can be effortlessly integrated with a real-time water management system or an autonomous weather station at the regional level.

Diagnosing and treating at-risk patients for chronic kidney disease (CKD) relies heavily on accurately classifying the disease. The use of deep learning models in healthcare research is receiving much interest due to recent developments in the field. CKD has many features; however, only some features contribute weightage for the classification task. Therefore, it is required to eliminate the irrelevant feature before applying the classification task. This paper proposed a hybrid feature selection method by combining the two feature selection techniques: the Boruta and the Recursive Feature Elimination (RFE) method. The features are ranked according to their importance for CKD classification using the Boruta algorithm and refined feature set using the RFE, which recursively eliminates the least important features. The hybrid feature selection method removes the feature with a low recursive score. Later, selected features are given input to the proposed ensemble deep learning method for classification. The experimental ensemble deep learning model with feature selection is compared to Support Vector Machine (SVM), Logistic Regression (LR), and Random Forest (RF) models with and without feature selection. When feature selection is used, the ensemble model improves accuracy by 2%. Experimental results found that these features, age, pus cell clumps, bacteria, and coronary artery disease, do not contribute much to accurate classification tasks. Accuracy, precision, and recall are used to evaluate the ensemble deep learning model.

Churn prediction has become one of the core concepts in customer relationship management within the insurances, telecom, and internet service provider industries, which is essential in customer retention. Therefore, this study attempts to analyze the effectiveness of the advanced machine learning and deep learning models for churn prediction in the evaluation of the models’ performance across different sectors. This would help conclude whether the varied patterns of the churn throughout different sectors to the level that affects the model performance and to what extent. The work includes three datasets: namely, insurance churn, internet service provider customer churn, and Telecom churn datasets. The implementation and comparison conducted in this study of models include XGBoost, Convolutional Neural Networks (CNNs), and Ensemble Deep Learning with the pre-trained hybrid approach. The results show that the ensemble deep learning model outperforms other models in terms of accuracy and F1-score, achieving accuracies of up to 95.96% in the insurance churn dataset and of 98.42% in the telecom churn dataset. Moreover, traditional machine learning models like XGBoost also produced competitive results for selected datasets. The proposed deep learning ensembles reveal the strength and possibility for churn prediction and provide a benchmark for future research relevant to customer retention strategies. Also, the proposed ensemble deep learning model shows stable performance across different sectors, which reflects its ability to capture the varied churn patterns of different sectors.

Order-up-to-level inventory optimization model using time-series demand forecasting with ensemble deep learning

With the development of information and technology, especially with the boom in big data, healthcare support systems are becoming much better. Patient data can be collected, retrieved, and stored in real time. These data are valuable and meaningful for monitoring, diagnosing, and further applications in data analysis and decision-making. Essentially, the data can be divided into three types, namely, statistical, image-based, and sequential data. Each type has a different method of retrieval, processing, and deployment. Additionally, the application of machine learning (ML) and deep learning (DL) in healthcare support systems is growing more rapidly than ever. Numerous high-performance architectures are proposed to optimize decision-making. As reliability and stability are the most important factors in the healthcare support system, enhancing the predicted performance and maintaining the stability of the model are always the top priority. The main idea of our study comes from ensemble techniques. Numerous studies and data science competitions show that by combining several weak models into one, ensemble models can attain outstanding performance and reliability. We propose three deep ensemble learning (DEL) approaches, each with stable and reliable performance, that are workable on the above-mentioned data types. These are deep-stacked generalization ensemble learning, gradient deep learning boosting, and deep aggregation learning. The experiment results show that our proposed approaches achieve more vigorous and reliable performance than traditional ML and DL techniques on statistical, image-based, and sequential benchmark datasets. In particular, on the Heart Disease UCI dataset, representing the statistical type, the gradient deep learning boosting approach dominates the others with accuracy, recall, F1-score, Matthews correlation coefficient, and area under the curve values of 0.87, 0.81, 0.83, 0.73, and 0.91, respectively. On the X-ray dataset, representing the image-based type, the deep aggregation learning approach shows the highest performance with values of 0.91, 0.97, 0.93, 0.80, and 0.94, respectively. On the Depresjon dataset, representing the sequence type, the deep-stacked generalization ensemble learning approach outperforms the others with values of 0.91, 0.84, 0.86, 0.8, and 0.94, respectively. Overall, we conclude that applying DL models using our proposed approaches is a promising method for the healthcare support system to enhance prediction and diagnosis performance. Furthermore, our study reveals that these approaches are flexible and easy to apply to achieve optimal performance.

Network congestion arises from factors like bandwidth misallocation and increased node density leading to issues such as reduced packet delivery ratios and energy efficiency, increased packet loss and delay, and diminished Quality of Service and Quality of Experience. This study highlights the potential of deep learning and ensemble learning for network congestion analysis, which has been less explored compared to packet-loss based, delay-based, hybrid-based, and machine learning approaches, offering opportunities for advancement through parameter tuning, data labeling, architecture simulation, and activation function experiments, despite challenges posed by the scarcity of labeled data due to the high costs, time, computational resources, and human effort required for labeling. In this paper, we investigate network congestion prediction using deep learning and observe the results individually, as well as analyze ensemble learning outcomes using majority voting, from data that we recorded and clustered using K-Means. We leverage deep learning models including BPNN, CNN, LSTM, and hybrid LSTM-CNN architectures on 12 scenarios formed out of the combination of level datasets, normalization techniques, and number of recommended clusters and the results reveal that ensemble methods, particularly those integrating LSTM and CNN models (LSTM-CNN), consistently outperform individual deep learning models, demonstrating higher accuracy and stability across diverse datasets. Besides that, it is preferably recommended to use the QoS level dataset and the combinations of 3 clusters due to the most consistent evaluation results across different configurations and normalization strategies. The ensemble learning evaluation results show consistently high performance across various metrics, with accuracy, Matthews Correlation Coefficient, and Cohen's Kappa values nearing 100%, indicates excellent predictive capability and agreement. Hamming Loss remains minimal highlighting the low misclassification rates. Notably, this study advances predictive modeling in network management, offering strategies to enhance network efficiency and reliability amidst escalating traffic demands for more sustainable network operations.

Document classification is a prevalent task in Natural Language Processing (NLP), which has an extensive range of applications in the biomedical domains such as biomedical literature indexing, automatic diagnosis codes assignment, tweets classification for public health topics, and patient safety reports classification. Nevertheless, manual classification of biomedical articles published every year into specific predefined categories becomes a cumbersome task. Hence, building an automatic document classification for biomedical databases emerges as a significant task among the scientific community. In recent years, Deep Learning (DL) models like Deep Neural Networks (DNN), Convolution Neural Networks (CNN), Recurrent Neural Networks (RNN), and Ensemble Deep Learning models are widely used in the area of text document classification for better classification performance compared to Machine Learning (ML) algorithms. The major advantage of using DL models in document classification is that it provides rich semantic and grammatical information for document representation through pre-trained word embedding. Hence, this paper investigates the deployment of the various state-of-the-art DL based classification models in automatic classification of benchmark biomedical datasets. Finally, the performance of all the aforementioned constitutional classifiers is compared and evaluated through the well-defined performance evaluation metrics such as accuracy, precision, recall, and f1-measure.

Accurate finding of Knee Osteoarthritis (KOA) from structural Magnetic Resonance Imaging (MRI) is a difficult task and is greatly subject to user variation. Furthermore, the identification of knee osteoarthritis (KOA) from MRI scans presents a challenge due to the limited information available. A novel methodology using an ensemble Deep Learning algorithm, combining EfficientNet-B3 and ResNext-101 architectures, aims to forecast KOA advancement, bridging the identified gap in clinical trials. The study aims to develop a precise predictive model for knee osteoarthritis using advanced deep-learning architectures and structural MRI scan data. By utilizing an ensemble technique, the model's accuracy in predicting disease development is enhanced, surpassing the limitations of traditional biomarkers. The study used the Osteoarthritis Initiative dataset to develop an ensemble Deep Learning model that combined EfficientNet-B3 and ResNext-101 architectures. Techniques like cropping, gamma correction, and in-slice rotation were used to expand the dataset and improve the model's generalization capacity. The Deep Learning model demonstrated 93% validation accuracy on the OAI dataset, accurately capturing subtle patterns of knee osteoarthritis progression. Augmentation approaches enhanced its resilience. Our ensemble Deep Learning approach, using ResNext-101 and EfficientNet-B3 architectures, accurately predicts knee osteoarthritis courses using structural MRI data, demonstrating the importance of data augmentation for improved predictive tools.

Deep learning models, especially deep convolutional neural networks (CNNs), have been intensively investigated for hyperspectral image (HSI) classification due to their powerful feature extraction ability. In the same manner, ensemble-based learning systems have demonstrated high potential to effectively perform supervised classification. In order to boost the performance of deep learning-based HSI classification, the idea of deep learning ensemble framework is proposed here, which is loosely based on the integration of deep learning model and random subspace-based ensemble learning. Specifically, two deep learning ensemble-based classification methods (i.e., CNN ensemble and deep residual network ensemble) are proposed. CNNs or deep residual networks are used as individual classifiers and random subspaces contribute to diversify the ensemble system in a simple yet effective manner. Moreover, to further improve the classification accuracy, transfer learning is investigated in this study to transfer the learnt weights from one individual classifier to another (i.e., CNNs). This mechanism speeds up the learning stage. Experimental results with widely used hyperspectral datasets indicate that the proposed deep learning ensemble system provides competitive results compared with state-of-the-art methods in terms of classification accuracy. The combination of deep learning and ensemble learning provides a significant potential for reliable HSI classification.

Securing Internet of Things (IoT)-enabled Cyber-Physical Systems (CPSs) can be challenging because security solutions intended for typical IT/OT systems may not be as effective in a CPS setting. The goal of this study is to create a mechanism for identifying and attributing two-level ensemble attacks that are specifically designed for use against Industrial Control Systems (ICSs). An original ensemble deep representation learning model is combined with decision tree algorithm to identify assaults on unbalanced ICS environments at the first level. An attack attribution network, which constitutes a collection of deep neural networks, is formed at the second level. The proposed model is tested using real-world datasets, notably those pertaining to water purification and gas pipelines. The results demonstrate that the proposed strategy outperforms other strategies with comparable computing complexity and that the recommended model outperforms the existing mechanisms.

To evaluate the feasibility of using deep learning models applied to digital breast tomosynthesis (DBT) images for non-invasive prediction of breast cancer biomarkers, including estrogen receptor (ER), progesterone receptor (PR), human epithelial growth factor receptor 2 (HER2), Ki-67 proliferation index, and triple-negative breast cancer (TNBC). In this retrospective study, patients with histopathologically-confirmed, invasive breast cancer were included. Furthermore, all included patients had complete, immunohistochemically-assessed biomarker data available. For each case, a representative DBT slice showing the tumor was selected and preprocessed using histogram equalization. Two pretrained convolutional neural networks (VGG19 and ResNet50) were fine-tuned for binary classification of each biomarker. Model performance was evaluated using accuracy, area under the curve (AUC), F1 score, and Matthews correlation coefficient. The study sample included 43 anonymized female patients. Deep learning models achieved strong predictive performance for ER (AUC = 0.81) and TNBC (AUC = 0.93). HER2 (AUC = 0.74) and Ki-67 index (AUC = 0.70) were predicted with moderate accuracy. PR results varied, with VGG19 reaching AUC = 0.76 while ResNet50 performed poorly (AUC = 0.24). Deep learning models applied to DBT images enabled non-invasive prediction of some key breast cancer biomarkers, especially ER status and TNBC type. This approach may function as a virtual biopsy to complement histopathology, guide biopsy targeting, and support treatment planning. Although preliminary, the findings highlight the potential of artificial intelligence-enhanced DBT assessment and warrant validation in larger, multi-center prospective studies.

Finding methods to accurately predict the final infarct volumes for acute ischemic stroke patients with full or no recanalization would significantly help to evaluate the potential benefits of thrombolytic therapy. We proposed such a method by constructing a model of ensemble deep learning and machine learning using diffusion-weighted imaging (DWI) only. The proposed prediction model (named AUNet) combines an adaptive linear ensemble model (ALEM) of machine learning and a deep U-Net network with an accelerated non-local module (U-NL-Net) to learn voxel-wise and spatial features, respectively. Of 40 patients with acute ischemic stroke who received thrombolytic therapy, 17 were fully recanalized, 14 were not recanalized, and nine were partially recanalized. The AUNet was separately trained for full recanalization conditions (AUNetR) and no recanalization (AUNetN) as the best and worst outcomes of thrombolysis, respectively. AUNet performed significantly better in predicting the final infarct volumes in both the recanalization and non-recanalization conditions [area under the receiver operating characteristic curve (AUC) =0.898±0.022, recanalization; AUC =0.875±0.036, non-recanalization: Matthew's correlation coefficient (MCC) =0.863±0.033, recanalization; MCC =0.851±0.025, non-recanalization] than the fixed-thresholding method (AUC =0.776±0.021, P<0.0001, recanalization; AUC =0.692±0.023, P<0.0001, non-recanalization: MCC =0.742±0.035, recanalization; MCC =0.671±0.024, non-recanalization), the logistic regression method (AUC =0.797±0.023, P<0.003, recanalization; AUC =0.751±0.030, P<0.003, non-recanalization: MCC =0.762±0.035, recanalization; MCC =0.730±0.031, non-recanalization), and a recently developed convolutional neural network (AUC =0.814±0.013, P<0.003, recanalization; AUC =0.781±0.027, P<0.003, non-recanalization: MCC =792±0.022, recanalization; MCC =0.758±0.016, non-recanalization). The potential benefit of thrombolysis calculated from AUNetR and AUNetN showed large individual differences (from 12.81% to 239.73%). AUNet improved predictive accuracy over current state-of-the-art methods. More importantly, the accurate prediction of infarct volumes under different recanalization conditions may provide benefitial information for physicians in selecting suitable patients for thrombolytic therapy.

Predicting benefit from immune checkpoint inhibitors in patients with non-small-cell lung cancer by CT-based ensemble deep learning: a retrospective study