Advancing IoT Cybersecurity: Adaptive Threat Identification with Deep Learning in Cyber-Physical Systems

Radiologists often face challenges in differentiating benign from malignant sacral bone lesions due to their similar imaging characteristics. This study aimed to develop an ensemble deep learning (DL) model that can preoperatively distinguish between benign and malignant sacral tumors using noncontrast computed tomography images. Preoperative sacral CT scans from 569 patients with confirmed sacral lesions were analyzed. Data from Center 1 were utilized in model development and internal test via fivefold cross-validation, and those from Centers 2 and 3 were employed in external test. Various ensemble models combining human-readable interpretation and DL were developed. The diagnostic performance of the models and radiologists was assessed using metrics such as precision, recall, accuracy, area under the curve (AUC), F1 score, and confusion matrix. Furthermore, the clinical benefits derived from radiologists' interpretations and supported by the DL model were evaluated. The ensemble model, which integrates 3D-DenseNet121 with human interpretation, exhibited the most robust performance. The ensemble model demonstrated high performance on the internal and external test sets and achieved AUCs of 0.9139 and 0.8713, F1 scores of 0.9054 and 0.8571, precision of 0.9041 and 0.8824, recall of 0.9136 and 0.8333, and accuracy of 0.8630 and 0.8182, respectively. Across the external test cohort, all radiologists experienced improvements in AUC, accuracy, sensitivity, and specificity. Notably, junior radiologists demonstrated significant improvements compared with senior radiologists. The potential clinical application of the DL model lies in its capacity to considerably enhance the diagnostic efficiency of radiologists. This study presents the first ensemble deep learning model integrating 3D-DenseNet121 with radiologists' interpretation for preoperative differentiation of sacral tumors on noncontrast CT that improved diagnostic performance across all experience levels, particularly for junior radiologists. First artificial intelligence-radiologist ensemble for noncontrast computed tomography (NCCT)-based sacral tumor classification. Boosts all radiologists' performance, with the greatest gains for juniors, potentially reducing referrals. Enables reliable NCCT diagnosis, overcoming contrast/magnetic resonance imaging dependency in musculoskeletal oncology.

The Internet of Things (IoT) has emerged as a crucial expansion of Internet services, linking numerous devices that produce and exchange substantial quantities of data. Strong defense against evolving cyberattacks that jeopardize the confidentiality, integrity, and availability (CIA Triad) of IoT systems is necessary due to this interconnectedness. Therefore, by controlling network activity to stop invasions and identify odd data patterns, intrusion detection systems can offer a robust layer of security for IoT systems. With a focus on identifying less frequent Advanced Persistent Threats (APT), this study aims to detect attacks directed at the CIA Triad. We examined two popular and unbalanced data sets, TON-IoT and BoT-IoT, for anomaly detection in IoT networks. By examining the usefulness of the ensemble learning and deep learning models in anomaly detection inside the IoT network data, we were able to study the security element of the network. The ensemble learning models considered are the following. Random Forest, XGBoost, and Stacking. The considered deep learning models are as follows: BiLSTM, GRU, Autoencoder, and Self-Taught Learning (STL: AE-SVM). The performance metrics studied are as follows: Accuracy, False Alert Rate, Recall, Precision, and F1-score. Regrettably, there were instances in which anomalies could not be detected using conventional techniques. This is because labeled data is scarce, particularly for APT attacks, and attack traffic is frequently sparse and noisy. We suggested a deep self-taught learning system (STL: AE-XGB) based on Transfer Learning to detect anomalies in IoT network data to tackle these difficult issues. To enhance classification performance and capture the changing landscape of attack methods in IoT networks. Our models automatically augment annotated data. The results show that our proposed model (STL: AE-XGB) outperforms the conventional models studied and provides interesting results in binary and multiclass classification.

Diagnosing and treating at-risk patients for chronic kidney disease (CKD) relies heavily on accurately classifying the disease. The use of deep learning models in healthcare research is receiving much interest due to recent developments in the field. CKD has many features; however, only some features contribute weightage for the classification task. Therefore, it is required to eliminate the irrelevant feature before applying the classification task. This paper proposed a hybrid feature selection method by combining the two feature selection techniques: the Boruta and the Recursive Feature Elimination (RFE) method. The features are ranked according to their importance for CKD classification using the Boruta algorithm and refined feature set using the RFE, which recursively eliminates the least important features. The hybrid feature selection method removes the feature with a low recursive score. Later, selected features are given input to the proposed ensemble deep learning method for classification. The experimental ensemble deep learning model with feature selection is compared to Support Vector Machine (SVM), Logistic Regression (LR), and Random Forest (RF) models with and without feature selection. When feature selection is used, the ensemble model improves accuracy by 2%. Experimental results found that these features, age, pus cell clumps, bacteria, and coronary artery disease, do not contribute much to accurate classification tasks. Accuracy, precision, and recall are used to evaluate the ensemble deep learning model.

Churn prediction has become one of the core concepts in customer relationship management within the insurances, telecom, and internet service provider industries, which is essential in customer retention. Therefore, this study attempts to analyze the effectiveness of the advanced machine learning and deep learning models for churn prediction in the evaluation of the models’ performance across different sectors. This would help conclude whether the varied patterns of the churn throughout different sectors to the level that affects the model performance and to what extent. The work includes three datasets: namely, insurance churn, internet service provider customer churn, and Telecom churn datasets. The implementation and comparison conducted in this study of models include XGBoost, Convolutional Neural Networks (CNNs), and Ensemble Deep Learning with the pre-trained hybrid approach. The results show that the ensemble deep learning model outperforms other models in terms of accuracy and F1-score, achieving accuracies of up to 95.96% in the insurance churn dataset and of 98.42% in the telecom churn dataset. Moreover, traditional machine learning models like XGBoost also produced competitive results for selected datasets. The proposed deep learning ensembles reveal the strength and possibility for churn prediction and provide a benchmark for future research relevant to customer retention strategies. Also, the proposed ensemble deep learning model shows stable performance across different sectors, which reflects its ability to capture the varied churn patterns of different sectors.

The potential of generalized deep learning models developed for crop water estimation was examined in the current study. This study was conducted in a semiarid region of India, i.e., Karnataka, with daily climatic data (maximum and minimum air temperatures, maximum and minimum relative humidity, wind speed, sunshine hours, and rainfall) of 44 years (1976–2020) for twelve locations. The Extreme Gradient Boosting (XGBoost), Gradient Boosting (GB), and Random Forest (RF) are three ensemble deep learning models that were developed using all of the climatic data from a single location (Bengaluru) from January 1976 to December 2017 and then immediately applied at eleven different locations (Ballari, Chikmaglur, Chitradurga, Devnagiri, Dharwad, Gadag, Haveri, Koppal, Mandya, Shivmoga, and Tumkuru) without the need for any local calibration. For the test period of January 2018–June 2020, the model’s capacity to estimate the numerical values of crop water requirement (Penman-Monteith (P-M) ETo values) was assessed. The developed ensemble deep learning models were evaluated using the performance criteria of mean absolute error (MAE), average absolute relative error (AARE), coefficient of correlation (r), noise to signal ratio (NS), Nash–Sutcliffe efficiency (ɳ), and weighted standard error of estimate (WSEE). The results indicated that the WSEE values of RF, GB, and XGBoost models for each location were smaller than 1 mm per day, and the model’s effectiveness varied from 96% to 99% across various locations. While all of the deep learning models performed better with respect to the P-M ETo approach, the XGBoost model was able to estimate ETo with greater accuracy than the GB and RF models. The XGBoost model’s strong performance was also indicated by the decreased noise-to-signal ratio. Thus, in this study, a generalized mathematical model for short-term ETo estimates is developed using ensemble deep learning techniques. Because of this type of model’s accuracy in calculating crop water requirements and its ability for generalization, it can be effortlessly integrated with a real-time water management system or an autonomous weather station at the regional level.

An ensemble deep learning model for exhaust emissions prediction of heavy oil-fired boiler combustion

Deep learning (DL) has become a powerful tool for the recognition and classification of biological sequences. However, conventional single-architecture models often struggle with suboptimal predictive performance and high computational costs. To address these challenges, we present EnsembleDL-Lipo, an innovative ensemble deep learning framework that combines Convolutional Neural Networks (CNNs) and Deep Neural Networks (DNNs) to enhance the identification of lipocalin sequences. Lipocalins are multifunctional extracellular proteins involved in various diseases and stress responses, and their low sequence similarity and occurrence in the 'twilight zone' of sequence alignment present significant hurdles for accurate classification. These challenges necessitate efficient computational methods to complement traditional, labor-intensive experimental approaches. EnsembleDL-Lipo overcomes these issues by leveraging a set of PSSM-based features to train a large ensemble of deep learning models. The framework integrates multiple feature representations derived from position-specific scoring matrices (PSSMs), optimizing classification performance across diverse sequence patterns. The model achieved superior results on the training dataset, with an accuracy (ACC) of 97.65%, recall of 97.10%, Matthews correlation coefficient (MCC) of 0.95, and area under the curve (AUC) of 0.99. Validation on an independent test set further confirmed the robustness of the model, yielding an ACC of 95.79%, recall of 90.48%, MCC of 0.92, and AUC of 0.97. These results demonstrate that EnsembleDL-Lipo is a highly effective and computationally efficient tool for lipocalin sequence identification, significantly outperforming existing methods and offering strong potential for applications in biomarker discovery.

This paper introduces ensemble deep learning (EDL) models for accurate demodulation of Fiber Bragg Grating (FBG) sensor spectra, with input training data enhanced through the numerical square method (NSM) technique. The technique combines numerical squaring with an EDL to improve prediction accuracy in noisy and overlapping spectral conditions. The proposed system uses an FBG interrogator with sensors embedded in transparent liquid containers to detect liquid level changes via Bragg wavelength shifts. However, changes in the liquid level shift the float position, causing overlap or cross-talk between adjacent sensors. To address this, the NSM enhances signal clarity by boosting authentic FBG responses and minimizing noise. Single, double, and triple NSM operations are applied to the experimental data to sharpen the reflection peaks and enhance overall signal quality, ensuring the data are well-prepared for training the EDL model. The EDL model combines CNNs for spectral feature extraction and LSTMs and GRUs for sequential pattern recognition and efficient learning, effectively capturing both spectral and sequential features. The EDL model is trained on the dataset enhanced by the NSM technique and validated using unseen experimental data to evaluate its performance. The results confirm that NSM efficiently sharpens reflection peaks, improves signal clarity, and reduces noise in FBG spectra, while the EDL model effectively demodulates the overlapping spectra. Experimental results demonstrate that the integrated NSM-EDL approach outperforms traditional machine learning and standalone deep learning models in terms of prediction accuracy, minimal errors, and computational time. The proposed method is cost-effective, hardware-efficient, and ideal for real-time multiplexed FBG sensing applications, such as liquid level monitoring.

Order-up-to-level inventory optimization model using time-series demand forecasting with ensemble deep learning

COVID-19 is highly infectious and causes acute respiratory disease. Machine learning (ML) and deep learning (DL) models are vital in detecting disease from computerized chest tomography (CT) scans. The DL models outperformed the ML models. For COVID-19 detection from CT scan images, DL models are used as end-to-end models. Thus, the performance of the model is evaluated for the quality of the extracted feature and classification accuracy. There are four contributions included in this work. First, this research is motivated by studying the quality of the extracted feature from the DL by feeding these extracted to an ML model. In other words, we proposed comparing the end-to-end DL model performance against the approach of using DL for feature extraction and ML for the classification of COVID-19 CT scan images. Second, we proposed studying the effect of fusing extracted features from image descriptors, e.g., Scale-Invariant Feature Transform (SIFT), with extracted features from DL models. Third, we proposed a new Convolutional Neural Network (CNN) to be trained from scratch and then compared to the deep transfer learning on the same classification problem. Finally, we studied the performance gap between classic ML models against ensemble learning models. The proposed framework is evaluated using a CT dataset, where the obtained results are evaluated using five different metrics The obtained results revealed that using the proposed CNN model is better than using the well-known DL model for the purpose of feature extraction. Moreover, using a DL model for feature extraction and an ML model for the classification task achieved better results in comparison to using an end-to-end DL model for detecting COVID-19 CT scan images. Of note, the accuracy rate of the former method improved by using ensemble learning models instead of the classic ML models. The proposed method achieved the best accuracy rate of 99.39%.

Accurate finding of Knee Osteoarthritis (KOA) from structural Magnetic Resonance Imaging (MRI) is a difficult task and is greatly subject to user variation. Furthermore, the identification of knee osteoarthritis (KOA) from MRI scans presents a challenge due to the limited information available. A novel methodology using an ensemble Deep Learning algorithm, combining EfficientNet-B3 and ResNext-101 architectures, aims to forecast KOA advancement, bridging the identified gap in clinical trials. The study aims to develop a precise predictive model for knee osteoarthritis using advanced deep-learning architectures and structural MRI scan data. By utilizing an ensemble technique, the model's accuracy in predicting disease development is enhanced, surpassing the limitations of traditional biomarkers. The study used the Osteoarthritis Initiative dataset to develop an ensemble Deep Learning model that combined EfficientNet-B3 and ResNext-101 architectures. Techniques like cropping, gamma correction, and in-slice rotation were used to expand the dataset and improve the model's generalization capacity. The Deep Learning model demonstrated 93% validation accuracy on the OAI dataset, accurately capturing subtle patterns of knee osteoarthritis progression. Augmentation approaches enhanced its resilience. Our ensemble Deep Learning approach, using ResNext-101 and EfficientNet-B3 architectures, accurately predicts knee osteoarthritis courses using structural MRI data, demonstrating the importance of data augmentation for improved predictive tools.

Predicting benefit from immune checkpoint inhibitors in patients with non-small-cell lung cancer by CT-based ensemble deep learning: a retrospective study

Background: Breast cancer is one of the leading causes of death in women, making early detection through mammography crucial for improving survival rates. However, human interpretation of mammograms is often prone to diagnostic errors. This study addresses the challenge of improving the accuracy of breast cancer detection by leveraging advanced machine learning techniques. Methods: We propose an extended ensemble deep learning model that integrates three state-of-the-art convolutional neural network (CNN) architectures: VGG16, DenseNet121, and InceptionV3. The model utilizes multi-scale feature extraction to enhance the detection of both benign and malignant masses in mammograms. This ensemble approach is evaluated on two benchmark datasets: INbreast and CBIS-DDSM. Results: The proposed ensemble model achieved significant performance improvements. On the INbreast dataset, the ensemble model attained an accuracy of 90.1%, recall of 88.3%, and an F1-score of 89.1%. For the CBIS-DDSM dataset, the model reached 89.5% accuracy and 90.2% specificity. The ensemble method outperformed each individual CNN model, reducing both false positives and false negatives, thereby providing more reliable diagnostic results. Conclusions: The ensemble deep learning model demonstrated strong potential as a decision support tool for radiologists, offering more accurate and earlier detection of breast cancer. By leveraging the complementary strengths of multiple CNN architectures, this approach can improve clinical decision making and enhance the accessibility of high-quality breast cancer screening.

Coronavirus disease (COVID-19) is a viral infection caused by SARS-CoV-2. The modalities such as computed tomography (CT) have been successfully utilized for the early stage diagnosis of COVID-19 infected patients. Recently, many researchers have utilized deep learning models for the automated screening of COVID-19 suspected cases. An ensemble deep learning and Internet of Things (IoT) based framework is proposed for screening of COVID-19 suspected cases. Three well-known pretrained deep learning models are ensembled. The medical IoT devices are utilized to collect the CT scans, and automated diagnoses are performed on IoT servers. The proposed framework is compared with thirteen competitive models over a four-class dataset. Experimental results reveal that the proposed ensembled deep learning model yielded 98.98% accuracy. Moreover, the model outperforms all competitive models in terms of other performance metrics achieving 98.56% precision, 98.58% recall, 98.75% F-score, and 98.57% AUC. Therefore, the proposed framework can improve the acceleration of COVID-19 diagnosis.

Electricity load forecasting is a crucial undertaking within all the deregulated markets globally. Among the research challenges on a global scale, the investigation of deep transfer learning (DTL) in the field of electricity load forecasting represents a fundamental effort that can inform artificial intelligence applications in general. In this paper, a comprehensive study is reported regarding day-ahead electricity load forecasting. For this purpose, three sequence-to-sequence (Seq2seq) deep learning (DL) models are used, namely the multilayer perceptron (MLP), the convolutional neural network (CNN) and the ensemble learning model (ELM), which consists of the weighted combination of the outputs of MLP and CNN models. Also, the study focuses on the development of different forecasting strategies based on DTL, emphasizing the way the datasets are trained and fine-tuned for higher forecasting accuracy. In order to implement the forecasting strategies using deep learning models, load datasets from three Greek islands, Rhodes, Lesvos, and Chios, are used. The main purpose is to apply DTL for day-ahead predictions (1–24 h) for each month of the year for the Chios dataset after training and fine-tuning the models using the datasets of the three islands in various combinations. Four DTL strategies are illustrated. In the first strategy (DTL Case 1), each of the three DL models is trained using only the Lesvos dataset, while fine-tuning is performed on the dataset of Chios island, in order to create day-ahead predictions for the Chios load. In the second strategy (DTL Case 2), data from both Lesvos and Rhodes concurrently are used for the DL model training period, and fine-tuning is performed on the data from Chios. The third DTL strategy (DTL Case 3) involves the training of the DL models using the Lesvos dataset, and the testing period is performed directly on the Chios dataset without fine-tuning. The fourth strategy is a multi-task deep learning (MTDL) approach, which has been extensively studied in recent years. In MTDL, the three DL models are trained simultaneously on all three datasets and the final predictions are made on the unknown part of the dataset of Chios. The results obtained demonstrate that DTL can be applied with high efficiency for day-ahead load forecasting. Specifically, DTL Case 1 and 2 outperformed MTDL in terms of load prediction accuracy. Regarding the DL models, all three exhibit very high prediction accuracy, especially in the two cases with fine-tuning. The ELM excels compared to the single models. More specifically, for conducting day-ahead predictions, it is concluded that the MLP model presents the best monthly forecasts with MAPE values of 6.24% and 6.01% for the first two cases, the CNN model presents the best monthly forecasts with MAPE values of 5.57% and 5.60%, respectively, and the ELM model achieves the best monthly forecasts with MAPE values of 5.29% and 5.31%, respectively, indicating the very high accuracy it can achieve.