Quantum Deep Reinforcement Learning for Digital Twin-Enabled 6G Networks and Semantic Communications: Considerations for Adoption and Security

The synchronization of digital twins (DT) serves as the cornerstone for effective operation of the DT framework. However, the limitations of channel capacity can greatly affect the data transmission efficiency of wireless communication. Unlike traditional communication methods, semantic communication transmits the intended meanings of physical objects instead of raw data, effectively saving bandwidth resource and reducing DT synchronization latency. Hence, we are committed to integrating semantic communication into the DT synchronization framework within the mobile edge computing system, aiming to enhance the DT synchronization efficiency of user devices (UDs). Our goal is to minimize the average DT synchronization latency of all UDs by jointly optimizing the synchronization strategy, transmission power of UDs, and computational resource allocation for both UDs and base station. The formulated problem involves sequential decision-making across multiple coherent time slots. Furthermore, the mobility of UDs introduces uncertainties into the decision-making process. To solve this challenging optimization problem efficiently, we propose a soft actor-critic-based deep reinforcement learning algorithm to optimize synchronization strategy and resource allocation. Numerical results demonstrate that our proposed algorithm can reduce synchronization latency by up to 13.2% and improve synchronization efficiency compared to other benchmark schemes.

Digital twins are virtual models of physical artefacts that may or may not be synchronously connected, and that can be used to simulate their behavior. They are widely used in several domains such as manufacturing and automotive to enable achieving specific quality goals. In the health domain, so-called digital patient twins have been understood as virtual models of patients generated from population data and/or patient data, including, for example, real-time feedback from wearables. Along with the growing impact of data science technologies like artificial intelligence, novel health data ecosystems centered around digital patient twins could be developed. This paves the way for improved health monitoring and facilitation of personalized therapeutics based on management, analysis, and interpretation of medical data via digital patient twins. The utility and feasibility of digital patient twins in routine medical processes are still limited, despite practical endeavors to create digital twins of physiological functions, single organs, or holistic models. Moreover, reliable simulations for the prediction of individual drug responses are still missing. However, these simulations would be one important milestone for truly personalized therapeutics. Another prerequisite for this would be individualized pharmaceutical manufacturing with subsequent obstacles, such as low automation, scalability, and therefore high costs. Additionally, regulatory challenges must be met thus calling for more digitalization in this area. Therefore, this narrative mini-review provides a discussion on the potentials and limitations of digital patient twins, focusing on their potential bridging function for personalized therapeutics and an individualized pharmaceutical manufacturing while also looking at the regulatory impacts.

PurposeTo conduct a head-to-head comparison between deep learning (DL) and radiomics models across institutions for predicting microvascular invasion (MVI) in hepatocellular carcinoma (HCC) and to investigate the model robustness and generalizability through rigorous internal and external validation.MethodsThis retrospective study included 2304 preoperative images of 576 HCC lesions from two centers, with MVI status determined by postoperative histopathology. We developed DL and radiomics models for predicting the presence of MVI using B-mode ultrasound, contrast-enhanced ultrasound (CEUS) at the arterial, portal, and delayed phases, and a combined modality (B + CEUS). For radiomics, we constructed models with enlarged vs. original regions of interest (ROIs). A cross-validation approach was performed by training models on one center’s dataset and validating the other, and vice versa. This allowed assessment of the validity of different ultrasound modalities and the cross-center robustness of the models. The optimal model combined with alpha-fetoprotein (AFP) was also validated. The head-to-head comparison was based on the area under the receiver operating characteristic curve (AUC).ResultsThirteen DL models and 25 radiomics models using different ultrasound modalities were constructed and compared. B + CEUS was the optimal modality for both DL and radiomics models. The DL model achieved AUCs of 0.802–0.818 internally and 0.667–0.688 externally across the two centers, whereas radiomics achieved AUCs of 0.749–0.869 internally and 0.646–0.697 externally. The radiomics models showed overall improvement with enlarged ROIs (P < 0.05 for both CEUS and B + CEUS modalities). The DL models showed good cross-institutional robustness (P > 0.05 for all modalities, 1.6–2.1% differences in AUC for the optimal modality), whereas the radiomics models had relatively limited robustness across the two centers (12% drop-off in AUC for the optimal modality). Adding AFP improved the DL models (P < 0.05 externally) and well maintained the robustness, but did not benefit the radiomics model (P > 0.05).ConclusionCross-institutional validation indicated that DL demonstrated better robustness than radiomics for preoperative MVI prediction in patients with HCC, representing a promising solution to non-standardized ultrasound examination procedures.

COVID-19 is highly infectious and causes acute respiratory disease. Machine learning (ML) and deep learning (DL) models are vital in detecting disease from computerized chest tomography (CT) scans. The DL models outperformed the ML models. For COVID-19 detection from CT scan images, DL models are used as end-to-end models. Thus, the performance of the model is evaluated for the quality of the extracted feature and classification accuracy. There are four contributions included in this work. First, this research is motivated by studying the quality of the extracted feature from the DL by feeding these extracted to an ML model. In other words, we proposed comparing the end-to-end DL model performance against the approach of using DL for feature extraction and ML for the classification of COVID-19 CT scan images. Second, we proposed studying the effect of fusing extracted features from image descriptors, e.g., Scale-Invariant Feature Transform (SIFT), with extracted features from DL models. Third, we proposed a new Convolutional Neural Network (CNN) to be trained from scratch and then compared to the deep transfer learning on the same classification problem. Finally, we studied the performance gap between classic ML models against ensemble learning models. The proposed framework is evaluated using a CT dataset, where the obtained results are evaluated using five different metrics The obtained results revealed that using the proposed CNN model is better than using the well-known DL model for the purpose of feature extraction. Moreover, using a DL model for feature extraction and an ML model for the classification task achieved better results in comparison to using an end-to-end DL model for detecting COVID-19 CT scan images. Of note, the accuracy rate of the former method improved by using ensemble learning models instead of the classic ML models. The proposed method achieved the best accuracy rate of 99.39%.

This study introduces a new method for optimizing evolutionary algorithms to learn characteristics of unknown solar energy systems or subsystems. When used together, Genetic Algorithm (GA) and Deep Learning methods allow for precise PV system calibration. By combining GA's optimization powers with deep learning models' pattern recognition abilities, this technique achieves precise system calibration by fine-tuning model parameters. One way to create a connection between variables like irradiance, temperature, and voltage, which are input parameters, and variables like power production or defect detection, is to employ a model that uses deep learning, and brain networks in particular. Feature extraction is a strong suit of CNNs, making them a promising tool for image analysis in the context of spatial pattern defect detection. Using GA, one may optimize the settings of either the PV system or the deep learning model. To do this, one must define a fitness function that represents the target performance metric. This metric might be anything from maximizing power production to reducing prediction error. In order to decrease the discrepancy between the expected and actual electrical output, GA may optimize the biases and weights of a neural network. To sum up, optimizing and calibrating PV systems has never been easier than with the help of a genetic algorithm and deep learning. Together, these things have the potential to make solar energy systems more efficient, provide better results, and enhance model accuracy. This method optimizes seven parameters of a PV system based on observed PV power: nominal power, tilt, azimuth, albedo, irradiance, temperature dependence, and the DC/AC ratio, which is the proportion of nominal module power to nominal inverter power. Through the optimization of these parameters, a digital twin is generated that faithfully mimics the real characteristics and actions of the unidentified solar power plants or subsystems. In order to create this method, we used data on the PV system's on-site power output, surrounding temperature, and irradiance as collected by satellite in southwest Germany. Here, we provide a method that, when applied to nominal power, produces a mean bias error of around 10%, 3° for azimuth with tilt angles, a temperature coefficient ranging from 0.01% to 0.09%, and now-casts having an accuracy of about 6%. As a conclusion, we provide a novel approach to properly parametrize and model PV systems with little to no prior information about their characteristics and elements.

We develop novel approaches to enhance the functionality and efficiency of digital twins through deep learning techniques. Digital twins, sophisticated virtual models of physical systems, serve as dynamic counterparts, mirroring real-world entities' behavior and performance. Their primary role includes real-time monitoring, flaw detection, automated control, and proactive maintenance. To perform such tasks it is necessary to use various methods from scientific computing to combine data with physics models. Furthermore, to assess the reliability of a digital twin, it is important to not only make predictions, but also quantify the uncertainty of estimations and predictions. In this regard, Bayesian methods serve as a natural framework to pose the problems. However, the complexity and real-time operation requirements pose significant computational challenges. The thesis explores the integration of deep learning in scientific computing for enabling digital twins. The focus is on overcoming the computational hurdles in real-time data assimilation and inverse problem-solving through surrogate modeling. Deep learning, with its capability to handle high-dimensional functions in both deterministic and stochastic settings is shown to be a viable solution to these challenges. This work investigates several key areas, beginning with a brief introduction to digital twins, data assimilation, and inverse problems. It proceeds to elaborate on the role of Bayesian inversion problems in static and dynamic contexts, highlighting the importance of uncertainty quantification. The focus is on the challenges of real-time computation and the potential of Bayesian methods, despite their computational intensity. The thesis investigates deep learning architectures like dense and residual neural networks, convolutional neural networks, recurrent neural networks and transformers. The core contributions are presented in the four main chapters of the thesis: 1) Reduced Order Modeling for Parameterized Time-Dependent Partial Differential Equations: Introducing a deep learning framework for solving parameterized PDEs using spatially and memory-aware neural networks, demonstrated on linear advection and incompressible Navier-Stokes equations. 2) Markov Chain Generative Adversarial Neural Networks: Enhancing Bayesian inversion for static problems with sparse observations using a unique algorithm combining MCMC and GANs, demonstrated on a Darcy flow problem and pipe flow leakage detection. 3) Probabilistic Digital Twin for Leak Localization: Developing a probabilistic framework using supervised Wasserstein Autoencoders for leak localization in water distribution networks, tested on several network models. 4) The Deep Latent Space Particle Filter: Proposing a real-time nonlinear data assimilation method for dynamic PDEs with uncertainty quantification, using transformer-based dimensionality reduction and time-stepping in particle filters, demonstrated on various dynamic systems. This thesis contributes significantly to digital twins, addressing computational limitations and opening new avenues for practical applications in various industries.

Electricity load forecasting is a crucial undertaking within all the deregulated markets globally. Among the research challenges on a global scale, the investigation of deep transfer learning (DTL) in the field of electricity load forecasting represents a fundamental effort that can inform artificial intelligence applications in general. In this paper, a comprehensive study is reported regarding day-ahead electricity load forecasting. For this purpose, three sequence-to-sequence (Seq2seq) deep learning (DL) models are used, namely the multilayer perceptron (MLP), the convolutional neural network (CNN) and the ensemble learning model (ELM), which consists of the weighted combination of the outputs of MLP and CNN models. Also, the study focuses on the development of different forecasting strategies based on DTL, emphasizing the way the datasets are trained and fine-tuned for higher forecasting accuracy. In order to implement the forecasting strategies using deep learning models, load datasets from three Greek islands, Rhodes, Lesvos, and Chios, are used. The main purpose is to apply DTL for day-ahead predictions (1–24 h) for each month of the year for the Chios dataset after training and fine-tuning the models using the datasets of the three islands in various combinations. Four DTL strategies are illustrated. In the first strategy (DTL Case 1), each of the three DL models is trained using only the Lesvos dataset, while fine-tuning is performed on the dataset of Chios island, in order to create day-ahead predictions for the Chios load. In the second strategy (DTL Case 2), data from both Lesvos and Rhodes concurrently are used for the DL model training period, and fine-tuning is performed on the data from Chios. The third DTL strategy (DTL Case 3) involves the training of the DL models using the Lesvos dataset, and the testing period is performed directly on the Chios dataset without fine-tuning. The fourth strategy is a multi-task deep learning (MTDL) approach, which has been extensively studied in recent years. In MTDL, the three DL models are trained simultaneously on all three datasets and the final predictions are made on the unknown part of the dataset of Chios. The results obtained demonstrate that DTL can be applied with high efficiency for day-ahead load forecasting. Specifically, DTL Case 1 and 2 outperformed MTDL in terms of load prediction accuracy. Regarding the DL models, all three exhibit very high prediction accuracy, especially in the two cases with fine-tuning. The ELM excels compared to the single models. More specifically, for conducting day-ahead predictions, it is concluded that the MLP model presents the best monthly forecasts with MAPE values of 6.24% and 6.01% for the first two cases, the CNN model presents the best monthly forecasts with MAPE values of 5.57% and 5.60%, respectively, and the ELM model achieves the best monthly forecasts with MAPE values of 5.29% and 5.31%, respectively, indicating the very high accuracy it can achieve.

We developed a multimodal ultrasound (US) deep learning (DL) fusion model to automatically classify early fibrosis in patients with chronic kidney disease (CKD). This prospective study included patients with CKD who underwent continuous gray-scale US, superb microvascular imaging, and strain elastography from May to November 2022. According to the pathological tubular atrophy and interstitial fibrosis score, patients were divided into minimal and mild groups (affected area ≤10% and 11 − 25% of the total cortical volume, respectively). The dataset was divided into training (70%) and test (30%) sets. A DL model combining the features of the three US modes was developed to predict early fibrosis in patients with CKD. We compared these findings with the area under the receiver operating characteristic curve (AUC) of the clinical model by analyzing the receiver operating characteristic curve in the test set. The AUC of single-mode DL based on gray-scale US, superb microvascular imaging, and strain elastography was 0.682, 0.745, and 0.648, respectively, while that of the multimodal US DL model was 0.86. The accuracy, specificity, and sensitivity of the multimodal US DL model were 0.779, 0.767, and 0.796, respectively, and the negative and positive predictive values were 0.842 and 0.706, respectively. The AUC of the multimodal US DL model was significantly better than that of the single-mode DL and clinical models. The DL algorithm developed using multimodal US images can effectively predict early fibrosis in patients with CKD with significantly greater accuracy than single-mode DL or clinical models.

Biopharma Is Going Digital … Bit by Bit

We aimed at comparing intratumoral and peritumoral deep learning, radiomics, and fusion models in predicting KRAS mutations in rectal cancer using endorectal ultrasound imaging. This study included 304 patients with rectal cancer from Fujian Medical University Union Hospital. The patients were randomly divided into a training group (213 patients) and a test group (91 patients) at a 7:3 ratio. Radiomics and deep learning models were established using primary tumor and peritumoral images. In the optimally performing regions-of-interest, two fusion strategies, a feature-based and a decision-based model, were employed to build the fusion models. The Shapley additive explanation (SHAP) method was used to evaluate the significance of features in the optimal radiomics, deep learning, and fusion models. The performance of each model was assessed using the area under the receiver operating characteristic curve (AUC) and decision curve analysis (DCA). In the test cohort, both the radiomics and deep learning models exhibited optimal performance with a 10-pixel patch extension, yielding AUC values of 0.824 and 0.856, respectively. The feature-based DLRexpand10_FB model attained the highest AUC (0.896) across all study sets. In addition, the DLRexpand10_FB model demonstrated excellent sensitivity, specificity, and DCA. SHAP analysis underscored the deep learning feature (DL_1) as the most significant factor in the hybrid model. The feature-based fusion model DLRexpand10_FB can be employed to predict KRAS gene mutations based on pretreatment endorectal ultrasound images of rectal cancer. The integration of peritumoral regions enhanced the predictive performance of both the radiomics and deep learning models.

Deep learning breast cancer risk models demonstrate improved accuracy compared with traditional risk models but have not been prospectively tested. We compared the accuracy of a deep learning risk score derived from the patient's prior mammogram to traditional risk scores to prospectively identify patients with cancer in a cohort due for screening. We collected data on 119 139 bilateral screening mammograms in 57 617 consecutive patients screened at 5 facilities between September 18, 2017, and February 1, 2021. Patient demographics were retrieved from electronic medical records, cancer outcomes determined through regional tumor registry linkage, and comparisons made across risk models using Wilcoxon and Pearson χ2 2-sided tests. Deep learning, Tyrer-Cuzick, and National Cancer Institute Breast Cancer Risk Assessment Tool (NCI BCRAT) risk models were compared with respect to performance metrics and area under the receiver operating characteristic curves. Cancers detected per thousand patients screened were higher in patients at increased risk by the deep learning model (8.6, 95% confidence interval [CI] = 7.9 to 9.4) compared with Tyrer-Cuzick (4.4, 95% CI = 3.9 to 4.9) and NCI BCRAT (3.8, 95% CI = 3.3 to 4.3) models (P &lt; .001). Area under the receiver operating characteristic curves of the deep learning model (0.68, 95% CI = 0.66 to 0.70) was higher compared with Tyrer-Cuzick (0.57, 95% CI = 0.54 to 0.60) and NCI BCRAT (0.57, 95% CI = 0.54 to 0.60) models. Simulated screening of the top 50th percentile risk by the deep learning model captured statistically significantly more patients with cancer compared with Tyrer-Cuzick and NCI BCRAT models (P &lt; .001). A deep learning model to assess breast cancer risk can support feasible and effective risk-based screening and is superior to traditional models to identify patients destined to develop cancer in large screening cohorts.

In the realm of proactive medical care, digital twin technology is rising to the forefront as a paradigm shift due to the combination of customised healthcare, 6G connectivity, and artificial intelligence (AI). This study investigates AI-driven, 6G-powered healthcare digital twin technology, focusing on patient-centric monitoring and predictive analytics. It must clearly outline the research objectives, such as real-time health data integration, adaptive decision-making, and improved treatment personalization. The proposed methodology should describe the use of 6G-enabled ultra-reliable low-latency communication, AI models for digital twin construction, and secure data management frameworks. Improved diagnostic accuracy, fast response time to clinical emergencies, and improved resource allocation in healthcare systems were main outcomes that are presented in this abstract. The technical contributions of this work with the integration of AI, 6G, and digital twin technologies are highlighted in the end of this study, and this may support the next-generation intelligent healthcare solutions. For self-learning healthcare digital twin, a model that leverages 6G-enabled medical informatics, real-time data streams, and artificial intelligence (AI) analytics is presented in this study. This presented model may support in clinical diagnosis, and classifying damages earlier. A real-time updated digital copy of the patient was created by the suggested method by integrating multimodal data from various sources including imaging, vital signs, consumer electronic devices such as smartwatches, wearable sensor metrics, and electronic health records (EHRs). The patient physiological patterns and previous injury profiles are constantly learnt by the integration of digital twin and deep learning (DL) algorithms. This integration is effective in predicting the occurrence, kind, and severity of injuries and internal trauma, falls, cardiovascular events, and neurodegenerative insults. The digital twin and healthcare providers are connected seamlessly with 6G communication infrastructure, guaranteeing ultra-low latency and great dependability. This allows for pre-hospital and critical care settings to implement instantaneous feedback loops, as well as for remote diagnostics and adaptive clinical decision-making. From home health monitoring solutions to smartphones and wearables, consumer electronics are ubiquitous and serve as common data collectors, greatly increase access and enabling the instant collection of accurate data using a digital twin. To keep data private while training AI models on dispersed healthcare data, the architecture offers federated learning. Based on severity level, the injury types are classified by a multi-layered decision engine, and this may promote a customized, context-aware responses like alerting emergency teams, modifying treatment regimens, or demanding robotic surgical support. The AI-driven, 6G-powered healthcare digital twin revolutionizes the diagnosis of injury, and it establish a path for a precise, preventive, and predictive medicine. Furthermore, scaling efficiencies were reduced by 95.81%, while reaction times decreased to 96.88%, resulting in significant gains. This technology is particularly helpful for populations who are elderly, at risk, or located far away, as it decreases reaction time, reduces diagnostic error, and provides scalable, continuous care.

The past few years have seen a surge of applying Deep Learning (DL) models for a wide array of tasks such as image classification, object detection, machine translation, etc. While DL models provide an opportunity to solve otherwise intractable tasks, their adoption relies on them being optimized to meet latency and resource requirements. Benchmarking is a key step in this process but has been hampered in part due to the lack of representative and up-to-date benchmarking suites. This is exacerbated by the fast-evolving pace of DL models. This paper proposes DLBricks, a composable benchmark generation design that reduces the effort of developing, maintaining, and running DL benchmarks on CPUs. DLBricks decomposes DL models into a set of unique runnable networks and constructs the original model's performance using the performance of the generated benchmarks. DLBricks leverages two key observations: DL layers are the performance building blocks of DL models and layers are extensively repeated within and across DL models. Since benchmarks are generated automatically and the benchmarking time is minimized, DLBricks can keep up-to-date with the latest proposed models, relieving the pressure of selecting representative DL models. Moreover, DLBricks allows users to represent proprietary models within benchmark suites. We evaluate DLBricks using $50$ MXNet models spanning $5$ DL tasks on $4$ representative CPU systems. We show that DLBricks provides an accurate performance estimate for the DL models and reduces the benchmarking time across systems (e.g. within $95\%$ accuracy and up to $4.4\times$ benchmarking time speedup on Amazon EC2 c5.xlarge).

Digital twin can well solve complex problems, especially in the case of mechanical failures. Digital twin technology can be applied in 3D IoT smart factories, new smart city construction, smart medical care, digital energy, digital archives, warehousing and logistics visualization and other fields. Deep learning covers a wide range of applications and is extremely common. This paper discusses the application of the two in the risk prediction of coal and gas outburst strength. This paper firstly describes the method of predicting coal and gas outburst intensity. For example, the BP neural network algorithm applied to the prediction of coal and gas outburst intensity in deep learning, the air flow control system model of digital twin for coal mines, and the risk assessment algorithm of coal and gas outburst intensity in coal mines based on grey relational analysis, and various ways to predict risk. And the system model is designed in this paper. Combined with the Formula, this paper describes the process of predicting risk in detail, and then conducts experiments based on digital twin and deep learning to predict coal and gas outburst intensity. In this paper, digital twin is used to systematically design coal and gas outburst intensity prediction, and a neural network prediction model based on optimized quantum gate nodes is established. In this paper, the practical application experiment and result analysis of the optimization algorithm in the coal and gas outburst prediction model are carried out, and the conclusion is drawn. After QGNN is optimized by the sdPSO algorithm, the error is extremely small, only 2.0914, and the specific value of the prediction accuracy in practical applications is as high as 95%. The experimental data verifies the feasibility of digital twin and deep learning technology in the prediction of coal and gas outburst intensity.

To develop and evaluate static ultrasound images of ovaries-based radiomics and deep learning models for predicting antral follicle count (AFC) and anti-Müllerian hormone (AMH). A retrospective analysis was performed on 395 infertility patients who completed AFC through transvaginal ultrasonography and AMH test in two branches of the hospital. The data set was divided into training cohort (n = 210, Branch A), internal test cohort (n = 91, Branch A), and external test cohort (n = 94, Branch B). The AFC results of static ultrasound images (AFCstatic) and dynamic scan (AFCdynamic) were analysed for correlation and consistency. The radiomics, deep learning and combined models (Rad-DL model) were developed to quantitatively predict AFC and AMH respectively, and the ordered 3-class classification (AFCdynamic≤5,6≤AFCdynamic≤19 and 20≤AFCdynamic) prediction was further carried out. There was a high degree of correlation and consistency between AFCstatic and AFCdynamic in both branches. In the quantitative prediction of AFCdynamic, the R2 of the radiomics, deep learning and Rad-DL models in the external test cohort were 0.572, 0.541, 0.583, respectively. In the quantitative prediction of AMH, the R2 of the radiomics, deep learning and Rad-DL models in the external test cohort were 0.420, 0.352, 0.509, respectively. For the ordered 3-class classification prediction, the radiomics, deep learning and Rad-DL models demonstrated a good discrimination performance with overall AUCs of 0.816 (95%CI, 0.747–0.887), 0.843 (95%CI, 0.772–0.901), and 0.881 (95%CI, 0.816–0.931) in the external test cohort, respectively. Radiomics and deep learning models developed by using static ultrasound images of ovaries may quantitatively predict AFCdynamic to a certain extent, and accurately screen infertility patients with different ovarian reserve levels. This study lays the foundation for further break through the rigid antral follicle count and explore an intelligent non-invasive ovarian reserve evaluation method.