A multi-source feature-driven deep learning method to generate linear energy transfer distribution for proton therapy.

In cancer care, determining the most beneficial treatment technique is a key decision affecting the patient's survival and quality of life. Patient selection for proton therapy (PT) over conventional radiotherapy (XT) currently entails comparing manually generated treatment plans, which requires time andexpertise. We developed an automatic and fast tool, AI-PROTIPP (Artificial Intelligence Predictive Radiation Oncology Treatment Indication to Photons/Protons), that assesses quantitatively the benefits of each therapeutic option. Our method uses deep learning (DL) models to directly predict the dose distributions for a given patient for both XT and PT. By using models that estimate the Normal Tissue Complication Probability (NTCP), namely the likelihood of side effects to occur for a specific patient, AI-PROTIPP can propose a treatment selection quickly andautomatically. A database of 60 patients presenting oropharyngeal cancer, obtained from the Cliniques Universitaires Saint Luc in Belgium, was used in this study. For every patient, a PT plan and an XT plan were generated. The dose distributions were used to train the two dose DL prediction models (one for each modality). The model is based on U-Net architecture, a type of convolutional neural network currently considered as the state of the art for dose prediction models. A NTCP protocol used in the Dutch model-based approach, including grades II and III xerostomia and grades II and III dysphagia, was later applied in order to perform automatic treatment selection for each patient. The networks were trained using a nested cross-validation approach with 11-folds. We set aside three patients in an outer set and each fold consists of 47 patients in training, five in validation and five for testing. This method allowed us to assess our method on 55 patients (five patients per test times the number of folds). The treatment selection based on the DL-predicted doses reached an accuracy of 87.4% for the threshold parameters set by the Health Council of the Netherlands. The selected treatment is directly linked with these threshold parameters as they express the minimal gain brought by the PT treatment for a patient to be indicated to PT. To validate the performance of AI-PROTIPP in other conditions, we modulated these thresholds, and the accuracy was above 81% for all the considered cases. The difference in average cumulative NTCP per patient of predicted and clinical dose distributions is very similar (less than 1% difference). AI-PROTIPP shows that using DL dose prediction in combination with NTCP models to select PT for patients is feasible and can help to save time by avoiding the generation of treatment plans only used for the comparison. Moreover, DL models are transferable, allowing, in the future, experience to be shared with centers that would not have PT planningexpertise.

Deep Learning (DL) techniques have enabled up to 6-fold acceleration in musculoskeletal magnetic resonance imaging (MRI) while preserving diagnostic image quality. Further, improvements in acceleration and generalization require novel approaches. We propose a DL framework that integrates Simultaneous Multislice (SMS) imaging with Parallel Imaging (PI) to enhance current DL-based reconstruction. To advance musculoskeletal Magnetic Resonance Imaging (MRI), by developing a DL reconstruction framework that combines SMS and PI, enabling acceleration of up to 8-fold and beyond, while maintaining image quality suitable for clinicalinterpretation. End-to-End (E2E) DL framework for reconstructing Turbo Spin Echo (TSE) MRI data acquired with SMS and PI acceleration. The method unrolls a proximal gradient algorithm with Nesterov momentum and integrates a novel DL network for joint regularization across simultaneously acquired slices. Slice separation and k-space-to-image reconstruction are unified by embedding the full SMS forward model into the DL architecture. Data Consistency (DC) is modulated to enhance denoising, and a super-resolution module improves image sharpness. The robust DL model was trained on over 200000 slices from 1.5T to 3T scans with diverse acquisitionsettings. The proposed E2E DL model outperforms prior methods at 8-fold and 12-fold acceleration, as measured by PSNR (peak signal-to-noise ratio) and SSIM (structural similarity index measure) metrics. Evaluation on prospectively acquired clinical scans by two radiologists confirms, that image quality and abnormality detection are comparable to standard acquisitions at loweracceleration. We extend state-of-the-art DL reconstruction frameworks by integrating slice separation directly into the model for SMS acquisitions. Our E2E DL approach achieves clinical-grade image quality at 8-fold acceleration across 20 subjects, reducing acquisition time by 27%. Preliminary results suggest potential for further acceleration up to 12-fold, demonstrating significant advancement beyond existing DLtechniques.

In film and television production, efficient and precise image processing is vital for achieving realistic visual effects. Therefore, exploring and applying advanced image processing technologies has become an essential method for elevating the production quality of film and television projects. This work investigates the application of artificial intelligence (AI) technology in the processing and production of animated images in film and television scenarios. By comparing the performance of standard Generative Adversarial Network (GAN), DenseNet, and CycleGAN models under different noise conditions, it is found that CycleGAN performs the best in image denoising and detail restoration. Experimental results demonstrate that CycleGAN achieves a Peak Signal-to-noise Ratio (PSNR) of 30.1dB and a Structural Similarity Index Measure (SSIM) of 0.88 under Gaussian noise conditions. Moreover, CycleGAN achieves a PSNR of 29.5dB and an SSIM of 0.85 under salt-and-pepper noise conditions. It outperforms the other models in both conditions. Additionally, CycleGAN’s mean absolute error is significantly lower than that of the other models. This work demonstrates that CycleGAN can more effectively handle complex noise and generate high-quality images under unsupervised learning conditions. These findings provide new directions for future image processing research and offer important references for model selection in practical applications. This work not only offers new perspectives on the development of animation image processing technology but also establishes a theoretical foundation for applying advanced AI techniques in film and television production. Through comparative analysis of various deep learning models, this work highlights the superior performance of CycleGAN under complex noise conditions. This advancement not only drives progress in image processing technology but also provides effective solutions for efficient production and quality enhancement of future film and television works.

The use of MRI-guided radiation therapy (MRIgRT) has shown considerable advantages. However, the acquisition of electron density information still relies on Computed tomography (CT) images. In the context of medical image synthesis, achieving high global and local accuracy, as well as efficient synthetic CT (sCT) generation, remains a key challenge. This study aims to improve the accuracy of sCT image generation from MRI by introducing a novel deep learning model that leverages global context and local detail, with a minimal increase in model complexity. In this study, we propose a novel generative adversarial network based on Mamba block and Residual Constraint (RC) strategy, namely RC-MambaGAN, which achieves a significant improvement in accuracy with only a minimal increase in computation time required for sCT generation. Specifically, we integrate Mamba block into the generator to enhance the model's ability to capture long-range contextual information, while preserving the precision of local features and maintaining the global structure of the image. The RC strategy improves the alignment between sCT and real CT by minimizing residuals. The model was validated on multi-center datasets that included both tumor and pelvic-region data. Furthermore, the dosimetric accuracy of the generated sCT images was evaluated to assess their clinical applicability. RC-MambaGAN achieved high performance with mean absolute error (MAE) of 37.094±8.761 HU, peak signal-to-noise ratio (PSNR) of 28.424±1.145dB, structural similarity index measure (SSIM) of 0.947±0.004, and Mutual Information (MI) of 1.426±0.012. External validation demonstrated that RC-MambaGAN outperformed existing state-of-the-art methods in both image quality and quantitative accuracy, even for structures visible in MRI but not in CT images. The proposed RC-MambaGAN markedly improves the image quality and anatomical accuracy of sCT images derived from MRI, thereby advancing the feasibility of MRI-only radiotherapy workflows.

Introduction: Decision-making regarding radiotherapy techniques for patients with nasopharyngeal cancer requires a comparison of photon and proton plans generated using planning software, which requires time and expertise. We developed a fully automated decision tool to select patients for proton therapy that predicts proton therapy (XT) and photon therapy (PT) dose distributions using only patient CT image data, predicts xerostomia and dysphagia probability using predicted critical organ mean doses, and makes decisions based on the Netherlands' National Indication Protocol Proton therapy (NIPP) to select patients likely to benefit from proton therapy. Methods: This study used 48 nasopharyngeal patients treated at the Cancer Hospital of the Chinese Academy of Medical Sciences. We manually generated a photon plan and a proton plan for each patient. Based on this dose distribution, photon and proton dose prediction models were trained using deep learning (DL) models. We used the NIPP model to measure xerostomia levels 2 and 3, dysphagia levels 2 and 3, and decisions were made according to the thresholds given by this protocol. Results: The predicted doses for both photon and proton groups were comparable to those for manual plan (MP). The Mean Absolute Error (MAE) for each organ at risk in the photon and proton plans did not exceed 5% and showed a good performance of the dose prediction model. For proton, the normal tissue complication probability (NTCP) of xerostomia and dysphagia performed well, p > 0.05. There was no statistically significant difference. For photon, the NTCP of dysphagia performed well, p > 0.05. For xerostomia p < 0.05 but the absolute deviation was 0.85% and 0.75%, which would not have a great impact on the prediction result. Among the 48 patients' decisions, 3 were wrong, and the correct rate was 93.8%. The area under curve (AUC) of operating characteristic curve (ROC) was 0.86, showing the good performance of the decision-making tool in this study. Conclusions: The decision tool based on DL and NTCP models can accurately select nasopharyngeal cancer patients who will benefit from proton therapy. The time spent generating comparison plans is reduced and the diagnostic efficiency of doctors is improved, and the tool can be shared with centers that do not have proton expertise. Trial registration: This study was a retrospective study, so it was exempt from registration.

Attenuation correction (AC), as an essential component of quantitative SPECT imaging, is highly challenging in the absence of CT or transmission scanning in dedicated cardiac SPECT devices. This study investigated the potential of direct attenuation correction (AC) of <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">99m</sup> Tc-sestamibi SPECT myocardial perfusion imaging (MPI) in the image domain using residual (ResNet) and UNet deep convolutional neural networks. MPI-SPECT <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">99m</sup> Tc-ses-tamibi images of 99 subjects were retrospectively used, considering non-attenuation corrected SPECT images as input and CT-based attenuation corrected (CT-AC) SPECT images as reference. Chang’s AC technique was also performed as a baseline for evaluating the performance of deep learning-based models. The quantitative accuracy of the attenuation corrected SPECT images was validated using the proposed methods considering the corresponding CT-AC SPECT images of 19 external datasets as reference. Image-derived metrics, including the voxel-wise mean absolute error (MAE), relative error (RE), structural similarity index measure (SSIM), as well as clinically relevant indices, such as total perfusion deficit (TPD), were calculated. The quantitative results obtained from the external validation dataset of the deep learning models exhibited good agreement with the reference CT-based attenuation corrected MPI-SPECT images. The deep learning models showed superior performance over Chang’s AC method. The ResNet and UNet models resulted in MAE of 20.24±17.63 and 13.65±11.23 and SSIM of 0.99±0.04 and 0.98±0.05, respectively. In comparison, Chang’s approach led to MAE and SSIM of 80.27±47.56 and 0.93±0.09, respectively. Similarly, the clinical evaluation showed a mean TPD of 12.78±9.22% and 12.57±8.93% for ResNet and UNet models, respectively, compared to TPD of 12.84±8.63% for reference SPECT CT-AC images. In contrast, Chang’s approach led to a mean TPD of 16.68±11.24%. The evaluation of deep learning approaches implemented in this work showed the promising potential of deep learning-based techniques in producing attenuation corrected MPI-SPECT images in the image domain without anatomical imaging.

Deep learning (DL) models are effective in leveraging latent representations from MR data, emerging as state-of-the-art solutions for accelerated MRI reconstruction. However, challenges arise due to the inherent uncertainties associated with undersampling in k-space, coupled with the over- or under-parameterized and opaque nature of DL models. Addressing uncertainty has thus become a critical issue in DL MRI reconstruction. Monte Carlo (MC) inference techniques are commonly employed to estimate uncertainty, involving multiple reconstructions of the same scan to compute variance as a measure of uncertainty. Nevertheless, these methods entail significant computational expenses, requiring multiple inferences through the DL model. In this context, we propose a novel approach to uncertainty estimation during MRI reconstruction using a pixel classification framework. Our method, PixCUE (Pixel Classification Uncertainty Estimation), generates both the reconstructed image and an uncertainty map in a single forward pass through the DL model. We validate the efficacy of this approach by demonstrating that PixCUE-generated uncertainty maps exhibit a strong correlation with reconstruction errors across various MR imaging sequences and under diverse adversarial conditions. We present an empirical relationship between uncertainty estimations using PixCUE and established reconstruction metrics such as NMSE, PSNR, and SSIM. Furthermore, we establish a correlation between the estimated uncertainties from PixCUE and the conventional MC method. Our findings affirm that PixCUE reliably estimates uncertainty in MRI reconstruction with minimal additional computational cost.

To develop a deep learning (DL) model to measure the sagittal Cobb angle of the cervical spine on computed tomography (CT). Two VB-Net-based DL models for cervical vertebra segmentation and key-point detection were developed. Four-points and line-fitting methods were used to calculate the sagittal Cobb angle automatically. The average value of the sagittal Cobb angle was manually measured by two doctors as the reference standard. The percentage of correct key points (PCK), matched samples t test, intraclass correlation coefficient (ICC), Pearson correlation coefficient, mean absolute error (MAE), and Bland‒Altman plots were used to evaluate the performance of the DL model and the robustness and generalization of the model on the external test set. A total of 991 patients were included in the internal data set, and 112 patients were included in the external data set. The PCK of the DL model ranged from 78 to 100% in the test set. The four-points method, line-fitting method, and reference standard measured sagittal Cobb angles were - 1.10 ± 18.29°, 0.30 ± 13.36°, and 0.50 ± 12.83° in the internal test set and 4.55 ± 20.01°, 3.66 ± 18.55°, and 1.83 ± 12.02° in the external test set, respectively. The sagittal Cobb angle calculated by the four-points method and the line-fitting method maintained high consistency with the reference standard (internal test set: ICC = 0.75 and 0.97; r = 0.64 and 0.94; MAE = 5.42° and 3.23°, respectively; external test set: ICC = 0.74 and 0.80, r = 0.66 and 0.974, MAE = 5.25° and 4.68°, respectively). The DL model can accurately measure the sagittal Cobb angle of the cervical spine on CT. The line-fitting method shows a higher consistency with the doctors and a minor average absolute error.

PurposeTo conduct a head-to-head comparison between deep learning (DL) and radiomics models across institutions for predicting microvascular invasion (MVI) in hepatocellular carcinoma (HCC) and to investigate the model robustness and generalizability through rigorous internal and external validation.MethodsThis retrospective study included 2304 preoperative images of 576 HCC lesions from two centers, with MVI status determined by postoperative histopathology. We developed DL and radiomics models for predicting the presence of MVI using B-mode ultrasound, contrast-enhanced ultrasound (CEUS) at the arterial, portal, and delayed phases, and a combined modality (B + CEUS). For radiomics, we constructed models with enlarged vs. original regions of interest (ROIs). A cross-validation approach was performed by training models on one center’s dataset and validating the other, and vice versa. This allowed assessment of the validity of different ultrasound modalities and the cross-center robustness of the models. The optimal model combined with alpha-fetoprotein (AFP) was also validated. The head-to-head comparison was based on the area under the receiver operating characteristic curve (AUC).ResultsThirteen DL models and 25 radiomics models using different ultrasound modalities were constructed and compared. B + CEUS was the optimal modality for both DL and radiomics models. The DL model achieved AUCs of 0.802–0.818 internally and 0.667–0.688 externally across the two centers, whereas radiomics achieved AUCs of 0.749–0.869 internally and 0.646–0.697 externally. The radiomics models showed overall improvement with enlarged ROIs (P < 0.05 for both CEUS and B + CEUS modalities). The DL models showed good cross-institutional robustness (P > 0.05 for all modalities, 1.6–2.1% differences in AUC for the optimal modality), whereas the radiomics models had relatively limited robustness across the two centers (12% drop-off in AUC for the optimal modality). Adding AFP improved the DL models (P < 0.05 externally) and well maintained the robustness, but did not benefit the radiomics model (P > 0.05).ConclusionCross-institutional validation indicated that DL demonstrated better robustness than radiomics for preoperative MVI prediction in patients with HCC, representing a promising solution to non-standardized ultrasound examination procedures.

This study investigates data standardization methods based on the grid search (GS) algorithm for energy load forecasting, including zero-mean, min-max, max, decimal, sigmoid, softmax, median, and robust, to determine the hyperparameters of deep learning (DL) models. The considered DL models are the convolutional neural network (CNN) and long short-term memory network (LSTMN). The procedure is made over (i) setting the configuration for CNN and LSTMN, (ii) establishing the hyperparameter values of CNN and LSTMN models based on epoch, batch, optimizer, dropout, filters, and kernel, (iii) using eight data standardization methods to standardize the input data, and (iv) using the GS algorithm to search the optimal hyperparameters based on the mean absolute error (MAE) and mean absolute percent error (MAPE) indexes. The effectiveness of the proposed method is verified on the power load data of the Australian state of Queensland and Vietnamese Ho Chi Minh city. The simulation results show that the proposed data standardization methods are appropriate, except for the zero-mean and min-max methods.

Purpose. This study aims to synthesize CT from MR images for radiotherapy planning of head and neck tumor using an improved three-dimensional conditional generative adversarial network (3D cGAN) based on dual-attention modules.Methods. A total of 212 paired CT and T1-weighted MRI datasets are utilized, including 180 publicly available cases and 32 clinical cases from our hospital. Building upon the 3D cGAN framework, we implement structural modifications to the generator, discriminator, and loss functions. In particular, a lightweight dual-attention mechanism module is introduced to the generator based on 3D residual network. The model is trained on 186 datasets and evaluated on 26 test cases. Quantitative metrics including normalized cross-correlation (NCC), structural similarity index measure (SSIM), peak signal-to-noise ratio (PSNR), and mean absolute error (MAE) are calculated to assess the similarity between synthetic CT (sCT) and ground-truth CT images. A comparative analysis with U-Net, CycleGAN and basic 3D cGAN is conducted to validate performance improvements.Results. The proposed dual-attention enhanced 3D cGAN generates clinically acceptable sCT images across all 26 test cases. Quantitative evaluations demonstrate high accuracy with NCC of 97.06%, SSIM of 90.24%, PSNR of 28.23 ± 0.42, and MAE of 32.53 ± 2.49 HU. In quantitative comparison, the proposed dual-attention enhanced 3D cGAN approach outperforms U-Net, CycleGAN and the basic 3D cGAN across all metrics.Conclusion. This study proposes an improved dual-attention enhanced 3D cGAN algorithm. The method can rapidly and automatically generate sCT images from MR images for patients of head and neck tumor, which holds significant importance for implementing MR-only radiotherapy planning.

PurposeThe purpose of this study was to evaluate the accuracy and perceptual quality of red-green-blue (RGB) fundus images generated from red-green (RG) images using Pix2Pix-based generative adversarial network (GAN).MethodsRG images were extracted from original RGB images recorded by a newer model of the Optos system by turning off the blue laser source. A conditional GAN based on the Pix2Pix architecture was trained to convert this extracted RG image to an RGB image. The performance was evaluated quantitatively by using a five-fold cross-validation and assessments with structural similarity index measure (SSIM), peak signal-to-noise ratio (PSNR), learned perceptual image patch similarity (LPIPS), mean absolute error (MAE), and root mean square error (RMSE). To assess the perceptual accuracy, 39 ophthalmologists examined 60 randomly selected images, 30 true and 30 generated RGB images, using Google Form-based interface. They were instructed to classify each image as a true or an artificial intelligence (AI)-generated image.ResultsThe generated images had high similarity to the original RGB images (SSIM = 0.97 ± 0.00; PSNR = 35.74 ± 0.17 decibel [dB]; LPIPS = 0.09 ± 0.00; MAE = 1.76 ± 0.04; and RMSE = 4.25 ± 0.10). The overall correct classification rate was 57.1%. Certain lesion types, such as epiretinal membrane and optic disc cupping, had higher discrimination rates. The receiver operating characteristic (ROC) analysis across all evaluations yielded an area under the curve (AUC) of 0.497 (P = 0.314).ConclusionsPix2Pix-based GANs can generate perceptually and structurally consistent RGB images from RG images without lesion-specific attention mechanisms.Translational RelevanceThis method enables a reinterpretation of legacy RG fundus images and may support future AI diagnostic applications.

Simultaneous attenuation and scatter correction of PET data in the image: quantitative and clinical assessment of image-to-image deep learning models.

COVID-19 is highly infectious and causes acute respiratory disease. Machine learning (ML) and deep learning (DL) models are vital in detecting disease from computerized chest tomography (CT) scans. The DL models outperformed the ML models. For COVID-19 detection from CT scan images, DL models are used as end-to-end models. Thus, the performance of the model is evaluated for the quality of the extracted feature and classification accuracy. There are four contributions included in this work. First, this research is motivated by studying the quality of the extracted feature from the DL by feeding these extracted to an ML model. In other words, we proposed comparing the end-to-end DL model performance against the approach of using DL for feature extraction and ML for the classification of COVID-19 CT scan images. Second, we proposed studying the effect of fusing extracted features from image descriptors, e.g., Scale-Invariant Feature Transform (SIFT), with extracted features from DL models. Third, we proposed a new Convolutional Neural Network (CNN) to be trained from scratch and then compared to the deep transfer learning on the same classification problem. Finally, we studied the performance gap between classic ML models against ensemble learning models. The proposed framework is evaluated using a CT dataset, where the obtained results are evaluated using five different metrics The obtained results revealed that using the proposed CNN model is better than using the well-known DL model for the purpose of feature extraction. Moreover, using a DL model for feature extraction and an ML model for the classification task achieved better results in comparison to using an end-to-end DL model for detecting COVID-19 CT scan images. Of note, the accuracy rate of the former method improved by using ensemble learning models instead of the classic ML models. The proposed method achieved the best accuracy rate of 99.39%.

Prediction of pink esthetic score using deep learning: A proof of concept.