Basal Strain Estimation in Transesophageal Echocardiography (TEE) using Deep Learning based Unsupervised Deformable Image Registration

Aims: This study aimed to explore the effect of training set diversity on the performance of deep learning models for predicting early gastric cancer (EGC) under endoscopy. Methods: Images of EGC and non-cancerous lesions under narrow-band imaging (ME-NBI) and magnifying blue laser imaging (ME-BLI) were retrospectively collected. Training set 1 was composed of 150 non-cancerous and 309 EGC ME-NBI images, training set 2 was composed of 1505 non-cancerous and 309 EGC ME-BLI images, and training set 3 was the combination of training set 1 and 2. Test set 1 was composed of 376 non-cancerous and 1052 EGC ME-NBI images, test set 2 consisted of 529 non-cancerous and 71 EGC ME-BLI images, and test set 3 was the combination of test set 1 and test set 2. Three deep learning models, convolutional neural network (CNN) 1, CNN 2 and CNN 3 (CNN 1, CNN 2 and CNN 3 were independently trained using training set 1, training set 2 and training set 3, respectively), were constructed, and their performances on each test set were respectively evaluated. One hundred and thirty-eight ME-NBI videos and 17 ME-BLI videos were further collected to evaluate and compare the performance of each model in real time. Results: On the whole, the performance of CNN 3 was the best. The accuracy (Acc), sensitivity (Sn), specificity (Sp) and area under the curve (AUC) of test set 1 in CNN 3 were 87.89% (1255/1428), 90.96% (342/376), 86.79% (913/1052) and 94.60%, respectively. The Acc, Sn, Sp and AUC of test set 2 in CNN 3 were 95% (570/600), 97.92% (518/529), 73.24% (52/71) and 90.93% respectively. The Acc, Sn, Sp and AUC of test set 3 in CNN 3 were 89.99% (1825/2028), 95.03% (860/905), 85.93% (965/1123) and 94.89%, respectively. The performance of CNN 3 was also the best in videos test set. The Acc, Sn and Sp of videos test set in CNN 3 were 91.03% (142/156), 90.58% (125/138) and 94.44% (17/18), respectively. Conclusions: The deep learning model with the most diverse training data has the best diagnostic effect.

Deep convolutional neural networks (DCNNs) for diagnosis of disease on chest radiographs (CXR) have been shown to be biased against males or females if the datasets used to train them have unbalanced sex representation. Prior work has suggested that DCNNs can predict sex on CXR, which could aid forensic evaluations, but also be a source of bias. To (1) evaluate the performance of DCNNs for predicting sex across different datasets and architectures and (2) evaluate visual biomarkers used by DCNNs to predict sex on CXRs. Chest radiographs were obtained from the Stanford CheXPert and NIH Chest XRay14 datasets which comprised of 224,316 and 112,120 CXRs, respectively. To control for dataset size and class imbalance, random undersampling was used to reduce each dataset to 97,560 images that were balanced for sex. Each dataset was randomly split into training (70%), validation (10%), and test (20%) sets. Four DCNN architectures pre-trained on ImageNet were used for transfer learning. DCNNs were externally validated using a test set from the opposing dataset. Performance was evaluated using area under the receiver operating characteristic curve (AUC). Class activation mapping (CAM) was used to generate heatmaps visualizing the regions contributing to the DCNN's prediction. On the internal test set, DCNNs achieved AUROCs ranging from 0.98 to 0.99. On external validation, the models reached peak cross-dataset performance of 0.94 for the VGG19-Stanford model and 0.95 for the InceptionV3-NIH model. Heatmaps highlighted similar regions of attention between model architectures and datasets, localizing to the mediastinal and upper rib regions, as well as to the lower chest/diaphragmatic regions. DCNNs trained on two large CXR datasets accurately predicted sex on internal and external test data with similar heatmap localizations across DCNN architectures and datasets. These findings support the notion that DCNNs can leverage imaging biomarkers to predict sex and potentially confound the accurate prediction of disease on CXRs and contribute to biased models. On the other hand, these DCNNs can be beneficial to emergency radiologists for forensic evaluations and identifying patient sex for patients whose identities are unknown, such as in acute trauma.

Removal of retained air during cardiac surgery with transesophageal echocardiography and capnography

Technical advances in echocardiography, especially the introduction of oesophageal probes, have led to the extension of its indications to the field of anaesthesia and intensive care. Transoesophageal echocardiography (TEE) provides high quality imaging of the left ventricle in patients on ventilators. In addition, manipulation of the probe is performed near the patient's head and so does not interfere with the surgeon's job. This enables echocardiographic monitoring of left ventricular function throughout the operation. The information so obtained not only provides data about anatomical changes which is sometimes invaluable, but also allows quantification of left ventricular function which is the main advantage for anaesthetists. The images allow study of regional and global left ventricular contraction and the deduction of certain parameters of ventricular performance. The ability to perform real time monitoring of ventricular function throughout anaesthesia and the immediate postoperative period is all the more valuable in patients with limited cardiac and coronary reserve. This new method of monitoring gives a better understanding of the physiopathology of peroperative cardiac events and provides information which may guide the conduct of the anaesthesia and postoperative care.

Caveolae are plasma membrane invaginations whose formation requires caveolin-1 (Cav1), the adaptor protein polymerase I, and the transcript release factor (PTRF or CAVIN1). Caveolae have an important role in cell functioning, signaling, and disease. In the absence of CAVIN1/PTRF, Cav1 forms non-caveolar membrane domains called scaffolds. In this work, we train machine learning models to automatically distinguish between caveolae and scaffolds from single molecule localization microscopy (SMLM) data. We apply machine and deep learning algorithms to discriminate biological structures from SMLM data. Our work is the first that is leveraging machine and deep learning approaches to automatically identifying biological structures from SMLM data. In particular, we develop and compare three binary classification methods to identify whether or not a given 3D cluster of Cav1 proteins is a caveolae. The first uses a random forest classifier applied to 28 hand-picked features, the second uses a convolutional neural net (CNN) applied to a projection of the point clouds onto three planes, and the third uses a PointNet model, a recent development that can directly take point clouds as its input. We validate our methods on a dataset of super-resolution microscopy images of PC3 prostate cancer cells labeled for Cav1. Specifically, we have images from two cell populations: 10 PC3 and 10 CAVIN1/PTRF-transfected PC3 cells (PC3-PTRF cells) that form caveolae. We obtained a balanced set of 1714 different cellular structures. Our results show that both the random forest on hand selected features and the deep learning approach achieve high accuracy in distinguishing the intrinsic features of the caveolae and non-caveolae biological structures. More specifically, both random forest and deep CNN classifiers achieve classification accuracy reaching 94% on our test set, while the PointNet model only reached 83% accuracy. We also discuss the pros and cons of the different approaches. Additional information about individual files is in the accompanying CSV file (item_metadata.csv). This dataset was originally deposited in the Simon Fraser University institutional repository.

Understanding the learning mechanism of convolutional neural networks in spectral analysis

We present an evaluation of different deep learning and machine learning approaches for tree health classification in the Black Forest, the Harz Mountains, and the Göttinger Forest on a unique, highly accurate tree-level dataset. The multispectral UAV data were collected from eight forest plots with diverse tree species, mostly conifers. As ground truth data (GTD), nearly 1500 tree polygons with related attribute information on the health status of the trees were used. This data were collected during extensive fieldwork using a mobile application and subsequent individual tree segmentation. Extensive preprocessing included normalization, NDVI calculations, data augmentation to deal with the underrepresented classes, and splitting the data into training, validation, and test sets. We conducted several experiments using a classical machine learning approach (random forests), as well as different convolutional neural networks (CNNs)—ResNet50, ResNet101, VGG16, and Inception-v3—on different datasets and classes to evaluate the potential of these algorithms for tree health classification. Our first experiment was a binary classifier of healthy and damaged trees, which did not consider the degree of damage or tree species. The best results of a 0.99 test accuracy and an F1 score of 0.99 were obtained with ResNet50 on four band composites using the red, green, blue, and infrared bands (RGBI images), while VGG16 had the worst performance, with an F1 score of only 0.78. In a second experiment, we also distinguished between coniferous and deciduous trees. The F1 scores ranged from 0.62 to 0.99, with the highest results obtained using ResNet101 on derived vegetation indices using the red edge band of the camera (NDVIre images). Finally, in a third experiment, we aimed at evaluating the degree of damage: healthy, slightly damaged, and medium or heavily damaged trees. Again, ResNet101 had the best performance, this time on RGBI images with a test accuracy of 0.98 and an average F1 score of 0.97. These results highlight the potential of CNNs to handle high-resolution multispectral UAV data for the early detection of damaged trees when good training data are available.

A dual-branch hybrid dilated CNN model for the AI-assisted segmentation of meningiomas in MR images

BackgroundIn digital pathology, image properties such as color, brightness, contrast and blurriness may vary based on the scanner and sample preparation. Convolutional Neural Networks (CNNs) are sensitive to these variations and may underperform on images from a different domain than the one used for training. Robustness to these image property variations is required to enable the use of deep learning in clinical practice and large scale clinical research.AimsCNN Stability Training (CST) is proposed and evaluated as a method to increase CNN robustness to scanner and Immunohistochemistry (IHC)-based image variability.MethodsCST was applied to segment epithelium in immunohistological cervical Whole Slide Images (WSIs). CST randomly distorts input tiles and factors the difference between the CNN prediction for the original and distorted inputs within the loss function. CNNs were trained using 114 p16-stained WSIs from the same scanner, and evaluated on 6 WSI test sets, each with 23 to 24 WSIs of the same tissue but different scanner/IHC combinations. Relative robustness (rAUC) was measured as the difference between the AUC on the training domain test set (i.e., baseline test set) and the remaining test sets.ResultsAcross all test sets, The AUC of CST models outperformed “No CST” models (AUC: 0.940–0.989 vs. 0.905–0.986, p < 1e − 8), and obtained an improved robustness (rAUC: [−0.038, −0.003] vs. [−0.081, −0.002]). At a WSI level, CST models showed an increase in performance in 124 of the 142 WSIs. CST models also outperformed models trained with random on-the-fly data augmentation (DA) in all test sets ([0.002, 0.021], p < 1e-6).ConclusionCST offers a path to improve CNN performance without the need for more data and allows customizing distortions to specific use cases. A python implementation of CST is publicly available at https://github.com/TIGACenter/CST_v1.

ObjectiveThis study aimed to develop and validate an Informer- Convolutional Neural Network (CNN) model to predict the gamma passing rate (GPR) for patient-specific quality assurance in volumetric modulated arc therapy (VMAT), enhancing treatment safety and efficacy by integrating multiple data sources.MethodsAnalyzing 465 VMAT treatment plans covering head &amp;amp; neck, chest, and abdomen, the study extracted data from 31 complexity indicators, 123 radiomics features, and 123 dosimetrics indices, along with daily linac performance data including 141 key performance indicators. A hybrid Informer-CNN architecture was used to handle both temporal and non-temporal data for predicting GPR.ResultsThe Informer-CNN model demonstrated superior predictive performance over traditional models like Convolutional Neural Networks (CNN), Long Short-Term Memory(LSTM), and Informer. Specifically, in the validation set, the model achieved a mean absolute error (MAE) of 0.0273 and a root mean square error (RMSE) of 0.0360 using the 3%/3mm criterion. In the test set, the MAE was 0.0327 and the RMSE was 0.0468. The model also showed high classification performance with AUC scores of 0.97 and 0.95 in test and validation sets, respectively.ConclusionThe developed Informer-CNN model significantly enhances the prediction accuracy and classification of gamma passing rates in VMAT treatment plans. It facilitates early integration of daily accelerator performance data, improving the assessment and verification of treatment plans for better patient-specific quality assurance.

Deep learning (DL) is now generally acknowledged as the benchmark and evolution in machine learning (ML) fields. Further more, it has steadily suited the most extensively in computational techniques for ML, delivering excellent outcomes in several challenging intellectual works that have equal or still outperformed human ability. DL has the advantage of learning from enormous amounts of data. Here, the DL approach has been applied to classify thunderstorms and non-thunderstorms with a meteorological dataset. Daily observational hourly data sets from 2016 to 2021 have been used to classify the incidence of thunderstorms and non-thunderstorms. Three DL approaches are used: Convolutional Neural Networks (CNN), Recurrent Neural Networks (RNN), and Long Short-Term Memory (LSTM) to classify thunderstorm incidences. These DL techniques are compared with conventional ML techniques, showing that DL approaches outperformed ML. The CNN method provides the highest performance with 92.01% accuracy among all DL and ML approaches.

Transesophageal echocardiography (TEE) is being increasingly utilized during orthotopic liver transplantation to guide perioperative management. Obstacles of improving TEE utilization include the challenge of becoming familiar with the TEE machine, optimizing TEE images, and translating skills acquired in didactic or simulator center-based training into clinical use. In an effort to expand TEE utilization and improve workflow among the liver transplantation (LT) anesthesiologists at our institution, a LT-specific TEE guide was created to serve as a reference and educational tool during LT. A 26-question survey was distributed to all LT anesthesiologists before and 6 months after implementation of the LT-specific TEE guide. All seven LT anesthesiologists completed the survey questions during the study period. No statistically significant difference was detected in participant-reported confidence in optimizing targeted TEE views, performing technical aspects of the exam, navigating the knobs on the TEE machine, or in ability to identify abnormal cardiac pathology during the study period. One participant became basic-TEE certified during the study period. Implementing a liver transplant-specific TEE guide is a strategy to expand TEE utilization, encourage longitudinal TEE education and reinforce concepts learned from hands-on education sessions; however, we did not detect a difference in participant-reported confidence of performing a TEE exam, ability to identify abnormal cardiac pathology or altering workflow. Further studies with larger sample sizes will be needed to evaluate the effectiveness of a LT-specific TEE guide.

Two-dimensional echocardiography is a useful tool for diagnosis of intracardiac masses. However, there are limitations to this technique, particularly when dealing with the left atrial appendage using a transthoracic approach. In this study, we compared the use of two techniques, transthoracic and transesophageal echocardiography, for detection of left atrial masses, in a series of 68 patients who were suspected of having mitral valvular lesion prior to heart surgery. Eleven of the 68 cases were found to have intracardiac masses. Six of the 11 had left atrial thrombi, 4 had left atrial myxoma, and 1 case had malignant sarcoma. We found a discrepancy between the two techniques. Using transthoracic echocardiography, we found 5 false-negative results (4 left atrial thrombi, 1 malignant sarcoma) and 1 false-positive (left atrial wall fold mimicking a thrombus). All the intracardiac masses were detected by transesophageal echocardiography. We concluded that transesophageal echocardiography is a particularly useful and more efficient approach for detection of left atrial masses than transthoracic echocardiography.

Gastrointestinal endoscopic image analysis presents significant challenges, such as considerable variations in quality due to the challenging in-body imaging environment, the often-subtle nature of abnormalities with low interobserver agreement, and the need for real-time processing. These challenges pose strong requirements on the performance, generalization, robustness and complexity of deep learning-based techniques in such safety–critical applications. While Convolutional Neural Networks (CNNs) have been the go-to architecture for endoscopic image analysis, recent successes of the Transformer architecture in computer vision raise the possibility to update this conclusion. To this end, we evaluate and compare clinically relevant performance, generalization and robustness of state-of-the-art CNNs and Transformers for neoplasia detection in Barrett’s esophagus. We have trained and validated several top-performing CNNs and Transformers on a total of 10,208 images (2,079 patients), and tested on a total of 7,118 images (998 patients) across multiple test sets, including a high-quality test set, two internal and two external generalization test sets, and a robustness test set. Furthermore, to expand the scope of the study, we have conducted the performance and robustness comparisons for colonic polyp segmentation (Kvasir-SEG) and angiodysplasia detection (Giana). The results obtained for featured models across a wide range of training set sizes demonstrate that Transformers achieve comparable performance as CNNs on various applications, show comparable or slightly improved generalization capabilities and offer equally strong resilience and robustness against common image corruptions and perturbations. These findings confirm the viability of the Transformer architecture, particularly suited to the dynamic nature of endoscopic video analysis, characterized by fluctuating image quality, appearance and equipment configurations in transition from hospital to hospital. The code is made publicly available at: https://github.com/BONS-AI-VCA-AMC/Endoscopy-CNNs-vs-Transformers.

Agricultural productivity is greatly influenced by plant health, including mustard greens (Brassica rapa), which are prone to leaf diseases and have high economic value. This study aims to develop a digital image-based classification system for mustard leaf diseases using a deep learning approach, particularly the Convolutional Neural Network (CNN) VGG19 architecture, and to compare its performance with ResNet50 and VGG16 models. The dataset used consists of 999 images divided into two classes: healthy mustard leaves and diseased mustard leaves. The images were processed through preprocessing steps (resized to 224×224 and normalized), then split into training, validation, and testing sets (80:10:10). The VGG19 architecture was customized with additional layers such as Global Average Pooling and Dense layers, and trained for 50 epochs with a configuration of 32 filters, a dropout rate of 0.5, and a learning rate of 0.0003. The results showed that the VGG19 model achieved the highest validation accuracy of 96%, followed by VGG16 with 95%, and ResNet50 with 74%. Evaluation using a confusion matrix demonstrated that VGG19 exhibited the most stable and accurate classification performance in recognizing both classes. These findings reinforce the potential of VGG19 for developing automated and real-time plant disease detection systems. Furthermore, this study opens up opportunities for integration into agricultural Internet of Things (IoT) systems for continuous plant health monitoring, thereby assisting farmers in making faster and more accurate preventive decisions.