Ischemic Stroke Segmentation by Transformer and Convolutional Neural Network Using Few-Shot Learning

PurposeSemantic segmentation is a fundamental part of the surgical application of deep learning. Traditionally, segmentation in vision tasks has been performed using convolutional neural networks (CNNs), but the transformer architecture has recently been introduced and widely investigated. We aimed to investigate the performance of deep learning models in segmentation in robot-assisted radical prostatectomy (RARP) and identify which of the architectures is superior for segmentation in robotic surgery.Materials and MethodsIntraoperative images during RARP were obtained. The dataset was randomly split into training and validation data. Segmentation of the surgical instruments, bladder, prostate, vas and seminal vesicle was performed using three CNN models (DeepLabv3, MANet, and U-Net++) and three transformers (SegFormer, BEiT, and DPT), and their performances were analyzed.ResultsThe overall segmentation performance during RARP varied across different model architectures. For the CNN models, DeepLabV3 achieved a mean Dice score of 0.938, MANet scored 0.944, and U-Net++ reached 0.930. For the transformer architectures, SegFormer attained a mean Dice score of 0.919, BEiT scored 0.916, and DPT achieved 0.940. The performance of CNN models was superior to that of transformer models in segmenting the prostate, vas, and seminal vesicle.ConclusionsDeep learning models provided accurate segmentation of the surgical instruments and anatomical structures observed during RARP. Both CNN and transformer models showed reliable predictions in the segmentation task; however, CNN models may be more suitable than transformer models for organ segmentation and may be more applicable in unusual cases. Further research with large datasets is needed.

What's New in Spine Surgery.

Medical Image Semantic Segmentation (MISS), the process of assigning a semantic label to each pixel in an image, is a foundational task in computational medicine, critical for quantitative diagnostics and treatment planning. However, developing robust MISS models faces two intertwined challenges. First, there is an architectural dilemma: Convolutional Neural Networks (CNNs), like U-Net, excel at learning local features but are limited by their receptive fields, failing to capture global context essential for segmenting organs with large deformations. Conversely, Vision Transformers (ViTs) effectively model long-range dependencies but lack the inductive biases of CNNs, leading to poor generalization on the small datasets typical in medicine without extensive pre-training. Second, the prohibitive cost and expertise required for creating pixel-level annotations create a severe data scarcity bottleneck. While Semi-Supervised Learning (SSL) aims to mitigate this by leveraging unlabeled data, existing methods often fail to learn high-level semantic relations and are susceptible to confirmation bias from noisy pseudo-labels, class imbalance, and suboptimal contrastive sample selection. This thesis presents a comprehensive investigation to systematically address these challenges, delivering a cohesive suite of novel deep learning frameworks. The contributions are four-fold: First, to resolve the architectural trade-off, this work introduces CS-Unet, a pure Transformer network built upon a U-Net-like architecture. Its core innovation is the Convolutional Swin Transformer (CST) block, which integrates convolutions directly within the Multi-Head Self-Attention and Feed-Forward Network modules. This design imbues the Transformer with inherent localized spatial context and strong inductive biases, enabling it to efficiently learn both local and global features. Without pre-training, CS-Unet outperforms existing Transformer and CNN-based models on multi-organ and cardiac datasets, achieving state-of-the-art performance with fewer parameters. Second, to address data scarcity, a novel Multi-Scale Cross Supervised Contrastive Learning (MCSC) framework for SSL is developed. MCSC jointly trains CNN and Transformer models, using a cross-teaching paradigm where each network provides pseudo-labels for the other. Crucially, it moves beyond simple output consistency by applying a contrastive loss to feature maps at multiple scales, enforcing hierarchical semantic consistency. To handle the class imbalance endemic to medical imaging, a class-prevalence-aware loss is used to ensure features for infrequent classes are learned robustly. Third, to fortify SSL against noisy pseudo-labels, a certainty-guided contrastive learning strategy is proposed. This approach mitigates the impact of inaccurate pseudo-labels by using a certainty metric to guide the selection of samples for contrastive learning. The framework’s computational efficiency is enhanced through novel sampling strategies that select a few representative samples for contrasting, and a negative memory bank is used to increase sample diversity and eliminate dependence on batch size. Fourth, this thesis introduces a new paradigm for SSL by leveraging external anatomical priors through the Contrastive Cross-Teaching with Registration (CCT-R) framework. CCT-R is the first method to integrate spatial registration transforms into the learning process. It features two novel modules: a Registration Supervision Loss (RSL), which uses transforms between labeled and unlabeled volumes to generate an additional, highly reliable source of pseudo-labels, and Registration-Enhanced Positive Sampling (REPS), which uses registration to identify anatomically-corresponding positive pairs across volumes for contrastive learning. Overall, these contributions provide a powerful toolkit that significantly alleviates the annotation bottleneck in medical AI. The proposed methods demonstrate state-of-the-art performance on challenging segmentation benchmarks, delivering a pathway to develop accurate, data-efficient models for real-world clinical applications and opening new avenues for research into fusing geometric priors with semantic segmentation.

Bone cancer is a significant health issue that leads to severe complications and deaths worldwide. Early detection can significantly improve patient outcomes, often resulting in a complete cure. Traditional approaches to managing bone cancer can be augmented with technology-driven methods, particularly those enabled by artificial intelligence. Convolutional Neural Networks (CNNs) and their variants have proven effective in medical data analysis. However, deep learning techniques require enhancements to improve performance in cancer detection across various medical imaging modalities. In this paper, we propose a deep-learning framework that utilizes advanced CNN models for the automatic screening of bone cancer. We have enhanced both the CNN model and the ResNet-50 model, which are integral components of the proposed framework. Additionally, we introduced an algorithm called Learning-based Bone Cancer Detection (LbBCD), designed to optimize the utilization of these enhanced deep learning models to improve bone cancer detection efficiency. Our research emphasizes a Region of Interest (ROI) based approach to enhance the screening process for bone cancer. Using a benchmark dataset known as the Bone CT Scan dataset, our empirical study demonstrated that the proposed deep learning framework, integrated with enhanced CNN and ResNet-50 models, achieved remarkable performance. Specifically, the enhanced CNN model reached an accuracy of 90.90%, while the enhanced ResNet-50 model achieved an accuracy of 92.20%, outperforming state-of-the-art deep learning models. Therefore, this proposed system can be integrated into healthcare applications for automatically screening bone cancer, contributing to a clinical decision support system for healthcare professionals.

Excessive withdrawal of groundwater for agricultural irrigation can cause seawater intrusion into coastal aquifers. Such a case will in turn results in deterioration of irrigation water quality. Determination of irrigation water quality with traditional methods is a time-consuming and costly process. However, machine learning algorithms can be useful tools for modeling and estimating groundwater quality used for irrigation water purposes. In this study, TDS, PS, SAR, and Cl parameters of groundwater were estimated with models based on EC and pH variables. For this purpose, prediction performances of two different deep learning methods (convolutional neural network (CNN) and deep neural network (DNN)) and two different classical machine learning (Random Forest (RF) and extreme gradient boosting (XGBoost)) methods were compared. In addition, predictive uncertainty of the models was determined by quantile regression (QR) analysis. Performance criteria and results of uncertainty analysis revealed that CNN (in testing phase, NSE = 0.95 for TDS, NSE = 0.96 for PS, NSE = 0.67 for SAR and NSE = 0.93 for CI) and DNN (in testing phase, NSE = 0.91 for TDS, NSE = 0.91 for PS, NSE = 0.57 for SAR and NSE = 0.94 for Cl) models had quite a close performance in estimation of TDS, PS, SAR, and Cl parameters and higher than the other two classical machine learning methods. As a result, the CNN model can be considered the best performing model in estimating all quality parameters due to the highest NSE and lowest RMSE values. In addition, the Taylor diagram showed that the values estimated using the CNN model had the highest correlation with the measured data. It was determined that the model with the lowest uncertainty based on the PICP statistics was DNN, followed by the CNN model. However, the CNN model has predicted outliers more accurately. Present findings proved that deep learning models could offer efficient tools for predicting irrigation water quality parameters.

Development in Computer vision applications, most prominently in Artificial Intelligence (AI) over the past decade, has encouraged researchers to use computational algorithms. Machine Learning (ML) and Deep Learning (DL) algorithms are getting attention in the biomedical area to early diagnosis and prognosis of lethiferous diseases with higher accuracy using statistical analysis. Cervical cancer is one of the predominant gynecological infectious forms of cancer in females ecumenically, generally transpiring in less-developed nations. DL algorithms are more optimized and convenient due to the automatic feature extraction in Cervical lesion detection at an early phase. Here we analyze the convolutional neural network (CNN) models' performance using variation in channels depth comprises in the convolution layers to classify multi-class liquid-based cytology (LBC) Whole slide images (WSI). We propose three CNN models having two convolution layers with the number of channels (4,8), (8,16), and (32,64) and two pooling layers. CNN model with the highest number of channels in the convolution layers gives the leading performance with an accuracy of 96.89%, precision of 93.38%, the sensitivity of 93.75%, and F-score of 94.15%.

Distinguishing benign inverted papilloma (IP) tumors from those that have undergone malignant transformation to squamous cell carcinoma (IP-SCC) is important but challenging to do preoperatively. Magnetic resonance imaging (MRI) can help differentiate these 2 entities, but no established method exists that can automatically synthesize all potentially relevant MRI image features to distinguish IP and IP-SCC. We explored a deep learning approach, using 3-dimensional convolutional neural networks (CNNs), to address this challenge. Retrospective chart reviews were performed at 2 institutions to create a data set of preoperative MRIs with corresponding surgical pathology reports. The MRI data set included all available MRI sequences in the axial plane, which were used to train, validate, and test 3 CNN models. Saliency maps were generated to visualize areas of MRIs with greatest influence on predictions. A total of 90 patients with IP (n=64) or IP-SCC (n=26) tumors were identified, with a total of 446 images of distinct MRI sequences for IP (n=329) or IP-SCC (n=117). The best CNN model, All-Net, demonstrated a sensitivity of 66.7%, specificity of 81.5%, overall accuracyof 77.9%, and receiver-operating characteristic area under the curve of 0.80 (95% confidence interval, 0.682-0.898) for test classification performance. The other 2 models, Small-All-Net and Elastic-All-Net, showed similar performance levels. A deep learning approach with 3-dimensional CNNs can distinguish IP and IP-SCC with moderate test classification performance. Although CNNs demonstrate promise to enhance the prediction of IP-SCC using MRIs, more data are needed before they can reach the predictive value already established by human MRI evaluation.

Backgroundand Purpose:This study aimed to develop and evaluate an efficient method to automatically segment T1- and T2-weighted brain magnetic resonance imaging (MRI) images. We specifically compared the segmentation performance of individual convolutional neural network (CNN) models against an ensemble approach to advance the accuracy of MRI-guided radiotherapy (RT) planning.Materials and Methods. The evaluation was conducted on a private clinical dataset and a publicly available dataset (HaN-Seg). Anonymized MRI data from 55 brain cancer patients, including T1-weighted, T1-weighted with contrast, and T2-weighted images, were used in the clinical dataset. We employed an EDL strategy that integrated five independently trained 2D neural networks, each tailored for precise segmentation of tumors and organs at risk (OARs) in the MRI scans. Class probabilities were obtained by averaging the final layer activations (Softmax outputs) from the five networks using a weighted-average method, which were then converted into discrete labels. Segmentation performance was evaluated using the Dice similarity coefficient (DSC) and Hausdorff distance at 95% (HD95). The EDL model was also tested on the HaN-Seg public dataset for comparison.Results. The EDL model demonstrated superior segmentation performance on both the clinical and public datasets. For the clinical dataset, the ensemble approach achieved an average DSC of 0.7 ± 0.2 and HD95 of 4.5 ± 2.5 mm across all segmentations, significantly outperforming individual networks which yielded DSC values ≤0.6 and HD95 values ≥14 mm. Similar improvements were observed in the HaN-Seg public dataset.Conclusions. Our study shows that the EDL model consistently outperforms individual CNN networks in both clinical and public datasets, demonstrating the potential of ensemble learning to enhance segmentation accuracy. These findings underscore the value of the EDL approach for clinical applications, particularly in MRI-guided RT planning.

Nowadays, deepfake is wreaking havoc on society. Deepfake content is created with the help of artificial intelligence and machine learning to replace one person’s likeness with another person in pictures or recorded videos. Although visual media manipulations are not new, the introduction of deepfakes has marked a breakthrough in creating fake media and information. These manipulated pictures and videos will undoubtedly have an enormous societal impact. Deepfake uses the latest technology like Artificial Intelligence (AI), Machine Learning (ML), and Deep Learning (DL) to construct automated methods for creating fake content that is becoming increasingly difficult to detect with the human eye. Therefore, automated solutions employed by DL can be an efficient approach for detecting deepfake. Though the “black-box” nature of the DL system allows for robust predictions, they cannot be completely trustworthy. Explainability is the first step toward achieving transparency, but the existing incapacity of DL to explain its own decisions to human users limits the efficacy of these systems. Though Explainable Artificial Intelligence (XAI) can solve this problem by interpreting the predictions of these systems. This work proposes to provide a comprehensive study of deepfake detection using the DL method and analyze the result of the most effective algorithm with Local Interpretable Model-Agnostic Explanations (LIME) to assure its validity and reliability. This study identifies real and deepfake images using different Convolutional Neural Network (CNN) models to get the best accuracy. It also explains which part of the image caused the model to make a specific classification using the LIME algorithm. To apply the CNN model, the dataset is taken from Kaggle, which includes 70 k real images from the Flickr dataset collected by Nvidia and 70 k fake faces generated by StyleGAN of 256 px in size. For experimental results, Jupyter notebook, TensorFlow, NumPy, and Pandas were used as software, InceptionResnetV2, DenseNet201, InceptionV3, and ResNet152V2 were used as CNN models. All these models’ performances were good enough, such as InceptionV3 gained 99.68% accuracy, ResNet152V2 got an accuracy of 99.19%, and DenseNet201 performed with 99.81% accuracy. However, InceptionResNetV2 achieved the highest accuracy of 99.87%, which was verified later with the LIME algorithm for XAI, where the proposed method performed the best. The obtained results and dependability demonstrate its preference for detecting deepfake images effectively.

This study compared popular Deep Learning (DL) architectures to classify machining surface roughness using sound and force data. The DL architectures considered in this study include Multi-Layer Perceptron (MLP), Convolution Neural Network (CNN), Long Short-Term Memory (LSTM), and transformer. The classification was performed on the sound and force data generated during machining aluminum sheets for different levels of spindle speed, feed rate, depth of cut, and end-mill diameter, and it was trained on 30 s machining data (10–40 s) of the machining experiments. Since a raw audio waveform is seldom used in DL models, Mel-Spectrogram and Mel Frequency Cepstral Coefficients (MFCCs) audio feature extraction techniques were used in the DL models. The results of DL models were compared for the training–validation accuracy, training epochs, and training parameters of each model. Although the roughness classification by all the DL models was satisfactory (except for CNN with Mel-Spectrogram), the transformer-based modes had the highest training (>96%) and validation accuracies (≈90%). The CNN model with Mel-Spectrogram exhibited the worst training and inference accuracy, which is influenced by limited training data. Confusion matrices were plotted to observe the classification accuracy visually. The confusion matrices showed that the transformer model trained on Mel-Spectrogram and the transformer model trained on MFCCs correctly predicted 366 (or 91.5%) and 371 (or 92.7%) out of 400 test samples. This study also highlights the suitability and superiority of the transformer model for time series sound and force data and over other DL models.

BACKGROUND AND AIM: Outdoor ultrafine particles (UFP; 100 nm) and black carbon (BC) vary greatly within cities and may have adverse impacts on human health such as cardiovascular mortality and brain tumour incidence. Traditional regression methods for estimating spatial distributions of ambient air pollution depend on extensive, curated geospatial information system databases. This has resulted in a disparity wherein little is known about local air pollution in the data-sparse settings of low- and middle- income countries. We used a hybrid approach to develop new models to estimate within-city spatial variations in outdoor UFP and BC concentrations across Bucaramanga, Colombia. METHODS: We conducted a large-scale mobile monitoring campaign over twenty days in 2019. Land use regression (LUR) models were developed using land use parameters from curated and open-source databases. Convolutional Neural Network (CNN) models were developed using Google Maps satellite view and street view images. Combined models were developed by combining predictions from the LUR and CNN models. After training in the training and validation sets, predictions were generated in the test set and compared to measured values. RESULTS:The combined UFP model (R2=0.54) outperformed the CNN (R2=0.47) and land use regression (LUR) models (R2=0.47) on their own. Similarly, the combined BC model also outperformed the CNN and LUR BC models (R2 = 0.51 vs 0.43 and 0.45 respectively). Spatial variations in model performance were more stable for the CNN and combined models compared to the LUR models suggesting that the combined approach may be less likely to contribute to differential exposure measurement error in epidemiological studies. CONCLUSIONS:Estimates from these models can then be applied to population-based cohorts in order to evaluate population health risks. Additionally, our findings demonstrated that satellite and street-level images can be combined with a traditional LUR modelling approach to improve predictions of within-city spatial variations in outdoor UFP and BC concentrations. KEYWORDS: Ultrafine particles, Black Carbon, Deep learning, Images, Land use regression

Transformer based architectures have emerged as powerful deep learning models compared to traditional convolutional neural networks (CNNs) for complex tasks like image classification and segmentation. In this study, the performance of transformer architectures on analysis of medical imaging datasets is investigated through quantitative comparison of three state-of-the-art transformer models which are Vision Transformer (ViT), SegFormer, and MaskFormer, with respect to CNN-based architectures. The models are evaluated on benchmark MRI datasets of human brain across two core tasks: classification and semantic segmentation for three tumor types namely glioma, meningioma and pituitary tumor. Our analysis takes into account multiple performance indicators, including accuracy, segmentation quality, model complexity, and model interpretability. The results reveal notable differences in trade-offs between accuracy, interpretability, and resource demands, offering practical guidance on the relative strengths and suitability of transformer and CNN models in clinical brain tumor diagnostics.

Abstract. Crowd counting, a critical component in the management and safety planning of large gatherings and public spaces, is essential for ensuring smooth event operations and preventing potential overcrowding issues. While the standard convolutional neural network (CNN) based model performs well in head counting tasks, it has certain drawbacks when applied to complex scenarios. With the rapid development of artificial intelligence, Transformer models that rely on self-attention mechanisms, as Swin Transformer, have demonstrated exceptional performance in visual tasks, such as image classification, and segmentation in recent times. This study examines the experimental findings of Swin Transformer's head counting tasks and contrasts them with the CNN-based model. Mean Absolute Error (MAE) and Mean Square Error (MSE) evaluation indicators show that the Transformer model outperforms the classic CNN model in terms of generalization abilities when dealing with complicated scenarios. Future research work will increase the diversity of data sets and focus on optimizing model structure and improving training efficiency.

The classification process is also used in artificial intelligence (AI), which is intelligence created by a computer, so that it can mimic actions like humans in general and can capture events that occur in the surrounding environment. Seeing the very high development of the international coffee trade, it can be concluded that if there is a type of coffee that has the best quality, it will be sought after by coffee importing countries. Batik is cloth that is painted using canting and liquid wax to form motifs or patterns that have high artistic value. Batik has a variety of unique and distinctive patterns that reflect the region where the batik motif originates. One area that has batik motifs in Indonesia is batik in the North Coastal Region of Java Island. In this area, various batiks are produced according to the characteristics of the area itself. Regarding information related to the introduction of types of batik motifs, perhaps it comes from people who have batik skills and batik craftsmen who best understand batik as a whole, while the general public does not really know about batik motifs. Because batik has different motifs and batik motifs in some areas have motifs that are almost uniform but not the same. One of the technologies that can be used is deep learning. The purpose of this study is to propose a Convolutional Neural Network (CNN) and Transfer Learning (TL) model to be implemented in an intelligent system for the process of image classification of coffee bean types. The method used in this study is the PSOCNN transfer learning model VGG16 From the results of tests carried out on 2 models, namely the CNN model, the PSOCNN transfer learning model VGG16. it was found that the highest accuracy was obtained when classifying batik motives images using the PSOCNN-transfer learning model VGG16, which was 83%. The level of accuracy that is increased when compared to the usual CNN model indicates that the use of transfer learning has a good effect on the level of accuracy obtained. Although an increase of 6% is significant, but with this increase it opens up opportunities to increase even higher by using other transfer learning models

BackgroundCurrently, there is no fully automated tool available for evaluating the degree of cervical spinal stenosis. The aim of this study was to develop and validate the use of artificial intelligence (AI) algorithms for the assessment of cervical spinal stenosis.MethodsIn this retrospective multi-center study, cervical spine magnetic resonance imaging (MRI) scans obtained from July 2020 to June 2023 were included. Studies of patients with spinal instrumentation or studies with suboptimal image quality were excluded. Sagittal T2-weighted images were used. The training data from the Fourth People’s Hospital of Shanghai (Hos. 1) and Shanghai Changzheng Hospital (Hos. 2) were annotated by two musculoskeletal (MSK) radiologists following Kang’s system as the standard reference. First, a convolutional neural network (CNN) was trained to detect the region of interest (ROI), with a second Transformer for classification. The performance of the deep learning (DL) model was assessed on an internal test set from Hos. 2 and an external test set from Shanghai Changhai Hospital (Hos. 3), and compared among six readers. Metrics such as detection precision, interrater agreement, sensitivity (SEN), and specificity (SPE) were calculated.ResultsOverall, 795 patients were analyzed (mean age ± standard deviation, 55±14 years; 346 female), with 589 in the training (75%) and validation (25%) sets, 206 in the internal test set, and 95 in the external test set. Four tasks with different clinical application scenarios were trained, and their accuracy (ACC) ranged from 0.8993 to 0.9532. When using a Kang system score of ≥2 as a threshold for diagnosing central cervical canal stenosis in the internal test set, both the algorithm and six readers achieved similar areas under the receiver operating characteristic curve (AUCs) of 0.936 [95% confidence interval (CI): 0.916–0.955], with a SEN of 90.3% and SPE of 93.8%; the AUC of the DL model was 0.931 (95% CI: 0.917–0.946), with a SEN in the external test set of 100%, and a SPE of 86.3%. Correlation analysis comparing the DL method, the six readers, and MRI reports between the reference standard showed a moderate correlation, with R values ranging from 0.589 to 0.668. The DL model produced approximately the same upgrades (9.2%) and downgrades (5.1%) as the six readers.ConclusionsThe DL model could fully automatically and reliably assess cervical canal stenosis using MRI scans.