TokenCut: Segmenting Objects in Images and Videos with Self-supervised Transformer and Normalized Cut

Contrastive learning, a successful form of representational learning, has shown promising results in pretraining deep learning (DL) models for downstream tasks. When working with limited annotation data, as in medical image segmentation tasks, learning domain-specific local representations can further improve the performance of DL models. In this work, we extend the contrastive learning framework to utilize domain-specific contrast information from unlabeled Magnetic Resonance (MR) images to improve the performance of downstream MR image segmentation tasks in the presence of limited labeled data. The contrast in MR images is controlled by underlying tissue properties (e.g., T1 or T2) and image acquisition parameters. We hypothesize that learning to discriminate local representations based on underlying tissue properties should improve subsequent segmentation tasks on MR images. We propose a novel constrained contrastive learning (CCL) strategy that uses tissue-specific information via a constraint map to define positive and negative local neighborhoods for contrastive learning, embedding this information in the representational space during pretraining. For a given MR contrast image, the proposed strategy uses local signal characteristics (constraint map) across a set of related multi-contrast MR images as a surrogate for underlying tissue information. We demonstrate the utility of the approach for downstream: (1) multi-organ segmentation tasks in T2-weighted images where a DL model learns T2 information with constraint maps from a set of 2D multi-echo T2-weighted images (n=101) and (2) tumor segmentation tasks in multi-parametric images from the public brain tumor segmentation (BraTS) (n=80) dataset where DL models learn T1 and T2 information from multi-parametric BraTS images. Performance is evaluated on downstream multi-label segmentation tasks with limited data in (1) T2-weighted images of the abdomen from an in-house Radial-T2 (Train/Test=30/20), (2) public Cartesian-T2 (Train/Test=6/12) dataset, and (3) multi-parametric MR images from the public brain tumor segmentation dataset (BraTS) (Train/Test=40/50). The performance of the proposed CCL strategy is compared to state-of-the-art self-supervised contrastive learning techniques. In each task, a model is also trained using all available labeled data for supervised baseline performance. The proposed CCL strategy consistently yielded improved Dice scores, Precision, and Recall metrics, and reduced HD95 values across all segmentation tasks. We also observed performance comparable to the baseline with reduced annotation effort. The t-SNE visualization of features for T2-weighted images demonstrates its ability to embed T2 information in the representational space. On the BraTS dataset, we also observed that using an appropriate multi-contrast space to learn T1+T2, T1, or T2 information during pretraining further improved the performance of tumor segmentation tasks. Learning to embed tissue-specific information that controls MR image contrast with the proposed constrained contrastive learning improved the performance of DL models on subsequent segmentation tasks compared to conventional self-supervised contrastive learning techniques. The use of such domain-specific local representations could help understand, improve performance, and mitigate the scarcity of labeled data in MR image segmentation tasks.

Medical image segmentation is a fundamental task in medical image analysis and has been widely applied in multiple medical fields. The latest transformer-based deep learning segmentation model, Segment Anything Model (SAM), has demonstrated outstanding performance in natural image segmentation tasks through large-scale pre-training, achieving zero-shot image semantic understanding and pixel-level segmentation. However, medical images present challenges such as style variability, ill-defined object boundaries, and feature ambiguities, limiting the direct applicability of the SAM to medical image segmentation tasks. To enhance the robustness of the SAM in the domain of medical segmentation, we propose the SAM-RCCF framework. This approach aims to enhance the generalizability and precision of segmentation performance across diverse intracranial tumor types, including gliomas, metastatic tumors, and meningiomas. The study collected 484 axial T1-weighted contrast-enhanced (T1CE) magnetic resonance imaging (MRI) data of brain tumor patients, including 164 cases of glioma, 158 cases of metastatic tumors, and 162 cases of meningioma. All imaging data were randomly divided into training and testing sets. We employed the proposed SAM-RCCF model to perform segmentation experiments on these data, and five-fold cross-validation was adopted to evaluate the model's performance. This framework integrates the RefineNet module and the conditional control field with a conditional controller and Mask generator, enabling precise feature recognition and tailored segmentation for medical images, optimizing segmentation accuracy RESULTS: In the glioma segmentation experiment, the SAM-RCCF model achieved outstanding performance with an IOU of 0.90, DSC of 0.912, and HD of 13.13. For the meningioma segmentation task, it obtained an IOU of 0.9214, DSC of 0.93, and HD of 11.41, significantly outperforming other classic segmentation models. The segmentation experiment results demonstrate that in the segmentation tasks of glioma, metastatic tumors, and meningioma MRI images, the SAM-RCCF algorithm significantly outperformed the original SAM in terms of DSC, HD, and IOU segmentation performance metrics. The experimental results verify the effectiveness of the SAM-RCCF framework in segmenting complex and variable brain tumor images, enhancing segmentation accuracy and robustness.

ObjectivesMedical photography is ubiquitous and plays an increasingly important role in the fields of medicine and surgery. Any assessment of these photographs by computer vision algorithms requires first that the area of interest can accurately be delineated from the background. We aimed to develop deep learning segmentation models for kidney and liver organ donation photographs where accurate automated segmentation has not yet been described.MethodsTwo novel deep learning models (Detectron2 and YoloV8) were developed using transfer learning and compared against existing tools for background removal (macBGRemoval, remBGisnet, remBGu2net). Anonymised photograph datasets comprised training/internal validation sets (821 kidney and 400 liver images) and external validation sets (203 kidney and 208 liver images). Each image had two segmentation labels: whole organ and clear view (parenchyma only). Intersection over Union (IoU) was the primary outcome, as the recommended metric for assessing segmentation performance.ResultsIn whole kidney segmentation, Detectron2 and YoloV8 outperformed other models with internal validation IoU of 0.93 and 0.94, and external validation IoU of 0.92 and 0.94, respectively. Other methods – macBGRemoval, remBGisnet and remBGu2net – scored lower, with highest internal validation IoU at 0.54 and external validation at 0.59. Similar results were observed in liver segmentation, where Detectron2 and YoloV8 both showed internal validation IoU of 0.97 and external validation of 0.92 and 0.91, respectively. The other models showed a maximum internal validation and external validation IoU of 0.89 and 0.59 respectively. All image segmentation tasks with Detectron2 and YoloV8 completed within 0.13–1.5 s per image.ConclusionsAccurate, rapid and automated image segmentation in the context of surgical photography is possible with open-source deep-learning software. These outperform existing methods and could impact the field of surgery, enabling similar advancements seen in other areas of medical computer vision.

BackgroundIn computer vision, image segmentation is crucial with applications ranging from autonomous driving to medical imaging.ObjectiveTo provide reliable segmentation across varied datasets, this study assesses the performance of an image segmentation model based on SegNet.MethodUsing a five-fold and a K-fold cross-validation method, the SegNet model is thoroughly validated. Intersection over Union (IOU), Dice Coefficient, Precision, Recall, Accuracy, and loss metrics are measured in the study to assess how well the model performs and is optimized throughout training.ResultsThe SegNet model consistently performs well throughout the folds, with Dice Coefficient values ranging from 88.32% to 89.8% and IOU scores ranging from 94.53% to 95.05%. The model's dependability is confirmed by metrics like precision, recall, and accuracy, all of which often exceed 90%. Loss values between 0.495 and 0.547 show that training optimized the system effectively.ConclusionBy enhancing the validation reliability, the K-fold cross-validation method highlights by what means the SegNet model segments objects in images across a range of datasets. These outcomes strengthen the confidence in the model's ability to generalize and highlight its potential for several practical uses in image segmentation.

Rheumatoid arthritis (RA) is a chronic autoimmune disease that can cause severe joint damage and functional impairment. Ultrasound imaging has shown promise in providing real-time assessment of synovium inflammation associated with the early stages of RA. Accurate segmentation of the synovium region and quantification of inflammation-specific imaging biomarkers are crucial for assessing and grading RA. However, automatic segmentation of the synovium in 3D ultrasound is challenging due to ambiguous boundaries, variability in synovium shape, and inhomogeneous intensity distribution. In this work, we introduce a novel network architecture, Swin Transformers with Deep Attentive Features for 3D segmentation (SwinDAF3D), which integrates Swin Transformers into a Deep Attentive Features framework. The developed architecture leverages the hierarchical structure and shifted windows of Swin Transformers to capture rich, multi-scale and attentive contextual information, improving the modeling of long-range dependencies and spatial hierarchies in 3D ultrasound images. In a six-fold cross-validation study with 3D ultrasound images of RA patients' finger joints (n = 72), our SwinDAF3D model achieved the highest performance with a Dice Score (DSC) of 0.838 ± 0.013, an Intersection over Union (IoU) of 0.719 ± 0.019, and Surface Dice Score (SDSC) of 0.852 ± 0.020, compared to 3D UNet (DSC: 0.742 ± 0.025; IoU: 0.589 ± 0.031; SDSC: 0.661 ± 0.029), DAF3D (DSC: 0.813 ± 0.017; IoU: 0.689 ± 0.022; SDSC: 0.817 ± 0.013), Swin UNETR (DSC: 0.808 ± 0.025; IoU: 0.678 ± 0.032; SDSC: 0.822 ± 0.039), UNETR++ (DSC: 0.810 ± 0.014; IoU: 0.684 ± 0.018; SDSC: 0.829 ± 0.027) and TransUNet (DSC: 0.818 ± 0.013; IoU: 0.692 ± 0.017; SDSC: 0.815 ± 0.016) models. This ablation study demonstrates the effectiveness of combining a Swin Transformers feature pyramid with a deep attention mechanism, improving the segmentation accuracy of the synovium in 3D ultrasound. This advancement shows great promise in enabling more efficient and standardized RA screening using ultrasound imaging.

Accurate segmentation of fingernail images is essential for biomedical applications like dermatological diagnostics and nail disease assessments. This study compares traditional methods (Sobel and Canny edge detectors) with an improved method using adaptive thresholding and morphological closing for fingernail image segmentation. The methodology includes data collection, preprocessing, edge detection, segmentation, and evaluation. A dataset of 500 fingernail images (free of nail polish) was captured using a digital camera. Preprocessing involves grayscale conversion to simplify analysis and Gaussian smoothing to reduce noise while preserving key features. For segmentation, thresholding and K-means clustering isolate the fingernail from the background. Evaluation combines qualitative and quantitative analyses. Qualitative results demonstrate that the improved method consistently outperforms traditional techniques under diverse conditions. Quantitative evaluation, based on accuracy, recall, F1 score, and Intersection over Union (IoU), further supports these findings. The Sobel method achieves 0.80 accuracy, 0.77 recall, 0.87 F1 score, and 0.77 IoU. The Canny method achieves 0.82 accuracy, 0.78 recall, 0.88 F1 score, and 0.78 IoU. In contrast, the improved method achieves 0.97 accuracy, 0.98 recall, 0.99 F1 score, and 0.98 IoU. The results clearly show that the improved method, using adaptive thresholding and morphological closing, provides superior segmentation performance. Additionally, the approach remains computationally efficient, making it suitable for real-time applications in medical diagnostics.

Due to the variety and complexity of objects in X-ray images, how to detect the prohibited items automatically and accurately is a challenging problem. In this paper, an X-ray image prohibited object detection method based on Dynamic Deformable Convolution (DyDC) and adaptive Intersection over Union (IoU) is proposed based on Cascade R-CNN framework. The main contributions are as follows. First, DyDC is proposed to cope with the diversity of the prohibited objects in X-ray images and to improve the feature representation capability. Then, adaptive IoU mechanism is proposed, which can dynamically adjust the IoU threshold during the training process to generate high quality proposals. The proposed method is extensively evaluated on two publicly available benchmark datasets, namely SIXray and OPIXray, and the experimental results show that it can achieve the state-of-the-art detection accuracy, compared with other existing methods.

Among the most difficult difficulties in contemporary satellite image-processing subjects is cloud and cloud shade segmentation. Due to substantial background noise interference, existing cloud and cloud shadow segmentation techniques would result in false detection and missing detection. We propose a Location Pooling Multi-Scale Network (LPMSNet) in this study. The residual network is utilised as the backbone in this method to acquire semantic info on various levels. Simultaneously, the Location Attention Multi-Scale Aggregation Module (LAMA) is introduced to obtain the image’s multi-scale info. The Channel Spatial Attention Module (CSA) is introduced to boost the network’s focus on segmentation goals. Finally, in view of the problem that the edge details of cloud as well as cloud shade are easily lost, this work designs the Scale Fusion Restoration Module (SFR). SFR can perform picture upsampling as well as the acquisition of edge detail information from cloud as well as cloud shade. The mean intersection over union (MIoU) accuracy of this network reached 94.36% and 81.60% on the Cloud and Cloud Shadow Dataset and the five-category dataset L8SPARCS, respectively. On the two-category HRC-WHU Dataset, the accuracy of the network on the intersection over union (IoU) reached 90.51%. In addition, in the Cloud and Cloud Shadow Dataset, our network achieves 97.17%, 96.83%, and 97.00% in precision (P), recall (R), and F1 score (F1) in cloud segmentation tasks, respectively. In the cloud shadow segmentation task, precision (P), recall (R), and F1 score (F1) reached 95.70%, 96.38%, and 96.04%, respectively. Therefore, this method has a significant advantage over the current cloud and cloud shade segmentation methods.

Using a Generative Adversarial Network (GAN) has proven its ability to successfully implement realistic images in image translation fields. It has its successful implementation in the sketch-to-image translation, too. Generative adversarial networks are widely used for the purpose of image translation. Most discriminators in generative adversarial networks use encoder or decoder blocks for image segmentation and classification tasks. U-net-based architecture is mostly used in the generator but rarely in the discriminator. If used in the discriminator, it is used for image resolution increment and segmentation tasks. In this research, a U-net-based discriminator is used for image translation tasks. U-net-based discriminator uses local and global differences between the real and fake images, which helps maintain global and local data representation. Resnet-9, used in the generator, uses skip connections, shortcuts, and concatenations, enabling information to flow from earlier to later layers. This helps preserve the original image features and solves the vanishing gradient problems in normal generators. The use of a strong discriminator and effective generator helps in the improvement system's performance. The available dataset was unpaired at the same time. Datasets from various sources were combined and formed a sketch-image pair. The input is a 512x256 human sketch and a corresponding real image pair. The image pair is split into sketch and image with dimensions 256x256. The system's output is the human face image of the corresponding sketch.

In the last few years, Deep Learning (DL) has been showing superior performance in different modalities of bio-medical image analysis. Several DL architectures have been proposed for classification, segmentation, and detection tasks in medical imaging and computational pathology. In this paper, we propose a new DL architecture, the NABLA-N network (∇N-Net), with better feature fusion techniques in decoding units for dermoscopic image segmentation tasks. The ∇N-Net has several advances for segmentation tasks. First, this model ensures better feature representation for semantic segmentation with a combination of low to high-level feature maps. Second, this network shows better quantitative and qualitative results with the same or fewer network parameters compared to other methods. In addition, the Inception Recurrent Residual Convolutional Neural Network (IRRCNN) model is used for skin cancer classification. The proposed ∇N-Net network and IRRCNN models are evaluated for skin cancer segmentation and classification on the benchmark datasets from the International Skin Imaging Collaboration 2018 (ISIC-2018). The experimental results show superior performance on segmentation tasks compared to the Recurrent Residual U-Net (R2U-Net). The classification model shows around 87% testing accuracy for dermoscopic skin cancer classification on ISIC2018 dataset.

Coarse-to-fine airway segmentation using multi information fusion network and CNN-based region growing

This experiment conducted a standardized head-to-head evaluation of six medical image segmentation architectures-UNet, TotalSegmentator (nnU-Net-based), MONAI SegResNet, KM-UNet (KAN+SSM), UniverSeg, and KonfAI—on two cellular-level subsets (White Blood Cell and Yeaz) of MedSegBench. All models are trained under identical settings (50 epochs; batch size 8; Adam with <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\text{lr}=2 \mathrm{e}-4$</tex>; DiceCE loss; AMP; gradient clipping =1.0) and assessed with Dice (foreground only), Intersection over Union (IoU), and surface distance. While mean Dice is identical across models (0.2420), spatial metrics separate them: MONAI achieves the best IoU (0.9005) and the lowest surface distance (3.0309), followed by UniverSeg and KM-UNet (IoU <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\approx 0.89$</tex>). We further study crossarchitecture knowledge distillation. UNet → KM-UNet collapses spatial fidelity (IoU 0.0039; infinite surface distance) despite unchanged Dice, whereas MONAI → UniverSeg preserves partial structure (IoU 0.5637; surface distance 22.42). These results show (i) multi-metric evaluation beyond Dice is essential for judging boundary quality, and (ii) naïve distillation can fail to transfer boundary information across heterogeneous architectures; architectural compatibility and distillation design are critical for reliable deployment.

Bias in medical image segmentation can lead to unequal performance across demographic subgroups, raising concerns about fairness and reliability in clinical AI systems. While deep learning models have achieved high segmentation accuracy, ensuring equitable performance across race and gender remains a significant challenge, particularly in privacy-sensitive healthcare environments. This study investigates fairness-aware medical image segmentation for hip and knee radiographs using deep learning models evaluated in both centralized and Federated Learning (FL) settings. We introduce Curriculum Learning (CL) strategies and Progressive Loss (PL) functions to regulate sample difficulty during training. In addition, we propose two novel fairness-oriented federated learning algorithms, Federated Intersection over Union (FedIoU) and Federated Intersection over Union with Outlier Analysis (FedIoUoutlier). Experiments are conducted using multiple segmentation backbones and simulated multi-site data partitions derived from the Osteoarthritis Initiative dataset. Model performance is evaluated using Intersection over Union (IoU), IoU standard deviation, Skewed Error Ratio (SER), and Min-Max Disparity across race and gender subgroups. Statistical significance was verified using paired t-tests to compare per-sample IoU performance against baseline configurations. Across both hip and knee segmentation tasks, curriculum learning and progressive loss strategies consistently improved segmentation accuracy and reduced demographic performance disparities in centralized training. In federated settings, fairness-aware aggregation further enhanced performance. Notably, FedIoUoutlier combined with balanced curriculum learning and tiered progressive loss achieved the highest mean IoU while yielding the lowest SER and Min-Max Disparity, indicating improved fairness without sacrificing accuracy. In several configurations, federated models matched or exceeded the performance of optimized centralized models, with statistically significant improvements in per-sample IoU over baseline configurations. The results demonstrate that structured training strategies and fairness-aware federated aggregation can jointly improve accuracy, stability, and demographic fairness in medical image segmentation. By integrating curriculum learning, progressive loss, and novel FL algorithms, this work provides a practical pathway toward equitable and privacy-preserving AI systems for medical imaging.

Floods are the most frequently occurring natural disasters, significantly impacting the environment and society. As part of natural disaster mitigation, the impacts could be reduced through predictive techniques using deep learning for semantic segmentation of inundation images. Therefore, this research aims to evaluate the performance of deep learning architectures in segmenting inundation images using the Flood Segmentation dataset, which comprised 290 aerial images. The following segmentation architectures, U-Net, SegNet, and LinkNet, were compared using backbones such as MobileNet, ResNet, EfficientNet, and VGG, as well as optimizers including Adam, SGD, AdaDelta, and RMSProp. Performance was assessed using Intersection over Union (IoU) score, precision, F1-score, recall, and accuracy metrics. The results showed that U-Net achieved the highest performance with IoU, precision, F1-score, recall, and accuracy of 0.767, 0.862, 0.866, 0.876, and 0.899, respectively. Regarding the backbones, MobileNet excelled with IoU, precision, F1-score, recall, and accuracy of 0.764, 0.866, 0.865, 0.869, and 0.898, respectively. The Adam optimizer outperformed others, yielding IoU, precision, F1-score, recall, and accuracy of 0.712, 0.807, 0.824, 0.873, and 0.843. In conclusion, the combination of U-Net with MobileNet backbone and Adam optimizer was the most effective architecture for flood inundation image segmentation, offering a robust foundation for prediction systems.

Accurate intracranial artery segmentation from digital subtraction angiography (DSA) is critical for neurovascular diagnosis and intervention planning. Vascular extraction, which combines preprocessing methods and deep learning models, yields a high level of results, but limited preprocessing results constrain the improvement of results. We propose a texture-based contrast enhancement preprocessing framework integrated with the nnU-Net model to improve vessel segmentation in time-sequential DSA images. The method generates a combined feature mask by fusing local contrast, local entropy, and brightness threshold maps, which is then used as input for deep learning–based segmentation. Segmentation performance was evaluated using the DIAS dataset with various standard quantitative metrics. The proposed preprocessing significantly improved segmentation across all metrics compared to both the baseline and contrast-limited adaptive histogram equalization (CLAHE). Using nnU-Net, the method achieved a Dice Similarity Coefficient (DICE) of 0.83 ± 0.20 and an Intersection over Union (IoU) of 0.72 ± 0.14, outperforming CLAHE (DICE 0.79 ± 0.41, IoU 0.70 ± 0.23) and the baseline (DICE 0.65 ± 0.15, IoU 0.47 ± 0.20). Most notably, vessel connectivity (VC) dropped by over 65% relative to unprocessed images, indicating marked improvements in VC and topological accuracy. This study demonstrates that combining texture-based preprocessing with nnU-Net delivers robust, noise-tolerant, and clinically interpretable segmentation of intracranial arteries from DSA.