Semi-TMS: an efficient regularization-oriented triple-teacher semi-supervised medical image segmentation model

Deep learning has achieved great success in the field of computer vision, and the precision in image classification and image detection has surpassed humans. Therefore, this paper combines deep learning and medical image segmentation, focusing on how to improve the accuracy and speed of segmentation algorithm of medical exercise rehabilitation image. Aiming at the shortcomings of traditional medical image recognition methods, a medical exercise rehabilitation image segmentation algorithm based on hierarchical features of convolutional neural networks is proposed, this paper calls it as hierarchical features of convolutional neural networks (HFCNN). The algorithm firstly samples the convolution output of multiple layers in the convolutional neural network to a unified scale and combines them to construct a hierarchical feature. This hierarchical feature combines the structural information of objects contained in the shallow layer of the network with the semantic information of objects contained in the deep layers of the network, so it has a strong ability to express. Secondly, the image can be segmented into multiple super pixels by the super pixel segmentation algorithm. The classifier is trained using the hierarchical features of the super pixel, and then the classification result is mapped back to the pixel. Finally, a fully connected conditional random field algorithm including one-potential potential energy and paired potential energy is constructed. The corresponding energy function is used to smooth the classification result of the pixel, and the regional consistency and continuity of the pixel mark are improved. Compared with many classical convolutional neural network algorithms, this algorithm not only accelerates the network convergence speed, shortens the training time, but also significantly improves the accuracy of segmentation algorithm of medical exercise rehabilitation image, showing good practical value.

Bridging 2D and 3D segmentation networks for computation-efficient volumetric medical image segmentation: An empirical study of 2.5D solutions.

The article examines the application of semi-supervised learning and computer vision methods for segmentation of spine MR images and diagnosis of deformation of intervertebral discs. Existing neural network architectures for spine MR image segmentation and semi-supervised learning methods used in medical image segmentation are reviewed. A system for diagnosis deformation of intervertebral discs is proposed, which consists of two modules: a segmentation module and a diagnostic module. The implementation of the proposed system using two convolutional neural networks is presented: U-Net for segmentation of spine MR images and ResNet for classification of the degree of deformation of each intervertebral disc based on the Pfirrmann classification of degenerative changes. The software implementation of the medical diagnosis system was developed in the Python programming language using the PyTorch library. Neural networks were trained on an open dataset of spine MR images using a variant of the Mean Teacher semi-supervised learning method. As a result of the verification, it was found that the system is capable of performing segmentation with high accuracy. It was found that the exact prediction of the grade of degenerative changes according to Pfirrmann remains a difficult task, but the introduction of another classification made it possible to increase the accuracy of diagnosis of deformation of intervertebral discs. The proposed medical system involves the addition of new diagnostic modules, which makes it possible to use it for the comprehensive analysis of various spine diseases

Segment anything model (SAM) has recently demonstrated tremendous potential in the artificial intelligence-based medical image segmentation. However, its outstanding performance is always accompanied by surging computation costs due to the extremely long patch sequences and their quadratic complexity in vision transformer, which hamper the development of real-time engineeringapplications. This work aims to accelerate SAM for medical image segmentation with better segmentation quality and less memoryusage. Instead of upscaling training images to larger resolutions to match the pre-trained SAM's input size, which may introduce image distortions, we propose a convolutional neural network (CNN)-assisted tuning strategy. This approach enables SAM to process smaller inputs directly, significantly reducing the number of patches and memory consumption. Moreover, based on the observation that the less-informative patches in medical images tend to be well-classified earlier than others, revealing the massive computation redundancy in the SAM, a token pausing strategy is presented to identify these easy-to-predicted patches in the feed-forward process and then pause their computation. Finally, we combine the CNN-assisted tuning strategy with the token pausing strategy, meets various run-time requirements by simply adjusting the pause parameters, and avoids repeatedtraining. Extensive experiments on two medical image segmentation benchmarks, namely Synapse and ACDC, demonstrate that our model can run 12 faster than existing SAM-based solutions, and meanwhile achieves a better segmentationperformance. This study identifies two practical limitations when applying SAM to medical imaging: resizing low-resolution inputs to match the pre-trained scale and uniformly processing all patches, which lead to redundant computation. Therefore, we present an efficient strategy that integrates adapter-based tuning with token pausing to enhance throughput while preserving segmentation performance. Relevant codes and models of this paper will be available at https://github.com/wuwen1994/FasterSAM.

Medical image segmentation is crucial for improving healthcare outcomes. Convolutional neural networks (CNNs) have been widely applied in medical image analysis; however, their inherent inductive biases limit their ability to capture global contextual information. Vision transformer (ViT) architectures address this limitation by leveraging attention mechanisms to model global relationships; however, they typically require large-scale datasets for effective training, which is challenging in the field of medical imaging due to limited data availability. This study aimed to integrate the advantages of CNN and ViT architectures to improve segmentation performance on small-scale medical image datasets. In this study, we established a U-shaped network architecture based on a Transformer-assisted convolutional neural network (TAC-UNet). The TAC-UNet is primarily composed of a hybrid structure integrating CNN and Transformer components. Specifically, the hybrid architecture follows a dual-path design in which the Transformer branch continuously conveys global contextual information to the CNN backbone. This allows the CNN backbone to enhance its global perception while building on the local features it extracts, thereby improving its ability to comprehend complex image structures. A channel cross-attention (CCA) module is also incorporated as a bridge between the encoder and decoder to better reconcile the semantic discrepancies between them. Detailed experiments on three public datasets were conducted. Specifically, our model was trained on 30 images from the Multi-organ Nucleus Segmentation (MoNuSeg) training dataset, 85 images from the Gland Segmentation (GlaS) training dataset, and 551 images from the Computer Vision Center Colorectal Cancer-Clinic Database (CVC-ClinicDB) dataset. We evaluated the performance of our model on the corresponding test sets. Our TAC-UNet achieved the best Dice scores (80.36%, 90.70%, and 91.81% on the MoNuSeg, GlaS, and CVC-ClinicDB datasets, respectively) of all the models. Compared to other CNN-based, Transformer-based, and hybrid methods, the TAC-UNet demonstrated significantly superior segmentation performance. Our TAC-UNet model showed advanced segmentation performance on small-scale medical image datasets. The detailed experimental results showed the effectiveness of the method. Our model's code is available at: https://github.com/hejlhello/TAC-UNet.

Due to the lack of quality annotation in medical imaging community, semi-supervised learning methods are highly valued in image semantic segmentation tasks. In this paper, an advanced consistency-aware pseudo-label-based self-ensembling approach is presented to fully utilize the power of Vision Transformer (ViT) and Convolutional Neural Network (CNN) in semi-supervised learning. Our proposed framework consists of a feature-learning module which is enhanced by ViT and CNN mutually, and a guidance module which is robust for consistency-aware purposes. The pseudo labels are inferred and utilized recurrently and separately by views of CNN and ViT in the feature-learning module to expand the data set and are beneficial to each other. Meanwhile, a perturbation scheme is designed for the feature-learning module, and averaging network weight is utilized to develop the guidance module. By doing so, the framework combines the feature-learning strength of CNN and ViT, strengthens the performance via dual-view co-training, and enables consistency-aware supervision in a semi-supervised manner. A topological exploration of all alternative supervision modes with CNN and ViT are detailed validated, demonstrating the most promising performance and specific setting of our method on semi-supervised medical image segmentation tasks. Experimental results show that the proposed method achieves state-of-the-art performance on a public benchmark data set with a variety of metrics. The code is publicly available (https://github.com/ziyangwang007/CV-SSL-MIS).

Medical image segmentation is an essential process in disease analysis and diagnosis, in which medical images are divided into specific areas of anatomic or pathological structures. However, this process has challenges, such as the quality, variability, and complexity of medical images. Thus, we propose a medical image segmentation method using the Convolutional Neural Network (CNN) and the Histogram Equalization (HE) image preprocessing. We evaluated medical image segmentation using 2 secondary datasets, namely Lung CT-Scan and Chest X-Rays, with 267 and 3616 images, respectively. The HE method uses an averaging of the optimal cumulative distribution function (CDF) values with a range of 0 to 39. The experimental results show that adding the HE method can increase the accuracy of CNN on the Lung CT-Scan and Chest X-rays data sets, respectively, by 0.73 percentage points from 91.28% to 92.08 % and 1.23 percentage points from 95.56% to 96.85% for the training process. The results of the testing process (validation) have an increase in accuracy of 2.82 percentage points from 89.77% to 92.59% and 0.91 percentage points from 96.06% to 96.97%. Implementing the HE method increases accuracy in CNN medical image segmentation based on the value of dice similarity coefficient (DSC) and structural similarity index measurement (SSIM) close to 0.95.

Deep learning models, e.g. supervised Encoder-Decoder style networks, exhibit promising performance in medical image segmentation, but come with a high labelling cost. We propose TriSegNet, a semi-supervised semantic segmentation framework. It uses triple-view feature learning on a limited amount of labelled data and a large amount of unlabeled data. The triple-view architecture consists of three pixel-level classifiers and a low-level shared-weight learning module. The model is first initialized with labelled data. Label processing, including data perturbation, confidence label voting and unconfident label detection for annotation, enables the model to train on labelled and unlabeled data simultaneously. The confidence of each model gets improved through the other two views of the feature learning. This process is repeated until each model reaches the same confidence level as its counterparts. This strategy enables triple-view learning of generic medical image datasets. Bespoke overlap-based and boundary-based loss functions are tailored to the different stages of the training. The segmentation results are evaluated on four publicly available benchmark datasets including Ultrasound, CT, MRI, and Histology images. Repeated experiments demonstrate the effectiveness of the proposed network compared against other semi-supervised algorithms, across a large set of evaluation measures.

Semi-supervised learning methods, compared to fully supervised learning, offer significant potential to alleviate the burden of manual annotations on clinicians. By leveraging unlabeled data, these methods can aid in the development of medical image segmentation systems for improving efficiency. Boundary segmentation is crucial in medical image analysis. However, accurate segmentation of boundary regions is under-explored in existing methods since boundary pixels constitute only a small fraction of the overall image, resulting in suboptimal segmentation performance for boundary regions. In this paper, we introduce boundary-guided contrastive learning for semi-supervised medical image segmentation (BoCLIS). Specifically, we first propose conservative-to-radical teacher networks with an uncertainty-weighted aggregation strategy to generate higher quality pseudo-labels, enabling more efficient utilization of unlabeled data. To further improve the performance of segmentation in boundary regions, we propose a boundary-guided patch sampling strategy to guide the framework in learning discriminative representations for these regions. Lastly, the patch-based contrastive learning is proposed to simultaneously compute the (dis)similarities of the discriminative representations across intra- and inter-images. Extensive experiments on three public datasets show that our method consistently outperforms existing methods, especially in the boundary region, with DSC improvements of 20.47%, 16.75%, and 17.18%, respectively. A comprehensive analysis is further performed to demonstrate the effectiveness of our approach. Our code is released publicly at https://github.com/youngyzzZ/BoCLIS.

ABSTRACTSemantic segmentation is a key task in computer vision, with medical image segmentation as a prominent downstream application that has seen significant advancements in recent years. However, the challenge of requiring extensive annotations in medical image segmentation remains exceedingly difficult. In addressing this issue, semi‐supervised semantic segmentation has emerged as a new approach to mitigate annotation burdens. Nonetheless, existing methods in semi‐supervised medical image segmentation still face challenges in fully exploiting unlabeled data and efficiently integrating labeled and unlabeled data. Therefore, this paper proposes a novel network model—feature similarity multilevel information fusion network (FSMIFNet). First, the feature similarity module is introduced to harness deep feature similarity among unlabeled images, predicting true label constraints and guiding segmentation features with deep feature relationships. This approach fully exploits deep feature information from unlabeled data. Second, the multilevel information fusion framework integrates labeled and unlabeled data to enhance segmentation quality in unlabeled images, ensuring consistency between original and feature maps for comprehensive optimization of detail and global information. In the ACDC dataset, our method achieves an mDice of 0.684 with 5% labeled data, 0.873 with 10%, 0.884 with 20%, and 0.897 with 50%. Experimental results demonstrate the effectiveness of FSMIFNet in semi‐supervised semantic segmentation of medical images, outperforming existing methods on public benchmark datasets. The code and models are available at https://github.com/liujiayin12/FSMIFNet.git.

Convolutional Neural Networks (CNNs) have achieved remarkable segmentation accuracy in medical image segmentation tasks. However, the Vision Transformer (ViT) model, with its capability of extracting global information, offers a significant advantage in contextual information compared to the limited receptive field of convolutional kernels in CNNs. Despite this, ViT models struggle to fully detect and extract high-frequency signals, such as textures and boundaries, in medical images. These high-frequency features are essential in medical imaging, as targets like tumors and pathological organs exhibit significant differences in texture and boundaries across different stages. Additionally, the high resolution of medical images leads to computational complexity in the self-attention mechanism of ViTs. To address these limitations, we propose a medical image segmentation network framework based on frequency domain decomposition using a Laplacian pyramid. This approach selectively computes attention features for high-frequency signals in the original image to enhance spatial structural information effectively. During attention feature computation, we introduce Singular Value Decomposition (SVD) to extract an effective representation matrix from the original image, which is then applied in the attention computation process for linear projection. This method reduces computational complexity while preserving essential features. We demonstrated the segmentation validity and superiority of our model on the Abdominal Multi-Organ Segmentation dataset and the Dermatological Disease dataset, and on the Synapse dataset our model achieved a score of 82.68 on the Dice metrics and 17.23 mm on the HD metrics. Experimental results indicate that our model consistently exhibits segmentation effectiveness and improved accuracy across various datasets.

Explainable multi-module semantic guided attention based network for medical image segmentation

Medical image segmentation is crucial for clinical diagnosis and treatment planning, but it requires models to capture fine local details and understand global contextual relationships. Although traditional convolutional neural networks (CNNs) perform well in extracting local features, their inherent local receptive fields limit their ability to model long-range dependencies. On the other hand, although Transformer models excel at capturing global information through self attention mechanisms, they may lose low-level spatial details that are crucial for accurate segmentation when processing images. To overcome these challenges, this study proposes a novel parallel branch architecture called Trans CNN Fusion Network (TCF Net).This architecture synergistically combines a CNN encoder and a Swin Transformer encoder to extract both local texture features and global semantic information in parallel. The core innovation we designed is a Multi Scale Attention Fusion (MSAF) module that can intelligently and adaptively merge feature streams from two parallel branches, significantly outperforming simple feature concatenation methods. In addition, to address the common issue of class imbalance in medical images, we employed a hybrid Dice Focal loss function for model optimization. The experimental results on the publicly available BraTS 2021 brain tumor segmentation dataset show that TCF Net outperforms existing baseline models in all evaluation metrics, with an average Dice coefficient improved by 3.5% compared to TransUNet, demonstrating its outstanding performance and effectiveness in segmenting complex tumor subregions. This work provides a powerful new paradigm for high-precision automated medical image analysis.

Medical image segmentation is a critical application of computer vision in the analysis of medical images. Its primary objective is to isolate regions of interest in medical images from the background, thereby assisting clinicians in accurately identifying lesions, their sizes, locations, and their relationships with surrounding tissues. However, compared to natural images, medical images present unique challenges, such as low resolution, poor contrast, inconsistency, and scattered target regions. Furthermore, the accuracy and stability of segmentation results are subject to more stringent requirements. In recent years, with the widespread application of Convolutional Neural Networks (CNNs) in computer vision, deep learning-based methods for medical image segmentation have become a focal point of research. This paper categorizes, reviews, and summarizes the current representative methods and research status in the field of medical image segmentation. A comparative analysis of relevant experiments is presented, along with an introduction to commonly used public datasets, performance evaluation metrics, and loss functions in medical image segmentation. Finally, potential future research directions and development trends in this field are predicted and analyzed.

In traditional machine learning approaches to classification, one uses only a labelled set to train the classifier. Labelled instances however are often difficult, expensive, or time consuming to obtain, as they require the efforts of experienced human annotators. Meanwhile unlabeled data may be relatively easy to collect, but there has been few ways to use them. Semi-supervised learning addresses this problem by using large amount of unlabeled data, together with the labelled data, to build better classifiers. Because semi-supervised learning requires less human effort and gives higher accuracy, it is of great interest both in theory and in practice. The paper discusses various important approaches to semi-supervised learning such as self-training, co-training(CO), expectation maximization (EM), CO-EM, Then how graph-based methods are useful is explained. All semi-supervised learning methods are classified into generative and discriminative methods. But experimental results show that the hybrid algorithm gives better classification accuracy.