How to build the best medical image segmentation algorithm using foundation models: a comprehensive empirical study with Segment Anything Model

Deep learning has achieved great success in the field of computer vision, and the precision in image classification and image detection has surpassed humans. Therefore, this paper combines deep learning and medical image segmentation, focusing on how to improve the accuracy and speed of segmentation algorithm of medical exercise rehabilitation image. Aiming at the shortcomings of traditional medical image recognition methods, a medical exercise rehabilitation image segmentation algorithm based on hierarchical features of convolutional neural networks is proposed, this paper calls it as hierarchical features of convolutional neural networks (HFCNN). The algorithm firstly samples the convolution output of multiple layers in the convolutional neural network to a unified scale and combines them to construct a hierarchical feature. This hierarchical feature combines the structural information of objects contained in the shallow layer of the network with the semantic information of objects contained in the deep layers of the network, so it has a strong ability to express. Secondly, the image can be segmented into multiple super pixels by the super pixel segmentation algorithm. The classifier is trained using the hierarchical features of the super pixel, and then the classification result is mapped back to the pixel. Finally, a fully connected conditional random field algorithm including one-potential potential energy and paired potential energy is constructed. The corresponding energy function is used to smooth the classification result of the pixel, and the regional consistency and continuity of the pixel mark are improved. Compared with many classical convolutional neural network algorithms, this algorithm not only accelerates the network convergence speed, shortens the training time, but also significantly improves the accuracy of segmentation algorithm of medical exercise rehabilitation image, showing good practical value.

Segment Any Tissue: One-shot reference guided training-free automatic point prompting for medical image segmentation.

Image segmentation is an important process in the field of medical imaging.Recently, much work has been reported in medical image segmentation. Among these techniques, finite Gaussian mixture models are considered to be more recent and accurate. However, in this approach, a number of segments that an image can be divided are taken through apriori and if these segments are not initiated properly it leads to misclassification. Hence, to overcome this disadvantage, we proposed an algorithm for Medical Image Segmentation using Hierarchical Clustering and Skew Gaussian Mixture. The experimentation is done with two different brain images and the results obtained are evaluated using both Image Quality metrics and Segmentation Quality Metrics.

The discovery of X-ray in 1895 initiated the era of medical imaging diagnostics. Since then, medical imaging systems have realized unprecedented advancements. These systems have also turned out to be invaluable tools in the practice of diagnostic medicine. However, despite the significant development in medical imaging technologies, processing medical images still pose a substantial challenge especially when it comes to image segmentation. That problem is gradually being alleviated by the implementation of digital medical image processing, especially in the diagnosis and treatment of brain tumors. But the capability of most of the contemporary image delineating algorithms remains limited. Therefore, there is a need to come up with the new medical image segmentation programs to fully utilize the power of digital image processing. In light that, this article reviews some of the contemporary algorithmic protocols for brain tumor delineation systems and how effective they are.

In recent years, generative adversarial networks (GANs) have gained tremendous popularity for various imaging related tasks such as artificial image generation to support AI training. GANs are especially useful for medical imaging-related tasks where training datasets are usually limited in size and heavily imbalanced against the diseased class. We present a systematic review, following the PRISMA guidelines, of recent GAN architectures used for medical image analysis to help the readers in making an informed decision before employing GANs in developing medical image classification and segmentation models. We have extracted 54 papers that highlight the capabilities and application of GANs in medical imaging from January 2015 to August 2020 and inclusion criteria for meta-analysis. Our results show four main architectures of GAN that are used for segmentation or classification in medical imaging. We provide a comprehensive overview of recent trends in the application of GANs in clinical diagnosis through medical image segmentation and classification and ultimately share experiences for task-based GAN implementations.

At present, the incidence of emergencies in obstetric care environment is gradually increasing, and different obstetric wards often have a variety of situations. Therefore, it can provide great help in clinical medicine to give early warning and plan coping plans according to different situations. This paper studied an obstetrics central surveillance system based on a medical image segmentation algorithm. Images obtained by central obstetrics monitoring are segmented, magnified in detail, and image features are extracted, collated, and trained. The normal distribution rule is used to classify the features, which are included in the feature library of the obstetric central monitoring system. In the gray space of the medical image, the statistical distribution of gray features of the medical image is described by the mixture model of Rayleigh distribution and Gaussian distribution. In the gray space of the medical image, Taylor series expansion is used to describe the linear geometric structure of medicine. The eigenvalues of Hessian matrix are introduced to obtain high-order multiscale features of medicine. The multiscale feature energy function is introduced into Markov random energy objective function to realize medical image segmentation. Compared with other segmentation algorithms, the accuracy and sensitivity of the proposed algorithm are 87.98% and 86.58%, respectively, which can clearly segment small medical features.

The availability of huge annotated datasets and deep learning have led to astounding Natural Language Processing (NLP) progress. However, there is a plethora of low-resource languages, lack the labelled corpora to train the models in the supervision. This kind of digital divide does not allow fair NLP development across language groups. Self-supervised learning (SSL) is a new paradigm in recent years that leverages large amounts of untagged text to learn powerful representations with little or no manual annotation. This paper is a review of self-supervised deep learning models in low-resource NLP tasks. It begins with the definition of the principles of the SSL and the difference between this approach and supervised and unsupervised approaches. We describe why methods such as masked language modeling, contrastive learning, and autoregressive modeling underlie modern pre-trained transformers such as BERT, GPT, and mBERT. A particular interest is paid to multilingual and cross lingual SSL schemes which allow knowledge transfer between high-resource and low resource languages. Low-resource tasks, including machine translation, sentiment analysis, speech-to-text, and information retrieval are reviewed. As per benchmark studies, even in small labelled samples, the SSL models can achieve big gains in accuracy and generalization. Computation cost, representational bias and morphologically rich and under-documented language evaluation, are however, problematic. The researchers observe that the study suffers certain limitations, such as over-reliance on high-resource pretraining information, inequity between linguistic groups, and the difficulty of deploying large-scale SSL models in resource-constrained circumstances. The future directions include lightweight multilingual models, federation of learning in NLP and symbolic linguistic knowledge and combination with SSL. Self-supervised deep learning bridges an essential gap between the high-resource and low-resource languages, and would be a highly valuable step toward inclusive, global NLP innovation.

The rapid advancement of prompt-based models in natural language processing and image generation has revolutionized the field of image segmentation. The introduction of the Segment Anything Model (SAM) has further invigorated this domain with its unprecedented versatility. However, its applicability to medical image segmentation remains uncertain due to significant disparities between natural and medical images, which demand careful consideration. This study comprehensively analyzes recent efforts to adapt SAM for medical image segmentation, including empirical benchmarking and methodological refinements aimed at bridging the gap between SAM's capabilities and the unique challenges of medical imaging. Furthermore, we explore future directions for SAM in this field. While direct application of SAM to complex, multimodal, and multi-target medical datasets may not yet yield optimal results, insights from these efforts provide crucial guidance for developing foundational models tailored to the intricacies of medical image analysis. Despite existing challenges, SAM holds considerable potential to demonstrate its unique advantages and robust capabilities in medical image segmentation in the near future.

DSEU-net: A novel deep supervision SEU-net for medical ultrasound image segmentation

The medical image is often noisy, which makes it difficult to extract the image features from the medical image segmentation model. Because the noise is often generated randomly, it is difficult to use supervised information for denoising. In this paper, we focus on this challenging problem and propose an IM-Net algorithm for medical image segmentation based on mutual information maximization. The IM-Net can remove the noise and therefore improve the quality of the extracted feature by maximizing the mutual information between the extracted feature and the input image. IM-Net uses the Binary Cross Entropy with Logits Estimation to approach the true value of mutual information and uses a bilinear interpolation function as a discriminator to maximize the mutual information estimator. Extensive experiments are conducted and the IM-Net is compared with different methods to demonstrate the effectiveness of our model. Experimental results show that the training efficiency and segmentation precision are greatly improved.

Accurate image segmentation is crucial in medical imaging for quantifying diseases, assessing prognosis, and evaluating treatment outcomes. However, existing methods often fall short in integrating global and local features in a meaningful way, failing to give sufficient attention to abnormal regions and boundary details in medical images. These limitations hinder the effectiveness of segmentation techniques in clinical settings. To address these issues, we propose a novel deep learning-based approach, MIPC-Net, designed for precise boundary segmentation in medical images. Our approach, inspired by radiologists' working patterns, introduces two distinct modules: 1. Mutual Inclusion of Position and Channel Attention (MIPC) Module: To improve boundary segmentation precision, we present the MIPC module. This module enhances the focus on channel information while extracting position features and vice versa, effectively enhancing the segmentation of boundaries in medical images. 2. Skip-Residue Module: To optimize the restoration of medical images, we introduce Skip-Residue, a global residual connection. This module improves the integration of the encoder and decoder by filtering out irrelevant information and recovering the most crucial information lost during the feature extraction process. We evaluate the performance of MIPC-Net on three publicly accessible datasets: Synapse, ISIC2018-Task, and Segpc. The evaluation uses metrics such as the Dice coefficient (DSC) and Hausdorff Distance (HD). Our ablation study confirms that each module contributes to the overall improvement of segmentation quality. Notably, with the integration of both modules, our model outperforms state-of-the-art methods across all metrics. Specifically, MIPC-Net achieves a 2.23 mm reduction in Hausdorff Distance on the Synapse dataset, highlighting the model's enhanced capability for precise image boundary segmentation. The introduction of the novel MIPC and Skip-Residue modules significantly improves feature extraction accuracy, leading to better boundary recognition in medical image segmentation tasks. Our approach demonstrates substantial improvements over existing methods, as evidenced by the results on benchmark datasets.

MA-SAM: Modality-agnostic SAM adaptation for 3D medical image segmentation

Medical image segmentation, the process of precisely delineating regions of interest (e.g. organs, lesions, cells) within medical images, is a pivotal technique in medical image analysis. It finds widespread application in computer‐aided diagnosis, surgical planning, radiation therapy, and pathological analysis, thus playing a crucial role in enabling precision medicine and enhancing the quality of clinical care. Traditional medical image segmentation methods often rely on hand‐crafted features and rule‐based approaches, which struggle to handle the inherent complexity and variability of medical imagery, leading to limitations in segmentation accuracy and robustness. Recently, deep learning methodologies, driven by their powerful capabilities in automatic feature learning and non‐linear modelling, have overcome the limitations of traditional methods and achieved significant advancements in the field of medical image segmentation. This review provides a comprehensive overview and summary of recent progress in deep learning‐based medical image segmentation, with a particular focus on fully supervised learning paradigms leveraging convolutional neural networks, transformers, and the segment anything model. We delve into the underlying principles, network architectures, advantages, and limitations of these approaches. Furthermore, we systematically compare their performance across diverse imaging modalities, anatomical structures, and pathological targets. We also present a curated compilation of commonly used datasets, evaluation metrics, and loss functions relevant to medical image segmentation. Finally, we discuss future research directions and potential challenges, offering insights into the evolving landscape of this critical field.

Computer vision and image processing find applications in various fields of medical imaging. Image segmentation and image fusion are important examples among them. Image fusion can help to detect and localize tumors by combining the information from different modalities, whereas image segmentation is useful in image detection. Combining these technologies would provide a robust way to detect abnormalities or tumors present in a particular organ. In this chapter, traditional image segmentation algorithms based on label fusion strategy such as local, majority, global weighted, and semi-local weighted voting are discussed. In addition, new medical image segmentation algorithms are also introduced.

Convolutional neural networks (CNNs), as a typical deep learning technique, have been widely used in image segmentation, but they often require a large amount of annotated data. However, the number of available pixel-wise labeled medical images is extremely small, and this prevents the application of CNNs in many medical image segmentation tasks. We proposed a self-paced learning with diversity SPLD) framework to boost the performances of medical image segmentation models with a limited amount of annotated data. Self-paced learning (SPL) is a learning regime that selects training samples in order from the easiest to the most difficult for model training. In addition, we took the diversity of the data into consideration. The proposed self-paced learning with diversity by query-by-committee (SPLD-QBC) algorithm dynamically and diversely selects the appropriate training data to boost the performance of an image segmentation model. SPLD-QBC incorporates the query-by-committee (QBC) technique for data selection and affinity propagation for optimizing the data diversity. By dynamically selecting the optimal sequence of training samples from different probability distributions, the segmentation models achieved improved performances. To verify the effectiveness of the proposed SPLD-QBC framework, we conducted experiments on three medical image segmentation tasks with five different datasets. The experimental results indicated that the proposed SPLD-QBC algorithm significantly improved upon the segmentation performances of the baseline models and resulted in a higher Dice score, surface distance and mean intersection over union (mIoU). The proposed SPLD significantly boosts the segmentation performances of models and is easily embedded into CNN-based image segmentation models.