Systematic Clinical Evaluation of a Deep Learning Method for Medical Image Segmentation: Radiosurgery Application.

Aggregating multi-level feature representation plays a critical role in achieving robust volumetric medical image segmentation, which is important for the auxiliary diagnosis and treatment. Unlike the recent neural architecture search (NAS) methods that typically searched the optimal operators in each network layer, but missed a good strategy to search for feature aggregations, this paper proposes a novel NAS method for 3D medical image segmentation, named UXNet, which searches both the scale-wise feature aggregation strategies as well as the block-wise operators in the encoder-decoder network. UXNet has several appealing benefits. (1) It significantly improves flexibility of the classical UNet architecture, which only aggregates feature representations of encoder and decoder in equivalent resolution. (2) A continuous relaxation of UXNet is carefully designed, enabling its searching scheme performed in an efficient differentiable manner. (3) Extensive experiments demonstrate the effectiveness of UXNet compared with recent NAS methods for medical image segmentation. The architecture discovered by UXNet outperforms existing state-of-the-art models in terms of Dice on several public 3D medical image segmentation benchmarks, especially for the boundary locations and tiny tissues. The searching computational complexity of UXNet is cheap, enabling to search a network with best performance less than 1.5 days on two TitanXP GPUs.

Three-dimensional (3D) medical image segmentation plays a crucial role in medical care applications. Although various two-dimensional (2D) and 3D neural network models have been applied to 3D medical image segmentation and achieved impressive results, a trade-off remains between efficiency and accuracy. To address this issue, a novel mixture convolutional network (MixConvNet) is proposed, in which traditional 2D/3D convolutional blocks are replaced with novel MixConv blocks. In the MixConv block, 3D convolution is decomposed into a mixture of 2D convolutions from different views. Therefore, the MixConv block fully utilizes the advantages of 2D convolution and maintains the learning ability of 3D convolution. It acts as 3D convolutions and thus can process volumetric input directly and learn intra-slice features, which are absent in the traditional 2D convolutional block. By contrast, the proposed MixConv block only contains 2D convolutions; hence, it has significantly fewer trainable parameters and less computation budget than a block containing 3D convolutions. Furthermore, the proposed MixConvNet is pre-trained with small input patches and fine-tuned with large input patches to improve segmentation performance further. In experiments on the Decathlon Heart dataset and Sliver07 dataset, the proposed MixConvNet outperformed the state-of-the-art methods such as UNet3D, VNet, and nnUnet.

Medical image segmentation plays a vital role in diagnostic imaging and treatment planning, especially for volumetric modalities such as MRI and CT scans. However, training high-performance deep learning models for 3D medical image segmentation requires large, annotated datasets, which are often siloed due to strict privacy laws like HIPAA and GDPR. Federated Learning (FL) offers a decentralized solution that enables collaborative training without sharing raw patient data, but its effectiveness is hindered by challenges such as data heterogeneity, communication overhead, and model degradation on non-IID datasets. In this study, we propose FedSegNet, a novel federated learning framework tailored for 3D medical image segmentation. FedSegNet integrates a Transformer-based U-Net architecture for capturing both local and global spatial features and introduces an Adaptive Aggregation Mechanism (AAM) to dynamically weigh client updates based on data quality, performance, and divergence. To reduce communication costs, the framework employs gradient sparsification and quantization techniques. We evaluate FedSegNet on multi-institutional datasets including BraTS, LiTS, and ACDC, using metrics such as Dice Similarity Coefficient and Hausdorff Distance. Results show that FedSegNet achieves up to 7.2% improvement in segmentation accuracy and 38% reduction in communication cost compared to existing methods, demonstrating its potential for secure, decentralized medical AI applications.

In this paper we present a new algorithm for 3D medical image segmentation. The algorithm is fast, relatively simple to implement, and semi-automatic. It is based on minimizing a global energy defined from a learned non-parametric estimation of the statistics of the region to be segmented. Implementation details are discussed and source code is freely available as part of the 3D Slicer project. In addition, a new unified set of validation metrics is proposed. Results on artificial and real MRI images show that the algorithm performs well on large brain structures both in terms of accuracy and robustness to noise.KeywordsGround TruthActive ContourMedical Image AnalysisMedical Image SegmentationGeodesic Active ContourThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Gliomas, the most aggressive primary tumors in the central nervous system, are characterized by high morphological heterogeneity and diffusely infiltrating boundaries. Such complexity poses significant challenges for accurate segmentation in clinical practice. Although deep learning methods have shown promising results, they often struggle to achieve a satisfactory trade-off among precise boundary delineation, robust multi-scale feature representation, and computational efficiency, particularly when processing high-resolution three-dimensional (3D) magnetic resonance imaging (MRI) data. Therefore, the aim of this study is to develop a novel 3D segmentation framework that specifically addresses these challenges, thereby improving clinical utility in brain tumor analysis. To accomplish this, we propose a multi-level channel-spatial attention and light-weight scale-fusion network (MCSLF-Net), which integrates a multi-level channel-spatial attention mechanism (MCSAM) and a light-weight scale-fusion module. By strategically enhancing subtle boundary features while maintaining a compact network design, our approach seeks to achieve high accuracy in delineating complex glioma morphologies, reduce computational burden, and provide a more clinically feasible segmentation solution. We propose MCSLF-Net, a network integrating two key components: (I) MCSAM: by strategically inserting a 3D channel-spatial attention module at critical semantic layers, the network progressively emphasizes subtle, infiltrative edges and small, easily overlooked contours. This avoids reliance on an additional edge detection branch while enabling fine-grained localization in ambiguous transitional regions. (II) Light-weight scale fusion unit (LSFU): leveraging depth-wise separable convolutions combined with multi-scale atrous (dilated) convolutions, LSFU enhances computational efficiency and adapts to varying feature requirements at different network depths. In doing so, it effectively captures small infiltrative lesions as well as extensive tumor areas. By coupling these two modules, MCSLF-Net balances global contextual information with local fine-grained features, simultaneously reducing the computational burden typically associated with 3D medical image segmentation. Extensive experiments on the BraTS 2019, BraTS 2020, and BraTS 2021 datasets validated the effectiveness of our approach. On BraTS 2021, MCSLF-Net achieved a mean Dice similarity coefficient (DSC) of 0.8974 and a mean 95th percentile Hausdorff distance (HD95) of 2.52 mm. Notably, it excels in segmenting intricate transitional areas, including the enhancing tumor (ET) region and the tumor core (TC), thereby demonstrating superior boundary delineation and multi-scale feature fusion capabilities relative to existing methods. These findings underscore the clinical potential of deploying multi-level channel-spatial attention and light-weight multi-scale fusion strategies in high-precision 3D glioma segmentation. By striking an optimal balance among boundary accuracy, multi-scale feature capture, and computational efficiency, the proposed MCSLF-Net offers a practical framework for further advancements in automated brain tumor analysis and can be extended to a range of 3D medical image segmentation tasks.

Proposes a methodology for 3D medical image segmentation based on the integration of 3D deformable and Markov random field models. Our method makes use of Markov random field theory to build Gibbs Prior models for the 3D medical image with arbitrary initial parameters to estimate the organ boundary. Then we use a 3D deformable model to fit the estimated boundary under the influence of gradient information in the initial 3D image and the balloon force. The result of the deformable model fit is used to update the Gibbs Prior model parameters, such as the gradient threshold of a boundary. Based on the updated parameters we restart the Gibbs Prior models. By integrating these processes recursively we achieve an automated segmentation of the initial 3D images. Our segmentation solution greatly reduces the time for 3D segmentation process and is capable of getting out of local minim. Results of the method are presented for several examples, including some MRI images with significant amount of noise.

Knee cartilage and bone segmentation is critical for physicians to analyze and diagnose articular damage and knee osteoarthritis (OA). Deep learning (DL) methods for medical image segmentation have largely outperformed traditional methods, but they often need large amounts of annotated data for model training, which is very costly and time-consuming for medical experts, especially on 3D images. In this paper, we report a new knee cartilage and bone segmentation framework, KCB-Net, for 3D MR images based on sparse annotation. KCB-Net selects a small subset of slices from 3D images for annotation, and seeks to bridge the performance gap between sparse annotation and full annotation. Specifically, it first identifies a subset of the most effective and representative slices with an unsupervised scheme; it then trains an ensemble model using the annotated slices; next, it self-trains the model using 3D images containing pseudo-labels generated by the ensemble method and improved by a bi-directional hierarchical earth mover's distance (bi-HEMD) algorithm; finally, it fine-tunes the segmentation results using the primal-dual Internal Point Method (IPM). Experiments on four 3D MR knee joint datasets (the SKI10 dataset, OAI ZIB dataset, Iowa dataset, and iMorphics dataset) show that our new framework outperforms state-of-the-art methods on full annotation, and yields high quality results for small annotation ratios even as low as 10%.

One of the most common tasks in medical imaging is semantic segmentation. Achieving this segmentation automatically has been an active area of research, but the task has been proven very challenging due to the large variation of anatomy across different patients. However, recent advances in deep learning have made it possible to significantly improve the performance of image recognition and semantic segmentation methods in the field of computer vision. Due to the data driven approaches of hierarchical feature learning in deep learning frameworks, these advances can be translated to medical images without much difficulty. Several variations of deep convolutional neural networks have been successfully applied to medical images. Especially fully convolutional architectures have been proven efficient for segmentation of 3D medical images. In this article, we describe how to build a 3D fully convolutional network (FCN) that can process 3D images in order to produce automatic semantic segmentations. The model is trained and evaluated on a clinical computed tomography (CT) dataset and shows state-of-the-art performance in multi-organ segmentation.

Objective. Over the past years, convolutional neural networks based methods have dominated the field of medical image segmentation. But the main drawback of these methods is that they have difficulty representing long-range dependencies. Recently, the Transformer has demonstrated super performance in computer vision and has also been successfully applied to medical image segmentation because of the self-attention mechanism and long-range dependencies encoding on images. To the best of our knowledge, only a few works focus on cross-modalities of image segmentation using the Transformer. Hence, the main objective of this study was to design, propose and validate a deep learning method to extend the application of Transformer to multi-modality medical image segmentation. Approach. This paper proposes a novel automated multi-modal Transformer network termed AMTNet for 3D medical image segmentation. Especially, the network is a well-modeled U-shaped network architecture where many effective and significant changes have been made in the feature encoding, fusion, and decoding parts. The encoding part comprises 3D embedding, 3D multi-modal Transformer, and 3D Co-learn down-sampling blocks. Symmetrically, the 3D Transformer block, upsampling block, and 3D-expanding blocks are included in the decoding part. In addition, a Transformer-based adaptive channel interleaved Transformer feature fusion module is designed to fully fuse features of different modalities. Main results. We provide a comprehensive experimental analysis of the Prostate and BraTS2021 datasets. The results show that our method achieves an average DSC of 0.907 and 0.851 (0.734 for ET, 0.895 for TC, and 0.924 for WT) on these two datasets, respectively. These values show that AMTNet yielded significant improvements over the state-of-the-art segmentation networks. Significance. The proposed 3D segmentation network exploits complementary features of different modalities during the feature extraction process at multiple scales to increase the 3D feature representations and improve the segmentation efficiency. This powerful network enriches the research of the Transformer to multi-modal medical image segmentation.

In this paper, we introduce an adaptive model-based segmentation framework, in which edge and region information are integrated and used adaptively while a solid model deforms toward the object boundary. Our 3D segmentation method stems from Metamorphs deformable models. The main novelty of our work is in that, instead of performing segmentation in an entire 3D volume, we propose model-based segmentation in an adaptively changing subvolume of interest. The subvolume is determined based on appearance statistics of the evolving object model, and within the subvolume, more accurate and object-specific edge and region information can be obtained. This local and adaptive scheme for computing edges and object region information makes our segmentation solution more efficient and more robust to image noise, artifacts and intensity inhomogeneity. External forces for model deformation are derived in a variational framework that consists of both edge-based and region-based energy terms, taking into account the adaptively changing environment. We demonstrate the performance of our method through extensive experiments using cardiac MR and liver CT images.

3D convolution neural networks (CNN) have been proved very successful in parsing organs or tumours in 3D medical images, but it remains sophisticated and time-consuming to choose or design proper 3D networks given different task contexts. Recently, Neural Architecture Search (NAS) is proposed to solve this problem by searching for the best network architecture automatically. However, the inconsistency between search stage and deployment stage often exists in NAS algorithms due to memory constraints and large search space, which could become more serious when applying NAS to some memory and time-consuming tasks, such as 3D medical image segmentation. In this paper, we propose a coarse-to-fine neural architecture search (C2FNAS) to automatically search a 3D segmentation network from scratch without inconsistency on network size or input size. Specifically, we divide the search procedure into two stages: 1) the coarse stage, where we search the macro-level topology of the network, i.e. how each convolution module is connected to other modules; 2) the fine stage, where we search at micro-level for operations in each cell based on previous searched macro-level topology. The coarse-to-fine manner divides the search procedure into two consecutive stages and meanwhile resolves the inconsistency. We evaluate our method on 10 public datasets from Medical Segmentation Decalthon (MSD) challenge, and achieve state-of-the-art performance with the network searched using one dataset, which demonstrates the effectiveness and generalization of our searched models.

MA-SAM: Modality-agnostic SAM adaptation for 3D medical image segmentation

Semantic segmentation of 3D medical images is a challenging task due to the high variability of the shape and pattern of objects (such as organs or tumors). Given the recent success of deep learning in medical image segmentation, Neural Architecture Search (NAS) has been introduced to find high-performance 3D segmentation network architectures. However, because of the massive computational requirements of 3D data and the discrete optimization nature of architecture search, previous NAS methods require a long search time or necessary continuous relaxation, and commonly lead to sub-optimal network architectures. While one-shot NAS can potentially address these disadvantages, its application in the segmentation domain has not been well studied in the expansive multi-scale multi-path search space. To enable one-shot NAS for medical image segmentation, our method, named HyperSegNAS, introduces a HyperNet to assist super-net training by incorporating architecture topology information. Such a HyperNet can be removed once the super-net is trained and introduces no overhead during architecture search. We show that HyperSegNAS yields better performing and more intuitive architectures compared to the previous state-of-the-art (SOTA) segmentation networks; furthermore, it can quickly and accurately find good architecture candidates under different computing constraints. Our method is evaluated on public datasets from the Medical Segmentation Decathlon (MSD) challenge, and achieves SOTA performances.

Learning with limited annotations: A survey on deep semi-supervised learning for medical image segmentation

Three-dimensional (3D) medical image segmentation is used to segment the target (a lesion or an organ) in 3D medical images. Through this process, 3D target information is obtained; hence, this technology is an important auxiliary tool for medical diagnosis. Although some methods have proved to be successful for two-dimensional (2D) image segmentation, their direct use in the 3D case has been unsatisfactory. To obtain more precise tumor segmentation results from 3D MR images, in this paper, we propose a method known as the 3D shape-weighted level set method (3D-SLSM). The proposed method first converts the LSM, which is superior with respect to 2D image segmentation, into a 3D algorithm that is suitable for overall calculations in 3D image models, and which improves the efficiency and accuracy of calculations. A 3D shape-weighted value is then added for each 3D-SLSM iterative process according to the changes in volume. Besides increasing the convergence rate and eliminating background noise, this shape-weighted value also brings the segmented contour closer to the actual tumor margins. To perform a quantitative analysis of 3D-SLSM and to examine its feasibility in clinical applications, we have divided our experiments into computer-simulated sequence images and actual breast MRI cases. Subsequently, we simultaneously compared various existing 3D segmentation methods. The experimental results demonstrated that 3D-SLSM exhibited precise segmentation results for both types of experimental images. In addition, 3D-SLSM showed better results for quantitative data compared with existing 3D segmentation methods.