TT-Net: Tensorized Transformer Network for 3D medical image segmentation.

When using convolutional neural networks (CNNs) for segmentation of organs and lesions in medical images, the conventional approach is to work with inputs and outputs either as single slice [two-dimensional (2D)] or whole volumes [three-dimensional (3D)]. One common alternative, in this study denoted as pseudo-3D, is to use a stack of adjacent slices as input and produce a prediction for at least the central slice. This approach gives the network the possibility to capture 3D spatial information, with only a minor additional computational cost. In this study, we systematically evaluate the segmentation performance and computational costs of this pseudo-3D approach as a function of the number of input slices, and compare the results to conventional end-to-end 2D and 3D CNNs, and to triplanar orthogonal 2D CNNs. The standard pseudo-3D method regards the neighboring slices as multiple input image channels. We additionally design and evaluate a novel, simple approach where the input stack is a volumetric input that is repeatably convolved in 3D to obtain a 2D feature map. This 2D map is in turn fed into a standard 2D network. We conducted experiments using two different CNN backbone architectures and on eight diverse data sets covering different anatomical regions, imaging modalities, and segmentation tasks. We found that while both pseudo-3D methods can process a large number of slices at once and still be computationally much more efficient than fully 3D CNNs, a significant improvement over a regular 2D CNN was only observed with two of the eight data sets. triplanar networks had the poorest performance of all the evaluated models. An analysis of the structural properties of the segmentation masks revealed no relations to the segmentation performance with respect to the number of input slices. A post hoc rank sum test which combined all metrics and data sets yielded that only our newly proposed pseudo-3D method with an input size of 13 slices outperformed almost all methods. In the general case, multislice inputs appear not to improve segmentation results over using 2D or 3D CNNs. For the particular case of 13 input slices, the proposed novel pseudo-3D method does appear to have a slight advantage across all data sets compared to all other methods evaluated in this work.

Accurate segmentation is a basic and crucial step for medical image processing and analysis. In the last few years, U-Net, and its variants, have become widely adopted models in medical image segmentation tasks. However, the multiple training parameters of these models determines high computation complexity, which is impractical for further applications. In this paper, by introducing depthwise separable convolution and attention mechanism into U-shaped architecture, we propose a novel lightweight neural network (DSCA-Net) for medical image segmentation. Three attention modules are created to improve its segmentation performance. Firstly, Pooling Attention (PA) module is utilized to reduce the loss of consecutive down-sampling operations. Secondly, for capturing critical context information, based on attention mechanism and convolution operation, we propose Context Attention (CA) module instead of concatenation operations. Finally, Multiscale Edge Attention (MEA) module is used to emphasize multi-level representative scale edge features for final prediction. The number of parameters in our network is 2.2 M, which is 71.6% less than U-Net. Experiment results across four public datasets show the potential and the dice coefficients are improved by 5.49% for ISIC 2018, 4.28% for thyroid, 1.61% for lung and 9.31% for nuclei compared with U-Net.

Reliable automatic target segmentation in Synthetic Aperture Radar (SAR) imagery has played an important role in the SAR fields. Different from the traditional methods, Spectral Residual (SR) and CFAR detector, with the recent advancements in machine learning theory, new methods based on machine learning show better performance in SAR target segmentation. In this paper, we proposed a deep deformable residual learning network for target segmentation, which attempts to preserve the precise contour of the target. The deformable convolutional layers and residual learning block are applied in this technique, which could extract and preserve the geometric information of the target as much as possible. The proposed method is evaluated based on the Moving and Stationary Target Acquisition and Recognition (MSTAR) dataset, and the experimental results have shown the superiority of the proposed network for the precise segmentation of the target.

MC-DC: An MLP-CNN Based Dual-path Complementary Network for Medical Image Segmentation

In the context of automated analysis of eye fundus images, it is an important common fallacy that prior works achieve very high scores in segmentation of lesions, and that fallacy is fueled by some reviews reporting very high scores, and perhaps some confusion with terms. A simple analysis of the detail of the few prior works that really do segmentation reveals scores between 7% and 70% in sensitivity for 1 FPI. That is clearly sub-par with medical doctors trained to detect signs of Diabetic Retinopathy, since they can distinguish well the contours of lesions in Eye Fundus Images (EFI). Still, a full segmentation of lesions could be an important step for both visualization and further automated analysis using rigorous quantification or areas and numbers of lesions to better diagnose. I discuss what prior work really does, using evidence-based analysis, and confront with segmentation networks, comparing on the terms used by prior work to show that the best performing segmentation network outperforms those prior works. I also compare architectures to understand how the network architecture influences the results. I conclude that, with the correct architecture and tuning, the semantic segmentation network improves up to 20 percentage points over prior work in the real task of segmentation of lesions. I also conclude that the network architecture and optimizations are important factors and that there are still important limitations in current work.

Bridging 2D and 3D segmentation networks for computation-efficient volumetric medical image segmentation: An empirical study of 2.5D solutions.

Accurate segmentation of medical images is crucial for disease diagnosis and understanding disease changes. Deep learning methods, utilizing encoder-decoder structures, have demonstrated cutting-edge performance in various medical image segmentation tasks. However, the pooling operation in the encoding stage results in feature loss, which makes the network lack the ability to fuse multi-scale information at different levels, hinders its effective perception of multi-scale information, and leads to poor segmentation performance. Drawing inspiration from the U-shaped network, this study introduces a multi-branch feature hybrid attention and adaptive receptive field network (MFHARFNet) for medical image segmentation. Building upon the encoder-decoder framework, we initially devise a multi-branch feature hybrid attention module (MFHAM) to seamlessly integrate feature maps of varying scales, capturing both fine-grained features and coarse-grained semantics across the entire scale. Furthermore, we redesign the skip connection to amalgamate feature information from different branches in the encoder stage and efficiently transmit it to the decoder, providing the decoder with global context feature maps at different levels. Finally, the adaptive receptive field (ARF) module is introduced in the decoder feature reconstruction stage to adapt and focus on related fields, ensuring the model’s adaptation to different segmentation target features, and achieving different weights for the output of different convolution kernels to improve segmentation performance. We comprehensively evaluate our method on medical image segmentation tasks, by using four public datasets across CT and MRI. Remarkably, MFHARFNet method consistently outperforms other state-of-the-art methods, exceeding UNet by 2.1%, 0.9%, 6.6% and 1.0% on Dice on ATLAS, LiTs, BraTs2019 and Spine and intervertebral disc datasets, respectively. In addition, MFHARFNet minimizes network parameters and computational complexity as much as possible. The source codes are in https://github.com/OneHundred99/MFHARFNet.

Attention classification-and-segmentation network for micro-crack anomaly detection of photovoltaic module cells

Hybrid architectures based on convolutional neural networks and Transformers, effectively captures both the local details and the overall structural context of lesion tissues and cells, achieving highly competitive segmentation results in microscopic hyperspectral image (MHSI) segmentation tasks. However, the fixed tokenization schemes and single-dimensional feature extraction and fusion in existing methods lead to insufficient global feature extraction in hyperspectral pathology images. Base on this, we propose a multi-scale token divided and spatial-channel fusion transformer network (MT-SCnet) for MHSIs segmentation. Specifically, we first designed a Multi-Scale Token Divided module. It divides token at different scale based on mirror padding and promotes information interaction and fusion between different tokens to obtain more representative features for subsequent global feature extraction. Secondly, a novel spatial channel fusion transformer was designed to capture richer features from spatial and channel dimensions, and eliminates the semantic gap between features from different dimensions based on cross-attention fusion block. Additionally, to better restore spatial information, deformable convolutions were introduced in decoder. The Experiments on two MHSI datasets demonstrate that MT-SCnet outperforms the comparison methods. This advance has significant implications for the field of MHSIs segmentation. Our code is freely available at https://github.com/sharycao/MT-SCnet.

A Stacked Generalization U-shape network based on zoom strategy and its application in biomedical image segmentation

Cancer, a disease with a high mortality rate, poses a great threat to patients' physical and mental health and can lead to huge medical costs and emotional damage. With the continuous development of artificial intelligence technologies, deep learning-based cancer image segmentation techniques are becoming increasingly important in cancer detection and accurate diagnosis. However, in segmentation tasks, there are differences in efficiency between large and small objects and limited segmentation effects on objects of individual sizes. The previous segmentation frameworks still have room for improvement in multi-scale collaboration when segmenting objects. This paper proposes a method to train a deep learning segmentation framework using a feature pyramid processing dataset to improve the average precision (AP) index, and realizes multi-scale cooperation in target segmentation. Pan-Cancer Histology Dataset for Nuclei Instance Segmentation and Classification (PanNuke) dataset was selected to include approximately 7500 pathology images with cells from 19 different types of tissues, including five classifications of cancer, non-cancer, inflammation, death, and connective tissue. First, the method uses whole-slide images in the pan-cancer histology dataset for nuclei instance segmentation and classification (PanNuke) dataset, combined with the mask region convolutional neural network (Mask R-CNN) segmentation framework and improved loss function to segment and detect each cellular tissue in cancerous sections. Second, to address the problem of non-synergistic object segmentation at different scales in cancerous tissue segmentation, a scheme using feature pyramids to process the dataset was adopted as part of the feature extraction module. Extensive experimental results on this dataset show that the method in this paper yields 0.269 AP and a boost of about 4% compared to the original Mask R-CNN framework. It is effective and feasible to use feature pyramid to process data set to improve the effect of medical image segmentation.

MultiTrans: Multi-branch transformer network for medical image segmentation

MHSU-Net: A more versatile neural network for medical image segmentation

The new coronavirus, which has become a global pandemic, has confirmed more than 88 million cases worldwide since the first case was recorded in December 2019, causing over 1.9 million deaths. Since COIVD-19 lesions have clear imaging features on CT images, it is suitable for the auxiliary diagnosis and treatment of COVID-19. Deep learning can be used to segment the lesions areas of COVID-19 in CT images to help monitor the epidemic situation. In this paper, we propose a multi-point supervision network (MPS-Net) for segmentation of COVID-19 lung infection CT image lesions to solve the problem of a variety of lesion shapes and areas. A multi-scale feature extraction structure, a sieve connection structure (SC), a multi-scale input structure and a multi-point supervised training structure were implemented into MPS-Net. In order to increase the ability to segment various lesion areas of different sizes, the multi-scale feature extraction structure and the sieve connection structure will use different sizes of receptive fields to extract feature maps of various scales. The multi-scale input structure is used to minimize the edge loss caused by the convolution process. In order to improve the accuracy of segmentation, we propose a multi-point supervision training structure to extract supervision signals from different up-sampling points on the network. Experimental results showed that the dice similarity coefficient (DSC), sensitivity, specificity and IOU of the segmentation results of our model are 0.8325, 0.8406, 09988 and 0.742, respectively. The experimental results demonstrated that the network proposed in this paper can effectively segment COVID-19 infection on CT images. It can be used to assist the diagnosis and treatment of new coronary pneumonia.

This chapter discusses multi-fractal texture estimation and characterization of brain lesions (necrosis, edema, enhanced tumor, non-enhanced tumor, etc.) in magnetic resonance (MR) images. This work formulates the complex texture of tumor in MR images using a stochastic model known as multi-fractional Brownian motion (mBm). Mathematical derivations of the mBm model and corresponding algorithm to extract the spatially varying multi-fractal texture feature are discussed. Extracted multi-fractal texture feature is fused with other effective features to enhance the tissue characteristics. Segmentation of the tissues is performed by using a feature-based classification method. The efficacy of the mBm texture feature in segmenting different abnormal tissues is demonstrated using a large-scale publicly available clinical dataset. Experimental results and performance of the methods confirm the efficacy of the proposed technique in an automatic segmentation of abnormal tissues in multimodal (T1, T2, Flair, and T1contrast) brain MRIs.