Global and Local Feature Reconstruction for Medical Image Segmentation.

Medical image segmentation is a fundamental task in medical image analysis, playing a crucial role in disease diagnosis, treatment planning, and clinical decision-making. Accurate segmentation of anatomical structures, such as blood vessels, organs, and lesions, is essential for reliable medical interpretations. However, existing segmentation models often face challenges in effectively capturing both local and global features within medical images. Traditional convolutional neural networks (CNNs) excel at extracting fine-grained local features but struggle to model long-range dependencies due to their limited receptive fields. Conversely, transformer-based models can capture global contextual relationships but often fail to preserve detailed local structures, leading to suboptimal segmentation performance. The purpose of this study is to develop SEFormer, a novel hybrid network that integrates SENet, ResNet, and Transformer to enhance medical image segmentation. SEFormer aims to effectively capture both local and global features, addressing the limitations of traditional CNNs and transformer-based models.SENet is incorporated to recalibrate feature maps by adaptively emphasizing informative features, thereby compensating for the global representation limitations of CNNs. Meanwhile, the ResNet backbone ensures deep feature extraction while maintaining computational efficiency. Additionally, the Transformer module is introduced to capture long-range dependencies and enhance contextual awareness, complementing the local feature extraction capabilities of CNNs. This hybrid approach allows SEFormer to balance fine-grained spatial details with broader contextual information, leading to more precise segmentation results.Furthermore, to mitigate information loss during feature extraction and ensure a more complete and hierarchical representation of image information, we introduce a feature pyramid structure inspired by multi-scale image pyramid models. By progressively increasing the receptive field across different scales and employing skip connections, SEFormer effectively fuses multi-scale local and global features at each stage of the pyramid. This hierarchical fusion mechanism ensures a richer and more robust feature representation, which is particularly beneficial for segmenting complex medical structures such as blood vessels. We evaluate SEFormer on the CHASE_DB1 dataset, a widely used benchmark for retinal vessel segmentation. Experimental results demonstrate that SEFormer outperforms existing state-of-the-art segmentation methods, achieving a 3.25% improvement in segmentation accuracy. Additionally, we conduct ablation studies to verify the contributions of different components within SEFormer. The results show that incorporating SENet enhances feature recalibration and channel attention, while the Transformer module significantly improves global context awareness. The feature pyramid structure further contributes to performance gains by ensuring a multi-scale representation of vascular structures. Compared to conventional CNN-based methods, our approach achieves better segmentation accuracy with improved robustness to variations in vessel thickness, noise, and image contrast. Furthermore, SEFormer maintains computational efficiency, making it suitable for real-world clinical applications where both accuracy and processing speed are critical. SEFormer provides an efficient and accurate approach for medical image segmentation by leveraging the complementary strengths of CNNs, SENet, and Transformers while integrating a feature pyramid structure to maximize feature representation. Future research directions include extending SEFormer to other medical imaging modalities, such as CT and MRI scans, as well as exploring further optimization techniques to reduce computational costs while maintaining high segmentation performance. Additionally, integrating SEFormer with active learning frameworks could further improve segmentation performance by leveraging expert-annotated data more efficiently.

Semantic image segmentation has a wide range of applications. When it comes to medical image segmentation, its accuracy is even more important than those of other areas because the performance gives useful information directly applicable to disease diagnosis, surgical planning, and history monitoring. The state-of-the-art models in medical image segmentation are variants of encoder-decoder architecture, which is called U-Net. To effectively reflect the spatial features in feature maps in encoder-decoder architecture, we propose a spatially adaptive weighting scheme for medical image segmentation. Specifically, the spatial feature is estimated from the feature maps, and the learned weighting parameters are obtained from the computed map, since segmentation results are predicted from the feature map through a convolutional layer. Especially in the proposed networks, the convolutional block for extracting the feature map is replaced with the widely used convolutional frameworks: VGG, ResNet, and Bottleneck Resent structures. In addition, a bilinear up-sampling method replaces the up-convolutional layer to increase the resolution of the feature map. For the performance evaluation of the proposed architecture, we used three data sets covering different medical imaging modalities. Experimental results show that the network with the proposed self-spatial adaptive weighting block based on the ResNet framework gave the highest IoU and DICE scores in the three tasks compared to other methods. In particular, the segmentation network combining the proposed self-spatially adaptive block and ResNet framework recorded the highest 3.01% and 2.89% improvements in IoU and DICE scores, respectively, in the Nerve data set. Therefore, we believe that the proposed scheme can be a useful tool for image segmentation tasks based on the encoder-decoder architecture.

Computational resources, which presents a significant challenge in resourceconstrained environments, particularly in developing countries. Consequently, the development of decoding mechanisms that are both computationally efficient and lightweight is imperative. However, the performance of medical image segmentation is frequently limited by the simplicity of decoder designs. Balancing the optimization of decoder architectures with the reduction of computational demands while maintaining high model accuracy remains a formidable challenge. In this context, we introduce a novel decoder that integrates line deformable convolution and multi-scale self-attention (LDMSD). The multi-scale self-attention enhancement module within LDMSD leverages two distinct multi-scale self-attention mechanisms, thereby substantially improving the representational capacity of the feature maps. Furthermore, the decoder incorporates a linear deformable convolution attention-guided mechanism to augment the feature maps derived from skip connections. This mechanism effectively mitigates the inherent limitations of conventional convolution and enhances the model's ability to capture complex semantic relationships within the feature maps. Through this collaborative mechanism, LDMSD is able to capture target information from both global and multiscale perspectives, accurately locate the target's boundaries and structures, while maintaining its lightweight nature. Experimental results demonstrate that LDMSD outperforms the state-of-the-art decoders in terms of performance metrics, achieving a reduction in Floating Point Operations (FLOPs) by 77.36% and in parameter count by 81.66% when compared to the Cascaded Attention Decoder (CASCADE). To substantiate the efficacy of the proposed method, extensive experiments are conducted on six publicly available datasets. The results validate that the proposed method surpasses existing approaches in medical image segmentation tasks, both in terms of accuracy and computational efficiency.

Multimodal activity recognition with local block CNN and attention-based spatial weighted CNN

RSKD: Enhanced medical image segmentation via multi-layer, rank-sensitive knowledge distillation in Vision Transformer models

Medical image segmentation assists in computer-aided diagnosis, surgeries, and treatment. Digitize tissue slide images are used to analyze and segment glands, nuclei, and other biomarkers which are further used in computer-aided medical applications. To this end, many researchers developed different neural networks to perform segmentation on histological images, mostly these networks are based on encoder-decoder architecture and also utilize complex attention modules or transformers. However, these networks are less accurate to capture relevant local and global features with accurate boundary detection at multiple scales, therefore, we proposed an Encoder-Decoder Network, Quick Attention Module and a Multi Loss Function (combination of Binary Cross Entropy (BCE) Loss, Focal Loss & Dice Loss). We evaluate the generalization capability of our proposed network on two publicly available datasets for medical image segmentation MoNuSeg and GlaS and outperform the state-of-the-art networks with 1.99% improvement on the MoNuSeg dataset and 7.15% improvement on the GlaS dataset. Implementation Code is available at this link: https://bit.ly/HistoSeg

Learning 3D global features by aggregating multiple views has been introduced as a successful strategy for 3D shape analysis. In recent deep learning models with end-to-end training, pooling is a widely adopted procedure for view aggregation. However, pooling merely retains the max or mean value over all views, which disregards the content information of almost all views and also the spatial information among the views. To resolve these issues, we propose Sequential Views To Sequential Labels (SeqViews2SeqLabels) as a novel deep learning model with an encoder-decoder structure based on recurrent neural networks (RNNs) with attention. SeqViews2SeqLabels consists of two connected parts, an encoder-RNN followed by a decoder-RNN, that aim to learn the global features by aggregating sequential views and then performing shape classification from the learned global features, respectively. Specifically, the encoder-RNN learns the global features by simultaneously encoding the spatial and content information of sequential views, which captures the semantics of the view sequence. With the proposed prediction of sequential labels, the decoder-RNN performs more accurate classification using the learned global features by predicting sequential labels step by step. Learning to predict sequential labels provides more and finer discriminative information among shape classes to learn, which alleviates the overfitting problem inherent in training using a limited number of 3D shapes. Moreover, we introduce an attention mechanism to further improve the discriminative ability of SeqViews2SeqLabels. This mechanism increases the weight of views that are distinctive to each shape class, and it dramatically reduces the effect of selecting the first view position. Shape classification and retrieval results under three large-scale benchmarks verify that SeqViews2SeqLabels learns more discriminative global features by more effectively aggregating sequential views than state-of-the-art methods.

Accurate segmentation of medical images is crucial for disease diagnosis and understanding disease changes. Deep learning methods, utilizing encoder-decoder structures, have demonstrated cutting-edge performance in various medical image segmentation tasks. However, the pooling operation in the encoding stage results in feature loss, which makes the network lack the ability to fuse multi-scale information at different levels, hinders its effective perception of multi-scale information, and leads to poor segmentation performance. Drawing inspiration from the U-shaped network, this study introduces a multi-branch feature hybrid attention and adaptive receptive field network (MFHARFNet) for medical image segmentation. Building upon the encoder-decoder framework, we initially devise a multi-branch feature hybrid attention module (MFHAM) to seamlessly integrate feature maps of varying scales, capturing both fine-grained features and coarse-grained semantics across the entire scale. Furthermore, we redesign the skip connection to amalgamate feature information from different branches in the encoder stage and efficiently transmit it to the decoder, providing the decoder with global context feature maps at different levels. Finally, the adaptive receptive field (ARF) module is introduced in the decoder feature reconstruction stage to adapt and focus on related fields, ensuring the model’s adaptation to different segmentation target features, and achieving different weights for the output of different convolution kernels to improve segmentation performance. We comprehensively evaluate our method on medical image segmentation tasks, by using four public datasets across CT and MRI. Remarkably, MFHARFNet method consistently outperforms other state-of-the-art methods, exceeding UNet by 2.1%, 0.9%, 6.6% and 1.0% on Dice on ATLAS, LiTs, BraTs2019 and Spine and intervertebral disc datasets, respectively. In addition, MFHARFNet minimizes network parameters and computational complexity as much as possible. The source codes are in https://github.com/OneHundred99/MFHARFNet.

Deep learning models, especially U-Net and its derivate models, have been widely used in medical image segmentation. These approaches have achieved promising results in many medical image segmentation tasks with a limited number of training samples. We aim on enhancing medical image segmentation by using spatial continuity information in a proposed Multi-Encoder Parse-Decoder Network (MEPDNet) based on the fact that most of the medical images are sampled continuously. Sequential images are input into parameter shared encoders for getting feature maps, which are then fused by a fusion block. A V$\Lambda$-block is structured to parse the fused feature map to extract the hidden continuity information. The reconstructed feature map is fed into a decoder for generating segmentation masks. Experiments on three datasets show MEPDNet outperforms other state-of-the-art segmentation models while using the least parameters.

The convolutional neural network architecture has different components like convolution and pooling. The pooling is crucial component placed after the convolution layer. It plays a vital role in visual recognition, detection and segmentation course to overcome the concerns like overfitting, computation time and recognition accuracy. The elementary pooling process involves down sampling of feature map by piercing into subregions. This piercing and down sampling is defined by the pooling hyperparameters, viz. stride and filter size. This down sampling process discards the irrelevant information and picks the defined global feature. The generally used global feature selection methods are average and max pooling. These methods decline, when the main element has higher or lesser intensity than the nonsignificant element. It also suffers with locus and order of nominated global feature, hence not suitable for every situation. The pooling variants are proposed by numerous researchers to overcome concern. This article presents the state of the art on selection of global feature for pooling process mainly based on four categories such as value, probability, rank and transformed domain. The value and probability-based methods use the criteria such as the way of down sampling, size of kernel, input output feature map, location of pooling, number stages and random selection based on probability value. The rank-based methods assign the rank and weight to activation; the feature is selected based on the defined criteria. The transformed domain pooling methods transform the image to other domains such as wavelet, frequency for pooling the feature.

PHCU-Net: A parallel hierarchical cascade U-Net for skin lesion segmentation

FDR-TransUNet: A novel encoder-decoder architecture with vision transformer for improved medical image segmentation

The bone marrow overproduces immature cells in the malignancy known as Acute Lymphoblastic Leukemia (ALL). In the United States, about 6500 occurrences of ALL are diagnosed each year in both children and adults, comprising nearly 25% of pediatric cancer cases. Recently, many computer-assisted diagnosis (CAD) systems have been proposed to aid hematologists in reducing workload, providing correct results, and managing enormous volumes of data. Traditional CAD systems rely on hematologists’ expertise, specialized features, and subject knowledge. Utilizing early detection of ALL can aid radiologists and doctors in making medical decisions. In this study, Deep Dilated Residual Convolutional Neural Network (DDRNet) is presented for the classification of blood cell images, focusing on eosinophils, lymphocytes, monocytes, and neutrophils. To tackle challenges like vanishing gradients and enhance feature extraction, the model incorporates Deep Residual Dilated Blocks (DRDB) for faster convergence. Conventional residual blocks are strategically placed between layers to preserve original information and extract general feature maps. Global and Local Feature Enhancement Blocks (GLFEB) balance weak contributions from shallow layers for improved feature normalization. The global feature from the initial convolution layer, when combined with GLFEB-processed features, reinforces classification representations. The Tanh function introduces non-linearity. A Channel and Spatial Attention Block (CSAB) is integrated into the neural network to emphasize or minimize specific feature channels, while fully connected layers transform the data. The use of a sigmoid activation function concentrates on relevant features for multiclass lymphoblastic leukemia classification The model was analyzed with Kaggle dataset (16,249 images) categorized into four classes, with a training and testing ratio of 80:20. Experimental results showed that DRDB, GLFEB and CSAB blocks’ feature discrimination ability boosted the DDRNet model F1 score to 0.96 with minimal computational complexity and optimum classification accuracy of 99.86% and 91.98% for training and testing data. The DDRNet model stands out from existing methods due to its high testing accuracy of 91.98%, F1 score of 0.96, minimal computational complexity, and enhanced feature discrimination ability. The strategic combination of these blocks (DRDB, GLFEB, and CSAB) are designed to address specific challenges in the classification process, leading to improved discrimination of features crucial for accurate multi-class blood cell image identification. Their effective integration within the model contributes to the superior performance of DDRNet.

The fully convolutional networks (FCNs) have been widely applied in numerous medical image segmentation tasks. However, tissue regions usually have large variations of shape and scale, so the ability of neural networks to learn multi-scale features is important to the segmentation performance. In this paper, we improve the network for multi-scale feature fusion, in the medical image segmentation by introducing two feature fusion modules: i) global attention multi-scale feature fusion module (GMF); ii) local dense multi-scale feature fusion module (LMF). GMF aims to use global context information to guide the recalibration of low-level features from both spatial and channel aspects, so as to enhance the utilization of effective multi-scale features and suppress the noise of low-level features. LMF adopts bottom-up top-down structure to capture context information, to generate semantic features, and to fuse feature information at different scales. LMF can integrate local dense multi-scale context features layer by layer in the network, thus improving the ability of network to encode interdependent relationships among boundary pixels. Based on the above two modules, we propose a novel medical image segmentation framework (GLF-Net). We evaluated the proposed network and modules on challenging brain tumor segmentation and pancreas segmentation datasets, and very competitive performance has been achieved.

Gastrointestinal (GI) disease examination presents significant challenges to doctors due to the intricate structure of the human digestive system. Colonoscopy and wireless capsule endoscopy are the most commonly used tools for GI examination. However, the large amount of data generated by these technologies requires the expertise and intervention of doctors for disease identification, making manual analysis a very time-consuming task. Thus, the development of a computer-assisted system is highly desirable to assist clinical professionals in making decisions in a low-cost and effective way. In this paper, we introduce a novel framework called InCoLoTransNet, designed for polyp segmentation. The study is based on a transformer and convolution-involution neural network, following the encoder-decoder architecture. We employed the vision transformer in the encoder section to focus on the global context, while the decoder involves a convolution-involution collaboration for resampling the polyp features. Involution enhances the model's ability to adaptively capture spatial and contextual information, while convolution focuses on local information, leading to more accurate feature extraction. The essential features captured by the transformer encoder are passed to the decoder through two skip connection pathways. The CBAM module refines the features and passes them to the convolution block, leveraging attention mechanisms to emphasize relevant information. Meanwhile, locality self-attention is employed to pass essential features to the involution block, reinforcing the model's ability to capture more global features in the polyp regions. Experiments were conducted on five public datasets: CVC-ClinicDB, CVC-ColonDB, Kvasir-SEG, Etis-LaribPolypDB, and CVC-300. The results obtained by InCoLoTransNet are optimal when compared with 15 state-of-the-art methods for polyp segmentation, achieving the highest mean dice score of 93% on CVC-ColonDB and 90% on mean intersection over union, outperforming the state-of-the-art methods. Additionally, InCoLoTransNet distinguishes itself in terms of polyp segmentation generalization performance. It achieved high scores in mean dice coefficient and mean intersection over union on unseen datasets as follows: 85% and 79% on CVC-ColonDB, 91% and 87% on CVC-300, and 79% and 70% on Etis-LaribPolypDB, respectively.