• Home
  • Search
  • Bidirectional Feature Fusion and Enhanced Alignment Based Multimodal Semantic Segmentation for Remote Sensing Images
  • Cite Icon7
  • https://doi.org/10.3390/rs16132289Copy DOI Icon

Bidirectional Feature Fusion and Enhanced Alignment Based Multimodal Semantic Segmentation for Remote Sensing Images

Show More
  • Abstract
  • Highlights & Summary
  • PDF
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

Image–text multimodal deep semantic segmentation leverages the fusion and alignment of image and text information and provides more prior knowledge for segmentation tasks. It is worth exploring image–text multimodal semantic segmentation for remote sensing images. In this paper, we propose a bidirectional feature fusion and enhanced alignment-based multimodal semantic segmentation model (BEMSeg) for remote sensing images. Specifically, BEMSeg first extracts image and text features by image and text encoders, respectively, and then the features are provided for fusion and alignment to obtain complementary multimodal feature representation. Secondly, a bidirectional feature fusion module is proposed, which employs self-attention and cross-attention to adaptively fuse image and text features of different modalities, thus reducing the differences between multimodal features. For multimodal feature alignment, the similarity between the image pixel features and text features is computed to obtain a pixel–text score map. Thirdly, we propose a category-based pixel-level contrastive learning on the score map to reduce the differences among the same category’s pixels and increase the differences among the different categories’ pixels, thereby enhancing the alignment effect. Additionally, a positive and negative sample selection strategy based on different images is explored during contrastive learning. Averaging pixel values across different training images for each category to set positive and negative samples compares global pixel information while also limiting sample quantity and reducing computational costs. Finally, the fused image features and aligned pixel–text score map are concatenated and fed into the decoder to predict the segmentation results. Experimental results on the ISPRS Potsdam, Vaihingen, and LoveDA datasets demonstrate that BEMSeg is superior to comparison methods on the Potsdam and Vaihingen datasets, with improvements in mIoU ranging from 0.57% to 5.59% and 0.48% to 6.15%, and compared with Transformer-based methods, BEMSeg also performs competitively on LoveDA dataset with improvements in mIoU ranging from 0.37% to 7.14%.

Loading PDF

Similar Papers
  • Research Article
  • Citations20

Difference-complementary Learning and Label Reassignment for Multimodal Semi-Supervised Semantic Segmentation of Remote Sensing Images.

  • Jan 01, 2025
  • IEEE transactions on image processing : a publication of the IEEE Signal Processing Society
  • Wenqi Han +3
  • PDF
  • Research Article
  • Citations4

LPASS-Net: Lightweight Progressive Attention Semantic Segmentation Network for Automatic Segmentation of Remote Sensing Images

  • Nov 29, 2022
  • Remote Sensing
  • Han Liang +1
  • Research Article
  • Citations87

Rethinking Transformers for Semantic Segmentation of Remote Sensing Images

  • Jan 01, 2023
  • IEEE Transactions on Geoscience and Remote Sensing
  • Yuheng Liu +3
  • PDF
  • Research Article
  • Citations18

Semi-Supervised Contrastive Learning for Few-Shot Segmentation of Remote Sensing Images

  • Aug 29, 2022
  • Remote Sensing
  • Yadang Chen +4
  • Research Article
  • Citations6

A Multilevel Multimodal Hybrid Mamba-Large Strip Convolution Network for Remote Sensing Semantic Segmentation

  • Aug 04, 2025
  • Remote Sensing
  • Lingyu Yan +5
  • Research Article
  • Citations44

Scale Sensitive Neural Network for Road Segmentation in High-Resolution Remote Sensing Images

  • Mar 23, 2020
  • IEEE Geoscience and Remote Sensing Letters
  • Xiaowei Tan +3
  • Research Article
  • Citations7

Vision Foundation Model Guided Multimodal Fusion Network for Remote Sensing Semantic Segmentation

  • Jan 01, 2025
  • IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing
  • Chen Pan +6
  • Research Article
  • Citations4

MMA-Net: A Semantic Segmentation Network for High-Resolution Remote Sensing Images Based on Multimodal Fusion and Multi-Scale Multi-Attention Mechanisms

  • Oct 28, 2025
  • Remote Sensing
  • Xuanxuan Huang +6
  • Research Article

Incomplete Modalities Restoration via Hierarchical Adaptation for Robust Multimodal Segmentation.

  • Jan 01, 2025
  • IEEE transactions on image processing : a publication of the IEEE Signal Processing Society
  • Yujia Sun +6
  • Research Article

Local-Contextual Feature Fusion Network Based on Nonlinear Spiking Neural Model for Semantic Segmentation of Remote Sensing Images.

  • Mar 24, 2026
  • International journal of neural systems
  • Junhao Du +3
  • Research Article
  • Citations90

Deep Bilateral Filtering Network for Point-Supervised Semantic Segmentation in Remote Sensing Images.

  • Jan 01, 2022
  • IEEE Transactions on Image Processing
  • Linshan Wu +5
  • Research Article
  • Citations3

MMFNet: A Mamba-Based Multimodal Fusion Network for Remote Sensing Image Semantic Segmentation.

  • Oct 08, 2025
  • Sensors (Basel, Switzerland)
  • Jingting Qiu +4
  • PDF
  • Research Article
  • Citations34

Learnable Gated Convolutional Neural Network for Semantic Segmentation in Remote-Sensing Images

  • Aug 17, 2019
  • Remote Sensing
  • Shichen Guo +5
  • Research Article

A Review of Semi-Supervised Learning Frameworks for Semantic Segmentation of Remote Sensing Images

  • Jan 01, 2024
  • Academic Journal of Computing & Information Science
  • Zekun Li +3
  • Research Article
  • Citations4

Onet: Twin U-Net Architecture for Unsupervised Binary Semantic Segmentation in Radar and Remote Sensing Images.

  • Jan 01, 2025
  • IEEE transactions on image processing : a publication of the IEEE Signal Processing Society
  • Yi Zhou +3
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.