• Home
  • Search
  • Integrating Adversarial Generative Network with Variational Autoencoders towards Cross-Modal Alignment for Zero-Shot Remote Sensing Image Scene Classification
  • Cite Icon11
  • https://doi.org/10.3390/rs14184533Copy DOI Icon

Integrating Adversarial Generative Network with Variational Autoencoders towards Cross-Modal Alignment for Zero-Shot Remote Sensing Image Scene Classification

Show More
  • Abstract
  • PDF
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

Remote sensing image scene classification takes image blocks as classification units and predicts their semantic descriptors. Because it is difficult to obtain enough labeled samples for all classes of remote sensing image scenes, zero-shot classification methods which can recognize image scenes that are not seen in the training stage are of great significance. By projecting the image visual features and the class semantic features into the latent space and ensuring their alignment, the variational autoencoder (VAE) generative model has been applied to address remote-sensing image scene classification under a zero-shot setting. However, the VAE model takes the element-wise square error as the reconstruction loss, which may not be suitable for measuring the reconstruction quality of the visual and semantic features. Therefore, this paper proposes to augment the VAE models with the generative adversarial network (GAN) to make use of the GAN’s discriminator in order to learn a suitable reconstruction quality metric for VAE. To promote feature alignment in the latent space, we have also proposed cross-modal feature-matching loss to make sure that the visual features of one class are aligned with the semantic features of the class and not those of other classes. Based on a public dataset, our experiments have shown the effects of the proposed improvements. Moreover, taking the ResNet models of ResNet18, extracting 512-dimensional visual features, and ResNet50 and ResNet101, both extracting 2048-dimensional visual features for testing, the impact of the different visual feature extractors has also been investigated. The experimental results show that better performance is achieved by ResNet18. This indicates that more layers of the extractors and larger dimensions of the extracted features may not contribute to the image scene classification under a zero-shot setting.

Loading PDF

Similar Papers
  • Research Article
  • Citations6

Mining Contrastive Relations Between Cross-Modal Features for Zero-Shot Remote Sensing Image Scene Classification

  • Jan 01, 2024
  • IEEE Geoscience and Remote Sensing Letters
  • Chun Liu +4
  • Supplementary Content
  • Citations2

Multi-level Cross-modal Feature Alignment via Contrastive Learning towards Zero-shot Classification of Remote Sensing Image Scenes

  • May 31, 2023
  • arXiv (Cornell University)
  • Chun Li +4
  • Conference Article
  • Citations15

Representation Learning of Remote Sensing Knowledge Graph for Zero-Shot Remote Sensing Image Scene Classification

  • Jul 11, 2021
  • Yansheng Li +4
  • Research Article
  • Citations154

Robust deep alignment network with remote sensing knowledge graph for zero-shot and generalized zero-shot remote sensing image scene classification

  • Aug 10, 2021
  • ISPRS Journal of Photogrammetry and Remote Sensing
  • Yansheng Li +4
  • Research Article
  • Citations16

A Scene Images Diversity Improvement Generative Adversarial Network for Remote Sensing Image Scene Classification

  • Dec 05, 2019
  • IEEE Geoscience and Remote Sensing Letters
  • Xin Pan +2
  • PDF
  • Research Article

Semantic Multigranularity Feature Learning for High-Resolution Remote Sensing Image Scene Classification

  • Oct 03, 2021
  • Applied Sciences
  • Xinyi Ma +3
  • Research Article
  • Citations46

A Supervised Progressive Growing Generative Adversarial Network for Remote Sensing Image Scene Classification

  • Jan 01, 2022
  • IEEE Transactions on Geoscience and Remote Sensing
  • Ailong Ma +4
  • Research Article
  • Citations105

Learning Deep Cross-Modal Embedding Networks for Zero-Shot Remote Sensing Image Scene Classification

  • Jan 14, 2021
  • IEEE Transactions on Geoscience and Remote Sensing
  • Yansheng Li +3
  • Research Article
  • Citations74

Robust Space–Frequency Joint Representation for Remote Sensing Image Scene Classification

  • Oct 01, 2019
  • IEEE Transactions on Geoscience and Remote Sensing
  • Jie Fang +3
  • Research Article
  • Citations26

Marginal Center Loss for Deep Remote Sensing Image Scene Classification

  • Sep 26, 2019
  • IEEE Geoscience and Remote Sensing Letters
  • Tianyu Wei +4
  • Research Article
  • Citations19

Spatial Information Considered Network for Scene Classification

  • May 29, 2020
  • IEEE Geoscience and Remote Sensing Letters
  • Chao Tao +3
  • Conference Article

Remote Sensing Image Scene Classification Based on SURF Feature and Deep Learning

  • Nov 01, 2019
  • Jinxiang Liang +4
  • PDF
  • Research Article
  • Citations7

Semi-supervised Remote Sensing Image Scene Classification Based on Generative Adversarial Networks

  • Oct 20, 2022
  • International Journal of Computational Intelligence Systems
  • Dongen Guo +3
  • Research Article
  • Citations970

Remote Sensing Image Scene Classification Meets Deep Learning: Challenges, Methods, Benchmarks, and Opportunities

  • Jan 01, 2020
  • IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing
  • Gong Cheng +4
  • Conference Article
  • Citations2

Remote Sensing Image Scene Classification Based on Densely Connected Multilayer Kernel ELM

  • Dec 01, 2018
  • X.W Jiang +5
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.