• Home
  • Search
  • Stacked cross-modal feature consolidation attention networks for image captioning
  • Open Access IconOpen Access
  • Cite Icon8
  • https://doi.org/10.1007/s11042-023-15869-xCopy DOI Icon

Stacked cross-modal feature consolidation attention networks for image captioning

Show More
  • Abstract
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

Recently, the attention-enriched encoder-decoder framework has aroused great interest in image captioning due to its overwhelming progress. Many visual attention models directly leverage meaningful regions to generate image descriptions. However, seeking a direct transition from visual space to text is not enough to generate fine-grained captions. This paper exploits a feature-compounding approach to bring together high-level semantic concepts and visual information regarding the contextual environment fully end-to-end. Thus, we propose a stacked cross-modal feature consolidation (SCFC) attention network for image captioning in which we simultaneously consolidate cross-modal features through a novel compounding function in a multi-step reasoning fashion. Besides, we jointly employ spatial information and context-aware attributes (CAA) as the principal components in our proposed compounding function, where our CAA provides a concise context-sensitive semantic representation. To make better use of consolidated features potential, we further propose an SCFC-LSTM as the caption generator, which can leverage discriminative semantic information through the caption generation process. The experimental results indicate that our proposed SCFC can outperform various state-of-the-art image captioning benchmarks in terms of popular metrics on the MSCOCO and Flickr30K datasets.

Similar Papers
  • Research Article
  • Citations4

Context-aware and co-attention network based image captioning model

  • Feb 23, 2023
  • The Imaging Science Journal
  • Himanshu Sharma +1
  • Research Article
  • Citations23

Parallel-fusion LSTM with synchronous semantic and visual information for image captioning

  • Feb 01, 2021
  • Journal of Visual Communication and Image Representation
  • Jing Zhang +2
  • Research Article
  • Citations148

Image Captioning with Deep Bidirectional LSTMs and Multi-Task Learning

  • Apr 25, 2018
  • ACM Transactions on Multimedia Computing, Communications, and Applications
  • Cheng Wang +2
  • Conference Article
  • Citations5

Image Captioning using Neural Network Model

  • Mar 25, 2022
  • Prachi Waghmare +2
  • Research Article
  • Citations25

Hierarchical Deep Neural Network for Image Captioning

  • Feb 18, 2019
  • Neural Processing Letters
  • Yuting Su +3
  • PDF
  • Research Article
  • Citations1

On the Principles and Decisions of New Word Translation in Sino-Japan Cross-Border e-Commerce: A Study in the Context of Cross-Cultural Communication

  • Apr 18, 2023
  • International Journal of Digital Multimedia Broadcasting
  • Gaowa Sulun
  • Research Article

Automatic Image Caption Generation with Deep Learning

  • Apr 27, 2024
  • INTERANTIONAL JOURNAL OF SCIENTIFIC RESEARCH IN ENGINEERING AND MANAGEMENT
  • Mr Vudumula Govinda Rao
  • Research Article
  • Citations55

Deliberate Attention Networks for Image Captioning

  • Jul 17, 2019
  • Proceedings of the AAAI Conference on Artificial Intelligence
  • Lianli Gao +5
  • Conference Article
  • Citations8

More Than Just Attention: Improving Cross-Modal Attentions with Contrastive Constraints for Image-Text Matching

  • Jan 01, 2023
  • Yuxiao Chen +6
  • Book Chapter
  • Citations10

Generation of Image Captions Using VGG and ResNet CNN Models Cascaded with RNN Approach

  • Jan 01, 2020
  • Madhuri Bhalekar +3
  • PDF
  • Research Article
  • Citations51

Image Captioning with Bidirectional Semantic Attention-Based Guiding of Long Short-Term Memory

  • Jan 11, 2019
  • Neural Processing Letters
  • Pengfei Cao +5
  • Book Chapter
  • Citations1

Enhancing Separate Encoding with Multi-layer Feature Alignment for Image-Text Matching

  • Jan 01, 2021
  • Keyu Wen +2
  • Research Article
  • Citations93

High-Order Interaction Learning for Image Captioning

  • Jul 01, 2022
  • IEEE Transactions on Circuits and Systems for Video Technology
  • Yanhui Wang +4
  • PDF
  • Research Article
  • Citations60

Neural Image Caption Generation with Weighted Training and Reference

  • Aug 08, 2018
  • Cognitive Computation
  • Guiguang Ding +5
  • Research Article

Scene Graph Knowledge Enhanced Hashing with Contrastive Learning for Image-Text Retrieval

  • Jan 01, 2026
  • IEEE Transactions on Multimedia
  • Hui Meng +5
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.