• Home
  • Search
  • SCA-CNN: Spatial and Channel-Wise Attention in Convolutional Networks for Image Captioning
  • Open Access IconOpen Access
  • Cite Icon2272
  • https://doi.org/10.1109/cvpr.2017.667Copy DOI Icon

SCA-CNN: Spatial and Channel-Wise Attention in Convolutional Networks for Image Captioning

  • Jul 1, 2017
  • Long Chen +6 more
Show More
  • Abstract
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

Visual attention has been successfully applied in structural prediction tasks such as visual captioning and question answering. Existing visual attention models are generally spatial, i.e., the attention is modeled as spatial probabilities that re-weight the last conv-layer feature map of a CNN encoding an input image. However, we argue that such spatial attention does not necessarily conform to the attention mechanism --- a dynamic feature extractor that combines contextual fixations over time, as CNN features are naturally spatial, channel-wise and multi-layer. In this paper, we introduce a novel convolutional neural network dubbed SCA-CNN that incorporates Spatial and Channel-wise Attentions in a CNN. In the task of image captioning, SCA-CNN dynamically modulates the sentence generation context in multi-layer feature maps, encoding where (i.e., attentive spatial locations at multiple layers) and what (i.e., attentive channels) the visual attention is. We evaluate the proposed SCA-CNN architecture on three benchmark image captioning datasets: Flickr8K, Flickr30K, and MSCOCO. It is consistently observed that SCA-CNN significantly outperforms state-of-the-art visual attention-based image captioning methods.

Similar Papers
  • Research Article
  • Citations521

Action Video Games Make Dyslexic Children Read Better

  • Feb 28, 2013
  • Current Biology
  • Sandro Franceschini +5
  • Research Article
  • Citations1

Channel and spatial attention mechanism for fashion image captioning

  • Oct 01, 2023
  • International Journal of Power Electronics and Drive Systems/International Journal of Electrical and Computer Engineering
  • Bao T Nguyen +2
  • Research Article
  • Citations83

Multi attention module for visual tracking

  • Oct 11, 2018
  • Pattern Recognition
  • Boyu Chen +5
  • Research Article
  • Citations165

Context-Aware Visual Policy Network for Fine-Grained Image Captioning

  • Apr 09, 2019
  • IEEE Transactions on Pattern Analysis and Machine Intelligence
  • Zheng-Jun Zha +4
  • Conference Article
  • Citations66

From Pixels to Objects: Cubic Visual Attention for Visual Question Answering

  • Jul 01, 2018
  • Jingkuan Song +3
  • Book Chapter
  • Citations7

Attention Mechanism for Fashion Image Captioning

  • Oct 28, 2020
  • Bao T Nguyen +2
  • Research Article
  • Citations37

Knowing What it is: Semantic-Enhanced Dual Attention Transformer

  • Jan 01, 2023
  • IEEE Transactions on Multimedia
  • Yiwei Ma +6
  • Research Article
  • Citations260

Hierarchical LSTMs with Adaptive Attention for Visual Captioning.

  • Jan 01, 2019
  • IEEE Transactions on Pattern Analysis and Machine Intelligence
  • Lianli Gao +3
  • Research Article
  • Citations3

A Survey on Image Captioning datasets and Evaluation Metrics

  • Apr 01, 2021
  • IOP Conference Series: Materials Science and Engineering
  • Himanshu Sharma
  • Conference Article
  • Citations16

Bengali Image Captioning with Visual Attention

  • Dec 19, 2020
  • Amit Saha Ami +4
  • Conference Article
  • Citations4

Attention systems and neural responses to visual and auditory stimuli: an fMRI study

  • May 01, 2007
  • Chunlin Li +5
  • Conference Article
  • Citations16

Weakly Labelled Audio Tagging Via Convolutional Networks with Spatial and Channel-Wise Attention

  • May 01, 2020
  • Sixin Hong +3
  • Conference Article
  • Citations5

MEDA: Multi-output Encoder-Decoder for Spatial Attention in Convolutional Neural Networks

  • Nov 01, 2019
  • Huayu Li +1
  • PDF
  • Research Article
  • Citations2

Structure Preserving Convolutional Attention for Image Captioning

  • Jul 19, 2019
  • Applied Sciences
  • Shichen Lu +4
  • Peer Review Report

Decision letter: Distinct roles of forward and backward alpha-band waves in spatial visual attention

  • Jan 27, 2023
  • Christian Keitel
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.