• Home
  • Search
  • Deliberate Attention Networks for Image Captioning
  • Cite Icon55
  • https://doi.org/10.1609/aaai.v33i01.33018320Copy DOI Icon

Deliberate Attention Networks for Image Captioning

  • Abstract
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

In daily life, deliberation is a common behavior for human to improve or refine their work (e.g., writing, reading and drawing). To date, encoder-decoder framework with attention mechanisms has achieved great progress for image captioning. However, such framework is in essential an one-pass forward process while encoding to hidden states and attending to visual features, but lacks of the deliberation action. The learned hidden states and visual attention are directly used to predict the final captions without further polishing. In this paper, we present a novel Deliberate Residual Attention Network, namely DA, for image captioning. The first-pass residual-based attention layer prepares the hidden states and visual attention for generating a preliminary version of the captions, while the second-pass deliberate residual-based attention layer refines them. Since the second-pass is based on the rough global features captured by the hidden layer and visual attention in the first-pass, our DA has the potential to generate better sentences. We further equip our DA with discriminative loss and reinforcement learning to disambiguate image/caption pairs and reduce exposure bias. Our model improves the state-of-the-arts on the MSCOCO dataset and reaches 37.5% BELU-4, 28.5% METEOR and 125.6% CIDEr. It also outperforms the-state-ofthe-arts from 25.1% BLEU-4, 20.4% METEOR and 53.1% CIDEr to 29.4% BLEU-4, 23.0% METEOR and 66.6% on the Flickr30K dataset.

Similar Papers
  • Conference Article
  • Citations2

Image caption based on Visual Attention Mechanism

  • Feb 25, 2019
  • Jinfei Zhou +2
  • Research Article
  • Citations1

Atypical visual selective attention in children with dyslexia: evidence from N2pc and PD.

  • Jul 01, 2025
  • NeuroImage
  • Hongyu Liu +5
  • Research Article
  • Citations67

Visuals to Text: A Comprehensive Review on Automatic Image Captioning

  • Aug 01, 2022
  • IEEE/CAA Journal of Automatica Sinica
  • Yue Ming +5
  • Research Article
  • Citations14

Parallel encoder–decoder framework for image captioning

  • Oct 11, 2023
  • Knowledge-Based Systems
  • Reyhane Saeidimesineh +3
  • PDF
  • Research Article
  • Citations39

Facial Expression Recognition Methods in the Wild Based on Fusion Feature of Attention Mechanism and LBP.

  • Apr 22, 2023
  • Sensors
  • Jun Liao +5
  • Research Article
  • Citations27

Modeling visual and word-conditional semantic attention for image captioning

  • Jun 15, 2018
  • Signal Processing: Image Communication
  • Chunlei Wu +4
  • Research Article
  • Citations12

PIEED: Position information enhanced encoder-decoder framework for scene text recognition

  • Feb 10, 2021
  • Applied Intelligence
  • Xitao Ma +3
  • Research Article
  • Citations8

Stacked cross-modal feature consolidation attention networks for image captioning

  • Jun 23, 2023
  • Multimedia Tools and Applications
  • Mozhgan Pourkeshavarz +3
  • Research Article
  • Citations30

Cortical Visual Impairment

  • Nov 01, 2009
  • Pediatrics in Review
  • L H Ospina
  • PDF
  • Research Article
  • Citations3

Semantic-Enhanced Cross-Modal Fusion for Improved Unsupervised Image Captioning

  • Aug 22, 2023
  • Electronics
  • Nan Xiang +4
  • PDF
  • Supplementary Content
  • Citations24

Divisive Normalization and Neuronal Oscillations in a Single Hierarchical Framework of Selective Visual Attention

  • May 04, 2012
  • Frontiers in Neural Circuits
  • Jorrit Steven Montijn +2
  • Research Article
  • Citations23

Parallel-fusion LSTM with synchronous semantic and visual information for image captioning

  • Feb 01, 2021
  • Journal of Visual Communication and Image Representation
  • Jing Zhang +2
  • Research Article
  • Citations3

Exploring Bengali Image Descriptions through the combination of diverse CNN Architectures and Transformer Decoders

  • Jan 20, 2025
  • Turkish Journal of Engineering
  • Biswajit Patra +1
  • Conference Article
  • Citations22

Image Caption Generation With Adaptive Transformer

  • May 28, 2019
  • Wei Zhang +3
  • Research Article
  • Citations155

YOLO-ACN: Focusing on Small Target and Occluded Object Detection

  • Jan 01, 2020
  • IEEE Access
  • Yongjun Li +5
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.