• Home
  • Search
  • HDAM: Heuristic Difference Attention Module for Convolutional Neural Networks
  • https://doi.org/10.32604/jiot.2022.025327Copy DOI Icon

HDAM: Heuristic Difference Attention Module for Convolutional Neural Networks

Show More
  • Abstract
  • Highlights & Summary
  • PDF
  • Literature Map
  • References
  • Similar Papers
Abstract

The attention mechanism is one of the most important priori knowledge to enhance convolutional neural networks. Most attention mechanisms are bound to the convolutional layer and use local or global contextual information to recalibrate the input. This is a popular attention strategy design method. Global contextual information helps the network to consider the overall distribution, while local contextual information is more general. The contextual information makes the network pay attention to the mean or maximum value of a particular receptive field. Different from the most attention mechanism, this article proposes a novel attention mechanism with the heuristic difference attention module (HDAM). HDAM's input recalibration is based on the difference between the local and global contextual information instead of the mean and maximum values. At the same time, to make different layers have a more suitable local receptive field sizes and increase the flexibility of the local receptive field design, we use genetic algorithm to heuristically produce local receptive fields. First, HDAM extracts the mean value of the global and local receptive fields as the corresponding contextual information. Then the difference between the global and local contextual information is calculated. Finally, HDAM uses this difference to recalibrate the input. In addition, we use the heuristic ability of genetic algorithm to search for the local receptive field size of each layer. Our experiments on CIFAR-10 and CIFAR-100 show that HDAM can use fewer parameters than other attention mechanisms to achieve higher accuracy. We implement HDAM with the Python library, Pytorch, and the code and models will be publicly available.

Loading PDF

Similar Papers
  • Book Chapter
  • Citations194

Keyword Extraction Using Support Vector Machine

  • Jan 01, 2006
  • Kuo Zhang +3
  • Research Article
  • Citations27

GC–HGNN: A global-context supported hypergraph neural network for enhancing session-based recommendation

  • Mar 01, 2022
  • Electronic Commerce Research and Applications
  • Dunlu Peng +1
  • Research Article
  • Citations1

Context‐based robust face detection algorithm for surveillance cameras

  • Sep 18, 2012
  • IEEJ Transactions on Electrical and Electronic Engineering
  • Shotaro Miwa +2
  • Research Article
  • Citations4

A Multi-Level SAR-Guided Contextual Attention Network for Satellite Images Cloud Removal

  • Dec 20, 2024
  • Remote Sensing
  • Ganchao Liu +2
  • Research Article
  • Citations12

A scene segmentation algorithm combining the body and the edge of the object

  • Dec 01, 2021
  • Information Processing & Management
  • Xianfeng Ou +4
  • Research Article

Temporal consistency-aware network for renal artery segmentation in X-ray angiography.

  • Aug 02, 2025
  • International journal of computer assisted radiology and surgery
  • Botao Yang +10
  • PDF
  • Research Article
  • Citations2

Few-Shot Object Detection with Local Feature Enhancement and Feature Interrelation

  • Sep 25, 2023
  • Electronics
  • Hefeng Lai +1
  • Research Article

Strategic adaptations to deceptive actions in sports: The anticipation-response-inhibition-accuracy trade-off.

  • Jul 01, 2026
  • Psychology of sport and exercise
  • Carolin Wickemeyer +2
  • Book Chapter
  • Citations34

Learning to Predict Context-Adaptive Convolution for Semantic Segmentation

  • Jan 01, 2020
  • Jianbo Liu +4
  • Conference Article
  • Citations53

Collaboratively Improving Topic Discovery and Word Embeddings by Coordinating Global and Local Contexts

  • Aug 04, 2017
  • Guangxu Xun +3
  • Research Article
  • Citations6

BIG-FUSION: Brain-Inspired Global-Local Context Fusion Framework for Multimodal Emotion Recognition in Conversations

  • Apr 11, 2025
  • Proceedings of the AAAI Conference on Artificial Intelligence
  • Yusong Wang +8
  • Conference Article
  • Citations6

Statistical Pyramid Dense Time Delay Neural Network for Speaker Verification

  • May 23, 2022
  • Zi-Kai Wan +3
  • Conference Article
  • Citations1

Document-Level Joint Biomedical Event Extraction Model Using Hypergraph Convolutional Networks

  • Oct 09, 2022
  • Jinghao Gong +3
  • Conference Article
  • Citations4

<title>Generation of video highlights using video context and perception</title>

  • Jan 01, 2001
  • Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE
  • Hang-Bong Kang
  • Research Article
  • Citations32

LSRFormer: Efficient Transformer Supply Convolutional Neural Networks With Global Information for Aerial Image Segmentation

  • Jan 01, 2024
  • IEEE Transactions on Geoscience and Remote Sensing
  • Renhe Zhang +2
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.