• Home
  • Search
  • Pairwise CNN-Transformer Features for Human-Object Interaction Detection.
  • Cite Icon9
  • https://doi.org/10.3390/e26030205Copy DOI Icon

Pairwise CNN-Transformer Features for Human-Object Interaction Detection.

  • Feb 27, 2024
  • Entropy
  • Hutuo Quan +5 more
Show More
  • Abstract
  • Highlights & Summary
  • PDF
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

Human-object interaction (HOI) detection aims to localize and recognize the relationship between humans and objects, which helps computers understand high-level semantics. In HOI detection, two-stage and one-stage methods have distinct advantages and disadvantages. The two-stage methods can obtain high-quality human-object pair features based on object detection but lack contextual information. The one-stage transformer-based methods can model good global features but cannot benefit from object detection. The ideal model should have the advantages of both methods. Therefore, we propose the Pairwise Convolutional neural network (CNN)-Transformer (PCT), a simple and effective two-stage method. The model both fully utilizes the object detector and has rich contextual information. Specifically, we obtain pairwise CNN features from the CNN backbone. These features are fused with pairwise transformer features to enhance the pairwise representations. The enhanced representations are superior to using CNN and transformer features individually. In addition, the global features of the transformer provide valuable contextual cues. We fairly compare the performance of pairwise CNN and pairwise transformer features in HOI detection. The experimental results show that the previously neglected CNN features still have a significant edge. Compared to state-of-the-art methods, our model achieves competitive results on the HICO-DET and V-COCO datasets.

Loading PDF

Similar Papers
  • Conference Article

An Optimization Model for Human-Object Interaction Detection Inspired by Multi-features

  • Apr 01, 2019
  • Hailan Kuang +3
  • Research Article
  • Citations8

Human–Object Interaction Detection: An Overview

  • Nov 01, 2024
  • IEEE Consumer Electronics Magazine
  • Jia Wang +3
  • Research Article
  • Citations1

Human object interaction detection in paintings using multi-task learning

  • Jul 24, 2024
  • Digital Applications in Archaeology and Cultural Heritage
  • Maya Antoun +1
  • Conference Article
  • Citations107

Detecting Human-Object Interaction via Fabricated Compositional Learning

  • Jun 01, 2021
  • Zhi Hou +4
  • Research Article
  • Citations5

Human–Object Interaction detection via Global Context and Pairwise-level Fusion Features Integration

  • Nov 13, 2023
  • Neural Networks
  • Haozhong Wang +2
  • Conference Article
  • Citations29

Discovering Human Interactions with Large-Vocabulary Objects via Query and Multi-Scale Detection

  • Oct 01, 2021
  • Suchen Wang +5
  • Research Article
  • Citations1

Visual tracking with complementary deep feature optimization

  • Aug 27, 2018
  • Journal of Electronic Imaging
  • Wei Wang +2
  • Research Article
  • Citations12

UAHOI: Uncertainty-aware robust interaction learning for HOI detection

  • Jul 20, 2024
  • Computer Vision and Image Understanding
  • Mu Chen +2
  • Book Chapter
  • Citations3

Human-Object Interaction Detection: A Survey of Deep Learning-Based Methods

  • Jan 01, 2022
  • Fang Li +3
  • Conference Article
  • Citations317

PPDM: Parallel Point Detection and Matching for Real-Time Human-Object Interaction Detection

  • Jun 01, 2020
  • Yue Liao +5
  • Research Article
  • Citations4

Effects of Motion-Relevant Knowledge From Unlabeled Video to Human-Object Interaction Detection.

  • Sep 01, 2023
  • IEEE Transactions on Neural Networks and Learning Systems
  • Xue Lin +4
  • Research Article
  • Citations23

From detection to understanding: A survey on representation learning for human-object interaction

  • Apr 27, 2023
  • Neurocomputing
  • Tianlun Luo +3
  • Conference Article

Learning human-object interactions by attention aggregation

  • Aug 06, 2021
  • Dongzhou Gu +2
  • Conference Article
  • Citations238

End-to-End Human Object Interaction Detection with HOI Transformer

  • Jun 01, 2021
  • Cheng Zou +10
  • PDF
  • Research Article
  • Citations32

Object Detection Network Based on Feature Fusion and Attention Mechanism

  • Jan 02, 2019
  • Future Internet
  • Ying Zhang +3
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.