• Home
  • Search
  • A Multi-Granularity Relation Graph Aggregation Framework with Multimodal Clues for Social Relation Reasoning
  • Cite Icon3
  • https://doi.org/10.1109/tmm.2025.3543054Copy DOI Icon

A Multi-Granularity Relation Graph Aggregation Framework with Multimodal Clues for Social Relation Reasoning

Show More
  • Abstract
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

The social relation is a fundamental attribute of human beings in daily life. The ability of humans to form large organizations and institutions stems directly from our complex social networks. Therefore, understanding social relationships in the context of multimedia is crucial for building domain-specific or general artificial intelligence systems. The key to reason social relations lies in understanding the human interactions between individuals through multimodal representations such as action and utterance. However, due to video editing techniques and various narrative sequences in videos, two individuals with social relationships may not appear together in the same frame or clip. Additionally, social relations may manifest in different levels of granularity in video expressions. Previous research has not effectively addressed these challenges. Therefore, this paper proposes a <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Multi-Granularity Relation Graph Aggregation Framework</i> (<bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">MGRG</b>) to enhance the inference ability for social relation reasoning in multimedia content, like video. Different from existing methods, our method considers the paradigm of jointly inferring the relations by constructing a social relation graph. We design a hierarchical multimodal relation graph illustrating the exchange of information between individuals' roles, capturing the complex interactions at multi-levels of granularity from fine to coarse. In MGRG, we propose two aggregation modules to cluster multimodal features in different granularity layer relation graph, considering temporal aspects and importance. Experimental results show that our method generates a logical and coherent social relation graph and improves the performance in accuracy.

Similar Papers
  • Research Article
  • Citations45

Retrieval, alignment, and clustering of computational models based on semantic annotations

  • Jan 01, 2011
  • Molecular Systems Biology
  • Marvin Schulz +4
  • Research Article
  • Citations10

Business Process Quality Computation: Computing Non-Functional Requirements to Improve Business Processes

  • Mar 04, 2015
  • Research Repository (Delft University of Technology)
  • Farideh Heidari
  • Research Article
  • Citations21

A theoretical study on object-oriented and property-oriented multi-scale formal concept analysis

  • Sep 20, 2019
  • International Journal of Machine Learning and Cybernetics
  • Yanhong She +4
  • PDF
  • Conference Article
  • Citations16

A Layered Language Model based Hybrid Approach to Automatic Full Diacritization of Arabic

  • Jan 01, 2017
  • Mohamed Al-Badrashiny +2
  • Book Chapter
  • Citations14

Schema Design Alternatives for Multi-granular Data Warehousing

  • Jan 01, 2010
  • Nadeem Iftikhar +1
  • Research Article
  • Citations18

Combining Geo‐SOMand Hierarchical Clustering to Explore Geospatial Data

  • Sep 16, 2013
  • Transactions in GIS
  • Chen‐Chieh Feng +2
  • Conference Article

Spatiotemporal Analysis of a Location Based Social Network Dataset based on Different Levels of Granularity

  • Oct 03, 2018
  • Felipe Araújo +2
  • Research Article
  • Citations62

Social relations and breath odour

  • Nov 01, 2003
  • International Journal of Dental Hygiene
  • L Mckeown
  • Research Article

Granularity Matters! Towards a Methodological Framework for Routine Studies

  • Jul 30, 2020
  • Academy of Management Proceedings
  • Jan Hoekzema +2
  • Conference Article
  • Citations5

Conformance checking with different levels of granularity : A case study on bariatric surgery

  • Oct 17, 2020
  • Denise M V Sato +5
  • Conference Article
  • Citations2

A novel ontology relation graph-based ontology module partition algorithm

  • Jun 01, 2012
  • Ji Xiaohui +1
  • Research Article
  • Citations1

Does playful teasing help great apes learn about social relationships?

  • Feb 05, 2026
  • Philosophical transactions of the Royal Society of London. Series B, Biological sciences
  • Sasha L Winkler +1
  • Research Article
  • Citations4

The role of baseline granularity for benchmarking citation impact. The case of CSS profiles

  • Apr 27, 2018
  • Scientometrics
  • Wolfgang Glänzel +1
  • PDF
  • Research Article
  • Citations9

Multi-Granularity Mission Negotiation for a Decentralized Remote Sensing Satellite Cluster

  • Nov 02, 2020
  • Remote Sensing
  • Xuelei Deng +2
  • Supplementary Content
  • Citations26

Object-Aware Process Management

  • Nov 07, 2013
  • OPen Access Repositorium der Universität Ulm (OPARU) (Ulm University)
  • Vera Künzle
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.