• Home
  • Search
  • HOLMES: Hyper-Relational Knowledge Graphs for Multi-hop Question Answering using LLMs
  • Open Access IconOpen Access
  • Cite Icon16
  • https://doi.org/10.18653/v1/2024.acl-long.717Copy DOI Icon

HOLMES: Hyper-Relational Knowledge Graphs for Multi-hop Question Answering using LLMs

  • Jan 1, 2024
  • Pranoy Panda +4 more
Show More
  • Abstract
  • Literature Map
  • Citations
  • Similar Papers
Abstract

Given unstructured text, Large Language Models (LLMs) are adept at answering simple (single-hop) questions. However, as the complexity of the questions increase, the performance of LLMs degrade. We believe this is due to the overhead associated with understanding the complex question followed by filtering and aggregating unstructured information in the raw text. Recent methods try to reduce this burden by integrating structured knowledge triples into the raw text, aiming to provide a structured overview that simplifies information processing. However, this simplistic approach is query-agnostic and the extracted facts are ambiguous as they lack context. To address these drawbacks and to enable LLMs to answer complex (multi-hop) questions with ease, we propose to use a knowledge graph (KG) that is context-aware and is distilled to contain query-relevant information. The use of our compressed distilled KG as input to the LLM results in our method utilizing up to 67% fewer tokens to represent the query relevant information present in the supporting documents, compared to the state-of-the-art (SoTA) method. Our experiments show consistent improvements over the SoTA across several metrics (EM, F1, BERTScore, and Human Eval) on two popular benchmark datasets (HotpotQA and MuSiQue).

Similar Papers
  • Front Matter
  • Citations1

Editorial: Large language models in work and business.

  • Nov 29, 2024
  • Frontiers in artificial intelligence
  • Şadi Evren Şeker
  • Book Chapter
  • Citations2

Enhancing Manufacturing Knowledge Access with LLMs and Context-Aware Prompting

  • Oct 16, 2024
  • Frontiers in artificial intelligence and applications
  • Sebastian Monka +6
  • Preprint Article
  • Citations1

Utilizing Large Language Models for Geoscience Literature Information Extraction

  • Jan 20, 2025
  • Peng Yu +3
  • Research Article

Influence of Knowledge Extraction Methods on the Effectiveness of Graph-Based Rag Systems

  • Nov 26, 2025
  • NaUKMA Research Papers. Computer Science
  • Maksym Androshchuk
  • Research Article
  • Citations4

Extracting epilepsy-related information from unstructured clinic letters using large language models.

  • Jul 10, 2025
  • Epilepsia
  • Shichao Fang +7
  • Research Article
  • Citations26

Knowledge graph–based thought: a knowledge graph–enhanced LLM framework for pan-cancer question answering

  • Jan 06, 2025
  • GigaScience
  • Yichun Feng +5
  • Research Article
  • Citations21

KNowNEt:Guided Health Information Seeking from LLMs via Knowledge Graph Integration.

  • Jan 01, 2025
  • IEEE transactions on visualization and computer graphics
  • Youfu Yan +4
  • Conference Article

Quantifying Informativeness in Knowledge Graph-Augmented In-Context Learning for Multiple Choice Query Answering

  • Nov 13, 2025
  • Maryam Ghanbari +1
  • Research Article

HIEA: Hierarchical Inference for Entity Alignment with Collaboration of Instruction-Tuned Large Language Models and Small Models

  • Jan 18, 2026
  • Electronics
  • Xinchen Shi +2
  • Conference Article
  • Citations3

MFORT-QA: Multi-hop Few-shot Open Rich Table Question Answering

  • Apr 26, 2024
  • Che Guan +2
  • Research Article
  • Citations1

Aiding data retrieval in clinical trials with large language models: The APOLLO 11 Consortium in advanced lung cancer patients.

  • Jun 01, 2025
  • Journal of Clinical Oncology
  • Federica Corso +19
  • Research Article
  • Citations1

Drug repurposing for Alzheimer’s disease using a graph-of-thoughts based large language model to infer drug-disease relationships in a comprehensive knowledge graph

  • Aug 05, 2025
  • BioData Mining
  • Zhiping Paul Wang +10
  • Preprint Article

Multi-hop Reasoning and Retrieval in Embedding Space: Leveraging Large Language Models with Knowledge

  • Feb 25, 2026
  • arXiv (Cornell University)
  • Lihui Liu
  • Conference Article
  • Citations4

Knowledge graph Extraction from Textual data using LLM

  • Jan 01, 2024
  • Research Article
  • Citations1

GraphRAG-Enabled Local Large Language Model for Gestational Diabetes Mellitus: Development of a Proof-of-Concept

  • Jan 05, 2026
  • JMIR Diabetes
  • Edmund Evangelista +4
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.