A complex question answering system based on knowledge Graph in Terahertz Domain

Over the past decade, Knowledge Graphs (KG) have emerged as a prominent repository for storing facts about the world in a linked data architecture. Providing machines with the capability of exploring such Knowledge Graphs and answering natural language questions over them, has been an active area of research. The purpose of this work, is to delve further into the research of retrieving information stored in KGs, based on the natural language questions posed by the user. Knowledge Graph Question Answering (KGQA) aims to produce a concise answer to a user question, such that the user is exempt from using KG vocabulary and overheads of learning a formal query language. Existing KGQA systems have achieved excellent results over Simple Questions, where the information required is limited to a single triple and a single formal query pattern. Our motivation is to improve the performance of KGQA over Complex Questions, where formal query patterns significantly vary, and a single triple is not confining for all the required information. Complex KGQA provides several challenges such as understanding semantics and syntactic structure of questions, Entity Linking, Relation Linking and Answer Representation. Lack of suitable datasets for complex question answering further adds to research gaps. Hence, in this thesis, we focus the research objective of laying the foundations for the advancement of the state-of-the-art for Complex Question Answering over Knowledge Graphs, by providing techniques to solve various challenges and provide resources to fill the research gaps. First, we propose Normalized Query Structure (NQS), which is a linguistic analyzer module that helps the QA system to detect inputs and intents and the relation between them in the users' question. NQS acts like an intermediate language between natural language questions and formal expressions to ease the process of query formulation for complex questions. We then developed a framework named LC-QuAD to generate large scale question answering dataset by reversing the process of question answering, thereby translating natural language questions from the formal query using intermediate templates. Our goal is to use this framework for high variations in the query patterns and create a large size dataset with minimum human effort. The first version of the dataset consists of 5,000 complex questions. By extending the LC-QuAD framework to support Reified KGs and crowd-sourcing, we published the second version of the dataset as LC-QuAD 2.0, consisting of 30,000 questions with their paraphrases and has higher complexity and new variations in the questions. To overcome the problem of Entity Linking and Relation Linking in KGQA, we develop EARL, a module performing these two tasks as a single joint task for complex question answering. We develop approaches for this module, first by formalizing the task as an instance of the Generalized Traveling Salesman Problem (GTSP) and the second approach uses machine learning to exploit the connection density between nodes in the Knowledge Graph. Lastly, we create another large scale dataset to answer verbalization and provide results for multiple baseline system s on it. The Verbalization dataset is introduced to make the system's response more human-like. The NQS based KGQA system was next to the best system in terms of accuracy on the QALD-5 dataset. We empirically prove that NQS is robust to tackle paraphrases of the questions. EARL achieves the state of the art results in Entity Linking and Relation Linking for question answering on several KGQA datasets. The dataset curated in this thesis has helped the research community to move forward in the direction of improving the accuracy of complex question answering as a task as other researchers too developed several KGQA systems and modules around these published datasets. With the large-scale datasets, we have encouraged the use of large scale machine learning, deep learning and emergence of new techniques to advance the state-of-the-art in complex question answering over knowledge graphs. We further developed core components for the KGQA pipeline to overcome the challenges of Question Understanding, Entity-Relation Linking and Answer Verbalization and thus achieve our research objective. All the proposed approaches mentioned in this thesis and the published resources are available at https://github.com/AskNowQA and are released under the umbrella project AskNow.

Processing knowledge graph-based complex questions through question decomposition and recomposition

Complex Question Answering over Knowledge Graph (C-KGQA) seeks to solve complex questions using knowledge graphs. Currently, KGQA systems achieve great success in answering simple questions, while complex questions still present challenging issues. As a result, an increasing number of novel methods have been proposed to remedy this challenge. In this survey, we proposed two mainstream categories of methods for C-KGQA, which are divided according to their use for knowledge graph representation and construction, namely, graph metric (GM)-Based Methods and graph neural network (GNN)-based methods. Additionally, we also acknowledge the influence of ChatGPT, which has prompted further research into utilizing knowledge graphs as a knowledge source to assist in answering complex questions. We also introduced methods based on pre-trained models and knowledge graph joint reasoning. Furthermore, we have compiled research achievements from the past three years to make it easier for researchers with similar interests to obtain state-of-the-art research. Finally, we discussed the resources and evaluation methods for tackling C-KGQA tasks and summarized several research prospects in this field.

With the growing availability of different knowledge graphs in a variety of domains, question answering over knowledge graph (KG-QA) becomes a prevalent information retrieval approach. Current KG-QA methods usually resort to semantic parsing, search or neural matching models. However, they cannot well tackle increasingly long input questions and complex information needs. In this work, we propose a new KG-QA approach, leveraging the rich domain context in the knowledge graph. We incorporate the new approach with question and answer domain context descriptions. Specifically, for questions, we enrich them with users’ subsequent input questions within a session and expand the input question representation. For the candidate answers, we equip them with surrounding context structures, i.e., meta-paths within the targeting knowledge graph. On top of these, we design a cross-attention mechanism to improve the question and answer matching performance. An experimental study on real datasets verifies these improvements. The new approach is especially beneficial for specific knowledge graphs with complex questions.

While conversing with chatbots, humans typically tend to ask many questions, a significant portion of which can be answered by referring to large-scale knowledge graphs (KG). While Question Answering (QA) and dialog systems have been studied independently, there is a need to study them closely to evaluate such real-world scenarios faced by bots involving both these tasks. Towards this end, we introduce the task of Complex Sequential QA which combines the two tasks of (i) answering factual questions through complex inferencing over a realistic-sized KG of millions of entities, and (ii) learning to converse through a series of coherently linked QA pairs. Through a labor intensive semi-automatic process, involving in-house and crowdsourced workers, we created a dataset containing around 200K dialogs with a total of 1.6M turns. Further, unlike existing large scale QA datasets which contain simple questions that can be answered from a single tuple, the questions in our dialogs require a larger subgraph of the KG. Specifically, our dataset has questions which require logical, quantitative, and comparative reasoning as well as their combinations. This calls for models which can: (i) parse complex natural language questions, (ii) use conversation context to resolve coreferences and ellipsis in utterances, (iii) ask for clarifications for ambiguous queries, and finally (iv) retrieve relevant subgraphs of the KG to answer such questions. However, our experiments with a combination of state of the art dialog and QA models show that they clearly do not achieve the above objectives and are inadequate for dealing with such complex real world settings. We believe that this new dataset coupled with the limitations of existing models as reported in this paper should encourage further research in Complex Sequential QA.

ObjectiveTo explore the potential and accuracy of the generative dialogue artificial intelligence tool GPT‐4.0 in answering questions related to paediatric emergency appendicitis.MethodsA cross‐sectional observational study design was used. We collected 134 appendicitis‐related questions from authoritative websites, such as Mayo Medical and APSA, covering all aspects of appendicitis, including simple and complex questions. These questions were answered by GPT‐4.0, and then evaluated by three paediatric surgical experts using a quality score ranging from 0 to 5. The answers were generated by GPT‐4.0 and then similarly evaluated by three experts for accuracy.ResultsWe found that GPT‐4.0 could achieve a high accuracy rate on simple questions with a quality score of 4.65 (standard deviation 0.51). For complex questions, the average score was 3.77 (standard deviation 0.68), and there was a significant difference between the two (P < .05). On clinical questions, the accuracy score of GPT‐4.0 was 4.00 (standard deviation 0.21). When answering actual questions from families of children with appendicitis, the accuracy score was 4.12 (standard deviation 0.59). Its accuracy lies between simple questions and complex questions, and it can basically meet the accuracy requirements of clinical questions. It's worth noting that GPT‐4.0 demonstrated empathy in answering some questions, which might further enhance patient satisfaction.ConclusionGPT‐4.0 showed its potential and accuracy in handling paediatric appendicitis questions, especially in simple and clinical questions. However, improvements are still needed in handling complex questions and updating information. Despite the limitations, this model is expected to improve the quality of medical services and enhance patient satisfaction.

Given unstructured text, Large Language Models (LLMs) are adept at answering simple (single-hop) questions. However, as the complexity of the questions increase, the performance of LLMs degrade. We believe this is due to the overhead associated with understanding the complex question followed by filtering and aggregating unstructured information in the raw text. Recent methods try to reduce this burden by integrating structured knowledge triples into the raw text, aiming to provide a structured overview that simplifies information processing. However, this simplistic approach is query-agnostic and the extracted facts are ambiguous as they lack context. To address these drawbacks and to enable LLMs to answer complex (multi-hop) questions with ease, we propose to use a knowledge graph (KG) that is context-aware and is distilled to contain query-relevant information. The use of our compressed distilled KG as input to the LLM results in our method utilizing up to 67% fewer tokens to represent the query relevant information present in the supporting documents, compared to the state-of-the-art (SoTA) method. Our experiments show consistent improvements over the SoTA across several metrics (EM, F1, BERTScore, and Human Eval) on two popular benchmark datasets (HotpotQA and MuSiQue).

A knowledge graph (KG) consists of numerous triples, in which each triple, i.e., (head entity, relation, tail entity), denotes a real-world assertion. Many large-scale KGs have been developed, e.g., general-purpose KGs Freebase and YAGO. Also, lots of domain-specific KGs are emerging, e.g., COVID-19 KGs, biomedical KGs, and agricultural KGs. By embedding KGs into low-dimensional vectors, i.e., representations of entities and relations, we could integrate KGs into machine learning models and enhance the performance of many prediction tasks, including search, recommendations, and question answering. During the construction, refinement, embedding, and application of KGs, a variety of KG learning algorithms have been developed to handle challenges in various real-world scenarios. Moreover, graph neural networks have also brought new opportunities to KG learning. This workshop aims to engage with active researchers from KG communities, recommendation communities, natural language processing communities, and other communities, and deliver state-of-the-art research insights into the core challenges in KG learning.

Knowledge graph embedding (KGE) models represent each entity and relation of a knowledge graph (KG) with low-dimensional embedding vectors. These methods have recently been applied to KG link prediction and question answering over incomplete KGs (KGQA). KGEs typically create an embedding for each entity in the graph, which results in large model sizes on real-world graphs with millions of entities. For downstream tasks these atomic entity representations often need to be integrated into a multi stage pipeline, limiting their utility. We show that an off-the-shelf encoder-decoder Transformer model can serve as a scalable and versatile KGE model obtaining state-of-the-art results for KG link prediction and incomplete KG question answering. We achieve this by posing KG link prediction as a sequence-to-sequence task and exchange the triple scoring approach taken by prior KGE methods with autoregressive decoding. Such a simple but powerful method reduces the model size up to 98% compared to conventional KGE models while keeping inference time tractable. After finetuning this model on the task of KGQA over incomplete KGs, our approach outperforms baselines on multiple large-scale datasets without extensive hyperparameter tuning.

Question answering (QA) systems are increasingly deployed across domains. However, their reliability is undermined when retrieved evidence is incomplete, noisy, or uncertain. Existing knowledge graph (KG) based QA frameworks typically represent facts as static and deterministic, failing to capture the evolving nature of information and the uncertainty inherent in reasoning. We present a demonstration of uncertainty-aware dynamic KGs, a framework that combines (i) dynamic construction of evolving KGs, (ii) confidence scoring and uncertainty-aware retrieval, and (iii) an interactive interface for reliable and interpretable QA. Our system highlights how uncertainty modeling can make QA more robust and transparent by enabling users to explore dynamic graphs, inspect confidence-annotated triples, and compare baseline versus confidence-aware answers. The target users of this demo are clinical data scientists and clinicians, and we instantiate the framework in healthcare: constructing personalized KGs from electronic health records, visualizing uncertainty across patient visits, and evaluating its impact on a mortality prediction task. This use case demonstrates the broader promise of uncertainty-aware dynamic KGs for enhancing QA reliability in high-stakes applications.

Besides the challenge that a human can ask one question in many different ways, a key aspect in Question Answering approaches over Knowledge Graphs (KGQA) is to deal with the vast amount of information present in the knowledge graphs. Modern real-world knowledge graphs contain nearly millions of entities and relationships. Additionally, they are enriched with new facts every day. However, not all facts are relevant for answering particular questions, thus fostering several challenges to KGQA systems, which require interpretable and query-able data. One solution to filtering the extra data in knowledge graphs is to rely on graph summarization techniques. Graph-based summarization approaches aim to resize knowledge graphs to be more concise and precise by storing only relevant information. In this paper, we propose a framework named LAUREN that applies different summarization techniques on knowledge graphs to be used in KGQA systems. Our experiments show that LAUREN summarizes large knowledge graphs such as DBpedia by 2 million entities and its summarization still achieves the same performance on both question answering and linking tasks compared to the complete DBpedia.

Knowledge graphs (KGs) are essential resources for many applications including Web search and question answering. As KGs are often automatically constructed, they may contain incorrect facts. Detecting them is a crucial, yet extremely expensive task. Prominent solutions detect and explain inconsistency in KGs with respect to accompanying ontologies that describe the KG domain of interest. Compared to machine learning methods they are more reliable and human-interpretable but scale poorly on large KGs. In this paper, we present a novel approach to dramatically speed up the process of detecting and explaining inconsistency in large KGs by exploiting KG abstractions that capture prominent data patterns. Though much smaller, KG abstractions preserve inconsistency and their explanations. Our experiments with large KGs (e.g., DBpedia and Yago) demonstrate the feasibility of our approach and show that it significantly outperforms the popular baseline.

Recently, there has been an upsurge in the number of knowledge graphs (KG) that can only be accessed by experts. Non-expert users lack an adequate understanding of the queried knowledge graph's vocabulary and structure, as well as the syntax of the structured query language used to express the user's information needs. To increase the user base of these KGs, a set of Question Answering (QA) systems that use natural language to query these knowledge graphs have been introduced. However, finding a benchmark that accurately evaluates the quality of a QA system is a difficult task due to (1) the high degree of variation in the fine-grained properties among the existing benchmarks, (2) the static nature of the existing benchmarks versus the evolving nature of KGs, and (3) the limited number of KGs targeted by existing benchmarks, which hinders the usability of QA systems in real-world deployment over KGs that are different from those that were used in the evaluation of the QA systems. In this paper, we introduce Maestro, a benchmark generation system for question answering over knowledge graphs. Maestro can generate a new benchmark for any KG given the KG and, optionally, a text corpus that covers this KG. The benchmark generated by Maestro is guaranteed to cover all the properties of the natural language questions and queries that were encountered in the literature as long as the targeted KG includes these properties. Maestro also generates high-quality natural language questions with various utterances that are on par with manually-generated ones to better evaluate QA systems.

While RDF was designed to make data easily readable by machines, it does not make data easily usable by end-users. Question Answering (QA) over Knowledge Graphs (KGs) is seen as the technology which is able to bridge this gap. It aims to build systems which are capable of extracting the answer to a user’s natural language question from an RDF dataset.In recent years, many approaches were proposed which tackle the problem of QA over KGs. Despite such efforts, it is hard and cumbersome to create a Question Answering system on top of a new RDF dataset. The main open challenge remains portability, i.e., the possibility to apply a QA algorithm easily on new and previously untested RDF datasets.In this publication, we address the problem of portability by presenting an architecture for a portable QA system. We present a novel approach called QAnswer KG, which allows the construction of on-demand QA systems over new RDF datasets. Hence, our approach addresses non-expert users in QA domain.In this paper, we provide the details of QA system generation process. We show that it is possible to build a QA system over any RDF dataset while requiring minimal investments in terms of training. We run experiments using 3 different datasets.To the best of our knowledge, we are the first to design a process for non-expert users. We enable such users to efficiently create an on-demand, scalable, multilingual, QA system on top of any RDF dataset.

BackgroundIn recent years, large language models (LLMs) have shown promise in various domains, notably in biomedical sciences. However, their real-world application is often limited by issues like erroneous outputs and hallucinatory responses.ResultsWe developed the knowledge graph–based thought (KGT) framework, an innovative solution that integrates LLMs with knowledge graphs (KGs) to improve their initial responses by utilizing verifiable information from KGs, thus significantly reducing factual errors in reasoning. The KGT framework demonstrates strong adaptability and performs well across various open-source LLMs. Notably, KGT can facilitate the discovery of new uses for existing drugs through potential drug–cancer associations and can assist in predicting resistance by analyzing relevant biomarkers and genetic mechanisms. To evaluate the knowledge graph question answering task within biomedicine, we utilize a pan-cancer knowledge graph to develop a pan-cancer question answering benchmark, named pan-cancer question answering.ConclusionsThe KGT framework substantially improves the accuracy and utility of LLMs in the biomedical field. This study serves as a proof of concept, demonstrating its exceptional performance in biomedical question answering.