Advancements in Complex Knowledge Graph Question Answering: A Survey

Over the past decade, Knowledge Graphs (KG) have emerged as a prominent repository for storing facts about the world in a linked data architecture. Providing machines with the capability of exploring such Knowledge Graphs and answering natural language questions over them, has been an active area of research. The purpose of this work, is to delve further into the research of retrieving information stored in KGs, based on the natural language questions posed by the user. Knowledge Graph Question Answering (KGQA) aims to produce a concise answer to a user question, such that the user is exempt from using KG vocabulary and overheads of learning a formal query language. Existing KGQA systems have achieved excellent results over Simple Questions, where the information required is limited to a single triple and a single formal query pattern. Our motivation is to improve the performance of KGQA over Complex Questions, where formal query patterns significantly vary, and a single triple is not confining for all the required information. Complex KGQA provides several challenges such as understanding semantics and syntactic structure of questions, Entity Linking, Relation Linking and Answer Representation. Lack of suitable datasets for complex question answering further adds to research gaps. Hence, in this thesis, we focus the research objective of laying the foundations for the advancement of the state-of-the-art for Complex Question Answering over Knowledge Graphs, by providing techniques to solve various challenges and provide resources to fill the research gaps. First, we propose Normalized Query Structure (NQS), which is a linguistic analyzer module that helps the QA system to detect inputs and intents and the relation between them in the users' question. NQS acts like an intermediate language between natural language questions and formal expressions to ease the process of query formulation for complex questions. We then developed a framework named LC-QuAD to generate large scale question answering dataset by reversing the process of question answering, thereby translating natural language questions from the formal query using intermediate templates. Our goal is to use this framework for high variations in the query patterns and create a large size dataset with minimum human effort. The first version of the dataset consists of 5,000 complex questions. By extending the LC-QuAD framework to support Reified KGs and crowd-sourcing, we published the second version of the dataset as LC-QuAD 2.0, consisting of 30,000 questions with their paraphrases and has higher complexity and new variations in the questions. To overcome the problem of Entity Linking and Relation Linking in KGQA, we develop EARL, a module performing these two tasks as a single joint task for complex question answering. We develop approaches for this module, first by formalizing the task as an instance of the Generalized Traveling Salesman Problem (GTSP) and the second approach uses machine learning to exploit the connection density between nodes in the Knowledge Graph. Lastly, we create another large scale dataset to answer verbalization and provide results for multiple baseline system s on it. The Verbalization dataset is introduced to make the system's response more human-like. The NQS based KGQA system was next to the best system in terms of accuracy on the QALD-5 dataset. We empirically prove that NQS is robust to tackle paraphrases of the questions. EARL achieves the state of the art results in Entity Linking and Relation Linking for question answering on several KGQA datasets. The dataset curated in this thesis has helped the research community to move forward in the direction of improving the accuracy of complex question answering as a task as other researchers too developed several KGQA systems and modules around these published datasets. With the large-scale datasets, we have encouraged the use of large scale machine learning, deep learning and emergence of new techniques to advance the state-of-the-art in complex question answering over knowledge graphs. We further developed core components for the KGQA pipeline to overcome the challenges of Question Understanding, Entity-Relation Linking and Answer Verbalization and thus achieve our research objective. All the proposed approaches mentioned in this thesis and the published resources are available at https://github.com/AskNowQA and are released under the umbrella project AskNow.

A knowledge graph (KG) consists of numerous triples, in which each triple, i.e., (head entity, relation, tail entity), denotes a real-world assertion. Many large-scale KGs have been developed, e.g., general-purpose KGs Freebase and YAGO. Also, lots of domain-specific KGs are emerging, e.g., COVID-19 KGs, biomedical KGs, and agricultural KGs. By embedding KGs into low-dimensional vectors, i.e., representations of entities and relations, we could integrate KGs into machine learning models and enhance the performance of many prediction tasks, including search, recommendations, and question answering. During the construction, refinement, embedding, and application of KGs, a variety of KG learning algorithms have been developed to handle challenges in various real-world scenarios. Moreover, graph neural networks have also brought new opportunities to KG learning. This workshop aims to engage with active researchers from KG communities, recommendation communities, natural language processing communities, and other communities, and deliver state-of-the-art research insights into the core challenges in KG learning.

Multitype view of knowledge contrastive learning for recommendation

Processing knowledge graph-based complex questions through question decomposition and recomposition

The automatic identification of the topology of power networks is important for the data-driven and situation-aware operation of power grids. Traditional methods of topology identification lack a data-tolerant mechanism, and the accuracy of their performance in terms of identification is thus affected by the quality of data. Topology identification is related to the link prediction problem. The graph neural network can be used to predict the state of unlabeled nodes (lines) through training on features of labeled nodes (lines) with fault tolerance. Inspired by the characteristics of the graph neural network, we applied it to topology identification in this study. We propose a method to identify the topology of a power network based on a knowledge graph and the graph neural network. Traditional knowledge graphs can quickly mine possible connections between entities and generate graph structure data, but in the case of errors or informational conflicts in the data, they cannot accurately determine whether the relationships between the entities really exist. The graph neural network can use data mining to determine whether a connection obtained between entities is based on their eigenvalues, and has a fault tolerance mechanism to adapt to errors and informational conflicts in the graph data, but needs the graph data as database. The combination of the knowledge graph and the graph neural network can compensate for the deficiency of the single knowledge graph method. We tested the proposed method by using the IEEE 118-bus system and a provincial network system. The results showed that our approach is feasible and highly fault tolerant. It can accurately identify network topology even in the presence of conflicting and missing measurement-related information.

Knowledge Graph Completion (KGC) is a popular topic in knowledge graph construction and related applications, aiming to complete the structure of knowledge graphs by predicting missing entities or relations and mining unknown facts in the knowledge graph. In the KGC task, Graph Neural Network (GNN)-based methods have achieved remarkable results due to their advantage of effectively capturing complex relations among entities and generating more accurate and rich entity representations by aggregating information from neighbouring nodes. These methods mainly focus on the representation of entities, and the representation of relations is obtained using simple dimensional transformations or initial embeddings. This treatment ignores the diversity and complex semantics of relations and restricts the efficiency of the model in utilizing relational information in the reasoning process. In this work, we propose the Relational Representation Augmented Graph Attention Network (RRA-GAT), which effectively identifies and weights neighbouring relations that actually contribute to the target relation by filtering out irrelevant information through an attention function based on the information and spatial domain. Furthermore, we capture complex patterns and features in the relational embedding by means of a feed-forward network consisting of a series of linear transformations and nonlinear activation functions. Experiments demonstrate the very advanced performance of RRA-GAT on the link prediction task on standard datasets FB15k-237 and WN18RR (e.g., improved the MRR metric on the WN18RR dataset by 7.8% relative improvement).

Arabic Knowledge Graph Construction: A close look in the present and into the future

Recent studies have shown that the problems of cold processing and data sparsity in recommender systems can be well alleviated with the help of knowledge graph information. Meanwhile, since the information in recommender systems can be represented by graph data structure, graph neural network has great advantages in recommendatory system. Therefore, graph neural network can better mine the deep relationship between objects in the recommender systems which combines with information from knowledge graph. This paper reviews the application of graph neural network in knowledge graph aided recommendation systems. Specifically, this paper firstly sorts out the related knowledge of knowledge graph, recommendation system and graph neural network, including the cold processing and data sparsity, the structure of knowledge graph, and the basic modules of graph neural network. This paper also summarizes the methods such as KGAT and CKAN used in knowledge graph aided recommendation systems in recent years, and gives some discussion on the development of this field.

With the rapid development of intelligent coal mine, the field of coal mine also ushered in a large number of heterogeneous data information. Aiming at the data information existing in the field of coal mine, a coal mine safety knowledge atlas centered on coal mine accident cases is constructed. The construction of knowledge graph mainly uses crawler technology to obtain text data of coal mine, Albert-BilSTM-CRF model to realize entity recognition, Albert-BilSTM-ATT to complete relation extraction, and obtain triplet form in the field of coal mine safety. Finally, the obtained triplet form is imported into Neo4J graph database. Form a structured knowledge graph.

Since the causes of aviation accidents and risks are complicated, concealed, unpredictable and difficult to be investigated, in order to achieve the efficient organization and knowledge sharing of the historical cases of aviation risk events, this paper put forward the method of constructing vertical knowledge graph for aviation risk field. Firstly, the data-driven incremental construction technology is used to build aviation risk event ontology model. Secondly, the pattern-based knowledge mapping mechanism, which transform structured data into RDF (Resource Description Framework) data for storage, is proposed. And then the application, update and maintenance of the knowledge graph are described. Finally, knowledge graph construction system in aviation risk field is developed; and the data from American Aviation Safety Reporting System (ASRS) is used as an example to verify the rationality and validity of the knowledge graph construction method. Practice has proved that the construction of knowledge graph has a guiding significance for the case information organization and sharing on the field of aviation risk.

With the growing availability of different knowledge graphs in a variety of domains, question answering over knowledge graph (KG-QA) becomes a prevalent information retrieval approach. Current KG-QA methods usually resort to semantic parsing, search or neural matching models. However, they cannot well tackle increasingly long input questions and complex information needs. In this work, we propose a new KG-QA approach, leveraging the rich domain context in the knowledge graph. We incorporate the new approach with question and answer domain context descriptions. Specifically, for questions, we enrich them with users’ subsequent input questions within a session and expand the input question representation. For the candidate answers, we equip them with surrounding context structures, i.e., meta-paths within the targeting knowledge graph. On top of these, we design a cross-attention mechanism to improve the question and answer matching performance. An experimental study on real datasets verifies these improvements. The new approach is especially beneficial for specific knowledge graphs with complex questions.

A review of graph neural networks and pretrained language models for knowledge graph reasoning

Knowledge graph (KG) is one of key technologies for intelligently answering questions, which can reduce customer service’s costs and improve its self-service capabilities. However, the description of questions is often ambiguous, and the operation and maintenance of online KG based QA services introduces a high cost. To address the above issues, this paper proposes a semantic enhancement based dynamic construction of domain knowledge graph for answering questions. We first employ a model combining LSTM and CRF to identify entities, and then propose a semantic enhancement method based on topic comparison to introduce external knowledge. We employ heuristic rules to get optimal answers, and then periodically update the global KG according to the integer linear programming solver’s results. Our approach can achieve a high precise answering results with a low response delay by accurately recognizing entities, automatically mapping domain knowledge to the KG, and online updating the KG. The experimental results show that our approach compared with the traditional method improves the precision, recall and F-measure by 6.41%, 16.46% and 11.17%, respectively.

In recent years, the related research of entity alignment has mainly focused on entity alignment via knowledge embeddings and graph neural networks; however, these proposed models usually suffer from structural heterogeneity and the large-scale problem of knowledge graph. A novel entity alignment model based on graph isomorphic network and compressed sensing is proposed. First, for the problem of structural heterogeneity, graph isomorphic network encoder is applied in knowledge graph to capture structural similarity of entity relation. Second, for the problem of large scale, key node and community are integrated for priority entity alignment to improve execution speed. However, the exiting node importance ranking algorithm cannot accurately identify key node in knowledge graph. So the compressed sensing is adopted in node importance ranking to improve the accuracy of identifying key node. The authors have carried out several experiments to test the effect and efficiency of the proposed entity alignment model.

Due to increased entrepreneurial awareness and policy incentives, the number of enterprises is growing and the relation between enterprises are complex. This poses challenges for enterprise managers and financial practitioners in terms of efficient enterprises supervision and discovery of hidden relation between enterprises. As one of the powerful tools of knowledge engineering, knowledge graph can solve this problem to a certain extent. However, the utilization rate of unstructured data is low in the construction of the enterprise knowledge graph. In addition, in most research work, an aspect of constructing the knowledge graph is emphasized in isolation, the entire processing flow is ignored. In order to solve this problem, this paper uses the deep learning model to propose a process and method for building a Chinese enterprise knowledge graph based on unstructured data on the web, and realizes the application of retrieval and question answering.