TAR-TKG: Temporal-aware representation for temporal knowledge graphs

Recently, large-scale knowledge graphs (KGs) have become a key asset for search, analytics, recommendations and data integration. Large-scale KGs provide millions of facts about the real world. Each fact is composed as (subject, relation, object), e.g., the triplet (“Cristiano Ronaldo”, playFor, “Real Madrid”). However, these facts are blind to the temporal dimension. Actually, knowledge in practice is time-variant and many relations are only valid for a certain period of time. This phenomenon highlights the importance of building temporal knowledge graphs. In particular, knowledge in temporal KG is represented as (subject, relation, object, valid time), e.g., (“Cristiano Ronaldo”, playFor, “Real Madrid”, “[2009, 2018]”. However, research on temporal KG is very current and there are still many problems needed to be addressed. One obvious problem is that the size of temporal KG is still very small. For example, only 6.6% of the facts are time-aware in one of the largest knowledge graphs, YAGO3. In addition, 71% of people have no known place of birth, and 75% have no known nationality in Freebase. Furthermore, over 87.7% of the facts are uncovered in Japanese DBpedia compared with English DBpedia. Therefore, in this thesis, we study how to enlarge and enrich temporal knowledge graphs from three aspects, namely, temporal KG enrichment. In particular, we study the enrichment problem from the following aspects: (1) volume, (2) completeness, and (3) coverage.Our first solution is temporal knowledge harvesting which extracts temporal knowledge from free text directly. However, text corpus is noisy, and extracting structured temporal facts with high accuracy and coverage is very challenging. Inspired by pattern-based systems, we propose a temporal knowledge harvesting framework. In particular, we propose various techniques to extract temporal patterns, including corpus annotation, pattern generation, scoring and clustering. These techniques can reduce ambiguity in the text corpus and can improve both the accuracy and coverage of the extracted patterns. Second, we leverage the extracted patterns to harvest temporal knowledge. To improve the accuracy, we propose a parse-tree-based method. And to increase the coverage, we consider the relationships between tree components, including part of speech (POS), clause types, constituency and dependency. Experiments on real-world datasets verify the effectiveness of our proposed framework.Our second solution is temporal knowledge graph completion. Temporal KG completion is the task of inferring unobserved edges between entity pairs. Generally, temporal KG completion relies on the temporal KG embeddings technique, which learns the low-dimensional representations of all KG components. As research on temporal KG embedding is very current, we study both the embedding and the completion problems. In particular, we observe that contexts are extremely useful for learning the representations of entities and for inferring the unknown time intervals. As a result, we propose a context-aware embedding model for KG embeddings and a context-based temporal inference model for KG completion. In our embedding model, we not only capture factual plausibility as traditional methods did, but also propose a new measure on contexts, named temporal consistency. It measures how well the target entity interacts with its surrounding contexts on the temporal dimension. Our completion model is based on the embedding model, and further captures the interactions on the entity dimension. Extensive experiments verify the effectiveness of our models.Our third solution is temporal knowledge graph alignment which aims to discover the SameAs edges across two temporal KGs. Not like previous attribute-based alignment models, we further divide attribute facts into character facts, digit facts and time facts. In particular, we observe that the context information is extremely useful for the identification of the same entities. Therefore, we propose an alignment model on leveraging temporal contexts to represent entities. However, contexts of the same object can be very different. For example, the value of career predicate changed from “football player” to “coach” for the football player “Zidane”. This is because entities are evolving over time and thus the predicate values can be different. In our framework, we propose an alignment module that simulates the entity evolving process. Specifically, this module captures the interactions between contexts and aggregates context information to represent the entity. Lastly, we found that not all contexts are relevant, e.g., height v.s. career. Actually, only the relevant contexts are useful for capturing the evolving. Therefore, we propose a clustering approach for grouping relevant contexts together. Our experimental results validate the superiority of our proposed alignment model.

Knowledge graphs’ reasoning is of great significance for the further development of artificial intelligence and information retrieval, especially for reasoning over temporal knowledge graphs. The rotation-based method has been shown to be effective at modeling entities and relations on a knowledge graph. However, due to the lack of temporal information representation capability, existing approaches can only model partial relational patterns and they cannot handle temporal combination reasoning. In this regard, we propose HTTR: Householder Transformation-based Temporal knowledge graph Reasoning, which focuses on the characteristics of relations that evolve over time. HTTR first fuses the relation and temporal information in the knowledge graph, then uses the Householder transformation to obtain an orthogonal matrix about the fused information, and finally defines the orthogonal matrix as the rotation of the head-entity to the tail-entity and calculates the similarity between the rotated vector and the vector representation of the tail entity. In addition, we compare three methods for fusing relational and temporal information. We allow other fusion methods to replace the current one as long as the dimensionality satisfies the requirements. We show that HTTR is able to outperform state-of-the-art methods in temporal knowledge graph reasoning tasks and has the ability to learn and infer all of the four relational patterns over time: symmetric reasoning, antisymmetric reasoning, inversion reasoning, and temporal combination reasoning.

FL-Evo: Jointly modeling fact and logic evolution patterns for temporal knowledge graph reasoning

Temporal knowledge graph embedding via sparse transfer matrix

There has been an increasing interest in inferring future links on temporal knowledge graphs (KG). While links on temporal KGs vary continuously over time, the existing approaches model the temporal KGs in discrete state spaces. To this end, we propose a novel continuum model by extending the idea of neural ordinary differential equations (ODEs) to multi-relational graph convolutional networks. The proposed model preserves the continuous nature of dynamic multi-relational graph data and encodes both temporal and structural information into continuous-time dynamic embeddings. In addition, a novel graph transition layer is applied to capture the transitions on the dynamic graph, i.e., edge formation and dissolution. We perform extensive experiments on five benchmark datasets for temporal KG reasoning, showing our model’s superior performance on the future link forecasting task.

One of the key requirements to facilitate the semantic analytics of information regarding contemporary and historical events on the Web, in the news and in social media is the availability of reference knowledge repositories containing comprehensive representations of events, entities and temporal relations. Existing knowledge graphs, with popular examples including DBpedia, YAGO and Wikidata, focus mostly on entity-centric information and are insufficient in terms of their coverage and completeness with respect to events and temporal relations. In this article we address this limitation, formalise the concept of a temporal knowledge graph and present its instantiation - EventKG. EventKG is a multilingual event-centric temporal knowledge graph that incorporates over 690 thousand events and over 2.3 million temporal relations obtained from several large-scale knowledge graphs and semi-structured sources and makes them available through a canonical RDF representation. Whereas popular entities often possess hundreds of relations within a temporal knowledge graph such as EventKG, generating a concise overview of the most important temporal relations for a given entity is a challenging task. In this article we demonstrate an application of EventKG to biographical timeline generation, where we adopt a distant supervision method to identify relations most relevant for an entity biography. Our evaluation results provide insights on the characteristics of EventKG and demonstrate the effectiveness of the proposed biographical timeline generation method.

Historical Trends and Normalizing Flow for One-shot Temporal Knowledge Graph Reasoning

Temporal knowledge graphs (TKGs) have been widely used in various fields, and predicting missing knowledge graph inference has been widely explored. However, the task of reasoning about potential future facts on TKGs is more challenging and has attracted the attention of researchers. As the unknowability of future events complicates inference, a thorough study of the characteristics of historical facts becomes crucial. The study of the concurrency of historical events and the underlying common patterns of relation-ships facilitates reasoning about future facts. In this paper, we propose a novel representation learning model based on Short-Term Sequential Patterns for TKG reasoning, namely STSP. By modeling TKG sequences recurrently and learning representations of entities and relations. Specifically, the STSP encoder uses three main modules. Concurrent facts for each timestamp are modeled using a convolution-based relation-aware GCN. The entity-aware attention module is used to integrate the entity representation of the current timestamp and the previous timestamp. The sliding window mechanism is used to learn different relations sequentially. The entity and relation representations are then handed over to a translation-based decoder for final reasoning. We use four benchmark datasets to evaluate the proposed approach. The experimental results show that STSP out-performs state-of-the-art TKG reasoning methods and obtains substantial performance.

Question answering over temporal knowledge graphs (TKGQA) is an emerging topic, which has attracted increasing interest since it considers the dynamic knowledge in the world.Several datasets along with model developments are proposed in the TKGQA research field.However, existing studies generally focus on fact-centered reasoning, with limited attention to temporal reasoning.To tackle the intricate and comprehensive nature of temporal reasoning, we propose a new TKGQA dataset, MusTQ, which contains 666K multistep temporal reasoning questions as well as a TKG.The multi-step temporal reasoning is established based on six basic temporal reasoning types derived from a well-established measure theory.Using MusTQ, we evaluate previous TKGQA methods and find that they typically fall short in multi-step temporal reasoning.Furthermore, we propose a TKGQA model, MusTKGQA, which enhances multistep reasoning ability with entity-time attention mechanism and optimized temporal knowledge graph representation.Extensive experiments on MusTQ show that our model achieves stateof-the-art multi-step temporal reasoning performance.

Temporal Knowledge Graph (TKG) representation learning embeds entities and relations into a low-dimensional space while preserving relational structures across time steps. Existing methods often neglect the critical role of timestamps in capturing evolving relational patterns. To bridge this gap, we propose MTT-TKG, a novel framework integrating three synergistic modules: (1) a Multi-Time Gate module modeling Knowledge Graph (KG) evolution across historical timestamps via multilayer gating; (2) a Time-Aware module capturing timestampspecific relational characteristics; (3) a Time-Guided module handling cross-graph temporal dependencies. An embeddingtime decoder completes the representation learning. Experiments on three real-world datasets demonstrate MTT-TKG’s superior performance in capturing temporal dynamics and relational structures.

Temporal Knowledge Graph (TKG) is composed of a series of facts related to timestamps in the real world and has become the basis of many artificial intelligence applications. However, the existing TKG is usually incomplete. It has become a hot research task to infer missing facts based on existing facts in a TKG; namely, Temporal Knowledge Graph Completion (TKGC). The current mainstream TKGC models are embedded models that predict missing facts by representing entities, relations and timestamps as low-dimensional vectors. In order to deal with the TKG structure information, there are some models that try to introduce attention mechanism into the embedding process. But they only consider the structure information of entities or relations, and ignore the structure information of the whole TKG. Moreover, most of them usually treat timestamps as a general feature and cannot take advantage of the potential time series information of the timestamp. To solve these problems, wo propose a new Hierarchical Self-Attention Embedding (HSAE) model which inspired by self-attention mechanism and diachronic embedding technique. For structure information of the whole TKG, we divide the TKG into two layers: entity layer and relation layer, and then apply the self-attention mechanism to the entity layer and relation layer respectively to capture the structure information. For time series information of the timestamp, we capture them by combining positional encoding and diachronic embedding technique into the above two self-attention layers. Finally, we can get the embedded representation vectors of entities, relations and timestamps, which can be combined with other models for better results. We evaluate our model on three TKG datasets: ICEWS14, ICEWS05-15 and GDELT. Experimental results on the TKGC (interpolation) task demonstrate that our model achieves state-of-the-art results.

PLEASING: Exploring the historical and potential events for temporal knowledge graph reasoning

Knowledge graph question answering is an important research direction for question answering tasks. In recent years, there has been an increasing amount of research on knowledge graph question answering, but temporal knowledge graph question answering is still a relatively unexplored area. In question answering tasks, many natural language questions have explicit or implicit temporal constraints, and most existing research methods lack the temporal awareness to deal with complex temporal questions. To address these challenges, this paper proposes a question answering model based on temporal knowledge graph embedding (TKGETQA). The model uses TKG embeddings to root a question in the entities, relations and time horizons it references, and perceives temporal information from the question to improve the accuracy of answer prediction. Experiments on the dataset CronQuestions in this paper show that the TKGETQA model exhibits better results compared to existing temporal knowledge graph question answering approaches.

Since the static Knowledge Graph cannot meet the dynamics of knowledge in the real world, Temporal Knowledge Graph has become a potential method for processing temporal knowledge. However, Temporal Knowledge Graph still faces the problem of incompleteness. Hence knowledge completion has become the main reasoning task for computing the missing facts in Knowledge Graph. In order to capture more richer historical information for predicting future events, we propose a new event forecasting model called Global-Recent Historical Network (GRHNet). In GRHNet, we use statistical methods to simulate the evolution of events. Furthermore, we design a global history learner to capture the repeatability of knowledge, and a recent history learner to capture time-variability of knowledge. GRHNet is evaluated on two benchmark datasets. The results demonstrate that, compared with state-of-the-art baselines, GRHNet achieves at least a 3% relative improvement in MRR.

Few-shot link prediction for temporal knowledge graphs based on time-aware translation and attention mechanism