Householder Transformation-Based Temporal Knowledge Graph Reasoning

Recently, large-scale knowledge graphs (KGs) have become a key asset for search, analytics, recommendations and data integration. Large-scale KGs provide millions of facts about the real world. Each fact is composed as (subject, relation, object), e.g., the triplet (“Cristiano Ronaldo”, playFor, “Real Madrid”). However, these facts are blind to the temporal dimension. Actually, knowledge in practice is time-variant and many relations are only valid for a certain period of time. This phenomenon highlights the importance of building temporal knowledge graphs. In particular, knowledge in temporal KG is represented as (subject, relation, object, valid time), e.g., (“Cristiano Ronaldo”, playFor, “Real Madrid”, “[2009, 2018]”. However, research on temporal KG is very current and there are still many problems needed to be addressed. One obvious problem is that the size of temporal KG is still very small. For example, only 6.6% of the facts are time-aware in one of the largest knowledge graphs, YAGO3. In addition, 71% of people have no known place of birth, and 75% have no known nationality in Freebase. Furthermore, over 87.7% of the facts are uncovered in Japanese DBpedia compared with English DBpedia. Therefore, in this thesis, we study how to enlarge and enrich temporal knowledge graphs from three aspects, namely, temporal KG enrichment. In particular, we study the enrichment problem from the following aspects: (1) volume, (2) completeness, and (3) coverage.Our first solution is temporal knowledge harvesting which extracts temporal knowledge from free text directly. However, text corpus is noisy, and extracting structured temporal facts with high accuracy and coverage is very challenging. Inspired by pattern-based systems, we propose a temporal knowledge harvesting framework. In particular, we propose various techniques to extract temporal patterns, including corpus annotation, pattern generation, scoring and clustering. These techniques can reduce ambiguity in the text corpus and can improve both the accuracy and coverage of the extracted patterns. Second, we leverage the extracted patterns to harvest temporal knowledge. To improve the accuracy, we propose a parse-tree-based method. And to increase the coverage, we consider the relationships between tree components, including part of speech (POS), clause types, constituency and dependency. Experiments on real-world datasets verify the effectiveness of our proposed framework.Our second solution is temporal knowledge graph completion. Temporal KG completion is the task of inferring unobserved edges between entity pairs. Generally, temporal KG completion relies on the temporal KG embeddings technique, which learns the low-dimensional representations of all KG components. As research on temporal KG embedding is very current, we study both the embedding and the completion problems. In particular, we observe that contexts are extremely useful for learning the representations of entities and for inferring the unknown time intervals. As a result, we propose a context-aware embedding model for KG embeddings and a context-based temporal inference model for KG completion. In our embedding model, we not only capture factual plausibility as traditional methods did, but also propose a new measure on contexts, named temporal consistency. It measures how well the target entity interacts with its surrounding contexts on the temporal dimension. Our completion model is based on the embedding model, and further captures the interactions on the entity dimension. Extensive experiments verify the effectiveness of our models.Our third solution is temporal knowledge graph alignment which aims to discover the SameAs edges across two temporal KGs. Not like previous attribute-based alignment models, we further divide attribute facts into character facts, digit facts and time facts. In particular, we observe that the context information is extremely useful for the identification of the same entities. Therefore, we propose an alignment model on leveraging temporal contexts to represent entities. However, contexts of the same object can be very different. For example, the value of career predicate changed from “football player” to “coach” for the football player “Zidane”. This is because entities are evolving over time and thus the predicate values can be different. In our framework, we propose an alignment module that simulates the entity evolving process. Specifically, this module captures the interactions between contexts and aggregates context information to represent the entity. Lastly, we found that not all contexts are relevant, e.g., height v.s. career. Actually, only the relevant contexts are useful for capturing the evolving. Therefore, we propose a clustering approach for grouping relevant contexts together. Our experimental results validate the superiority of our proposed alignment model.

There has been an increasing interest in inferring future links on temporal knowledge graphs (KG). While links on temporal KGs vary continuously over time, the existing approaches model the temporal KGs in discrete state spaces. To this end, we propose a novel continuum model by extending the idea of neural ordinary differential equations (ODEs) to multi-relational graph convolutional networks. The proposed model preserves the continuous nature of dynamic multi-relational graph data and encodes both temporal and structural information into continuous-time dynamic embeddings. In addition, a novel graph transition layer is applied to capture the transitions on the dynamic graph, i.e., edge formation and dissolution. We perform extensive experiments on five benchmark datasets for temporal KG reasoning, showing our model’s superior performance on the future link forecasting task.

Temporal Knowledge Graph (TKG) representation learning aims to project entities and relations from high-dimensional spaces into low-dimensional ones while preserving dynamic relational characteristics. However, many existing methods primarily focus on single-time-stamp Knowledge Graphs, neglecting the importance of time in capturing the evolving relationships within TKGs. To address this limitation, we introduce TAR-TKG (Temporal-Aware Representation for Temporal Knowledge Graph), a novel framework that consists of three core modules. The first module, the Temporal Dynamics Steering Module, enhances dynamic temporal features by employing a multi-time-awareness network to capture changes between time stamps, thereby improving the understanding of temporal data evolution. The second module, the Cross-Time Domain Gating Module, applies cross-time domain graph convolution and adaptive gating to learn relationships between different time stamps, facilitating the integration of information across multiple time spans to improve the accuracy of temporal reasoning. The third module, the Temporal Adaptive Relation Perception Module, combines temporal embeddings, causal reasoning, and multi-modal relation fusion to enhance the model’s ability to perceive temporal relationships, particularly in managing causal dependencies and complex time-based interactions. Experimental results demonstrate that TAR-TKG outperforms existing baseline methods on three real-world datasets, proving its effectiveness in capturing dynamic relationship evolution and improving temporal reasoning within TKGs.

Historical Trends and Normalizing Flow for One-shot Temporal Knowledge Graph Reasoning

FL-Evo: Jointly modeling fact and logic evolution patterns for temporal knowledge graph reasoning

PLEASING: Exploring the historical and potential events for temporal knowledge graph reasoning

Temporal knowledge graphs (TKGs) have been widely used in various fields, and predicting missing knowledge graph inference has been widely explored. However, the task of reasoning about potential future facts on TKGs is more challenging and has attracted the attention of researchers. As the unknowability of future events complicates inference, a thorough study of the characteristics of historical facts becomes crucial. The study of the concurrency of historical events and the underlying common patterns of relation-ships facilitates reasoning about future facts. In this paper, we propose a novel representation learning model based on Short-Term Sequential Patterns for TKG reasoning, namely STSP. By modeling TKG sequences recurrently and learning representations of entities and relations. Specifically, the STSP encoder uses three main modules. Concurrent facts for each timestamp are modeled using a convolution-based relation-aware GCN. The entity-aware attention module is used to integrate the entity representation of the current timestamp and the previous timestamp. The sliding window mechanism is used to learn different relations sequentially. The entity and relation representations are then handed over to a translation-based decoder for final reasoning. We use four benchmark datasets to evaluate the proposed approach. The experimental results show that STSP out-performs state-of-the-art TKG reasoning methods and obtains substantial performance.

Traditional Chinese medicine (TCM) is an important intangible cultural heritage of China. To enhance the services of TCM, many works focus on constructing various types of TCM knowledge graphs according to the concrete requirements such as information retrieval. However, most of them ignored several key issues. One is temporal information that is very important for TCM clinical diagnosis and treatment. For example, a herb needs to be boiled for different periods in different prescriptions, but existing methods cannot represent this temporal information very well. The other is that current TCM-based retrieval systems cannot effectively deal with the temporal intentions of search sentences, which leads to bad experiences for users in retrieval services. To solve these issues, we propose a new model tailored for TCM based on the temporal knowledge graph in this paper, which can effectively represent the clinical knowledge changing dynamically over time. Moreover, we implement a temporal semantic search system and employ reasoning rules based on our proposed model to complete the temporal intentions of search sentences. The preliminary result indicates that our system can obtain better results than existing methods in terms of precision.

Knowledge graph question answering is an important research direction for question answering tasks. In recent years, there has been an increasing amount of research on knowledge graph question answering, but temporal knowledge graph question answering is still a relatively unexplored area. In question answering tasks, many natural language questions have explicit or implicit temporal constraints, and most existing research methods lack the temporal awareness to deal with complex temporal questions. To address these challenges, this paper proposes a question answering model based on temporal knowledge graph embedding (TKGETQA). The model uses TKG embeddings to root a question in the entities, relations and time horizons it references, and perceives temporal information from the question to improve the accuracy of answer prediction. Experiments on the dataset CronQuestions in this paper show that the TKGETQA model exhibits better results compared to existing temporal knowledge graph question answering approaches.

Temporal knowledge graph embedding via sparse transfer matrix

Knowledge graphs can be constructed by extracting triples from documents, which denotes document-level relation extraction. Each triple illustrates a fact composed of two entities and a relation. However, temporal information corresponding to these facts is ignored. Incorporating temporal information exhibits the temporal connections between facts. Constructing a temporal knowledge graph (TKG) from documents is relatively unexplored. To address this limitation, we built a new dataset for this task based on a document-level relation extraction dataset. We mine the combination relation patterns and construct temporal quadruples by combining facts and timestamps. Additionally, two large language models (LLMs) are adopted to generate quadruples for the rest of the triples without timestamps. Multiple filters and manual annotation are used to ensure the quality of the data. To evaluate the dataset, we propose an LLM-based framework for extracting relations with temporal information from documents. The framework transforms relation extraction to a seq-to-seq task and fine-tunes LLMs to predict the relation with timestamps between entities. Experiments show the performance of LLMs on the proposed dataset.

Anomalous behaviors in social networks can lead to privacy leaks and the spread of false information. In this paper, we propose an anomalous behavior detection method based on optimized graph embedding representation. Specifically, the user behavior logs are first extracted into a social network user behavior temporal knowledge graph, based on which the graph embedding representation method is used to transform the network topology and temporal information in the user behavior knowledge graph into structural embedding vectors and temporal information embedding vectors, and the hybrid attention mechanism is used to merge the two types of vectors to obtain the final entity embedding to complete the prediction and complementation of the temporal knowledge graph of user behavior. We use graph neural networks, which use the temporal information of user behaviors as a time constraint and capture both user behavioral and semantic information. It converts the two parts of information into vectors for concatenation and linear transformation to obtain a comprehensive representation vector of the whole subgraph, as well as joint deep learning models to evaluate abnormal behavior. Finally, we perform experiments on the Yelp dataset to validate that our method achieves a 9.56% improvement in the F1-score.

Recent years, there is a growing interest in knowledge graph embedding (KGE), which maps symbolic entities and relations into low-dimensional vector space to effectively represent structured data from the knowledge graph. In addition, the concept of temporal knowledge graph is proposed to document dynamically changing facts in the real world. Existing works attempt to incorporate temporal information into static KGE methods to accomplish temporal knowledge representations. However, existing static or temporal KGE approaches focus on the single query fact and ignore the query-relevant contextual information in the graph structure. This paper moves beyond the traditional way of scoring facts in distinct vector space and proposes a unified framework with pre-trained language models (PLM) to learn dynamic <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">co</b>ntextualized <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">s</b>tatic/<bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">t</b>emporal <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">k</b>nowledge <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">g</b>raph <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">e</b>mbeddings, called CoS/TKGE. Given the query-specific subgraph, our model transforms it into an input sequence and uses the PLM to obtain the contextualized knowledge representations, which is flexible adaptive to the input graph contexts. We reformulate the link prediction task as a mask prediction problem to fine-tune the pre-trained language model. And the contrastive learning technique is employed to align dynamic contextual embeddings with static global embeddings. Experimental results on three widely used static and temporal KG datasets show the superiority of our model.

Since the static Knowledge Graph cannot meet the dynamics of knowledge in the real world, Temporal Knowledge Graph has become a potential method for processing temporal knowledge. However, Temporal Knowledge Graph still faces the problem of incompleteness. Hence knowledge completion has become the main reasoning task for computing the missing facts in Knowledge Graph. In order to capture more richer historical information for predicting future events, we propose a new event forecasting model called Global-Recent Historical Network (GRHNet). In GRHNet, we use statistical methods to simulate the evolution of events. Furthermore, we design a global history learner to capture the repeatability of knowledge, and a recent history learner to capture time-variability of knowledge. GRHNet is evaluated on two benchmark datasets. The results demonstrate that, compared with state-of-the-art baselines, GRHNet achieves at least a 3% relative improvement in MRR.

One of the key requirements to facilitate the semantic analytics of information regarding contemporary and historical events on the Web, in the news and in social media is the availability of reference knowledge repositories containing comprehensive representations of events, entities and temporal relations. Existing knowledge graphs, with popular examples including DBpedia, YAGO and Wikidata, focus mostly on entity-centric information and are insufficient in terms of their coverage and completeness with respect to events and temporal relations. In this article we address this limitation, formalise the concept of a temporal knowledge graph and present its instantiation - EventKG. EventKG is a multilingual event-centric temporal knowledge graph that incorporates over 690 thousand events and over 2.3 million temporal relations obtained from several large-scale knowledge graphs and semi-structured sources and makes them available through a canonical RDF representation. Whereas popular entities often possess hundreds of relations within a temporal knowledge graph such as EventKG, generating a concise overview of the most important temporal relations for a given entity is a challenging task. In this article we demonstrate an application of EventKG to biographical timeline generation, where we adopt a distant supervision method to identify relations most relevant for an entity biography. Our evaluation results provide insights on the characteristics of EventKG and demonstrate the effectiveness of the proposed biographical timeline generation method.