Anomalous behavior detection based on optimized graph embedding representation in social networks

Recently, large-scale knowledge graphs (KGs) have become a key asset for search, analytics, recommendations and data integration. Large-scale KGs provide millions of facts about the real world. Each fact is composed as (subject, relation, object), e.g., the triplet (“Cristiano Ronaldo”, playFor, “Real Madrid”). However, these facts are blind to the temporal dimension. Actually, knowledge in practice is time-variant and many relations are only valid for a certain period of time. This phenomenon highlights the importance of building temporal knowledge graphs. In particular, knowledge in temporal KG is represented as (subject, relation, object, valid time), e.g., (“Cristiano Ronaldo”, playFor, “Real Madrid”, “[2009, 2018]”. However, research on temporal KG is very current and there are still many problems needed to be addressed. One obvious problem is that the size of temporal KG is still very small. For example, only 6.6% of the facts are time-aware in one of the largest knowledge graphs, YAGO3. In addition, 71% of people have no known place of birth, and 75% have no known nationality in Freebase. Furthermore, over 87.7% of the facts are uncovered in Japanese DBpedia compared with English DBpedia. Therefore, in this thesis, we study how to enlarge and enrich temporal knowledge graphs from three aspects, namely, temporal KG enrichment. In particular, we study the enrichment problem from the following aspects: (1) volume, (2) completeness, and (3) coverage.Our first solution is temporal knowledge harvesting which extracts temporal knowledge from free text directly. However, text corpus is noisy, and extracting structured temporal facts with high accuracy and coverage is very challenging. Inspired by pattern-based systems, we propose a temporal knowledge harvesting framework. In particular, we propose various techniques to extract temporal patterns, including corpus annotation, pattern generation, scoring and clustering. These techniques can reduce ambiguity in the text corpus and can improve both the accuracy and coverage of the extracted patterns. Second, we leverage the extracted patterns to harvest temporal knowledge. To improve the accuracy, we propose a parse-tree-based method. And to increase the coverage, we consider the relationships between tree components, including part of speech (POS), clause types, constituency and dependency. Experiments on real-world datasets verify the effectiveness of our proposed framework.Our second solution is temporal knowledge graph completion. Temporal KG completion is the task of inferring unobserved edges between entity pairs. Generally, temporal KG completion relies on the temporal KG embeddings technique, which learns the low-dimensional representations of all KG components. As research on temporal KG embedding is very current, we study both the embedding and the completion problems. In particular, we observe that contexts are extremely useful for learning the representations of entities and for inferring the unknown time intervals. As a result, we propose a context-aware embedding model for KG embeddings and a context-based temporal inference model for KG completion. In our embedding model, we not only capture factual plausibility as traditional methods did, but also propose a new measure on contexts, named temporal consistency. It measures how well the target entity interacts with its surrounding contexts on the temporal dimension. Our completion model is based on the embedding model, and further captures the interactions on the entity dimension. Extensive experiments verify the effectiveness of our models.Our third solution is temporal knowledge graph alignment which aims to discover the SameAs edges across two temporal KGs. Not like previous attribute-based alignment models, we further divide attribute facts into character facts, digit facts and time facts. In particular, we observe that the context information is extremely useful for the identification of the same entities. Therefore, we propose an alignment model on leveraging temporal contexts to represent entities. However, contexts of the same object can be very different. For example, the value of career predicate changed from “football player” to “coach” for the football player “Zidane”. This is because entities are evolving over time and thus the predicate values can be different. In our framework, we propose an alignment module that simulates the entity evolving process. Specifically, this module captures the interactions between contexts and aggregates context information to represent the entity. Lastly, we found that not all contexts are relevant, e.g., height v.s. career. Actually, only the relevant contexts are useful for capturing the evolving. Therefore, we propose a clustering approach for grouping relevant contexts together. Our experimental results validate the superiority of our proposed alignment model.

Knowledge graphs’ reasoning is of great significance for the further development of artificial intelligence and information retrieval, especially for reasoning over temporal knowledge graphs. The rotation-based method has been shown to be effective at modeling entities and relations on a knowledge graph. However, due to the lack of temporal information representation capability, existing approaches can only model partial relational patterns and they cannot handle temporal combination reasoning. In this regard, we propose HTTR: Householder Transformation-based Temporal knowledge graph Reasoning, which focuses on the characteristics of relations that evolve over time. HTTR first fuses the relation and temporal information in the knowledge graph, then uses the Householder transformation to obtain an orthogonal matrix about the fused information, and finally defines the orthogonal matrix as the rotation of the head-entity to the tail-entity and calculates the similarity between the rotated vector and the vector representation of the tail entity. In addition, we compare three methods for fusing relational and temporal information. We allow other fusion methods to replace the current one as long as the dimensionality satisfies the requirements. We show that HTTR is able to outperform state-of-the-art methods in temporal knowledge graph reasoning tasks and has the ability to learn and infer all of the four relational patterns over time: symmetric reasoning, antisymmetric reasoning, inversion reasoning, and temporal combination reasoning.

There has been an increasing interest in inferring future links on temporal knowledge graphs (KG). While links on temporal KGs vary continuously over time, the existing approaches model the temporal KGs in discrete state spaces. To this end, we propose a novel continuum model by extending the idea of neural ordinary differential equations (ODEs) to multi-relational graph convolutional networks. The proposed model preserves the continuous nature of dynamic multi-relational graph data and encodes both temporal and structural information into continuous-time dynamic embeddings. In addition, a novel graph transition layer is applied to capture the transitions on the dynamic graph, i.e., edge formation and dissolution. We perform extensive experiments on five benchmark datasets for temporal KG reasoning, showing our model’s superior performance on the future link forecasting task.

Knowledge graph question answering is an important research direction for question answering tasks. In recent years, there has been an increasing amount of research on knowledge graph question answering, but temporal knowledge graph question answering is still a relatively unexplored area. In question answering tasks, many natural language questions have explicit or implicit temporal constraints, and most existing research methods lack the temporal awareness to deal with complex temporal questions. To address these challenges, this paper proposes a question answering model based on temporal knowledge graph embedding (TKGETQA). The model uses TKG embeddings to root a question in the entities, relations and time horizons it references, and perceives temporal information from the question to improve the accuracy of answer prediction. Experiments on the dataset CronQuestions in this paper show that the TKGETQA model exhibits better results compared to existing temporal knowledge graph question answering approaches.

Temporal knowledge graph embedding via sparse transfer matrix

Wireless networks and personalized mobile devices are deeply integrated and embedded in our lives. Such wide adoptions of new technologies will impact user behavior and in turn will affect network performance. It is imperative to characterize the fundamental structure of wireless user behavior in order to model, manage, leverage and design efficient mobile networks. One major challenge in characterizing user behavior stems from the significant size and complexity of user behavioral data. Without summarization and dimension reduction, the sheer amount of data does not provide much useful information. The key contribution of the paper is a novel similarity metric based on a matrix representation of mobility preferences and its decomposition. This method provides an efficient way to reduce important spatiotemporal dynamics in user mobility into a few eigen-behavior vectors. This also facilitates nodes to exchange their mobility summaries and determine their mutual similarity locally. Without any assumption on the properties of user population, we use unsupervised learning (clustering) techniques to classify WLAN users. Such a user grouping scheme based on learned user behavior is crucial for applications relying on the usage context of each mobile device (e.g., participatory sensing, social-relationship-aware message forwarding). In this study, using our systematic TRACE approach, we analyze wireless users' behavioral patterns by extensively mining wireless network logs from two major university campuses to showcase its efficacy. While our findings partly validate intuitive repetitive behavioral trends and user grouping, it is surprising to find the qualitative commonalities and striking consistency of user behavior from the two universities. We discover multimodal user behavior for more than 60 percent of the users, and there are hundreds of distinct groups with unique behavioral patterns in both campuses. The sizes of the major groups follow a power-law distribution.

Knowledge graphs can be constructed by extracting triples from documents, which denotes document-level relation extraction. Each triple illustrates a fact composed of two entities and a relation. However, temporal information corresponding to these facts is ignored. Incorporating temporal information exhibits the temporal connections between facts. Constructing a temporal knowledge graph (TKG) from documents is relatively unexplored. To address this limitation, we built a new dataset for this task based on a document-level relation extraction dataset. We mine the combination relation patterns and construct temporal quadruples by combining facts and timestamps. Additionally, two large language models (LLMs) are adopted to generate quadruples for the rest of the triples without timestamps. Multiple filters and manual annotation are used to ensure the quality of the data. To evaluate the dataset, we propose an LLM-based framework for extracting relations with temporal information from documents. The framework transforms relation extraction to a seq-to-seq task and fine-tunes LLMs to predict the relation with timestamps between entities. Experiments show the performance of LLMs on the proposed dataset.

Traditional Chinese medicine (TCM) is an important intangible cultural heritage of China. To enhance the services of TCM, many works focus on constructing various types of TCM knowledge graphs according to the concrete requirements such as information retrieval. However, most of them ignored several key issues. One is temporal information that is very important for TCM clinical diagnosis and treatment. For example, a herb needs to be boiled for different periods in different prescriptions, but existing methods cannot represent this temporal information very well. The other is that current TCM-based retrieval systems cannot effectively deal with the temporal intentions of search sentences, which leads to bad experiences for users in retrieval services. To solve these issues, we propose a new model tailored for TCM based on the temporal knowledge graph in this paper, which can effectively represent the clinical knowledge changing dynamically over time. Moreover, we implement a temporal semantic search system and employ reasoning rules based on our proposed model to complete the temporal intentions of search sentences. The preliminary result indicates that our system can obtain better results than existing methods in terms of precision.

Temporal Knowledge Graph (TKG) is composed of a series of facts related to timestamps in the real world and has become the basis of many artificial intelligence applications. However, the existing TKG is usually incomplete. It has become a hot research task to infer missing facts based on existing facts in a TKG; namely, Temporal Knowledge Graph Completion (TKGC). The current mainstream TKGC models are embedded models that predict missing facts by representing entities, relations and timestamps as low-dimensional vectors. In order to deal with the TKG structure information, there are some models that try to introduce attention mechanism into the embedding process. But they only consider the structure information of entities or relations, and ignore the structure information of the whole TKG. Moreover, most of them usually treat timestamps as a general feature and cannot take advantage of the potential time series information of the timestamp. To solve these problems, wo propose a new Hierarchical Self-Attention Embedding (HSAE) model which inspired by self-attention mechanism and diachronic embedding technique. For structure information of the whole TKG, we divide the TKG into two layers: entity layer and relation layer, and then apply the self-attention mechanism to the entity layer and relation layer respectively to capture the structure information. For time series information of the timestamp, we capture them by combining positional encoding and diachronic embedding technique into the above two self-attention layers. Finally, we can get the embedded representation vectors of entities, relations and timestamps, which can be combined with other models for better results. We evaluate our model on three TKG datasets: ICEWS14, ICEWS05-15 and GDELT. Experimental results on the TKGC (interpolation) task demonstrate that our model achieves state-of-the-art results.

Since the static Knowledge Graph cannot meet the dynamics of knowledge in the real world, Temporal Knowledge Graph has become a potential method for processing temporal knowledge. However, Temporal Knowledge Graph still faces the problem of incompleteness. Hence knowledge completion has become the main reasoning task for computing the missing facts in Knowledge Graph. In order to capture more richer historical information for predicting future events, we propose a new event forecasting model called Global-Recent Historical Network (GRHNet). In GRHNet, we use statistical methods to simulate the evolution of events. Furthermore, we design a global history learner to capture the repeatability of knowledge, and a recent history learner to capture time-variability of knowledge. GRHNet is evaluated on two benchmark datasets. The results demonstrate that, compared with state-of-the-art baselines, GRHNet achieves at least a 3% relative improvement in MRR.

One of the key requirements to facilitate the semantic analytics of information regarding contemporary and historical events on the Web, in the news and in social media is the availability of reference knowledge repositories containing comprehensive representations of events, entities and temporal relations. Existing knowledge graphs, with popular examples including DBpedia, YAGO and Wikidata, focus mostly on entity-centric information and are insufficient in terms of their coverage and completeness with respect to events and temporal relations. In this article we address this limitation, formalise the concept of a temporal knowledge graph and present its instantiation - EventKG. EventKG is a multilingual event-centric temporal knowledge graph that incorporates over 690 thousand events and over 2.3 million temporal relations obtained from several large-scale knowledge graphs and semi-structured sources and makes them available through a canonical RDF representation. Whereas popular entities often possess hundreds of relations within a temporal knowledge graph such as EventKG, generating a concise overview of the most important temporal relations for a given entity is a challenging task. In this article we demonstrate an application of EventKG to biographical timeline generation, where we adopt a distant supervision method to identify relations most relevant for an entity biography. Our evaluation results provide insights on the characteristics of EventKG and demonstrate the effectiveness of the proposed biographical timeline generation method.

PLEASING: Exploring the historical and potential events for temporal knowledge graph reasoning

Few-shot link prediction for temporal knowledge graphs based on time-aware translation and attention mechanism

Inferring missing facts in temporal knowledge graphs (TKGs) is a fundamental and challenging task. Previous works have approached this problem by augmenting methods for static knowledge graphs to leverage time-dependent representations. However, these methods do not explicitly leverage multi-hop structural information and temporal facts from recent time steps to enhance their predictions. Additionally, prior work does not explicitly address the temporal sparsity and variability of entity distributions in TKGs. We propose the Temporal Message Passing (TeMP) framework to address these challenges by combining graph neural networks, temporal dynamics models, data imputation and frequency-based gating techniques. Experiments on standard TKG tasks show that our approach provides substantial gains compared to the previous state of the art, achieving a 10.7% average relative improvement in Hits@10 across three standard benchmarks. Our analysis also reveals important sources of variability both within and across TKG datasets, and we introduce several simple but strong baselines that outperform the prior state of the art in certain settings.

Temporal Knowledge Graph (TKG) representation learning aims to project entities and relations from high-dimensional spaces into low-dimensional ones while preserving dynamic relational characteristics. However, many existing methods primarily focus on single-time-stamp Knowledge Graphs, neglecting the importance of time in capturing the evolving relationships within TKGs. To address this limitation, we introduce TAR-TKG (Temporal-Aware Representation for Temporal Knowledge Graph), a novel framework that consists of three core modules. The first module, the Temporal Dynamics Steering Module, enhances dynamic temporal features by employing a multi-time-awareness network to capture changes between time stamps, thereby improving the understanding of temporal data evolution. The second module, the Cross-Time Domain Gating Module, applies cross-time domain graph convolution and adaptive gating to learn relationships between different time stamps, facilitating the integration of information across multiple time spans to improve the accuracy of temporal reasoning. The third module, the Temporal Adaptive Relation Perception Module, combines temporal embeddings, causal reasoning, and multi-modal relation fusion to enhance the model’s ability to perceive temporal relationships, particularly in managing causal dependencies and complex time-based interactions. Experimental results demonstrate that TAR-TKG outperforms existing baseline methods on three real-world datasets, proving its effectiveness in capturing dynamic relationship evolution and improving temporal reasoning within TKGs.