Transformer-Style Relational Reasoning with Dynamic Memory Updating for Temporal Network Modeling

Network representation learning (NRL) maps vertices into latent vector space for further network inference. The existing algorithms concern more about whether the vectors of two similar nodes be close in latent vector space while the hierarchy proximity has been largely neglected by them. The distribution of the representation vectors needs to reflect the hierarchical structural properties which widely exist in networks. In this paper, we propose a novel network representation learning framework that can encode the interpretable hierarchical structural semantics into the representation vectors. Specifically, we measure the distance and importance degree of nodes in the original network and map the nodes to a tree space. This makes the hierarchical structural relations in the original network be clearly revealed by the tree which is also of good interpretability. In this paper, the local structural proximities and the interpretable hierarchy knowledge are encoded into vector space by optimizing the objective function. Extensive experiments conducted on the realistic data sets demonstrate that the proposed approach outperforms the existing state-of-the-art approaches on tasks of node classification, link prediction, and visualization. Finally, a case study is conducted for further analysis about how the proposed model works.

Network embedding methods have obtained great progresses on many tasks, such as node classification and link prediction. Sampling strategy is very important in network embedding. It is still a challenge for sampling in a network with complicated topology structure. In this paper, we propose a high-order Markov chain Sampling strategy for Network Embedding (MSNE). MSNE selects the next sampled node based on a distance metric between nodes. Due to high-order sampling, it can exploit the whole sampled path to capture network properties and generate expressive node sequences which are beneficial for downstream tasks. We conduct the experiments on several benchmark datasets. The results show that our model can achieve substantial improvements in two tasks of node classification and link prediction. (Datasets and code are available at https://github.com/SongY123/MSNE.)

Real-world networked systems often show dynamic properties with continuously evolving network nodes and topology over time. When learning from dynamic networks, it is beneficial to correlate all temporal networks to fully capture the similarity/relevance between nodes. Recent work for dynamic network representation learning typically trains each single network independently and imposes relevance regularization on the network learning at different time steps. Such a snapshot scheme fails to leverage topology similarity between temporal networks for progressive training. In addition to the static node relationships within each network, nodes could show similar variation patterns (e.g., change of local structures) within the temporal network sequence. Both static node structures and temporal variation patterns can be combined to better characterize node affinities for unified embedding learning. In this paper, we propose Graph Attention Evolving Networks (GAEN) for dynamic network embedding with preserved similarities between nodes derived from their temporal variation patterns. Instead of training graph attention weights for each network independently, we allow model weights to share and evolve across all temporal networks based on their respective topology discrepancies. Experiments and validations, on four real-world dynamic graphs, demonstrate that GAEN outperforms the state-of-the-art in both link prediction and node classification tasks.

We propose a novel unsupervised node embedding method leveraging the free energy distance. More precisely, we build a matrix to encode pairwise node similarities based on the free energy distance, which interpolates between the shortest path distance and the commute time distance, thus providing an additional degree of flexibility. Moreover, we propose a method to factorize the constructed similarity matrix by generalizing the loss function of the skip-gram model with negative sampling to arbitrary similarity matrices. The proposed matrix factorization method can be easily implemented using advanced automatic differentiation toolkits and computed efficiently by leveraging GPU resources. Numerical experiments on four real-world datasets demonstrate that the proposed embedding method outperforms five state-of-the-art alternatives in tasks of node classification and link prediction.

In this paper, we propose a model using a graph convolution network for predicting the accuracy of the automatically generated convolution neural network (CNN). In recent years, there have been many excellent results in the automatic designing of deep learning network structures and their hyperparameters. These works mainly use evolutionary algorithms and reinforcement learning methods. But to verify the accuracy of newly generated neural network samples, it requires a lot of computational time and large scale of hardware. Therefore, how to reduce training time and improve the utilization of samples has become an important problem of automatic design for deep learning networks. On the other hand, the effectiveness of the graph convolution network (GCN) has been shown in processing graph data on the tasks of graph node classification, link prediction, and clustering. We are considering using GCN to predict the accuracy of the automatically designed neural networks. We compare GCN, a multi-layer neural network(NN), and CNN on the accuracy prediction task of automatically generated CNN for image classification tasks on the CIFAR-10 dataset. Our experiment demonstrates the advantages of GCN on small datasets compared to the multi-layer neural network and CNN, while GCN achieves the equivalent accuracy to the other methods on large datasets.

Network representation learning is receiving increasing attention from scholars. Among them, methods based on graph neural networks have become particularly popular. However, most existing methods currently only focus on networks with a single type of relations. In the real world, networks contain a wealth of diverse information, with multiple types of relationships between nodes. In this paper, we propose a graph neural network-based multiplex network representation learning model (GNMRL). We model nodes within each layer of the multiplex network by aggregating neighbor information. Additionally, since nodes are influenced by different network layers, we integrate node interaction information across network layers. We conducted systematic experiments on four datasets, and the results show that GNMRL outperforms other comparison methods in both link prediction and node classification tasks.

Heterogeneous information network (HIN) contains multiple types of entities and relations. Most of existing HIN embedding methods learn the semantic information based on the heterogeneous structures between different entities, which are implicitly assumed to be complete. However, in real world, it is common that some relations are partially observed due to privacy or other reasons, resulting in a sparse network, in which the structure may be incomplete, and the "unseen" links may also be positive due to the missing relations in data collection. To address this problem, we propose a novel and principled approach: a Multi-View Adversarial Completion Model (MV-ACM). Each relation space is characterized in a single viewpoint, enabling us to use the topological structural information in each view. Based on the multi-view architecture, an adversarial learning process is utilized to learn the reciprocity (i.e., complementary information) between different relations: In the generator, MV-ACM generates the complementary views by computing the similarity of the semantic representation of the same node in different views; while in the discriminator, MV-ACM discriminates whether the view is complementary by the topological structural similarity. Then we update the node's semantic representation by aggregating neighborhoods information from the syncretic views. We conduct systematical experiments1 on six real-world networks from varied domains: AMiner, PPI, YouTube, Twitter, Amazon and Alibaba. Empirical results show that MV-ACM significantly outperforms the state-of-the-art approaches for both link prediction and node classification tasks.

Recently, representation learning on temporal graphs has drawn increasing attention, which aims at learning temporal patterns to characterize the evolving nature of dynamic graphs in real-world applications. Despite effectiveness, these methods commonly ignore the individual- and combinatorial-level patterns derived from different types of interactions (e.g.,user-item), which are at the heart of the representation learning on temporal graphs. To fill this gap, we propose MERIT, a novel multi-level graph attention network for inductive representation learning on temporal graphs.We adaptively embed the original timestamps to a higher, continuous dimensional space for learn-ing individual-level periodicity through Personalized Time Encoding (PTE) module. Furthermore, we equip MERIT with Continuous time and Con-text aware Attention (Coco-Attention) mechanism which chronologically locates most relevant neighbors by jointly capturing multi-level context on temporal graphs. Finally, MERIT performs multiple aggregations and propagations to explore and exploit high-order structural information for down-stream tasks. Extensive experiments on four public datasets demonstrate the effectiveness of MERITon both (inductive / transductive) link prediction and node classification task.

Many link prediction methods have been developed to infer unobserved links or predict missing links based on the observed network structure that is always incomplete and subject to interfering noise. Thus, the performance of existing methods is usually limited in that their computation depends only on input graph structures, and they do not consider external information. The effects of social influence and homophily suggest that both network structure and node attribute information should help to resolve the task of link prediction. This work proposes SASNMF, a link prediction unified framework based on non-negative matrix factorization that considers not only graph structure but also the internal and external auxiliary information, which refers to both the node attributes and the structural latent feature information extracted from the network. Furthermore, three different combinations of internal and external information are proposed and input into the framework to solve the link prediction problem. Extensive experimental results on thirteen real networks, five node attribute networks and eight non-attribute networks show that the proposed framework has competitive performance compared with benchmark methods and state-of-the-art methods, indicating the superiority of the presented algorithm.

Both the metaverse and its underlying blockchain technology have attracted extensive attention in the past few years. It becomes a natural problem to extract, process, and analyze the tremendous data generated by the blockchain systems for various metaverse applications though it also poses diverse challenges. Amongst those challenges, this paper mainly focuses on modeling and understanding the blockchain transaction network from a structural identity perspective, which represents the entire network structure and reveals the relations among multiple entities. In this paper, we propose a novel representation learning method named Structure-to-Vector with Random Pace (SVRP) for learning both latent representation and structural identity of blockchain transaction networks. We then conduct node classification and link prediction tasks with integration with Graph Neural Networks (GNNs). Empirical results on three representative blockchain data sets, namely Non-fungible token (NFT), Ethereum (ETH), and Bitcoin (BTC), demonstrate that our proposed SVRP outperforms other existing methods in multiple tasks. In particular, our SVRP achieves the highest node classification accuracy (99.3%) while only requiring original non-attributed graphs (i.e., graphs without node features).

In recent years, network embedding has attracted much attention from researchers and achieved excellent performance. But few works investigate the adaptability of network embedding, especially for performance in different network structures. Heterogeneity, as a universal topological characteristic, plays a prominent role in network behaviors. In this study, we investigate the effect of heterogeneity on the effectiveness of existing network embedding approaches. We conduct experiments in scale-free networks with varying power exponents from both macro and micro perspectives to address link prediction and node similarity tasks, respectively. The results indicate that network embedding approaches can be divided into two classes according to their performance in the link prediction task. As the network heterogeneity decreases, the performance of approaches in the first class declines, while the performance of approaches in the second class initially improves and then declines. Moreover, our simulation discovers that, based on the node similarity metric, nodes are partitioned into two clusters by approaches, corresponding to large-degree nodes and small-degree nodes, respectively. Furthermore, approaches in the same class present similar characteristics between large-degree nodes and small-degree nodes, and the embedding is interpreted to some extent. Performance variations in the link prediction task can be explained by the characteristics of approaches, and similar characteristics are confirmed in experiments on real networks. Based on the findings for link prediction, we offer a brief guide for choosing an appropriate method based on the extent of heterogeneity. The investigation provides insight into network embedding and offers some interpretation of embedding.

Node Embedding, which uses low-dimensional non-linear feature vectors to represent nodes in the network, has shown a great promise, not only because it is easy-to-use for downstream tasks, but also because it has achieved great success on many network analysis tasks. One of the challenges has been how to develop a node embedding method for integrating topological information from multiple networks. To address this critical problem, we propose a novel node embedding, called DeepMNE, for multi-network integration using a deep semi-supervised autoencoder. The key point of DeepMNE is that it captures complex topological structures of multiple networks and utilizes correlation among multiple networks as constraints. We evaluate DeepMNE in node classification task and link prediction task on four real-world datasets. The experimental results demonstrate that DeepMNE shows superior performance over seven state-of-the-art single-network and multi-network embedding algorithms.

The random walk process on network data is a widely-used approach for network representation learning. However, we argue that the sampling of node sequences and the subsampling for the Skip-gram’s contexts have two drawbacks. One is less possible to precisely find the most correlated context nodes for every central node with only uniform graph search. The other is not easily controlled due to the expensive cost of hyperparameter tuning. Such two drawbacks lead to higher training cost and lower accuracy due to abundant and irrelevant samples. To solve these problems, we compute the adaptive probability of random walk based on Personalized PageRank (PPR), and propose an <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Adaptive SKip-gram</i> (ASK) model without using complicated sampling process and negative sampling. We utilize <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$k$ </tex-math></inline-formula> -most important neighbors for positive samples selection, and attach their corresponding PPR probability into the objective function. Based on benchmark datasets with three citation networks and three social networks, we demonstrate the improvement of our ASK model for network representation learning in tasks of link prediction, node classification, and embedding visualization. The results achieve more effective performance and efficient learning time.

Link prediction is a key tool for studying the structure and evolution mechanism of complex networks. Recommending new friend relationships through accurate link prediction is one of the important factors in the evolution, development, and popularization of social networks. At present, scholars have proposed many link prediction algorithms based on the similarity of local information and random walks. These algorithms help identify actual missing and false links in various networks. However, the prediction results significantly differ in networks with various structures, and the prediction accuracy is low. This study proposes a method for improving the accuracy of link prediction. Before link prediction, k‐shell decomposition method is used to layer the network, and the nodes that are in 1‐shell and the nodes that are not linked to the high‐shell in the 2‐shell are deleted. The experiments on four real network datasets verify the effectiveness of the proposed method.

Dynamic network link prediction is a hot research problem in complex networks. Network representation learning can effectively learn the similarity of nodes and can be used for link prediction. Existing dynamic network representation learning methods mainly discrete window dynamic networks and then model them on static network snapshot graphs, which is difficult to effectively deal with dynamic networks with fine-grained temporal characteristics. In this paper, we propose a link prediction model that can learn complex temporal properties in dynamic networks. The model uses continuous time event sequences to represent dynamic networks, learns continuous temporal information and structural evolution features in the networks, and proposes a temporal attention-based information transfer mechanism to model the diffusion and aggregation of information in the networks. Finally, it transforms link prediction into a classification problem. The experiments are conducted on four real dynamic network datasets and simulated networks, using ap and auc as evaluation metrics. The experimental results of real networks demonstrate that the model can effectively learn the continuity of network evolution and obtain an effective node representation, thus improving the link prediction effect. The experimental results of the simulated network show that the effect of link prediction is related to the network model, but the model in this paper can still obtain better prediction results.