REFINE

Network embedding is to learn effective low-dimensional vector representations for nodes in a network and has attracted considerable attention in recent years. To date, existing methods mainly focus on network structure information and cannot leverage abundant label information, which is potentially valuable in learning better vector representations. Due to the noise and incompleteness of label information, it is intractable to integrate label information into the vector representations in a partially labeled network. To address this issue, we investigate the effects of label information based on the label homophily. Briefly, label homophily can not only drive nodes sharing similar labels to be connected to each other, but also produce a division of a network into densely-connected, homogeneous parts that are weakly connected to each other. Furthermore, we propose a novel Label Homophily Oriented Network Embedding (LHONE) model to make the best of label homophily by converting a partially labeled network to two bipartite networks, and learning vector representations combined with a Gaussian mixture model (GMM). Extensive experiments on two real-world networks demonstrate the effectiveness of LHONE compared to state-of-the-art network embedding approaches.

Since the invention of word2vec, the skip-gram model has significantly advanced the research of network embedding, such as the recent emergence of the DeepWalk, LINE, PTE, and node2vec approaches. In this work, we show that all of the aforementioned models with negative sampling can be unified into the matrix factorization framework with closed forms. Our analysis and proofs reveal that: (1) DeepWalk empirically produces a low-rank transformation of a network's normalized Laplacian matrix; (2) LINE, in theory, is a special case of DeepWalk when the size of vertices' context is set to one; (3) As an extension of LINE, PTE can be viewed as the joint factorization of multiple networks' Laplacians; (4) node2vec is factorizing a matrix related to the stationary distribution and transition probability tensor of a 2nd-order random walk. We further provide the theoretical connections between skip-gram based network embedding algorithms and the theory of graph Laplacian. Finally, we present the NetMF method as well as its approximation algorithm for computing network embedding. Our method offers significant improvements over DeepWalk and LINE for conventional network mining tasks. This work lays the theoretical foundation for skip-gram based network embedding methods, leading to a better understanding of latent network representation learning.

Attributed network embedding aims to learn low-dimensional vector representations for nodes in a network, where each node contains rich attributes/features describing node content. Because network topology structure and node attributes often exhibit high correlation, incorporating node attribute proximity into network embedding is beneficial for learning good vector representations. In reality, large-scale networks often have incomplete/missing node content or linkages, yet existing attributed network embedding algorithms all operate under the assumption that networks are complete. Thus, their performance is vulnerable to missing data and suffers from poor scalability. In this paper, we propose a Scalable Incomplete Network Embedding (SINE) algorithm for learning node representations from incomplete graphs. SINE formulates a probabilistic learning framework that separately models pairs of node-context and node-attribute relationships. Different from existing attributed network embedding algorithms, SINE provides greater flexibility to make the best of useful information and mitigate negative effects of missing information on representation learning. A stochastic gradient descent based online algorithm is derived to learn node representations, allowing SINE to scale up to large-scale networks with high learning efficiency. We evaluate the effectiveness and efficiency of SINE through extensive experiments on real-world networks. Experimental results confirm that SINE outperforms state-of-the-art baselines in various tasks, including node classification, node clustering, and link prediction, under settings with missing links and node attributes. SINE is also shown to be scalable and efficient on large-scale networks with millions of nodes/edges and high-dimensional node features. The source code of this paper is available at https://github.com/daokunzhang/SINE.

In recent years, network embedding has attracted much attention from researchers and achieved excellent performance. But few works investigate the adaptability of network embedding, especially for performance in different network structures. Heterogeneity, as a universal topological characteristic, plays a prominent role in network behaviors. In this study, we investigate the effect of heterogeneity on the effectiveness of existing network embedding approaches. We conduct experiments in scale-free networks with varying power exponents from both macro and micro perspectives to address link prediction and node similarity tasks, respectively. The results indicate that network embedding approaches can be divided into two classes according to their performance in the link prediction task. As the network heterogeneity decreases, the performance of approaches in the first class declines, while the performance of approaches in the second class initially improves and then declines. Moreover, our simulation discovers that, based on the node similarity metric, nodes are partitioned into two clusters by approaches, corresponding to large-degree nodes and small-degree nodes, respectively. Furthermore, approaches in the same class present similar characteristics between large-degree nodes and small-degree nodes, and the embedding is interpreted to some extent. Performance variations in the link prediction task can be explained by the characteristics of approaches, and similar characteristics are confirmed in experiments on real networks. Based on the findings for link prediction, we offer a brief guide for choosing an appropriate method based on the extent of heterogeneity. The investigation provides insight into network embedding and offers some interpretation of embedding.

Network embedding (graph embedding) has become the focus of studying graph structure in recent years. In addition to the research on homogeneous networks and heterogeneous networks, there are also some methods to attempt to solve the problem of dynamic network embedding. However, in dynamic networks, there is no research method specifically for heterogeneous networks. Therefore, this paper proposes a heterogeneous dynamic graph attention network (HDGAN), which attempts to use the attention mechanism to take the heterogeneity and dynamics of the network into account at the same time, so as to better learn network embedding. Our method is based on three levels of attention, namely structural-level attention, semantic-level attention and time-level attention. Structural-level attention pays attention to the network structure itself, and obtains the representation of structural-level nodes by learning the attention coefficients of neighbor nodes. Semantic-level attention integrates semantic information into the representation of nodes by learning the optimal weighted combination of different meta-paths. Time-level attention is based on the time decay effect, and the time feature is introduced into the node representation by neighborhood formation sequence. Through the above three levels of attention mechanism, the final network embedding can be obtained.Through experiments on two real-world heterogeneous dynamic networks, our models have the best results, proving the effectiveness of the HDGAN model.

Network embedding, aiming to learn the low-dimensional representations of nodes in networks, is of paramount importance in many real applications. One basic requirement of network embedding is to preserve the structure and inherent properties of the networks. While previous network embedding methods primarily preserve the microscopic structure, such as the first- and second-order proximities of nodes, the mesoscopic community structure, which is one of the most prominent feature of networks, is largely ignored. In this paper, we propose a novel Modularized Nonnegative Matrix Factorization (M-NMF) model to incorporate the community structure into network embedding. We exploit the consensus relationship between the representations of nodes and community structure, and then jointly optimize NMF based representation learning model and modularity based community detection model in a unified framework, which enables the learned representations of nodes to preserve both of the microscopic and community structures. We also provide efficient updating rules to infer the parameters of our model, together with the correctness and convergence guarantees. Extensive experimental results on a variety of real-world networks show the superior performance of the proposed method over the state-of-the-arts.

In this paper we address the problem of matrix factorization on compressively-sampled measurements which are obtained by random projections. While this approach improves the scalability of matrix factorization, its performance is not satisfactory. We present a matrix co-factorization method where compressed measurements and a small number of uncompressed measurements are jointly decomposed, sharing a factor matrix. We evaluate the performance of three matrix factorization methods in terms of Cramer-Rao bounds, including: (1) matrix factorization on uncompressed data (MF); (2) matrix factorization on compressed data (CS-MF); (3) matrix co-factorization on compressed and uncompressed data (CS-MCF). Numerical experiments demonstrate that CS-MCF improves the performance of CS-MF, emphasizing the useful behavior of exploiting side information (a small number of uncompressed measurements).

Network embedding is an effective method to learn a low-dimensional feature vector representation for each node of a given network. In this paper, we propose a deep network embedding model with aggregated proximity preserving (DNE-APP). Firstly, an overall network proximity matrix is generated to capture both local and global network structural information, by aggregating different k-th order network proximities between different nodes. Then, a semi-supervised stacked auto-encoder is employed to learn the hidden representations which can best preserve the aggregated proximity in the original network, and also map the node pairs with higher proximity closer to each other in the embedding space. With the hidden representations learned by DNE-APP, we apply vector-based machine learning techniques to conduct node classification and link label prediction tasks on the real-world datasets. Experimental results demonstrate the superiority of our proposed DNE-APP model over the state-of-the-art network embedding algorithms.

As dynamic systems, social networks exhibit continuous topological changes over time, and are typically modeled as temporal networks. In order to understand their dynamic characteristics, it is essential to investigate temporal community detection (TCD), which poses significant challenges compared to static network analysis. These challenges arise from the need to simultaneously detect community structures and track their evolutionary behaviors. To address these issues, we propose TCDA-NE, a novel TCD algorithm that combines evolutionary clustering with convex non-negative matrix factorization (Convex-NMF). Our method innovatively integrates community structure into network embedding, preserving both microscopic details and community-level information in node representations while effectively capturing the evolutionary dynamics of networks. A distinctive feature of TCDA-NE is its utilization of a common-neighbor similarity matrix, which significantly enhances the algorithm’s ability to identify meaningful community structures in temporal networks. By establishing coherent relationships between node representations and community structures, we optimize both the Convex-NMF-based representation learning model and the evolutionary clustering-based TCD model within a unified framework. We derive the updating rules and provide rigorous theoretical proofs for the algorithm’s validity and convergence. Extensive experiments on synthetic and real-world social networks, including email and phone call networks, demonstrate the superior performance of our model in community detection and tracking temporal network evolution. Notably, TCDA-NE achieves a maximum improvement of up to 0.1 in the normalized mutual information (NMI) index compared to state-of-the-art methods, highlighting its effectiveness in temporal community detection.

Recently, network embedding has received a large amount of attention in network analysis. Although some network embedding methods have been developed from different perspectives, on one hand, most of the existing methods only focus on leveraging the plain network structure, ignoring the abundant attribute information of nodes. On the other hand, for some methods integrating the attribute information, only the lower-order proximities (e.g., microscopic proximity structure) are taken into account, which may suffer if there exists the sparsity issue and the attribute information is noisy. To overcome this problem, the attribute information and mesoscopic community structure are utilized. In this article, we propose a novel network embedding method termed Attributed Network Embedding with Micro-Meso structure, which is capable of preserving both the attribute information and the structural information including the microscopic proximity structure and mesoscopic community structure. In particular, both the microscopic proximity structure and node attributes are factorized by Nonnegative Matrix Factorization (NMF), from which the low-dimensional node representations can be obtained. For the mesoscopic community structure, a community membership strength matrix is inferred by a generative model (i.e., BigCLAM) or modularity from the linkage structure, which is then factorized by NMF to obtain the low-dimensional node representations. The three components are jointly correlated by the low-dimensional node representations, from which two objective functions (i.e., ANEM_B and ANEM_M) can be defined. Two efficient alternating optimization schemes are proposed to solve the optimization problems. Extensive experiments have been conducted to confirm the superior performance of the proposed models over the state-of-the-art network embedding methods.

Cross-scale contrastive triplet networks for graph representation learning

Combining Matrix Factorization (MF) with Network Embedding (NE) has been a promising solution to social recommender systems. However, in most of the current combined schemes, the user-specific linking proportions learned by NE are fed to the downstream MF, but not reverse, which is sub-optimal as the rating information is not utilized to discover the linking features for users. Furthermore, the existing combined models mainly focus on enhancing the representation learning for users by exploiting user–user network, yet ignore the representation improvement for items. In this paper, we propose a novel social recommendation scheme, called MF with dual-network collaborative embedding (MF-decoding), which jointly optimizes an integrated objective function of MF and NE, in which both MF and NE tasks can be mutually reinforced in a unified learning process. In particular, the explicit user–user network and the implicit item–item network are collaboratively used by MF-decoding to enhance the representation learning for users and items simultaneously. Our encouraging experimental results on three benchmarks validate the superiority of the proposed MF-decoding model over state-of-the-art social recommendation methods.

OLMNE+FT: Multiplex network embedding based on overlapping links

Network analysis has been widely applied in many real-world tasks, such as gene analysis and targeted marketing. To extract effective features for these analysis tasks, network embedding automatically learns a low-dimensional vector representation for each node, such that the meaningful topological proximity is well preserved. While the embedding algorithms on pure topological structure have attracted considerable attention, in practice, nodes are often abundantly accompanied with other types of meaningful information, such as node attributes, second-order proximity, and link directionality. A general framework for incorporating the heterogeneous information into network embedding could be potentially helpful in learning better vector representations. However, it remains a challenging task to jointly embed the geometrical structure and a distinct type of information due to the heterogeneity. In addition, the real-world networks often contain a large number of nodes, which put demands on the scalability of the embedding algorithms. To bridge the gap, in this article, we propose a general embedding framework named Heterogeneous Information Learning in Large-scale networks (HILL) to accelerate the joint learning. It enables the simultaneous node proximity assessing process to be done in a distributed manner by decomposing the complex modeling and optimization into many simple and independent sub-problems. We validate the significant correlation between the heterogeneous information and topological structure, and illustrate the generalizability of HILL by applying it to perform attributed network embedding and second-order proximity learning. A variation is proposed for link directionality modeling. Experimental results on real-world networks demonstrate the effectiveness and efficiency of HILL.

Network embedding, which aims at learning distributed representations for nodes in networks, is a critical task with wide downstream applications. Most existing studies focus on networks with a single type of edges, whereas in many cases, the edges of networks can be derived from two opposite relationships, yielding signed networks. This paper studies network embedding for the signed network, and a novel approach called TEA is proposed. Similar to existing methods, TEA (Triad+Edge+Attention) learns node representations by predicting the sign of each edge in the network. However, many existing methods only consider the local structural information (i.e., the representations of nodes in an edge) for prediction, which can be biased especially for sparse networks. By contrast, TEA seeks to leverage the high-order structures by drawing inspirations from the Structural Balance Theory. More specifically, for an edge linking two nodes, TEA predicts the edge sign by considering the triangles connecting the two nodes as features. Meanwhile, an attention mechanism is proposed, which assigns different weights to the different triangles before aggregating their predictions for more precise results. We conduct experiments on several real-world signed networks, and the results prove the effectiveness of TEA over many strong baseline approaches.