Deep Attributed Network Embedding by Preserving Structure and Attribute Information

Attributed network embedding aims to learn low-dimensional vector representations for nodes in a network, where each node contains rich attributes/features describing node content. Because network topology structure and node attributes often exhibit high correlation, incorporating node attribute proximity into network embedding is beneficial for learning good vector representations. In reality, large-scale networks often have incomplete/missing node content or linkages, yet existing attributed network embedding algorithms all operate under the assumption that networks are complete. Thus, their performance is vulnerable to missing data and suffers from poor scalability. In this paper, we propose a Scalable Incomplete Network Embedding (SINE) algorithm for learning node representations from incomplete graphs. SINE formulates a probabilistic learning framework that separately models pairs of node-context and node-attribute relationships. Different from existing attributed network embedding algorithms, SINE provides greater flexibility to make the best of useful information and mitigate negative effects of missing information on representation learning. A stochastic gradient descent based online algorithm is derived to learn node representations, allowing SINE to scale up to large-scale networks with high learning efficiency. We evaluate the effectiveness and efficiency of SINE through extensive experiments on real-world networks. Experimental results confirm that SINE outperforms state-of-the-art baselines in various tasks, including node classification, node clustering, and link prediction, under settings with missing links and node attributes. SINE is also shown to be scalable and efficient on large-scale networks with millions of nodes/edges and high-dimensional node features. The source code of this paper is available at https://github.com/daokunzhang/SINE.

Attributed network embedding aims to learn low dimensional node representations by combining both the network's topological structure and node attributes. Most of the existing methods either propagate the attributes over the network structure or learn the node representations by an encoder-decoder framework. However, propagation based methods tend to prefer network structure to node attributes, whereas encoder-decoder methods tend to ignore the longer connections beyond the immediate neighbors. In order to address these limitations while enjoying the best of the two worlds, we design cross fusion layers for unsupervised attributed network embedding. Specifically, we first construct two separate views to handle network structure and node attributes, and then design cross fusion layers to allow flexible information exchange and integration between the two views. The key design goals of the cross fusion layers are three-fold: 1) allowing critical information to be propagated along the network structure, 2) encoding the heterogeneity in the local neighborhood of each node during propagation, and 3) incorporating an additional node attribute channel so that the attribute information will not be overshadowed by the structure view. Extensive experiments on three datasets and three downstream tasks demonstrate the effectiveness of the proposed method.

Network embedding leverages the node proximity manifested to learn a\nlow-dimensional node vector representation for each node in the network. The\nlearned embeddings could advance various learning tasks such as node\nclassification, network clustering, and link prediction. Most, if not all, of\nthe existing works, are overwhelmingly performed in the context of plain and\nstatic networks. Nonetheless, in reality, network structure often evolves over\ntime with addition/deletion of links and nodes. Also, a vast majority of\nreal-world networks are associated with a rich set of node attributes, and\ntheir attribute values are also naturally changing, with the emerging of new\ncontent patterns and the fading of old content patterns. These changing\ncharacteristics motivate us to seek an effective embedding representation to\ncapture network and attribute evolving patterns, which is of fundamental\nimportance for learning in a dynamic environment. To our best knowledge, we are\nthe first to tackle this problem with the following two challenges: (1) the\ninherently correlated network and node attributes could be noisy and\nincomplete, it necessitates a robust consensus representation to capture their\nindividual properties and correlations; (2) the embedding learning needs to be\nperformed in an online fashion to adapt to the changes accordingly. In this\npaper, we tackle this problem by proposing a novel dynamic attributed network\nembedding framework - DANE. In particular, DANE first provides an offline\nmethod for a consensus embedding and then leverages matrix perturbation theory\nto maintain the freshness of the end embedding results in an online manner. We\nperform extensive experiments on both synthetic and real attributed networks to\ncorroborate the effectiveness and efficiency of the proposed framework.\n

Recently, network embedding has received a large amount of attention in network analysis. Although some network embedding methods have been developed from different perspectives, on one hand, most of the existing methods only focus on leveraging the plain network structure, ignoring the abundant attribute information of nodes. On the other hand, for some methods integrating the attribute information, only the lower-order proximities (e.g., microscopic proximity structure) are taken into account, which may suffer if there exists the sparsity issue and the attribute information is noisy. To overcome this problem, the attribute information and mesoscopic community structure are utilized. In this article, we propose a novel network embedding method termed Attributed Network Embedding with Micro-Meso structure, which is capable of preserving both the attribute information and the structural information including the microscopic proximity structure and mesoscopic community structure. In particular, both the microscopic proximity structure and node attributes are factorized by Nonnegative Matrix Factorization (NMF), from which the low-dimensional node representations can be obtained. For the mesoscopic community structure, a community membership strength matrix is inferred by a generative model (i.e., BigCLAM) or modularity from the linkage structure, which is then factorized by NMF to obtain the low-dimensional node representations. The three components are jointly correlated by the low-dimensional node representations, from which two objective functions (i.e., ANEM_B and ANEM_M) can be defined. Two efficient alternating optimization schemes are proposed to solve the optimization problems. Extensive experiments have been conducted to confirm the superior performance of the proposed models over the state-of-the-art network embedding methods.

Network embedding aims to learn the low-dimensional representations of nodes in networks and preserve the features of networks simultaneously. In this paper, we propose a novel approach of network embedding, named Attributed Network Embedding with Data Distribution Adaptation (ANEDDA). In our model, we consider network structure and node attributes as two kinds of data which have different probability distributions. This is because that they come from different aspects or views and capture different kinds of features. However, because they come from the same network, there must exist some latent features which are related to both two kinds of information, and these features are significant to detect the community structure and analyze the network. We utilize a domain adaptation model named Transfer Component Analysis (TCA) to identify the latent common features. The proposed method is designed for unweighted networks (graphs), and it integrates the structure information and nodes attributes.

In today's networked society, many real-world problems can be formalized as predicting links in networks, such as Facebook friendship suggestions, e-commerce recommendations, and the prediction of scientific collaborations in citation networks. Increasingly often, link prediction problem is tackled by means of network embedding methods, owing to their state-of-the-art performance. However, these methods lack transparency when compared to simpler baselines, and as a result their robustness against adversarial attacks is a possible point of concern: could one or a few small adversarial modifications to the network have a large impact on the link prediction performance when using a network embedding model? Prior research has already investigated adversarial robustness for network embedding models, focused on classification at the node and graph level. Robustness with respect to the link prediction downstream task, on the other hand, has been explored much less. This paper contributes to filling this gap, by studying adversarial robustness of Conditional Network Embedding (CNE), a state-of-the-art probabilistic network embedding model, for link prediction. More specifically, given CNE and a network, we measure the sensitivity of the link predictions of the model to small adversarial perturbations of the network, namely changes of the link status of a node pair. Thus, our approach allows one to identify the links and non-links in the network that are most vulnerable to such perturbations, for further investigation by an analyst. We analyze the characteristics of the most and least sensitive perturbations, and empirically confirm that our approach not only succeeds in identifying the most vulnerable links and non-links, but also that it does so in a time-efficient manner thanks to an effective approximation.

Network embedding, which aims to learn low-dimensional representations of nodes, has been used for various graph related tasks including visualization, link prediction and node classification. Most existing embedding methods rely solely on network structure. However, in practice we often have auxiliary information about the nodes and/or their interactions, e.g., content of scientific papers in co-authorship networks, or topics of communication in Twitter mention networks. Here we propose a novel embedding method that uses both network structure and edge attributes to learn better network representations. Our method jointly minimizes the reconstruction error for higher-order node neighborhood, social roles and edge attributes using a deep architecture that can adequately capture highly non-linear interactions. We demonstrate the efficacy of our model over existing state-of-the-art methods on a variety of real-world networks including collaboration networks, and social networks. We also observe that using edge attributes to inform network embedding yields better performance in downstream tasks such as link prediction and node classification.

Network embedding which aims to embed a given network into a low-dimensional vector space has been proved effective in various network analysis and mining tasks such as node classification, link prediction and network visualization. The emerging network embedding methods have shifted of emphasis in utilizing mature deep learning models. The neural-network based network embedding has become a mainstream solution because of its high efficiency and capability of preserving the nonlinear characteristics of the network. In this paper, we propose Adversarial Network Embedding using Structural Similarity (ANESS), a novel, versatile, low-complexity GAN-based network embedding model which utilizes the inherent vertex-to-vertex structural similarity attribute of the network. ANESS learns robustness and effective vertex embeddings via a adversarial training procedure. Specifically, our method aims to exploit the strengths of generative adversarial networks in generating high-quality samples and utilize the structural similarity identity of vertexes to learn the latent representations of a network. Meanwhile, ANESS can dynamically update the strategy of generating samples during each training iteration. The extensive experiments have been conducted on the several benchmark network datasets, and empirical results demonstrate that ANESS significantly outperforms other state-of-the-art network embedding methods.

Network embedding maps the nodes of a network to a continuous vector space, which can then be used as the input to downstream tasks, such as node classification, node clustering, link prediction, and similarity search. To learn network embedding more effectively, many technologies adopt the approach of random walk to obtain the network structure. As the meta-path of heterogeneous networks emerges, network embedding will be equipped with more semantic interpretation. Consequently, various random walks, based on meta-path strategies, have been proposed for network embedding. However, the combination of semantic and structure in a heterogeneous network cannot achieve ideal results. To overcome this challenge, we start from a task-guided issue by combining the timestamps information in the heterogeneous network, and then employing the method of temporal segmentation to decompose the network into a continuous temporal sequence. Finally, the set of context-paths between nodes is calculated in a continuous vector by the depth-first meta-path search algorithm. More precisely, we propose a Temporal Sliding Density Walk (TSDW) algorithm by combining network semantics and structure effectively. Empirical results for network data show that TSDW could significantly outperform the state-of-the-art representation learning models, including DeepWalk, LINE, Node2vec, PTE, Meapath2vec, HIN2vec, HTNE, and CTDNE by 3.02% to 44.9% of Macro-F1, 0.9% to 18.92% of Micro-F1 in multi-class node classification and 21% to 47% of NMI in node clustering.

Learning network embedding for large-scale networks have been attracting increasing attention due to their importance in supporting numerous network analytic and data mining tasks such as node classification, clustering and visualization. In this paper, we present a novel framework for learning large-scale network embedding incorporating network topology and community structural information. Most existing network embedding methods tend to embed network topology and ignore the partially labeled community structure information that exist in real-world networks and thus are unable to efficiently learn and capture the community structure of real-world networks. Unlike existing works, our framework integrates the network topology and community structure into the learning process. We propose a deep autoencoder model to generate low-dimensional feature representations efficiently through learning network reconstruction and community classification tasks. The experimental results on several real-world networks show that our framework outperforms the state-of-the-art methods.

This paper studies the problem of Representation Learning for network with textual information, which aims to learn low dimensional vectors for nodes by leveraging network structure and textual information. Most existing works only focus on one aspect of network structure and cannot fuse network first-order proximity, second-order proximity and textual information. In this paper, we propose a novel network embedding method NE-FLGC: Network Embedding based on Fusing Local (first-order) and Global (second-order) network structure with node Content. Especially, we adopt context-enhance method that obtains node embedding by concatenating the vector of itself and the context vectors. In experiments, we compare our model with existing network embedding models on four real-world datasets. The experimental results demonstrate that NE-FLGC is stable and significantly outperforms state-of-the-art methods.

In recent years, network embedding has attracted much attention from researchers and achieved excellent performance. But few works investigate the adaptability of network embedding, especially for performance in different network structures. Heterogeneity, as a universal topological characteristic, plays a prominent role in network behaviors. In this study, we investigate the effect of heterogeneity on the effectiveness of existing network embedding approaches. We conduct experiments in scale-free networks with varying power exponents from both macro and micro perspectives to address link prediction and node similarity tasks, respectively. The results indicate that network embedding approaches can be divided into two classes according to their performance in the link prediction task. As the network heterogeneity decreases, the performance of approaches in the first class declines, while the performance of approaches in the second class initially improves and then declines. Moreover, our simulation discovers that, based on the node similarity metric, nodes are partitioned into two clusters by approaches, corresponding to large-degree nodes and small-degree nodes, respectively. Furthermore, approaches in the same class present similar characteristics between large-degree nodes and small-degree nodes, and the embedding is interpreted to some extent. Performance variations in the link prediction task can be explained by the characteristics of approaches, and similar characteristics are confirmed in experiments on real networks. Based on the findings for link prediction, we offer a brief guide for choosing an appropriate method based on the extent of heterogeneity. The investigation provides insight into network embedding and offers some interpretation of embedding.

Network analysis has been widely applied in many real-world tasks, such as gene analysis and targeted marketing. To extract effective features for these analysis tasks, network embedding automatically learns a low-dimensional vector representation for each node, such that the meaningful topological proximity is well preserved. While the embedding algorithms on pure topological structure have attracted considerable attention, in practice, nodes are often abundantly accompanied with other types of meaningful information, such as node attributes, second-order proximity, and link directionality. A general framework for incorporating the heterogeneous information into network embedding could be potentially helpful in learning better vector representations. However, it remains a challenging task to jointly embed the geometrical structure and a distinct type of information due to the heterogeneity. In addition, the real-world networks often contain a large number of nodes, which put demands on the scalability of the embedding algorithms. To bridge the gap, in this article, we propose a general embedding framework named Heterogeneous Information Learning in Large-scale networks (HILL) to accelerate the joint learning. It enables the simultaneous node proximity assessing process to be done in a distributed manner by decomposing the complex modeling and optimization into many simple and independent sub-problems. We validate the significant correlation between the heterogeneous information and topological structure, and illustrate the generalizability of HILL by applying it to perform attributed network embedding and second-order proximity learning. A variation is proposed for link directionality modeling. Experimental results on real-world networks demonstrate the effectiveness and efficiency of HILL.

Link prediction, which utilizes the information of endpoint and network structure to predict the unknown links between two nodes, has attracted much attention in recent years. The network topological attributes contain the structure attributes and node attributes. However, some existing methods focus on the node attributes, while others focus on the structure attributes. To solve this problem, we propose a prediction method based on attributes fusion which combines node attributes and structure attributes. In our proposed method, we first analyze the structural attributes based on common neighbors in directed networks and define the structural attribute similarity. Then the similarity contribution of the influence of the common neighbors to the predicted nonadjacent nodes is analyzed. Experimental results on 9 directed networks show that our proposed index achieves higher performance than existing mainstream baselines under the precision evaluation.

Many real world networks are very large and constantly change over time. These dynamic networks exist in various domains such as social networks, traffic networks and biological interactions. To handle large dynamic networks in downstream applications such as link prediction and anomaly detection, it is essential for such networks to be transferred into a low dimensional space. Recently, network embedding, a technique that converts a large graph into a low-dimensional representation, has become increasingly popular due to its strength in preserving the structure of a network. Efficient dynamic network embedding, however, has not yet been fully explored. In this paper, we present a dynamic network embedding method that integrates the history of nodes over time into the current state of nodes. The key contribution of our work is 1) generating dynamic network embedding by combining both dynamic and static node information 2) tracking history of neighbors of nodes using LSTM 3) significantly decreasing the time and memory by training an autoencoder LSTM model using temporal walks rather than adjacency matrices of graphs which are the common practice. We evaluate our method in multiple applications such as anomaly detection, link prediction and node classification in datasets from various domains.