Attributed Network Embedding with Micro-Meso Structure

Network embedding aims to learn distributed vector representations of nodes in a network. The problem of network embedding is fundamentally important. It plays crucial roles in many applications, such as node classification, link prediction, and so on. As the real-world networks are often sparse with few observed links, many recent works have utilized the local and global network structure proximity with shallow models for better network embedding. In reality, each node is usually associated with rich attributes. Some attributed network embedding models leveraged the node attributes in these shallow network embedding models to alleviate the data sparsity issue. Nevertheless, the underlying structure of the network is complex. What is more, the connection between the network structure and node attributes is also hidden. Thus, these previous shallow models fail to capture the nonlinear deep information embedded in the attributed network, resulting in the suboptimal embedding results. In this paper, we propose a deep attributed network embedding framework to capture the complex structure and attribute information. Specifically, we first adopt a personalized random walk-based model to capture the interaction between network structure and node attributes from various degrees of proximity. After that, we construct an enhanced matrix representation of the attributed network by summarizing the various degrees of proximity. Then, we design a deep neural network to exploit the nonlinear complex information in the enhanced matrix for network embedding. Thus, the proposed framework could capture the complex attributed network structure by preserving both the various degrees of network structure and node attributes in a unified framework. Finally, empirical experiments show the effectiveness of our proposed framework on a variety of network embedding-based tasks.

Network embedding assigns nodes in a network to low-dimensional representations and effectively preserves the network structure. Recently, a significant amount of progresses have been made toward this emerging network analysis paradigm. In this survey, we focus on categorizing and then reviewing the current development on network embedding methods, and point out its future research directions. We first summarize the motivation of network embedding. We discuss the classical graph embedding algorithms and their relationship with network embedding. Afterwards and primarily, we provide a comprehensive overview of a large number of network embedding methods in a systematic manner, covering the structure- and property-preserving network embedding methods, the network embedding methods with side information and the advanced information preserving network embedding methods. Moreover, several evaluation approaches for network embedding and some useful online resources, including the network data sets and softwares, are reviewed, too. Finally, we discuss the framework of exploiting these network embedding methods to build an effective system and point out some potential future directions.

Attributed network embedding aims to learn low-dimensional vector representations for nodes in a network, where each node contains rich attributes/features describing node content. Because network topology structure and node attributes often exhibit high correlation, incorporating node attribute proximity into network embedding is beneficial for learning good vector representations. In reality, large-scale networks often have incomplete/missing node content or linkages, yet existing attributed network embedding algorithms all operate under the assumption that networks are complete. Thus, their performance is vulnerable to missing data and suffers from poor scalability. In this paper, we propose a Scalable Incomplete Network Embedding (SINE) algorithm for learning node representations from incomplete graphs. SINE formulates a probabilistic learning framework that separately models pairs of node-context and node-attribute relationships. Different from existing attributed network embedding algorithms, SINE provides greater flexibility to make the best of useful information and mitigate negative effects of missing information on representation learning. A stochastic gradient descent based online algorithm is derived to learn node representations, allowing SINE to scale up to large-scale networks with high learning efficiency. We evaluate the effectiveness and efficiency of SINE through extensive experiments on real-world networks. Experimental results confirm that SINE outperforms state-of-the-art baselines in various tasks, including node classification, node clustering, and link prediction, under settings with missing links and node attributes. SINE is also shown to be scalable and efficient on large-scale networks with millions of nodes/edges and high-dimensional node features. The source code of this paper is available at https://github.com/daokunzhang/SINE.

Network embedding, aiming to learn the low-dimensional representations of nodes in networks, is of paramount importance in many real applications. One basic requirement of network embedding is to preserve the structure and inherent properties of the networks. While previous network embedding methods primarily preserve the microscopic structure, such as the first- and second-order proximities of nodes, the mesoscopic community structure, which is one of the most prominent feature of networks, is largely ignored. In this paper, we propose a novel Modularized Nonnegative Matrix Factorization (M-NMF) model to incorporate the community structure into network embedding. We exploit the consensus relationship between the representations of nodes and community structure, and then jointly optimize NMF based representation learning model and modularity based community detection model in a unified framework, which enables the learned representations of nodes to preserve both of the microscopic and community structures. We also provide efficient updating rules to infer the parameters of our model, together with the correctness and convergence guarantees. Extensive experimental results on a variety of real-world networks show the superior performance of the proposed method over the state-of-the-arts.

Attributed network embedding aims to learn low dimensional node representations by combining both the network's topological structure and node attributes. Most of the existing methods either propagate the attributes over the network structure or learn the node representations by an encoder-decoder framework. However, propagation based methods tend to prefer network structure to node attributes, whereas encoder-decoder methods tend to ignore the longer connections beyond the immediate neighbors. In order to address these limitations while enjoying the best of the two worlds, we design cross fusion layers for unsupervised attributed network embedding. Specifically, we first construct two separate views to handle network structure and node attributes, and then design cross fusion layers to allow flexible information exchange and integration between the two views. The key design goals of the cross fusion layers are three-fold: 1) allowing critical information to be propagated along the network structure, 2) encoding the heterogeneity in the local neighborhood of each node during propagation, and 3) incorporating an additional node attribute channel so that the attribute information will not be overshadowed by the structure view. Extensive experiments on three datasets and three downstream tasks demonstrate the effectiveness of the proposed method.

Network embedding: Taxonomies, frameworks and applications

Network analysis has been widely applied in many real-world tasks, such as gene analysis and targeted marketing. To extract effective features for these analysis tasks, network embedding automatically learns a low-dimensional vector representation for each node, such that the meaningful topological proximity is well preserved. While the embedding algorithms on pure topological structure have attracted considerable attention, in practice, nodes are often abundantly accompanied with other types of meaningful information, such as node attributes, second-order proximity, and link directionality. A general framework for incorporating the heterogeneous information into network embedding could be potentially helpful in learning better vector representations. However, it remains a challenging task to jointly embed the geometrical structure and a distinct type of information due to the heterogeneity. In addition, the real-world networks often contain a large number of nodes, which put demands on the scalability of the embedding algorithms. To bridge the gap, in this article, we propose a general embedding framework named Heterogeneous Information Learning in Large-scale networks (HILL) to accelerate the joint learning. It enables the simultaneous node proximity assessing process to be done in a distributed manner by decomposing the complex modeling and optimization into many simple and independent sub-problems. We validate the significant correlation between the heterogeneous information and topological structure, and illustrate the generalizability of HILL by applying it to perform attributed network embedding and second-order proximity learning. A variation is proposed for link directionality modeling. Experimental results on real-world networks demonstrate the effectiveness and efficiency of HILL.

Recent research on community detection focuses on learning representations of nodes using different network embedding methods, and then feeding them as normal features to clustering algorithms. However, we find that though one may have good results by direct clustering based on such network embedding features, there is ample room for improvement. More seriously, in many real networks, some statisticallysignificant nodes which play pivotal roles are often divided into incorrect communities using network embedding methods. This is because while some distance measures are used to capture the spatial relationship between nodes by embedding, the nodes after mapping to feature vectors are essentially not coupled any more, losing important structural information. To address this problem, we propose a general Markov Random Field (MRF) framework to incorporate coupling in network embedding which allows better detecting network communities. By smartly utilizing properties of MRF, the new framework not only preserves the advantages of network embedding (e.g. low complexity, high parallelizability and applicability for traditional machine learning), but also alleviates its core drawback of inadequate representations of dependencies via making up the missing coupling relationships. Experiments on real networks show that our new approach improves the accuracy of existing embedding methods (e.g. Node2Vec, DeepWalk and MNMF), and corrects most wrongly-divided statistically-significant nodes, which makes network embedding essentially suitable for real community detection applications. The new approach also outperforms other state-of-the-art conventional community detection methods.

Network embedding which aims to embed a given network into a low-dimensional vector space has been proved effective in various network analysis and mining tasks such as node classification, link prediction and network visualization. The emerging network embedding methods have shifted of emphasis in utilizing mature deep learning models. The neural-network based network embedding has become a mainstream solution because of its high efficiency and capability of preserving the nonlinear characteristics of the network. In this paper, we propose Adversarial Network Embedding using Structural Similarity (ANESS), a novel, versatile, low-complexity GAN-based network embedding model which utilizes the inherent vertex-to-vertex structural similarity attribute of the network. ANESS learns robustness and effective vertex embeddings via a adversarial training procedure. Specifically, our method aims to exploit the strengths of generative adversarial networks in generating high-quality samples and utilize the structural similarity identity of vertexes to learn the latent representations of a network. Meanwhile, ANESS can dynamically update the strategy of generating samples during each training iteration. The extensive experiments have been conducted on the several benchmark network datasets, and empirical results demonstrate that ANESS significantly outperforms other state-of-the-art network embedding methods.

Compact network embedding for fast node classification

Network embedding aims to learn the low-dimensional representations of nodes in networks and preserve the features of networks simultaneously. In this paper, we propose a novel approach of network embedding, named Attributed Network Embedding with Data Distribution Adaptation (ANEDDA). In our model, we consider network structure and node attributes as two kinds of data which have different probability distributions. This is because that they come from different aspects or views and capture different kinds of features. However, because they come from the same network, there must exist some latent features which are related to both two kinds of information, and these features are significant to detect the community structure and analyze the network. We utilize a domain adaptation model named Transfer Component Analysis (TCA) to identify the latent common features. The proposed method is designed for unweighted networks (graphs), and it integrates the structure information and nodes attributes.

Network embedding is a useful tool to map graph structures into vector spaces, which facilitates graph analysis tasks including node classification, graph visualization, similarity calculation etc. Existing network embedding methods calculate embedding vectors based on node series generated by random walks. These methods treat all the links equally during the random walk procedure, which leads to the missing of structural information that is key to the embedding performance. We therefore propose in this paper a novel random walk-based network embedding method called Self-Adjusting Random Walk (SARW). SARW utilizes a self-adjusting strategy that makes the walking biased towards the links that are more strongly connected in order to better capture the structural information. Further more, the strengths of links are updated using the embedding output as feedback. Through experiments we have verified that our method out performs state-of-the-art network embedding methods, in node classification tasks and link prediction tasks.

Network embedding enables to apply off-the-shelf machine learning methods to the nodes on the network. Leveraging the textual information associated with nodes into network embedding methods is advantageous. However, only a few works try to leverage textual information to network embeddings. Moreover, the structure of networks and associated texts could be dynamically changed over time, this property leads shifts of the vector representation of nodes and words. However, to the best of our knowledge, none of previous network embedding methods considers chronological changes of vector representations of nodes and words. In this paper, we propose a dynamic text attribute network embedding method, which embeds the nodes and the words in a cooperative manner and takes chronological changes of vector representations of nodes and words into account. Experimental results show that (1) vector representations of nodes of our method achieve higher accuracy than baseline methods in classification and clustering tasks; (2) The vector representations of nodes and words successfully capture semantic similarity; And (3) our method successfully capture the chronological change of the vector representations over time.

Network Embedding, which represents nodes in networks with efficient low-dimensional vectors, has been proved useful in a variety of applications. However, most existing approaches study single-view networks but not the multi-view networks with multiple types of relationships between nodes. Meanwhile, they ignore the rich features associated with the nodes, which is common in real world. In this paper, we propose a novel network embedding method, Intra-view and Inter-view attention for Multi-view Network Embedding (I2MNE), which leverages both the multi-view network structure and the node features to efficiently generate node representations. Specially, we introduce the intra-view attention when aggregating node features from neighbors for each single view and the inter-view attention when integrating representations across different views. Experiments on two real-world networks show that our approach outperforms other counterpart network embedding methods.

Road networks are a type of spatial network, where edges may be associated with qualitative information such as road type and speed limit. Unfortunately, such information is often incomplete; for instance, OpenStreetMap only has speed limits for 13% of all Danish road segments. This is problematic for analysis tasks that rely on such information for machine learning. To enable machine learning in such circumstances, one may consider the application of network embedding methods to extract structural information from the network. However, these methods have so far mostly been used in the context of social networks, which differ significantly from road networks in terms of, e.g., node degree and level of homophily (which are key to the performance of many network embedding methods). We analyze the use of network embedding methods, specifically node2vec, for learning road segment embeddings in road networks. Due to the often limited availability of information on other relevant road characteristics, the analysis focuses on leveraging the spatial network structure. Our results suggest that network embedding methods can indeed be used for deriving relevant network features (that may, e.g, be used for predicting speed limits), but that the qualities of the embeddings differ from embeddings for social networks.