Attributed Network Embedding with Data Distribution Adaptation

Network embedding aims to learn distributed vector representations of nodes in a network. The problem of network embedding is fundamentally important. It plays crucial roles in many applications, such as node classification, link prediction, and so on. As the real-world networks are often sparse with few observed links, many recent works have utilized the local and global network structure proximity with shallow models for better network embedding. In reality, each node is usually associated with rich attributes. Some attributed network embedding models leveraged the node attributes in these shallow network embedding models to alleviate the data sparsity issue. Nevertheless, the underlying structure of the network is complex. What is more, the connection between the network structure and node attributes is also hidden. Thus, these previous shallow models fail to capture the nonlinear deep information embedded in the attributed network, resulting in the suboptimal embedding results. In this paper, we propose a deep attributed network embedding framework to capture the complex structure and attribute information. Specifically, we first adopt a personalized random walk-based model to capture the interaction between network structure and node attributes from various degrees of proximity. After that, we construct an enhanced matrix representation of the attributed network by summarizing the various degrees of proximity. Then, we design a deep neural network to exploit the nonlinear complex information in the enhanced matrix for network embedding. Thus, the proposed framework could capture the complex attributed network structure by preserving both the various degrees of network structure and node attributes in a unified framework. Finally, empirical experiments show the effectiveness of our proposed framework on a variety of network embedding-based tasks.

Attributed network embedding aims to learn low dimensional node representations by combining both the network's topological structure and node attributes. Most of the existing methods either propagate the attributes over the network structure or learn the node representations by an encoder-decoder framework. However, propagation based methods tend to prefer network structure to node attributes, whereas encoder-decoder methods tend to ignore the longer connections beyond the immediate neighbors. In order to address these limitations while enjoying the best of the two worlds, we design cross fusion layers for unsupervised attributed network embedding. Specifically, we first construct two separate views to handle network structure and node attributes, and then design cross fusion layers to allow flexible information exchange and integration between the two views. The key design goals of the cross fusion layers are three-fold: 1) allowing critical information to be propagated along the network structure, 2) encoding the heterogeneity in the local neighborhood of each node during propagation, and 3) incorporating an additional node attribute channel so that the attribute information will not be overshadowed by the structure view. Extensive experiments on three datasets and three downstream tasks demonstrate the effectiveness of the proposed method.

Recently, network embedding has received a large amount of attention in network analysis. Although some network embedding methods have been developed from different perspectives, on one hand, most of the existing methods only focus on leveraging the plain network structure, ignoring the abundant attribute information of nodes. On the other hand, for some methods integrating the attribute information, only the lower-order proximities (e.g., microscopic proximity structure) are taken into account, which may suffer if there exists the sparsity issue and the attribute information is noisy. To overcome this problem, the attribute information and mesoscopic community structure are utilized. In this article, we propose a novel network embedding method termed Attributed Network Embedding with Micro-Meso structure, which is capable of preserving both the attribute information and the structural information including the microscopic proximity structure and mesoscopic community structure. In particular, both the microscopic proximity structure and node attributes are factorized by Nonnegative Matrix Factorization (NMF), from which the low-dimensional node representations can be obtained. For the mesoscopic community structure, a community membership strength matrix is inferred by a generative model (i.e., BigCLAM) or modularity from the linkage structure, which is then factorized by NMF to obtain the low-dimensional node representations. The three components are jointly correlated by the low-dimensional node representations, from which two objective functions (i.e., ANEM_B and ANEM_M) can be defined. Two efficient alternating optimization schemes are proposed to solve the optimization problems. Extensive experiments have been conducted to confirm the superior performance of the proposed models over the state-of-the-art network embedding methods.

Attributed network embedding aims to learn low-dimensional vector representations for nodes in a network, where each node contains rich attributes/features describing node content. Because network topology structure and node attributes often exhibit high correlation, incorporating node attribute proximity into network embedding is beneficial for learning good vector representations. In reality, large-scale networks often have incomplete/missing node content or linkages, yet existing attributed network embedding algorithms all operate under the assumption that networks are complete. Thus, their performance is vulnerable to missing data and suffers from poor scalability. In this paper, we propose a Scalable Incomplete Network Embedding (SINE) algorithm for learning node representations from incomplete graphs. SINE formulates a probabilistic learning framework that separately models pairs of node-context and node-attribute relationships. Different from existing attributed network embedding algorithms, SINE provides greater flexibility to make the best of useful information and mitigate negative effects of missing information on representation learning. A stochastic gradient descent based online algorithm is derived to learn node representations, allowing SINE to scale up to large-scale networks with high learning efficiency. We evaluate the effectiveness and efficiency of SINE through extensive experiments on real-world networks. Experimental results confirm that SINE outperforms state-of-the-art baselines in various tasks, including node classification, node clustering, and link prediction, under settings with missing links and node attributes. SINE is also shown to be scalable and efficient on large-scale networks with millions of nodes/edges and high-dimensional node features. The source code of this paper is available at https://github.com/daokunzhang/SINE.

As an important data type, the demand of network analysis and learning is increasingly prominent. A key problem of network analysis is to study how to reasonably represent the feature information in the network, that is network embedding. However, in the study of network embedding, only the first-order proximity relationship of nodes is characterized, while the second-order proximity relationship hidden in the network nodes is ignored. Therefore, we propose an algorithm, called SINE, to realize the representation learning of nodes in the network which fuses the first-order and second-order proximity of nodes in the original network. Through applying SINE to three real networks, we obtain the feature representation of the nodes in the networks and cluster the nodes based on these features. Comparing with the existing network embedding learning methods-Node2vec, Large-Scale Information Network Embedding (LINE) and Structural Deep Network Embedding (SDNE), the SINE algorithm showed better performance in clustering tasks.

Network embedding, aiming to learn the low-dimensional representations of nodes in networks, is of paramount importance in many real applications. One basic requirement of network embedding is to preserve the structure and inherent properties of the networks. While previous network embedding methods primarily preserve the microscopic structure, such as the first- and second-order proximities of nodes, the mesoscopic community structure, which is one of the most prominent feature of networks, is largely ignored. In this paper, we propose a novel Modularized Nonnegative Matrix Factorization (M-NMF) model to incorporate the community structure into network embedding. We exploit the consensus relationship between the representations of nodes and community structure, and then jointly optimize NMF based representation learning model and modularity based community detection model in a unified framework, which enables the learned representations of nodes to preserve both of the microscopic and community structures. We also provide efficient updating rules to infer the parameters of our model, together with the correctness and convergence guarantees. Extensive experimental results on a variety of real-world networks show the superior performance of the proposed method over the state-of-the-arts.

Compact network embedding for fast node classification

To alleviate the problem caused by the sparsity of network structure which is often the case in large-scale network, attributed network embedding has attracted an increasing amount of attention. Some existing attributed network embedding models integrate linkage structure and node attribute by adding a consistency criterion on the structure representation and attribute representation, whereby the similarity between them can be ensured. However, to enforce the structure and attribute representations to be similar may cause the information distortion due to the inherent difference between these two kinds of information. Additionally, the existing models mainly focus on learning the inter-relation between structure and attribute information, while the intra-characteristic of each information is ignored, leading to the information loss. To address the above two problems, we propose a novel model named <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">I</b> nter- <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">I</b> ntra Information Preserving Attributed <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">N</b> etwork <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">E</b> mbedding (IINE) to effectively learn the node representations of attributed network, which can <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">not only</i> capture the inter-relation between structure and attribute information with less information distortion <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">but also</i> effectively preserve the intra-characteristic of each information. The proposed model is composed of a primary model named <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">coupled autoencoder</b> and two auxiliary models named <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">structure miner</b> and <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">attribute miner</b> . The coupled autoencoder trains the node representation by smoothly combining both structure and attribute information, while the structure miner and attribute miner are utilized to further mine the intra-characteristic from the corresponding information so as to assist the primary model. The Extensive experiments are conducted on seven real-world datasets, and the results confirm the superior performance of IINE over several state-of-the-art models.

Network embedding (NE) maps a network into a low-dimensional space while preserving intrinsic features of the network. Variational Auto-Encoder (VAE) has been actively studied for NE. These VAE-based methods typically utilize both network topologies and node semantics and treat these two types of data in the same way. However, the information of network topology and information of node semantics are orthogonal and are often from different sources; the former quantifies coupling relationships among nodes, whereas the latter represents node specific properties. Ignoring this difference affects NE. To address this issue, we develop a network-specific VAE for NE, named as NetVAE. In the encoding phase of our new approach, compression of network structures and compression of node attributes share the same encoder in order to perform co-training to achieve transfer learning and information integration. In the decoding phase, a dual decoder is introduced to reconstruct network topologies and node attributes separately. Specifically, as a part of the dual decoder, we develop a novel method based on a Gaussian mixture model and the block model to reconstruct network structures. Extensive experiments on large real-world networks demonstrate a superior performance of the new approach over the state-of-the-art methods.

Network Embedding, which represents nodes in networks with efficient low-dimensional vectors, has been proved useful in a variety of applications. However, most existing approaches study single-view networks but not the multi-view networks with multiple types of relationships between nodes. Meanwhile, they ignore the rich features associated with the nodes, which is common in real world. In this paper, we propose a novel network embedding method, Intra-view and Inter-view attention for Multi-view Network Embedding (I2MNE), which leverages both the multi-view network structure and the node features to efficiently generate node representations. Specially, we introduce the intra-view attention when aggregating node features from neighbors for each single view and the inter-view attention when integrating representations across different views. Experiments on two real-world networks show that our approach outperforms other counterpart network embedding methods.

Network embedding leverages the node proximity manifested to learn a\nlow-dimensional node vector representation for each node in the network. The\nlearned embeddings could advance various learning tasks such as node\nclassification, network clustering, and link prediction. Most, if not all, of\nthe existing works, are overwhelmingly performed in the context of plain and\nstatic networks. Nonetheless, in reality, network structure often evolves over\ntime with addition/deletion of links and nodes. Also, a vast majority of\nreal-world networks are associated with a rich set of node attributes, and\ntheir attribute values are also naturally changing, with the emerging of new\ncontent patterns and the fading of old content patterns. These changing\ncharacteristics motivate us to seek an effective embedding representation to\ncapture network and attribute evolving patterns, which is of fundamental\nimportance for learning in a dynamic environment. To our best knowledge, we are\nthe first to tackle this problem with the following two challenges: (1) the\ninherently correlated network and node attributes could be noisy and\nincomplete, it necessitates a robust consensus representation to capture their\nindividual properties and correlations; (2) the embedding learning needs to be\nperformed in an online fashion to adapt to the changes accordingly. In this\npaper, we tackle this problem by proposing a novel dynamic attributed network\nembedding framework - DANE. In particular, DANE first provides an offline\nmethod for a consensus embedding and then leverages matrix perturbation theory\nto maintain the freshness of the end embedding results in an online manner. We\nperform extensive experiments on both synthetic and real attributed networks to\ncorroborate the effectiveness and efficiency of the proposed framework.\n

Network embedding, which aims at learning distributed representations for nodes in networks, is a critical task with wide downstream applications. Most existing studies focus on networks with a single type of edges, whereas in many cases, the edges of networks can be derived from two opposite relationships, yielding signed networks. This paper studies network embedding for the signed network, and a novel approach called TEA is proposed. Similar to existing methods, TEA (Triad+Edge+Attention) learns node representations by predicting the sign of each edge in the network. However, many existing methods only consider the local structural information (i.e., the representations of nodes in an edge) for prediction, which can be biased especially for sparse networks. By contrast, TEA seeks to leverage the high-order structures by drawing inspirations from the Structural Balance Theory. More specifically, for an edge linking two nodes, TEA predicts the edge sign by considering the triangles connecting the two nodes as features. Meanwhile, an attention mechanism is proposed, which assigns different weights to the different triangles before aggregating their predictions for more precise results. We conduct experiments on several real-world signed networks, and the results prove the effectiveness of TEA over many strong baseline approaches.

Real-time data processing applications demand dynamic resource provisioning and efficient service discovery, which is particularly challenging in resource-constraint edge computing environments. Network embedding techniques can potentially aid effective resource discovery services in edge environments, by achieving a proximity-preserving representation of the network resources. Most of the existing techniques of network embedding fail to capture accurate proximity information among the network nodes and further lack exploiting information beyond the second-order neighbourhood. This paper leverages artificial intelligence for network representation and proposes a deep learning model, named inductive content augmented network embedding (ICANE), which integrates the network structure and resource content attributes into a feature vector. Secondly, a hierarchical aggregation approach is introduced to explicitly learn the network representation through sampling the nodes and aggregating features from the higher-order neighbourhood. A semantic proximity search model is then designed to generate the top-k ranking of relevant nodes using the learned network representation. Experiments conducted on real-world datasets demonstrate the superiority of the proposed model over the existing popular methods in terms of resource discovery and the query resolving performance.

Data driven method has been investigated in fault diagnosis. This kind of methods usually need of a large number of labeled data in order to obtain a good model. However, the unlabeled data dominates the data availability in the real industries, which impedes the successfully application of these methods. To remedy this situation, this research investigates a new semi-supervised fault diagnosis method, and it aims at training an adaptive model by using the small labeled data and large unlabeled data. Firstly, the fault signal is transformed to the RGB format. Then, Convolutional Neural Network (CNN) is trained to generate the fault features combining with the Correlation Alignment (CORAL) loss. Finally, the Transfer Component Analysis (TCA) is adopted to classify the fault features to improve the prediction accuracy. The proposed semi-supervised method is conducted on three bearing datasets. The experiments with different proportions of labeled data are done to validate the proposed CNN with CORAL and TCA. The comparisons between the proposed CNN model with other popular deep learning methods are also presented. The results show that CNN with CORAL and TCA is potential for fault diagnosis, and the prediction accuracies of proposed method outperform other comparisons methods.

In recent years, network embedding has attracted much attention from researchers and achieved excellent performance. But few works investigate the adaptability of network embedding, especially for performance in different network structures. Heterogeneity, as a universal topological characteristic, plays a prominent role in network behaviors. In this study, we investigate the effect of heterogeneity on the effectiveness of existing network embedding approaches. We conduct experiments in scale-free networks with varying power exponents from both macro and micro perspectives to address link prediction and node similarity tasks, respectively. The results indicate that network embedding approaches can be divided into two classes according to their performance in the link prediction task. As the network heterogeneity decreases, the performance of approaches in the first class declines, while the performance of approaches in the second class initially improves and then declines. Moreover, our simulation discovers that, based on the node similarity metric, nodes are partitioned into two clusters by approaches, corresponding to large-degree nodes and small-degree nodes, respectively. Furthermore, approaches in the same class present similar characteristics between large-degree nodes and small-degree nodes, and the embedding is interpreted to some extent. Performance variations in the link prediction task can be explained by the characteristics of approaches, and similar characteristics are confirmed in experiments on real networks. Based on the findings for link prediction, we offer a brief guide for choosing an appropriate method based on the extent of heterogeneity. The investigation provides insight into network embedding and offers some interpretation of embedding.