Learning Universal Network Representation via Link Prediction by Graph Convolutional Neural Network

Link prediction is a technology to find existing but unobserved links (static) or predict new links (dynamic) by studying the network topology. In real life, most networks change with time. Some recent works focus on the dynamic prediction of networks, but they do not solve the weight problem in link prediction well. At the same time, the methods based on matrix decomposition and node embedding used in most link prediction tasks have some problems, such as huge amount of calculation or can’t better represent the time evolution. In this paper, we extend the method of predicting the weight between nodes in dynamic networks and propose a generative adversarial model based on multi-layer graph convolution and recurrent neural network. The model consists of a generator and discriminator trained during an adversarial process. Multi-layer graph convolution used to approximate the high order similarity of network graph, so as to compensate the time evolution characteristic that RNN network can only deal with low order similarity. Using the process of confrontation training, the model can learn the representation of robust time evolution and predict with high accuracy. Experiments on several real data sets show that our model has good adaptability better than several baseline models.

With regards to network structured data nowadays, link prediction stands out as a key problem. It has been observed that interactions between the entities act as the basic foundations of many applications in various fields such as chemistry, biology, or social networks. In the previous studies on this subject, most of the approaches used heuristics methods with an evaluation function to find the similarity index between entities, thereby predicting the possibility of links between them. However, heuristics methods are based on assumptions about the existence of associations between entities; thus its shortcoming is that when these assumptions are not accurate, the results of the algorithm would be reduced significantly. Taking those into considerations, in this paper, we propose a method to solve this problem based on deep graph convolutional neural network. Experimental results on paper citation network dataset have shown that particular and this method is promising.KeywordsGraph neural networksDeep graph convolutional neural networkLink predictionPaper citation network

Cross-subject emotion recognition in brain-computer interface based on frequency band attention graph convolutional adversarial neural networks

Video-based human action recognition is one of the most important and challenging areas of research in the field of computer vision. Human action recognition has found many pragmatic applications in video surveillance, human-computer interaction, entertainment, autonomous driving, etc. Owing to the recent development of deep learning methods for human action recognition, the performance of action recognition has significantly enhanced for challenging datasets. Deep learning techniques are mainly used for recognizing actions in images and videos comprising of Euclidean data. A recent development in deep learning methods is the extension of these techniques to non-Euclidean data or graph data with many nodes and edges. Human body skeleton resembles a graph, therefore, the graph convolutional network (GCN) is applicable to the non-Euclidean body skeleton. In the past few years, GCN has emerged as an important tool for skeleton-based action recognition. Therefore, we conduct a survey using GCN methods for action recognition. Herein, we present a comprehensive overview of recent GCN techniques for action recognition, propose a taxonomy for the categorization of GCN techniques for action recognition, carry out a detailed study of the benchmark datasets, enlist relevant resources and open-source codes, and finally provide an outline for future research directions and trends. To the best of authors' knowledge, this is the first survey for action recognition using GCN techniques.

In recent years, network embedding has attracted much attention from researchers and achieved excellent performance. But few works investigate the adaptability of network embedding, especially for performance in different network structures. Heterogeneity, as a universal topological characteristic, plays a prominent role in network behaviors. In this study, we investigate the effect of heterogeneity on the effectiveness of existing network embedding approaches. We conduct experiments in scale-free networks with varying power exponents from both macro and micro perspectives to address link prediction and node similarity tasks, respectively. The results indicate that network embedding approaches can be divided into two classes according to their performance in the link prediction task. As the network heterogeneity decreases, the performance of approaches in the first class declines, while the performance of approaches in the second class initially improves and then declines. Moreover, our simulation discovers that, based on the node similarity metric, nodes are partitioned into two clusters by approaches, corresponding to large-degree nodes and small-degree nodes, respectively. Furthermore, approaches in the same class present similar characteristics between large-degree nodes and small-degree nodes, and the embedding is interpreted to some extent. Performance variations in the link prediction task can be explained by the characteristics of approaches, and similar characteristics are confirmed in experiments on real networks. Based on the findings for link prediction, we offer a brief guide for choosing an appropriate method based on the extent of heterogeneity. The investigation provides insight into network embedding and offers some interpretation of embedding.

Knowledge graph embedding models are used to learn low‐dimensional representations of entities and relations in knowledge graphs. In this paper, we propose Multi‐RAttE, an attention‐based learning method for multiple relational knowledge graph embedding representation, which divides the information transfer in the knowledge graph into cross‐relational information transfer and relation‐specific information transfer, and divides the embedding of knowledge graph entities into structural embedding and multi‐relational embedding for joint learning. To objectively analyse the performance of the Multi‐RAttE model, we select two typical datasets and different representative baseline models for experimental evaluation on several tasks such as link prediction, multi‐relation prediction and node classification. The experimental results show that the Multi‐RAttE model improves 8% over the state‐of‐the‐art model Composition‐based Multi‐Relational Graph Convolutional Networks (CompGCN) in terms of Hits@1 metric on the link prediction task on FB15k‐237 dataset; on the multi‐relation prediction task, the accuracy improves by 1.8% and 3.7% in the auc metric and F1 metric, respectively. The experimental results have proved that the Multi‐RAttE model can effectively perform the representation of multiple relations.

In this paper, we propose a new approach named D2NE, short for Deep Dynamic Network Embedding, to learn the vertex representations for dynamic networks. The algorithm utilizes the graph attention mechanism to refresh embeddings efficiently, in which each update associate with local information only. To address the missing data, which is a common phenomenon in real-world networks, we model the auxiliary side information to capture more information on vertex relations. D2NE is highly efficient to refresh embeddings for dynamic networks, even with standard real-world sparse networks. We conduct extensive experiments on several real-world dynamic networks to validate the performance of D2NE in the link prediction task. Both quantitative results and qualitative analysis verify the effectiveness and rationality of our D2NE method.

Messenger RNA (mRNA) has emerged as a critical global technology that requires global joint efforts from different entities to develop a COVID-19 vaccine. However, the chemical properties of RNA pose a challenge in utilizing mRNA as a vaccine candidate. For instance, the molecules are prone to degradation, which has a negative impact on the distribution of mRNA among patients. In addition, little is known of the degradation properties of individual RNA bases in a molecule. Therefore, this study aims to investigate whether a hybrid deep learning can predict RNA degradation from RNA sequences. Two deep hybrid neural network models were proposed, namely GCN_GRU and GCN_CNN. The first model is based on graph convolutional neural networks (GCNs) and gated recurrent unit (GRU). The second model is based on GCN and convolutional neural networks (CNNs). Both models were computed over the structural graph of the mRNA molecule. The experimental results showed that GCN_GRU hybrid model outperform GCN_CNN model by a large margin during the test time. Validation of proposed hybrid models is performed by well-known evaluation measures. Among different deep neural networks, GCN_GRU based model achieved best scores on both public and private MCRMSE test scores with 0.22614 and 0.34152, respectively. Finally, GCN_GRU pre-trained model has achieved the highest AuC score of 0.938. Such proven outperformance of GCNs indicates that modeling RNA molecules using graphs is critical in understanding molecule degradation mechanisms, which helps in minimizing the aforementioned issues. To show the importance of the proposed GCN_GRU hybrid model, in silico experiments has been contacted. The in-silico results showed that our model pays local attention when predicting a given position’s reactivity and exhibits interesting behavior on neighboring bases in the sequence.

Network embedding, which aims to learn low-dimensional representations of nodes, has been used for various graph related tasks including visualization, link prediction and node classification. Most existing embedding methods rely solely on network structure. However, in practice we often have auxiliary information about the nodes and/or their interactions, e.g., content of scientific papers in co-authorship networks, or topics of communication in Twitter mention networks. Here we propose a novel embedding method that uses both network structure and edge attributes to learn better network representations. Our method jointly minimizes the reconstruction error for higher-order node neighborhood, social roles and edge attributes using a deep architecture that can adequately capture highly non-linear interactions. We demonstrate the efficacy of our model over existing state-of-the-art methods on a variety of real-world networks including collaboration networks, and social networks. We also observe that using edge attributes to inform network embedding yields better performance in downstream tasks such as link prediction and node classification.

Drug repositioning (also called drug repurposing) is a strategy for identifying new therapeutic targets for existing drugs. This approach is of great importance in pharmacology as it is a faster and cheaper way to develop new medical treatments. In this paper, we present, to our knowledge, the first application of multiplex-heterogeneous network embedding to drug repositioning. Network embedding learns the vector representations of nodes, opening the whole machine learning toolbox for a wide variety of applications including link prediction, node labelling or clustering. So far, the application of network embedding for drug repositioning focused on heterogeneous networks. Our approach for drug repositioning is based on multiplex-heterogeneous network embedding. Such method allows the richness and complexity of multiplex and heterogeneous networks to be projected in the same vector space. In other words, multiplex-heterogeneous networks aggregate different multi-omics data in the same network representation. We validate the approach on a task of link prediction and on a case study for SARS-CoV2 drug repositioning. Experimental results show that our approach is highly robust and effective for finding new drug-target associations.KeywordsNetwork embeddingMultiplex-heterogeneous networkMulti-layer networkDrug repositioningGraph representation learningSARS-CoV2COVID-19

GCNFusion: An efficient graph convolutional network based model for information diffusion

Neural link predictors learn distributed representations of entities and relations in a knowledge graph. They are remarkably powerful in the link prediction and knowledge base completion tasks, mainly due to the learned representations that capture important statistical dependencies in the data. Recent works in the area have focused on either designing new scoring functions or incorporating extra information into the learning process to improve the representations. Yet the representations are mostly learned from the observed links between entities, ignoring commonsense or schema knowledge associated with the relations in the graph. A fundamental aspect of the topology of relational data is the cardinality information, which bounds the number of predictions given for a relation between a minimum and maximum frequency. In this paper, we propose a new regularisation approach to incorporate relation cardinality constraints to any existing neural link predictor without affecting their efficiency or scalability. Our regularisation term aims to impose boundaries on the number of predictions with high probability, thus, structuring the embeddings space to respect commonsense cardinality assumptions resulting in better representations. Experimental results on Freebase, WordNet and YAGO show that, given suitable prior knowledge, the proposed method positively impacts the predictive accuracy of downstream link prediction tasks.

Geometric deep learning provides a principled and universal way for the integration of imaging and non-imaging modes in the medical field. Graph Convolutional Networks (GCNs) have been widely explored in a variety of problems, such as disease prediction, segmentation, and matrix completion. Using large, multi-modal data sets, graphs can capture the interaction of individual elements represented as nodes in the graphs. In particular, in medical applications, nodes can represent individuals (patients or healthy controls) in a potentially large population and are accompanied by a set of features, while the edges of the graph contain the associations between subjects in an intuitive way. This representation allows the inclusion of rich imaging and non-imaging information as well as individual subject characteristics in the task of disease classification. This article gives an overview of graph convolutional neural networks, focusing on the application of graph convolutional neural networks in disease prediction, and discusses the challenges and prospects of graph convolutional neural networks in disease prediction.

Graph, as an important data representation, is ubiquitous in many real world applications ranging from social network analysis to biology. How to correctly and effectively learn and extract information from graph is essential for a large number of machine learning tasks. Graph embedding is a way to transform and encode the data structure in high dimensional and non-euclidean feature space to a low dimensional and structural space, which is easily exploited by other machine learning algorithms. We have witnessed a huge surge of such embedding methods, from statistical approaches to recent deep learning methods such as the graph convolutional networks (GCN). Deep learning approaches usually outperform the traditional methods in most graph learning benchmarks by building an end-to-end learning framework to optimize the loss function directly. However, most of the existing GCN methods can only perform convolution operations with node features, while ignoring the handy information in edge features, such as relations in knowledge graphs. To address this problem, we present CensNet, Convolution with Edge-Node Switching graph neural network, for learning tasks in graph-structured data with both node and edge features. CensNet is a general graph embedding framework, which embeds both nodes and edges to a latent feature space. By using line graph of the original undirected graph, the role of nodes and edges are switched, and two novel graph convolution operations are proposed for feature propagation. Experimental results on real-world academic citation networks and quantum chemistry graphs show that our approach achieves or matches the state-of-the-art performance in four graph learning tasks, including semi-supervised node classification, multi-task graph classification, graph regression, and link prediction.

The link prediction problem is a time-evolving model in network science that has simultaneously abetted myriad applications and experienced extensive methodological improvement. Inferring the possibility of emerging links in dynamic social networks, also known as the dynamic link prediction task, is complex and challenging. In contrast to the link prediction in cross-sectional networks, dynamic link prediction methods need to cater to the actor-level temporal changes and associated evolutionary information regarding their micro- (i.e., link formation/deletion) and mesoscale (i.e., community formation) network structure. With the advent of abundant community detection algorithms, the research community has examined community-aware link prediction strategies in static networks. However, the same task in dynamic networks where, apart from the actors and links among them, their community pattern is also dynamic, is yet to be explored. Evolutionary community-aware information, including the associated link structure and temporal neighborhood changes, can effectively be mined to build dynamic similarity metrics for dynamic link prediction. This study aims to develop and integrate such dynamic features with machine learning algorithms for link prediction tasks in dynamic social networks. It also compares the performances of these features against well-known similarity metrics (i.e., ResourceAllocation) for static networks and a time series-based link prediction strategy in dynamic networks. These proposed features achieved high-performance scores, representing them as prospective candidates for both dynamic link prediction tasks and modeling the network growth.