Is a Single Vector Enough?

In today's networked society, many real-world problems can be formalized as predicting links in networks, such as Facebook friendship suggestions, e-commerce recommendations, and the prediction of scientific collaborations in citation networks. Increasingly often, link prediction problem is tackled by means of network embedding methods, owing to their state-of-the-art performance. However, these methods lack transparency when compared to simpler baselines, and as a result their robustness against adversarial attacks is a possible point of concern: could one or a few small adversarial modifications to the network have a large impact on the link prediction performance when using a network embedding model? Prior research has already investigated adversarial robustness for network embedding models, focused on classification at the node and graph level. Robustness with respect to the link prediction downstream task, on the other hand, has been explored much less. This paper contributes to filling this gap, by studying adversarial robustness of Conditional Network Embedding (CNE), a state-of-the-art probabilistic network embedding model, for link prediction. More specifically, given CNE and a network, we measure the sensitivity of the link predictions of the model to small adversarial perturbations of the network, namely changes of the link status of a node pair. Thus, our approach allows one to identify the links and non-links in the network that are most vulnerable to such perturbations, for further investigation by an analyst. We analyze the characteristics of the most and least sensitive perturbations, and empirically confirm that our approach not only succeeds in identifying the most vulnerable links and non-links, but also that it does so in a time-efficient manner thanks to an effective approximation.

A novel quality prediction model based on dual-layer graph supervised embedding with multi-granularity attention enhancement mechanisms.

Link prediction is a common problem in network science that cuts across many disciplines. The goal is to forecast the appearance of new links or to find links missing in the network. Typical methods for link prediction use the topology of the network to predict the most likely future or missing connections between a pair of nodes. However, network evolution is often mediated by higher-order structures involving more than pairs of nodes; for example, cliques on three nodes (also called triangles) are key to the structure of social networks, but the standard link prediction framework does not directly predict these structures. To address this gap, in recent work, we propose a new link prediction task called “pairwise link prediction” that directly targets the prediction of new triangles, where one is tasked with finding which nodes are most likely to form a triangle with a given edge. We extend this work in this manuscript, and we evaluate a variety of natural extensions to link prediction methods including neighborhood and PageRank-based methods. A key difference from our previous work is the definition of the neighborhood of an edge, which has a surprisingly large impact on the empirical performance. Our experiments on a variety of networks show that diffusion-based methods are less sensitive to the type of graphs used and more consistent in their results. We also show how our pairwise link prediction framework can be used to get better predictions within the context of standard link prediction evaluation.

Language models, such as BERT, construct multiple, contextualized embeddings for each word occurrence in a corpus. Understanding how the contextualization propagates through the model's layers is crucial for deciding which layers to use for a specific analysis task. Currently, most embedding spaces are explained by probing classifiers; however, some findings remain inconclusive. In this paper, we present LMFingerprints, a novel scoring‐based technique for the explanation of contextualized word embeddings. We introduce two categories of scoring functions, which measure (1) the degree of contextualization, i.e., the layerwise changes in the embedding vectors, and (2) the type of contextualization, i.e., the captured context information. We integrate these scores into an interactive explanation workspace. By combining visual and verbal elements, we provide an overview of contextualization in six popular transformer‐based language models. We evaluate hypotheses from the domain of computational linguistics, and our results not only confirm findings from related work but also reveal new aspects about the information captured in the embedding spaces. For instance, we show that while numbers are poorly contextualized, stopwords have an unexpected high contextualization in the models' upper layers, where their neighborhoods shift from similar functionality tokens to tokens that contribute to the meaning of the surrounding sentences.

Network embedding aims to learn distributed vector representations of nodes in a network. The problem of network embedding is fundamentally important. It plays crucial roles in many applications, such as node classification, link prediction, and so on. As the real-world networks are often sparse with few observed links, many recent works have utilized the local and global network structure proximity with shallow models for better network embedding. In reality, each node is usually associated with rich attributes. Some attributed network embedding models leveraged the node attributes in these shallow network embedding models to alleviate the data sparsity issue. Nevertheless, the underlying structure of the network is complex. What is more, the connection between the network structure and node attributes is also hidden. Thus, these previous shallow models fail to capture the nonlinear deep information embedded in the attributed network, resulting in the suboptimal embedding results. In this paper, we propose a deep attributed network embedding framework to capture the complex structure and attribute information. Specifically, we first adopt a personalized random walk-based model to capture the interaction between network structure and node attributes from various degrees of proximity. After that, we construct an enhanced matrix representation of the attributed network by summarizing the various degrees of proximity. Then, we design a deep neural network to exploit the nonlinear complex information in the enhanced matrix for network embedding. Thus, the proposed framework could capture the complex attributed network structure by preserving both the various degrees of network structure and node attributes in a unified framework. Finally, empirical experiments show the effectiveness of our proposed framework on a variety of network embedding-based tasks.

This paper aims to accelerate scientific discovery by studying link prediction in a semantic network. The nodes are unidentified concepts in machine learning, and the time-stamped edges indicate co-occurrence in scientific papers. Taking advantage of this temporal information, we perform node embedding on the graph at every year from 1994 to 2017, and apply two methods to find features for node pairs: the first method uses a transformer, while the other uses distance metrics combined with known link prediction features. The latter feature extraction technique with a 3-layer multi-layer perceptron achieved an AUC of 0.902 on predicting edges in the 2020 graph. Inspection of the resulting features suggests that the model does indeed pay attention to the dynamic nature of the features, e.g., how node-pair distance in embedding space changes over the years.

Vector embedding models have recently become popular for encoding both structured and unstructured data. In the context of knowledge graphs such models often serve as additional evidence supporting various tasks related to the knowledge base population: e.g., information extraction or link prediction to expand the original dataset. However, the embedding models themselves are often not used directly alongside structured data: they merely serve as additional evidence for structured knowledge extraction. In the metaphactory knowledge graph management platform, we use federated hybrid SPARQL queries for combining explicit information stated in the graph, implicit information from the associated embedding models, and information extracted using vector embeddings in a transparent way for the end user. In this paper we show how we integrated RDF data with vector space models to construct an augmented knowledge graph to be used in customer applications.

Online users are typically active on multiple social media networks (SMNs), which constitute a multiplex social network. With improvements in cybersecurity awareness, users increasingly choose different usernames and provide different profiles on different SMNs. Thus, it is becoming increasingly challenging to determine whether given accounts on different SMNs belong to the same user; this can be expressed as an interlayer link prediction problem in a multiplex network. To address the challenge of predicting interlayer links, feature or structure information is leveraged. Existing methods that use network embedding techniques to address this problem focus on learning a mapping function to unify all nodes into a common latent representation space for prediction; positional relationships between unmatched nodes and their common matched neighbors (CMNs) are not utilized. Furthermore, the layers are often modeled as unweighted graphs, ignoring the strengths of the relationships between nodes. To address these limitations, we propose a framework based on multiple types of consistency between embedding vectors (MulCEVs). In MulCEV, the traditional embedding-based method is applied to obtain the degree of consistency between the vectors representing the unmatched nodes, and a proposed distance consistency index based on the positions of nodes in each latent space provides additional clues for prediction. By associating these two types of consistency, the effective information in the latent spaces is fully utilized. In addition, MulCEV models the layers as weighted graphs to obtain representation. In this way, the higher the strength of the relationship between nodes, the more similar their embedding vectors in the latent representation space will be. The results of our experiments on several real-world and synthetic datasets demonstrate that the proposed MulCEV framework markedly outperforms current embedding-based methods, especially when the number of training iterations is small.

Counterfeit and substandard product crimes pose significant threats to consumer safety and market stability, yet the incompleteness of knowledge graphs in this domain hampers effective case analysis and decision-making. Link prediction (LP) captures both structural and semantic representations of entities to infer latent relationships, thereby mitigating graph incompleteness and enabling more accurate investigation and law enforcement. To achieve this, most existing works employ graph neural networks to aggregate neighbor information of entities, aiming to capture both structural and semantic features in the embedding process. However, due to the heterogeneity of the neighbor information, directly aggregating it can negatively impact the prediction accuracy. In this paper, we propose a neighbor-aware embedding interaction (NEI) model that computes interactions between semantic and structural neighbors to sufficiently aggregate heterogeneous neighbor information, which improves LP performance. Specifically, we first propose dependency direction enhancement module that extracts high-quality semantic neighbors to improve understanding for entities with sparse relationships. Subsequently, a neighbor-aware interaction module captures valuable hidden information from both semantic and structural neighbors, enriching the neighbor embedding representation. Then, we propose a neighbor-aware convolution method to improve the prediction performance by incorporating rich neighbor embedding vectors for head and relation embedding vectors of each triple through interaction. We conducted a series of experiments on three public LP datasets to evaluate the effectiveness of the NEI model. The NEI model outperforms the suboptimal baseline by 7. 3% and 5. 3% in the WN18RR dataset in terms of MRR and Hits@1, respectively.

Proximity measures for link prediction based on temporal events

Introduction Network representation can give a holistic view of relationships for biomedical entities through network topology. Link prediction estimates the probability of link formation between the pair of unconnected nodes. In the drug discovery process, the link prediction method not only enables the detection of connectivity patterns but also predicts the effects of one biomedical entity to multiple entities simultaneously and vice versa, which is useful for many applications. Areas covered The authors provide a comprehensive overview of network link prediction in drug discovery. Link prediction methodologies such as similarity-based approaches, embedding-based approaches, probabilistic model-based approaches, and preprocessing methods are summarized with examples. In addition to describing their properties and limitations, the authors discuss the applications of link prediction in drug discovery based on the relationship between biomedical concepts. Expert opinion Link prediction is a powerful method to infer the existence of novel relationships in drug discovery. However, link prediction has been hampered by the sparsity of data and the lack of negative links in biomedical networks. With preprocessing to balance positive and negative samples and the collection of more data, the authors believe it is possible to develop more reliable link prediction methods that can become invaluable tools for successful drug discovery.

Link prediction aims to identify missing links within static networks or estimate the probability of emerging links in dynamic networks, representing a critical and challenging research direction in complex network. Many similarity-based methods have been established from various viewpoints, however, there are relatively few methods that take into account both the path information between node pairs and the nodes along the path. To fill this gap, we propose two novel link prediction algorithms, namely LPRA and LPH. The core idea is that the similarity between node pairs is closely related to the local paths connecting the two nodes and the resource transition capability of the nodes along those paths. The algorithms utilize the local paths with adjustable lengths between node pairs and the topological information of the nodes on the path to calculate the similarity index, which incorporate the resource transition capabilities of all the nodes on the possible paths between node pairs. We conducted multiple groups of comparative experiments on 10 real-world networks to validate the effectiveness of the proposed algorithm. Experimental results demonstrate that LPRA and LPH outperform nine classical methods and five recently popular methods. Moreover, LPRA demonstrates a significant difference level (P-value [Formula: see text]) compared to most of the methods in the ANOVA of AUC accuracy, indicating a statistically significant performance advantage of LPRA.

Various social networks have become important part of the people. This social network can be represented as graph. which change with respect to time different nodes and edges are added. In this graph users can be represented as nodes and edges can be thought as relationship between these nodes (User). The goal of link prediction is to finding missing links and prediction future relationships. The task of recommending new relationships(edges) to users(nodes) can be framed as task to Link prediction in a graph. We will use supervised Machine learning approaches with a set of manually extracted features along with node embedding generated using node2vec to increase the performance. Then the trained model is used for performing link Prediction between nodes on the Twitch dataset, which is collected from SNAP. recommendation problem can be mapped to a binary classification problem wherein the two classes are - recommend or do not recommend. We propose solving this by generating link predictions and then using the predicted links to recommend streamers. Thus, we aim to train classification models using machine learning to predict whether a link exists between any given pair of nodes and use this prediction for recommendations. of game. The performance of the model is assessed utilizing prediction performance metric

Link prediction of time-evolving network based on node ranking

Knowledge graph embedding (KGE) models represent each entity and relation of a knowledge graph (KG) with low-dimensional embedding vectors. These methods have recently been applied to KG link prediction and question answering over incomplete KGs (KGQA). KGEs typically create an embedding for each entity in the graph, which results in large model sizes on real-world graphs with millions of entities. For downstream tasks these atomic entity representations often need to be integrated into a multi stage pipeline, limiting their utility. We show that an off-the-shelf encoder-decoder Transformer model can serve as a scalable and versatile KGE model obtaining state-of-the-art results for KG link prediction and incomplete KG question answering. We achieve this by posing KG link prediction as a sequence-to-sequence task and exchange the triple scoring approach taken by prior KGE methods with autoregressive decoding. Such a simple but powerful method reduces the model size up to 98% compared to conventional KGE models while keeping inference time tractable. After finetuning this model on the task of KGQA over incomplete KGs, our approach outperforms baselines on multiple large-scale datasets without extensive hyperparameter tuning.