Exploiting Mutual Information for Substructure-aware Graph Representation Learning

Nowadays, graph structure data has played a key role in machine learning because of its simple topological structure, and therefore, the graph representation learning methods have attracted great attention. And it turns out that the low-dimensional embedding representation obtained by graph representation learning is extremely useful in various typical tasks, such as node classification and content recommendation. However, most of the existing methods do not further dig out potential structural information on the original graph structure. Here, we propose wGCN, which utilizes random walk to obtain the node-specific mesoscopic structures (high-order local structure) of the graph and utilizes these mesoscopic structures to enhance the graph and organize the characteristic information of the nodes. Our method can effectively generate node embedding for data of previously unknown categories, which has been proven in a series of experiments conducted on many types of graph networks. And compared to baselines, our method shows the best performance on most datasets and achieves competitive results on others. It is believed that combining the mesoscopic structure to further explore the structural information of the graph will greatly improve the learning efficiency of the graph neural network.

Cross-scale contrastive triplet networks for graph representation learning

The prediction of drug–target interactions is always a key task in the field of drug redirection. However, traditional methods of predicting drug–target interactions are either mediocre or rely heavily on data stacking. In this work, we proposed our model named GraphMS. We merged heterogeneous graph information and obtained effective node information and substructure information based on mutual information in graph embeddings. We then learned high quality representations for downstream tasks, and proposed an end–to–end auto–encoder model to complete the task of link prediction. Experimental results show that our method outperforms several state–of–the–art models. The model can achieve the area under the receiver operating characteristics (AUROC) curve of 0.959 and area under the precise recall curve (AUPR) of 0.847. We found that the mutual information between the substructure and graph–level representations contributes most to the mutual information index in a relatively sparse network. And the mutual information between the node–level and graph–level representations contributes most in a relatively dense network.

Vision-language representation learning largely benefits from image-text alignment through contrastive losses (e.g., InfoNCE loss). The success of this alignment strategy is attributed to its capability in maximizing the mutual information (MI) between an image and its matched text. However, simply performing cross-modal alignment (CMA) ignores data potential within each modality, which may result in degraded representations. For instance, although CMA-based models are able to map image-text pairs close together in the embedding space, they fail to ensure that similar inputs from the same modality stay close by. This problem can get even worse when the pre-training data is noisy. In this paper, we propose triple contrastive learning (TCL) for vision-language pre-training by leveraging both cross-modal and intra-modal self-supervision. Besides CMA, TCL introduces an intra-modal contrastive objective to provide complementary benefits in representation learning. To take advantage of localized and structural information from image and text input, TCL further maximizes the average MI between local regions of image/text and their global summary. To the best of our knowledge, ours is the first work that takes into account local structure information for multi-modality representation learning. Experimental evaluations show that our approach is competitive and achieves the new state of the art on various common downstream vision-language tasks such as image-text retrieval and visual question answering.

Dynamic heterogeneous graph representation learning with neighborhood type modeling

<p dir="ltr">With the development of new neural network architectures for graph representation learning in recent years, the use of graphs to store, represent and process data has become increasingly more important. This thesis focuses on both structural and semantic aspects of graphs in order to design and develop neural representation learning models that are capable of performing effective and efficient search on graphs in order to identify and retrieve relevant subgraph structures. Graph search is an NP-hard problem, with existing methods struggling to balance accuracy and efficiency. This work aims to design robust neural representations that address these challenges. The first contribution examines team formation as a case study of search within complete graphs. Team formation is concerned with the identification of a group of experts who have a high likelihood of effectively collaborating with each other in order to satisfy a collection of input skills. This thesis proposes a variational Bayesian neural network architecture that learns representations for teams whose members have collaborated with each other in the past. The learnt representations allow our proposed approach to mine teams that have a past collaborative history and collectively cover the requested desirable set of skills. Through our experiments on DBLP and Dota2 datasets, we demonstrate that our approach shows stronger performance compared to a range of strong team formation techniques from both quantitative and quality perspectives. Furthermore, we redefine team discovery as a task of learning subgraph representations from heterogeneous collaboration networks. Our method captures both local (node interactions within teams) and global (subgraph interactions between teams) characteristics, enabling seamless mapping between homogeneous and heterogeneous subgraphs to effectively discover teams. Extensive experiments on two real-world datasets confirm the effectiveness of the approach. In the context of incomplete graphs, we introduce a novel graph neural network representation learning technique specifically tailored for graphs with missing information. We propose a novel keyword graph representation learning method that incorporates complementary aspects of graphs: global, local, adjusted, and feature semantics. Considering these multiple aspects, our approach remains robust and resilient to missing information. We adopt and fine-tune a transformer-based model to aggregate the various features of a graph to generate rich representations, recognizing the pivotal role of keywords in this task. We show through experiments on real-world data that our method outperforms the state-ofthe-art approaches and is particularly robust in the face of missing values, underscoring its ability to effectively handle incomplete graphs.</p>

Graph neural networks (GNNs) have emerged as a powerful tool in graph representation learning. However, they are increasingly challenged by over-smoothing as network depth grows, compromising their ability to capture and represent complex graph structures. Additionally, some popular GNN variants only consider local neighbor information during node updating, ignoring the global structural information and leading to inadequate learning and differentiation of graph structures. To address these challenges, we introduce a novel graph neural network framework, GraphSAGE++. Our model extracts the representation of the target node at each layer and then concatenates all layer weighted representations to obtain the final result. In addition, the strategies combining double aggregations with weighted concatenation are proposed, which significantly enhance the model’s discernment and preservation of structural information. Empirical results on various datasets demonstrate that GraphSAGE++ excels in vertex classification, link prediction, and visualization tasks, surpassing existing methods in effectiveness.

Frontiers Events is a rapidly growing calendar management system dedicated to the scheduling of academic events. This includes announcements and invitations, participant listings and search functionality, abstract handling and publication, related events and post-event exchanges. Whether an organizer or participant, make your event a Frontiers Event!

The corrugated layer structure bismuth has been successfully tailored into negative thermal expansion along c axis by size effect. Pair distribution function and extended X-ray absorption fine structure are combined to reveal the local structural distortion for nanosized bismuth. The comprehensive method to identify the local structure of nanomaterials can benefit the regulating and controlling of thermal expansion in nanodivices.

In recent years, graph representation learning has played a huge role in the fields and research of node clustering, node classification, link prediction, etc., among which many excellent models and methods have emerged. These methods can achieve better results for model training and verification of data in a single space domain. However, in real scenarios, the solution of cross-domain problems of multiple information networks is very practical and important, and the existing methods cannot be applied to cross-domain scenarios, so we research on cross-domain representation is based on multi-network space integration. This paper conducts representation learning research for cross-domain scenarios. First, we use different network representation learning methods to perform representation learning in a single network space. Second, we use the attention mechanism to fuse representations in different spaces to obtain a fusion representation of multiple network spaces; Finally, the model is verified through cross-domain experiments. The experimental results show that the fusion model proposed in this paper can improve the performance of cross-domain scenarios.

BLoG: Bootstrapped graph representation learning with local and global regularization for recommendation

People often express opinions towards others in a social network, causing sentiment links to form among users. To develop effective methods for discovering implicit sentiment links among users, the extraction and exploitation of structural semantic information from heterogeneous social networks are of great importance. We propose a novel heterogeneous social network embedding-based approach for sentiment link prediction that takes both global structural information with multi-dimensional relations and heterogeneous context information into consideration to leverage rich and intrinsic association information. Specifically, the attributed heterogeneous social network and Sentic LSTM-based sentiment link network are employed to incorporate various explicit context knowledge and implicit multi-dimensional user interaction association knowledge into representation learning and sentiment link prediction. The experimental results on a real-world dataset show that the proposed approach has advantages over the state-of-the-art baselines. The results show the effectiveness of incorporating social relations and profile context information into sentiment link prediction, especially in cold-start scenarios. The learned embedding representation that incorporates both structural information with multi-dimensional relations and context information from heterogeneous social networks can improve sentiment link prediction performance. The proposed approach is effective and feasible for detecting unobserved sentiment links from online social networks and outperforms the state-of-the-art baselines in sentiment link prediction tasks.

A Comprehensive Survey on Deep Graph Representation Learning

The problem of privacy-preserving data mining has attracted considerable attention in recent years because of increasing concerns about the privacy of the underlying data. In recent years, an important data domain which has emerged is that of graphs and structured data. Many data sets such as XML data, transportation networks, traffic in IP networks, social networks and hierarchically structured data are naturally represented as graphs. Existing work on graph privacy has focussed on the problem of anonymizing nodes or edges of a single graph, in which the identity is assumed to be associated with individual nodes. In this paper, we examine the more complex case, where we have a collection of graphs, and the identity is associated with individual graphs rather than nodes or edges. In such cases, the problem of identity anonymization is extremely difficult, since we need to not only anonymize the labels on the nodes, but also the underlying global structural information. In such cases, both the global and local structural information can be a challenge to the anonymization process, since any combination of such information can be used in order to de-identify the underlying graphs. In order to achieve this goal, we will create synthesized representations of the underlying graphs based on aggregate structural analytics of the collection of graphs. The synthesized graphs retain the properties of the original data while satisfying the k-anonymity requirement. Our experimental results show that the synthesized graphs maintain a high level of structural information and compatible classification accuracies with the original data.

Graph representation learning is an emerging area for graph analysis and inference. However, existing approaches for large-scale graphs either sample nodes in sequential walks or manipulate the adjacency matrices of graphs. The former approach can cause sampling bias against less-connected nodes, whereas the latter may suffer from sparsity that exists in many real-world graphs. To learn from structural information in a graph more efficiently and comprehensively, this paper proposes a new graph representation learning approach inspired by the cognitive model of spreading-activation mechanisms in human memory. This approach learns node embeddings by adopting a graph activation model that allows nodes to “activate” their neighbors and spread their own structural information to other nodes through the paths simultaneously. Comprehensive experiments demonstrate that the proposed model performs better than existing methods on several empirical datasets for multiple graph inference tasks. Meanwhile, the spreading-activation-based model is computationally more efficient than existing approaches–the training process converges after only a small number of iterations, and the training time is linear in the number of edges in a graph. The proposed method works for both homogeneous and heterogeneous graphs.