Hierarchical Multi-View Graph Pooling with Structure Learning

Graph Neural Networks have been popularly investigated and achieved promising performance in node-wise and graph-level tasks, like node classification and graph classification. For graph classification, correctly defining pooling operations to coarsen an input graph for distilling hierarchical knowledge from node representations is demanded. Some works have devised pooling strategies to summarize a graph by evaluating and gathering the ranked top-k representative nodes. Yet there is few literature to explicitly consider the node's degree-specific structure information in pooling procedure. One intuition is the larger the node's degree is, the more important it probably is in the entire graph. In this work, we propose a novel Degree-Induced Pooling (DIPool), which can be integrated into various graph neural network architectures. Specifically, the DIPool layer devises two approaches for explicitly incorporating the characteristics and degree-specific structures of nodes into a unified criterion to evaluate the importance of the nodes within an input graph. This setting permits to retain more representative nodes to generate the newly pooled graph. Experimental results on a collection of public benchmarks have demonstrated the effectiveness of the proposed DIPool operator. Especially, DIPool achieves relative improvements of 3.69% on NCI1 and 3.24% on NCI109 compared with the best baseline ASAP.

This study introduces a novel approach to drug discovery in regenerative medicine through the utilization of a graph neural network (GNN). The research methodology integrates the development and training of the GNN with a subsequent evaluation of its performance metrics. The first phase involves the generation of synthetic data simulating a biological network, employing networkX and NumPy libraries to construct a random graph with Erdos-Renyi topology. The data, representing cellular responses to biomaterials, is then converted into PyTorch tensors for compatibility with the GNN architecture. The GNN model, characterized by two fully connected layers with ReLU and log-softmax activations, captures intricate relationships within the graph-structured data. The second phase employs a stochastic gradient descent algorithm, specifically the Adam optimizer, to train the GNN over 100 epochs using the cross-entropy loss for multi-class classification. The research methodology extends to the evaluation phase, producing three distinct output graphs for analysis: Visualization of the graph structure, a comparison between predicted and true labels, and a plot illustrating training loss over epochs. Performance metrics, including accuracy, precision, recall, and F1-score, are computed to assess the model’s predictive capabilities quantitatively. The study concludes with a discussion on the nuances revealed by each graph and their implications for refining GNN models in the context of drug discovery for regenerative medicine.

Graph neural networks (GNNs) is an information - processing system that uses message passing among graph nodes. In recent years, GNN variants including graph attention network (GAT), graph convolutional network (GCN), and graph recurrent network (GRN) have shown revolutionary performance in computer vision applications using deep learning and artificial intelligence. These neural network model extensions, collect information in the form of graphs. GNN may be divided into three groups based on the challenges it solves: link prediction, node classification, graph classification. Machines can differentiate and recognise objects in image and video using standard CNNs. Extensive amount of research work needs to be done before robots can have same visual intuition as humans. GNN architectures, on the other hand, may be used to solve various image categorization and video challenges. The number of GNN applications in computer vision not limited, continues to expand. Human-object interaction, actin understanding, image categorization from a few shots and many more. In this paper use of GNN in image and video understanding, design aspects, architecture, applications and implementation challenges towards computer vision is described. GNN is a strong tool for analysing graph data and is still a relatively active area that needs further researches attention to solve many computer vision applications.

Graph neural networks (GNNs) have recently achieved remarkable success on a variety of graph-related tasks, while such success relies heavily on a given graph structure that may not always be available in real-world applications. To address this problem, graph structure learning (GSL) is emerging as a promising research topic where task-specific graph structure and GNN parameters are jointly learned in an end-to-end unified framework. Despite their great progress, existing approaches mostly focus on the design of similarity metrics or graph construction, but directly default to adopting downstream objectives as supervision, which lacks deep insight into the power of supervision signals. More importantly, these approaches struggle to explain how GSL helps GNNs, and when and why this help fails. In this article, we conduct a systematic experimental evaluation to reveal that GSL and GNNs enjoy consistent optimization goals in terms of improving the graph homophily. Furthermore, we demonstrate theoretically and experimentally that task-specific downstream supervision may be insufficient to support the learning of both graph structure and GNN parameters, especially when the labeled data are extremely limited. Therefore, as a complement to downstream supervision, we propose homophily-enhanced self-supervision for GSL (HES-GSL), a method that provides more supervision for learning an underlying graph structure. A comprehensive experimental study demonstrates that HES-GSL scales well to various datasets and outperforms other leading methods. Our code will be available in https://github.com/LirongWu/Homophily-Enhanced-Self-supervision.

Graph neural networks (GNNs) have achieved effective performance in many graph-related tasks involving recommendation systems, social networks, and bioinformatics. Recent studies have proposed several graph pooling operators to obtain graph-level representations from node representations. Nevertheless, they usually adopt a single strategy to evaluate the importance of nodes, which may generate node rankings with weak robustness. Also, they cannot capture the different substructures of a graph since they shrink the graph layer by layer. To solve the above problems, this article proposes a Multistructure graph classification method with Attention mechanism and Convolutional neural network (CNN), called MAC. In particular, we propose a novel pooling operator, which adopts multiple strategies to evaluate the importance of nodes and updates node representations through an attention mechanism. Also, we design a hierarchical architecture for MAC to capture multiple different substructures of a graph. To further reduce the loss of graph information, we utilize 2-D CNN to generate a graph-level representation. Comparative experiments are performed on public benchmark datasets deriving from social systems, and the experimental results indicate that our method outperforms a range of state-of-the-art graph classification methods.

This study presents a novel approach that synergizes community detection algorithms with various Graph Neural Network (GNN) models to bolster link prediction in scientific literature networks. By integrating the Louvain community detection algorithm into our GNN frameworks, we consistently enhanced the performance across all models tested. For example, integrating the Louvain model with the GAT model resulted in an AUC score increase from 0.777 to 0.823, exemplifying the typical improvements observed. Similar gains were noted when the Louvain model was paired with other GNN architectures, confirming the robustness and effectiveness of incorporating community-level insights. This consistent increase in performance—reflected in our extensive experimentation on bipartite graphs of scientific collaborations and citations—highlights the synergistic potential of combining community detection with GNNs to overcome common link prediction challenges such as scalability and resolution limits. Our findings advocate for the integration of community structures as a significant step forward in the predictive accuracy of network science models, offering a comprehensive understanding of scientific collaboration patterns through the lens of advanced machine learning techniques.

The rapid advancements in machine learning have opened up new possibilities for solving complex problems, including in highly specialized applications with notable impact on relevant diseases, such as cancer. A critical aspect of cancer treatment is histopathological image analysis, a field that analyzes large zoomable images at a microscope level, to understand how the disease affected the patient, enabling patient-tailored treatment strategies. However, progress in this area relies heavily on interdisciplinary collaboration and the development of methods that utilize highquality annotated datasets effectively. In this thesis, we address these challenges by proposing a new collaborative annotation software for Whole Slides Images. Using the software, we aid in the construction of novel annotated datasets with application-specific goals. Then, inspired by the pathologist\'s work methodology, we develop a novel Graph Neural Network (GNN) technique based on activation maps that models neighborhood representation based on the presence or absence of features in the neighborhood. Our approach is based on the Message Passing architecture, which although very popular, may limit graph representation by suffering from issues such as limited injectability and oversquashing, depending on the GNN architecture and task. Moreover, increasing traditional GNN\'s receptive field requires deepening the neural network, which introduces additional challenges, such as higher computational requirements. In contrast, our approach projects the graph\'s features to a fixed-size map representation with Self-Organizing Maps, which are activated based on the presence and distance of features from the central node, within the model\'s configurable receptive field. Lastly, the activation map keeps the graph\'s features within their original domain during aggregation, which has benefits for interpretability. We conclude this work by training the proposed GNN on a novel annotated dataset developed with the proposed tool, and using its interpretability advantages to highlight the patterns it identifies, contrasting it with a pathologist\'s expectations. This work contributes to the advancement of histopathological image analysis by proposing novel research tools, annotated data, and methods, paving the way for more accurate and reliable AI-based personalized medicine.

Graph Neural Networks (GNNs) have revolutionized graph learning through efficiently learned node embeddings and achieved promising results in various graph-related tasks such as node and graph classification. Within GNNs, a pooling operation reduces the size of the input graph by grouping nodes that share commonalities intending to generate more robust and expressive latent representations. For this reason, pooling is a critical operation that significantly affects downstream tasks. Existing global pooling methods mostly use readout functions like max or sum to perform the pooling operations, but these methods neglect the hierarchical information of graphs. Clique-based hierarchical pooling methods have recently been developed to overcome global pooling issues. Such clique pooling methods perform a hard partition between nodes, which destroys the topological structural relationship of nodes, assuming that a node should belong to a single cluster. However, overlapping clusters widely exist in many real-world networks since a node can belong to more than one cluster. Here we introduce a new hierarchical graph pooling method to address this issue. Our pooling method, named Quasi-CliquePool, builds on the concept of a quasi-clique, which generalizes the notion of cliques to extract dense incomplete subgraphs of a graph. We also introduce a soft peel-off strategy to find the overlapping cluster nodes to keep the topological structural relationship of nodes. For a fair comparison, we follow the same procedure and training settings used by state-of-the-art pooling techniques. Our experiments demonstrate that combining the Quasi-Clique Pool with existing GNN architectures yields an average improvement of 2% accuracy on four out of six graph classification benchmarks compared to other existing pooling methods.

Anomaly Detection in Dynamic Networks plays a critical role in various real-world applications such as cyberse-curity, e-commerce, and social media. Recent approaches based on graph neural networks have achieved fruitful results in static networks, and most of the researches focus on learning node embeddings to represent the large-scale complex networks, thereby facilitating downstream anomaly detection task. However, these methods require the whole network to extract intrinsic properties, so they are sensitive to the frequent changes of nodes and incapable of capturing the dynamism as realworld networks evolve over time. In this paper, we propose a novel framework DynAD for anomalous edge detection on time-evolving networks, which performs adaptive parameter learning in an end-to-end manner. In particular, DynAD first extracts fixed-size node embedding from each snapshot with temporal graph convolution and pooling operations. Then, it captures the temporal information of the graph sequence using Gated recurrent units (GRU) for anomaly detection, where an attention mechanism is employed to highlight the differences between the dynamic patterns. Experimental results on three real-world datasets illustrate that DynAD significantly outperforms the state-of-the-art baseline methods in anomaly detection.

Graph Neural Networks (GNNs) have demonstrated significant success in various graph-related tasks. However, their performance is highly dependent on the availability of sufficient, class-balanced annotated data and an accurate graph structure. GNNs are vulnerable to substantial performance degradation when confronted with limited annotated samples, class imbalance, or noisy graph structures. First, a Dual GNN learning framework is introduced for semi-supervised node classification under limited supervision. The framework comprises two GNN-based node prediction modules: a primary module, which uses the input graph structure to induce standard node embeddings and predictions, and an auxiliary module, which leverages spectral clustering to construct a new graph structure and learn new node embeddings and predictions. These modules collaborate to enable end-to-end learning of discriminative node representations. Next, a graph augmentation method called Graph Dual Mixup (GDM) is proposed for graph classification with scarce labels. GDM employs a graph structural auto-encoder to learn structural embeddings, applying mixup in the learned structural embedding space to generate new graph structures. Additionally, mixup is applied to input node features, generating node features for new graph instances. Together, these generated node features and graph structures contribute new graphs that increase the size and diversity of the labeled dataset, improving classification performance. To enhance robustness against adversarial attacks on graph structures, an Efficient Low-Rank GNN is introduced. This approach learns robust low-rank sparse graph structures through a two-stage process. In the first stage, Singular Value Decomposition is used to estimate a low-rank approximation of the graph structure. This estimate is refined in the second stage by jointly learning a low-rank sparse graph structure alongside the GNN model, resulting in enhanced robustness. Finally, to address class-imbalanced node classification, a Unified GNN Learning (Uni-GNN) framework is proposed. Uni-GNN integrates structural and semantic connectivity representations to extend the propagation of embeddings to non-adjacent structural neighbors and semantically similar nodes, facilitating the diffusion of discriminative information. Additionally, it incorporates a balanced pseudo-label generation mechanism to improve the representations of minority classes. Comprehensive evaluations on benchmark datasets demonstrate that these methods outperform state-of-the-art approaches and hold significant potential for advancing GNN learning with limited supervision.

In this paper we survey our recent results characterizing various graph neural network (GNN) architectures in terms of their ability to classify nodes over graphs, for classifiers based on unary logical formulas- or queries. We focus on the language FOC2, a well-studied fragment of FO. This choice is motivated by the fact that FOC2 is related to theWeisfeiler-Lehman (WL) test for checking graph isomorphism, which has the same ability as GNNs for distinguishing nodes on graphs. We unveil the exact relationship between FOC2 and GNNs in terms of node classification. To tackle this problem, we start by studying a popular basic class of GNNs, which we call AC-GNNs, in which the features of each node in a graph are updated, in successive layers, according only to the features of its neighbors. We prove that the unary FOC2 formulas that can be captured by an AC-GNN are exactly those that can be expressed in its guarded fragment, which in turn corresponds to graded modal logic. This result implies in particular that ACGNNs are too weak to capture all FOC2 formulas. We then seek for what needs to be added to AC-GNNs for capturing all FOC2. We show that it suffices to add readouts layers, which allow updating the node features not only in terms of its neighbors, but also in terms of a global attribute vector. We call GNNs with readouts ACR-GNNs. We also describe experiments that validate our findings by showing that, on synthetic data conforming to FOC2 but not to graded modal logic, AC-GNNs struggle to fit in while ACR-GNNs can generalise even to graphs of sizes not seen during training.

Ovonic threshold switching (OTS) selectors are pivotal in nonvolatile memory devices due to their nonlinear electrical characteristics and polarity-dependent threshold voltages. However, the atomic-scale origins of the defect states responsible for these behaviors remain unclear. In this study, we systematically investigate defects in amorphous GeSe using molecular dynamics simulations accelerated by machine learning interatomic potentials (MLIPs). We first benchmark several MLIP architectures, including descriptor-based potentials and graph neural network (GNN)-based potentials. Our results demonstrate that capturing higher-order interactions, at least four-body correlations, and medium-range structural order is essential for accurately representing amorphous GeSe structures. Our analysis indicates that GNN architectures with multiple interaction layers effectively capture higher-order correlations and medium-range order, thereby preventing spurious defects easily introduced by less descriptive MLIPs. Utilizing the optimized GNN model, we identify two distinct defect motifs across 20 independent 960-atom amorphous GeSe structures: aligned Ge chains associated with defect states near the conduction band, and overcoordinated Ge chains linked to defect states near the valence band. Moreover, we establish correlations between electronic defect levels and specific structural features─namely, the average alignment of bond angles in aligned chains and the degree of local Peierls distortion around overcoordinated Ge atoms. These insights provide a theoretical framework for interpreting experimental observations and deepening the understanding of defect-driven OTS phenomena in amorphous GeSe.

At present, Graph Neural Network (GNN) methods usually follow the node centered message passing process and rely heavily on smooth node characteristics rather than graph structure. In view of this limitation, based on the heuristic method and graph attention mechanism, a feature fusion link prediction model (SAFL) combined with graph neural network is proposed. This model extracts the enclosing subgraphs around the target, combines the attention mechanism to assign neighbor weights to learn useful structural features, considers the impact of different nodes on the link, and fuses the graph neural network with the characteristics of input nodes to predict the link. The experiment on OGB dataset shows that the proposed link prediction model based on heuristic method enhances the graph structure characteristics, effectively represents the connectivity of enclosing subgraphs, and achieves better performance in link prediction.

Graph classification is an important problem with applications across many domains, for which graph neural networks (GNNs) have been state-of-the-art (SOTA) methods. In the literature, to adopt GNNs for the graph classification task, there are two groups of methods: global pooling and hierarchical pooling. The global pooling methods obtain the graph representation vectors by globally pooling all of the node embeddings together at the end of several GNN layers, whereas the hierarchical pooling methods provide one extra pooling operation between the GNN layers to extract hierarchical information and improve the graph representations. Both global and hierarchical pooling methods are effective in different scenarios. Due to highly diverse applications, it is challenging to design data-specific pooling methods with human expertise. To address this problem, we propose PAS (Pooling Architecture Search) to design adaptive pooling architectures by using the neural architecture search (NAS). To enable the search space design, we propose a unified pooling framework consisting of four modules: Aggregation, Pooling, Readout, and Merge. Two variants, PAS-G and PAS-NE, are provided to design the pooling operations in different scales. A set of candidate operations is designed in the search space using this framework. Then, existing human-designed pooling methods, including global and hierarchical ones, can be incorporated. To enable efficient search, a coarsening strategy is developed to continuously relax the search space, and then a differentiable search method can be adopted. We conduct extensive experiments on six real-world datasets, including the large-scale datasets MR and ogbg-molhiv. Experimental results in this article demonstrate the effectiveness and efficiency of the proposed PAS in designing the pooling architectures for graph classification. The Top-1 performance on two Open Graph Benchmark (OGB) datasets 1 further indicates the utility of PAS when facing diverse realistic data. The implementation of PAS is available at: https://github.com/AutoML-Research/PAS.

Graph Neural Networks (GNNs) are de facto solutions to structural data learning. However, it is susceptible to low-quality and unreliable structure, which has been a norm rather than an exception in real-world graphs. Existing graph structure learning (GSL) frameworks still lack robustness and interpretability. This paper proposes a general GSL framework, SE-GSL, through structural entropy and the graph hierarchy abstracted in the encoding tree. Particularly, we exploit the one-dimensional structural entropy to maximize embedded information content when auxiliary neighbourhood attributes are fused to enhance the original graph. A new scheme of constructing optimal encoding trees is proposed to minimize the uncertainty and noises in the graph whilst assuring proper community partition in hierarchical abstraction. We present a novel sample-based mechanism for restoring the graph structure via node structural entropy distribution. It increases the connectivity among nodes with larger uncertainty in lower-level communities. SE-GSL is compatible with various GNN models and enhances the robustness towards noisy and heterophily structures. Extensive experiments show significant improvements in the effectiveness and robustness of structure learning and node representation learning.