Learning and Understanding User Interface Semantics from Heterogeneous Networks with Multimodal and Positional Attributes

In recent years, heterogeneous network/graph representation learning/embedding (HNE) has drawn tremendous attentions from research communities in multiple disciplines. HNE has shown its outstanding performances in various networked data analysis and mining tasks. In fact, most of real-world information networks in multiple fields can be modelled as the heterogeneous information networks (HIN). Thus, the HNE-based techniques can sufficiently capture rich-structured and semantic latent features from the given information network in order to facilitate for different task-driven learning tasks. This is considered as fundamental success of HNE-based approach in comparing with previous traditional homogeneous network/graph based embedding techniques. However, there are recent studies have also demonstrated that the heterogeneous network/graph modelling and embedding through graph neural network (GNN) is not usually reliable. This challenge is original come from the fact that most of real-world heterogeneous networks are considered as incomplete and normally contain a large number of feature noises. Therefore, multiple attempts have proposed recently to overcome this limitation. Within this approach, the meta-path-based heterogeneous graph-structured latent features and GNN-based parameters are jointly learnt and optimized during the embedding process. However, this integrated GNN and heterogeneous graph structure (HGS) learning approach still suffered a challenge of effectively parameterizing and fusing different graph-structured latent features from both GNN- and HGS-based sides into better task-driven friendly and noise-reduced embedding spaces. Therefore, in this paper we proposed a novel attention-supplemented heterogeneous graph structure embedding approach, called as: AGSE. Our proposed AGSE model supports to not only achieve the combined rich heterogeneous structural and GNN-based aggregated node representations but also transform achieved node embeddings into noise-reduced and task-driven friendly embedding space. Extensive experiments in benchmark heterogeneous networked datasets for node classification task showed the effectiveness of our proposed AGSE model in comparing with state-of-the-art network embedding baselines.

Graph Neural Networks (GNNs) are the current state-of-the-art models in learning node representations for many predictive tasks on graphs. Typically, GNNs reuses the same set of model parameters across all nodes in the graph to improve the training efficiency and exploit the translationally-invariant properties in many datasets. However, the parameter sharing scheme prevents GNNs from distinguishing two nodes that are isomorphic and that the translation invariance property may not exhibit in real-world graphs. In this paper, we present Graph Representation Learning with Adaptive Mixtures (GRAM), a novel approach for learning node representations in a graph by introducing multiple independent GNN models and a trainable mixture distribution for each node. GRAM is a general framework that can be readily embedded into existing models for enhancing representation learning. It allows for adaptive node clustering with mixtures and joint optimization of all GNN models simultaneously. To achieve this, we develop an Expectation-Maximization algorithm with stochastic gradient descent in a scalable manner. Specifically, in the E-Step, we approximate the posterior probability distribution of the latent cluster membership based on the k-th independent GNN models in the last iteration. In the M-step, we update all GNN models and the mixture parameters based on the refined latent cluster membership for downstream tasks, e.g., node classification. To fully exploit the graph structural information, we further implement a regularization term on unlabeled nodes, which encourages adjacent nodes to have the same mixture distribution. By jointly optimizing the model in an end-to-end manner, GRAM learns to capture different abstract patterns on graphs for different node clusters, which yields discriminative representation for each node. We evaluate GRAM on five benchmark datasets with extensive experiments. GRAM is demonstrated to consistently boost state-of-the-art GNN variants in node classification tasks.

Developing fast and accurate computational models to simulate intricate physical phenomena has been a persistent research challenge. Recent studies have demonstrated remarkable capabilities in predicting various physical outcomes through machine learning-assisted approaches. However, it remains challenging to generalize current methods, usually crafted for a specific problem, to other more complex or broader scenarios. To address this challenge, we developed graph neural network (GNN) models with enhanced generalizability derived from the distinct GNN architecture and neural operator techniques. As a proof of concept, we employ our GNN models to predict finite element (FE) simulation results for three-dimensional solid mechanics problems with varying boundary conditions. Results show that our GNN model achieves accurate and robust performance in predicting the stress and deformation profiles of structures compared with FE simulations. Furthermore, the neural operator embedded GNN approach enables learning and predicting various solid mechanics problems in a generalizable fashion, making it a promising approach for surrogate modeling.

Sex classification is a major benchmark of previous work in learning on the structural connectome, a naturally occurring brain graph that has proven useful for studying cognitive function and impairment. While graph neural networks (GNNs), specifically graph convolutional networks (GCNs), have gained popularity lately for their effectiveness in learning on graph data, achieving strong performance in adult sex classification tasks, their application to pediatric populations remains unexplored. We seek to characterize the capacity for GNN models to learn connectomic patterns on pediatric data through an exploration of training techniques and architectural design choices. Two datasets comprising an adult BRIGHT dataset (N = 147 Hodgkin's lymphoma survivors and N = 162 age similar controls) and a pediatric Human Connectome Project in Development (HCP-D) dataset (N = 135 healthy subjects) were utilized. Two GNN models (GCN simple and GCN residual), a deep neural network (multi-layer perceptron), and two standard machine learning models (random forest and support vector machine) were trained. Architecture exploration experiments were conducted to evaluate the impact of network depth, pooling techniques, and skip connections on the ability of GNN models to capture connectomic patterns. Models were assessed across a range of metrics including accuracy, AUC score, and adversarial robustness. GNNs outperformed other models across both populations. Notably, adult GNN models achieved 85.1% accuracy in sex classification on unseen adult participants, consistent with prior studies. The extension of the adult models to the pediatric dataset and training on the smaller pediatric dataset were sub-optimal in their performance. Using adult data to augment pediatric models, the best GNN achieved comparable accuracy across unseen pediatric (83.0%) and adult (81.3%) participants. Adversarial sensitivity experiments showed that the simple GCN remained the most robust to perturbations, followed by the multi-layer perceptron and the residual GCN. These findings underscore the potential of GNNs in advancing our understanding of sex-specific neurological development and disorders and highlight the importance of data augmentation in overcoming challenges associated with small pediatric datasets. Further, they highlight relevant tradeoffs in the design landscape of connectomic GNNs. For example, while the simpler GNN model tested exhibits marginally worse accuracy and AUC scores in comparison to the more complex residual GNN, it demonstrates a higher degree of adversarial robustness.

Deep learning has led to major advances in fields like natural language processing, computer vision, and other Euclidean data domains. Yet, many important fields have data defined on irregular domains, requiring graphs to be explicitly modeled. One such application is drug discovery. Recently, research has found that using graph neural network (GNN) models, given enough data, can perform better than using human-engineered fingerprints or descriptors in predicting molecular properties of potential antibiotics.We explore these state-of-the-art AI models on predicting desirable molecular properties for drugs that can inhibit SARS-CoV-2. We build upon the GNN models with ideas from recent breakthroughs in geometric deep learning, inspired by the topologies of the molecules. In this poster paper, we present an overview of the drug discovery framework, drug-target interaction framework, and GNNs. Preliminary results on two COVID-19 related datasets are encouraging, achieving a ROC-AUC of 0.72 for FDA-approved chemical library screened against SARS-CoV-2 in vitro.

Recently, heterogeneous information network (HIN) analysis has attracted a lot of attention, and many data mining tasks have been exploited on HIN. As an important data mining task, recommender system includes a lot of object types (e.g., users, movies, actors, and interest groups in movie recommendation) and the rich relations among object types, which naturally constitute an HIN. The comprehensive information integration and rich semantic information of HIN make it promising to generate better recommendation. In this chapter, we introduce three works on recommendation with HIN. One work recommends items with semantic meta paths, and the other two works extend traditional matrix factorization with rich heterogeneous information.

PurposeDrug-induced liver injury (DILI) is a significant concern in drug development, often leading to the discontinuation of clinical trials and the withdrawal of drugs from the market. This study explores the application of graph neural networks (GNNs) for DILI prediction, using molecular graph representations as the primary input.MethodsWe evaluated several GNN architectures, including Graph Convolutional Networks (GCNs), Graph Attention Networks (GATs), Graph Sample and Aggregation (GraphSAGE), and Graph Isomorphism Networks (GINs), using the latest FDA DILI dataset and other molecular property prediction datasets. We introduce a novel approach that creates a custom graph dataset, driven by molecular optimisation, that incorporates detailed and realistic chemical features such as bond lengths and partial charges as input into the GNN models. We have named our model approach DILIGeNN.ResultsDILIGeNN achieved an AUC of 0.897 on the DILI dataset, surpassing the current state-of-the-art model in the DILI prediction task. Furthermore, DILIGeNN outperformed the state-of-the-art in other graph-based molecular prediction tasks, achieving an AUC of 0.918 on the Clintox dataset, 0.993 on the BBBP dataset, and 0.953 on the BACE dataset, indicating strong generalisation and performance across different datasets.ConclusionDILIGeNN, utilising a single graph representation as input, outperforms the state-of-the-art methods in DILI prediction that incorporate both molecular fingerprint and graph-structured data. These findings highlight the effectiveness of our molecular graph generation and the GNN training approach as a powerful tool for early-stage drug development and drug repurposing pipeline.Scientific Contribution: DILIGeNN is a GNN framework that extracts graph features from 3D optimised molecular structures as is done in target-based drug discovery and molecular docking simulation. Our method is the first to encode spatial and electrostatic information into a single graph representation, as opposed to other work that require multiple graphs or additional chemical descriptors for feature representation. Our approach, using warm starts following repeated early stopping during training, outperforms the current state-of-the-art methods in liver toxicity (DILI), permeability (BBBP) and activity (BACE) prediction tasks.Graphic

Traditional recommender systems (RS) only consider homogeneous data and cannot fully model heterogeneous information of complex objects and relations. Recent advances in the study of Heterogeneous Information Network (HIN) have shed some light on how to leverage heterogeneous information in RS. However, existing HIN-based recommendation models assume HIN is invariable and merely use HIN as a data source for assisting recommendation, which limits their performance. In this paper, we propose a multi-task learning framework, called MTRec, for recommendation over HIN. MTRec relies on self-attention mechanism to learn the semantics of meta-paths in HIN and jointly optimizes the tasks of both recommendation and link prediction. Using a Bayesian task weight learner, MTRec is able to achieve the balance of two tasks during optimization automatically. Moreover, MTRec provides good interpretabilities of recommendation through a "translation" mechanism which is used to model the three-way interactions among users, items and the meta-paths connecting them. Experimental results demonstrate the superiority of MTRec over state-of-the-art HIN-based recommendation models, and the case studies we provide illustrate that MTRec enhances the explainability of RS.

Generalized predictive analysis of reactions in paper devices via graph neural networks

Graph Neural Networks (GNNs) have become indispensable tools for learning from graph-structured data, catering to various applications such as social network analysis and fraud detection for financial services. At the heart of these networks are the edges, which are crucial in guiding GNN models' predictions. In many scenarios, these edges represent sensitive information, such as personal associations or financial dealings, which require privacy assurance. However, their contributions to GNN model predictions may, in turn, be exploited by the adversary to compromise their privacy. Motivated by these conflicting requirements, this paper investigates edge privacy in contexts where adversaries possess only black-box access to the target GNN model, restricted further by access controls, preventing direct insights into arbitrary node outputs. Moreover, we are the first to extensively examine situations where the target graph continuously evolves—a common trait of many real-world graphs. In this setting, we present a range of attacks that leverage the message-passing mechanism of GNNs. We evaluated the effectiveness of our attacks using nine real-world datasets, encompassing both static and dynamic graphs, across four different GNN architectures. The results demonstrate that our attack outperforms existing methods across various GNN architectures, consistently achieving an F1 score of at least 0.8 in static scenarios. Furthermore, our attack retains robustness in dynamic graph scenarios, maintaining F1 scores up to 0.8, unlike previous methods that only achieve F1 scores around 0.2.

Past events have revealed that widespread blackouts are mostly a result of cascading failures in the power grid. Understanding the underlining mechanisms of cascading failures can help in developing strategies to minimize the risk of such events. Moreover, a real-time detection of precursors to cascading failures will help operators take measures to prevent their propagation. Currently, the well-established probabilistic and physics-based models of cascading failures offer low computational efficiency, hindering them to be used only as offline tools. In this work, we develop a data-driven methodology for online estimation of the risk of cascading failures. We utilize a physics-based cascading failure model to generate a cascading failure dataset considering different operating conditions and failure scenarios, thus obtaining a sample space covering a large set of power grid states that are labeled as safe or unsafe. We use the synthetic data to train deep learning architectures, namely Feed-forward Neural Networks (FNN) and Graph Neural Networks (GNN). With the development of GNNs, improved performance is achieved with graph-structured data, and GNNs can generalize to graphs of diverse sizes. A comparison between FNN and GNN is made and the GNNs inductive capability is demonstrated via test grids. Furthermore, we apply transfer learning to improve the performance of a pre-trained GNN model on power grids not seen in the training process. The GNN model shows accuracy and balanced accuracy above 96% on selected test datasets not used in the training. Conversely, the FNN shows accuracy above 85% and balanced accuracy above 81% on test datasets unseen during training. Overall, the GNN model is successful in determining, if one or several simultaneous outages result in a critical grid state, under specific grid operating conditions.

A heterogeneous graph neural network model for list recommendation

This article proposes a direction-of-arrival (DOA) estimation algorithm for a random sparse linear array based on a novel graph neural network (GNN). Unlike convolutional layers and fully connected layers, which do not interact well with information between different antennas, the GNN model can adapt to the goniometry problem of non-uniform random sparse linear arrays without any prior information by applying neighbor nodes’ aggregation and update operations. This helps the model in learning signal features under complex environmental conditions. We train the model in an end-to-end way to reduce the complexity of the network. Experiments are conducted on the uniform and sparse linear arrays for various signal-to-noise ratio (SNR) and numbers of snapshots for comparison. We prove that the GNN model has superior angle estimation performance on arrays with large sparsity that cannot be used by traditional algorithms and surpasses existing deep learning models based on convolutional or fully connected structures. The proposed algorithm shows excellent DOA estimation performance under the complex conditions of limited snapshots, low signal-to-noise ratio, and large array sparsity as well. In addition, the algorithm has a low time calculation cost and is suitable for scenarios that require low latency.

Most distribution system operators may not accurately record or completely maintain the phase connections for the numerous LV customers. Different consumer phase identification (CPI) approaches based on voltages, powers or other measurements are proposed in the literature. Due to the technical challenges in collecting voltage measurements, power measurement based approaches are preferable. Hence, this paper proposes a novel power based CPI methodology applying Graph Neural Networks (GNNs). The CPI methodology generates synthetic transformer power profiles assuming random combinations of phases for the measured load profiles, which are used altogether to train the GNN model. The GNN model is then tested using measured transformer and load power profiles. The performance of the methodology is evaluated in a test low voltage grid of 55 loads under various conditions of Photovoltaic penetration, Photovoltaics under maintenance, measurement errors, unmetered consumption, uncertain grid asset parameters and inaccurate phase connections. Further tests on a real low voltage grid with 111 loads prove the scalability of the methodology. The attained results show that the GNN model can achieve accuracy above 90% in most cases, outperforming various state-of-the-art methods. • The Graph Neural Network identifies the phase labels with accuracy higher than 90%. • Under noisy or unmetered demand, the Graph Neural Network achieves accuracy over 90%. • Under various Photovoltaic cases, the Graph Neural Network yields accuracy over 90%. • The Graph Neural Network yields accuracy of 92% in case of multi-phase connections. • The grid losses and uncertain assets’ parameters have minor impact on model accuracy.

Many underlying relationships among data in several areas of science and engineering, e.g., computer vision, molecular chemistry, molecular biology, pattern recognition, and data mining, can be represented in terms of graphs. In this paper, we propose a new neural network model, called graph neural network (GNN) model, that extends existing neural network methods for processing the data represented in graph domains. This GNN model, which can directly process most of the practically useful types of graphs, e.g., acyclic, cyclic, directed, and undirected, implements a function tau(G,n) is an element of IR(m) that maps a graph G and one of its nodes n into an m-dimensional Euclidean space. A supervised learning algorithm is derived to estimate the parameters of the proposed GNN model. The computational cost of the proposed algorithm is also considered. Some experimental results are shown to validate the proposed learning algorithm, and to demonstrate its generalization capabilities.