Workflow Anomaly Detection with Graph Neural Networks

Sex classification is a major benchmark of previous work in learning on the structural connectome, a naturally occurring brain graph that has proven useful for studying cognitive function and impairment. While graph neural networks (GNNs), specifically graph convolutional networks (GCNs), have gained popularity lately for their effectiveness in learning on graph data, achieving strong performance in adult sex classification tasks, their application to pediatric populations remains unexplored. We seek to characterize the capacity for GNN models to learn connectomic patterns on pediatric data through an exploration of training techniques and architectural design choices. Two datasets comprising an adult BRIGHT dataset (N = 147 Hodgkin's lymphoma survivors and N = 162 age similar controls) and a pediatric Human Connectome Project in Development (HCP-D) dataset (N = 135 healthy subjects) were utilized. Two GNN models (GCN simple and GCN residual), a deep neural network (multi-layer perceptron), and two standard machine learning models (random forest and support vector machine) were trained. Architecture exploration experiments were conducted to evaluate the impact of network depth, pooling techniques, and skip connections on the ability of GNN models to capture connectomic patterns. Models were assessed across a range of metrics including accuracy, AUC score, and adversarial robustness. GNNs outperformed other models across both populations. Notably, adult GNN models achieved 85.1% accuracy in sex classification on unseen adult participants, consistent with prior studies. The extension of the adult models to the pediatric dataset and training on the smaller pediatric dataset were sub-optimal in their performance. Using adult data to augment pediatric models, the best GNN achieved comparable accuracy across unseen pediatric (83.0%) and adult (81.3%) participants. Adversarial sensitivity experiments showed that the simple GCN remained the most robust to perturbations, followed by the multi-layer perceptron and the residual GCN. These findings underscore the potential of GNNs in advancing our understanding of sex-specific neurological development and disorders and highlight the importance of data augmentation in overcoming challenges associated with small pediatric datasets. Further, they highlight relevant tradeoffs in the design landscape of connectomic GNNs. For example, while the simpler GNN model tested exhibits marginally worse accuracy and AUC scores in comparison to the more complex residual GNN, it demonstrates a higher degree of adversarial robustness.

Developing fast and accurate computational models to simulate intricate physical phenomena has been a persistent research challenge. Recent studies have demonstrated remarkable capabilities in predicting various physical outcomes through machine learning-assisted approaches. However, it remains challenging to generalize current methods, usually crafted for a specific problem, to other more complex or broader scenarios. To address this challenge, we developed graph neural network (GNN) models with enhanced generalizability derived from the distinct GNN architecture and neural operator techniques. As a proof of concept, we employ our GNN models to predict finite element (FE) simulation results for three-dimensional solid mechanics problems with varying boundary conditions. Results show that our GNN model achieves accurate and robust performance in predicting the stress and deformation profiles of structures compared with FE simulations. Furthermore, the neural operator embedded GNN approach enables learning and predicting various solid mechanics problems in a generalizable fashion, making it a promising approach for surrogate modeling.

Many underlying relationships among data in several areas of science and engineering, e.g., computer vision, molecular chemistry, molecular biology, pattern recognition, and data mining, can be represented in terms of graphs. In this paper, we propose a new neural network model, called graph neural network (GNN) model, that extends existing neural network methods for processing the data represented in graph domains. This GNN model, which can directly process most of the practically useful types of graphs, e.g., acyclic, cyclic, directed, and undirected, implements a function tau(G,n) is an element of IR(m) that maps a graph G and one of its nodes n into an m-dimensional Euclidean space. A supervised learning algorithm is derived to estimate the parameters of the proposed GNN model. The computational cost of the proposed algorithm is also considered. Some experimental results are shown to validate the proposed learning algorithm, and to demonstrate its generalization capabilities.

Past events have revealed that widespread blackouts are mostly a result of cascading failures in the power grid. Understanding the underlining mechanisms of cascading failures can help in developing strategies to minimize the risk of such events. Moreover, a real-time detection of precursors to cascading failures will help operators take measures to prevent their propagation. Currently, the well-established probabilistic and physics-based models of cascading failures offer low computational efficiency, hindering them to be used only as offline tools. In this work, we develop a data-driven methodology for online estimation of the risk of cascading failures. We utilize a physics-based cascading failure model to generate a cascading failure dataset considering different operating conditions and failure scenarios, thus obtaining a sample space covering a large set of power grid states that are labeled as safe or unsafe. We use the synthetic data to train deep learning architectures, namely Feed-forward Neural Networks (FNN) and Graph Neural Networks (GNN). With the development of GNNs, improved performance is achieved with graph-structured data, and GNNs can generalize to graphs of diverse sizes. A comparison between FNN and GNN is made and the GNNs inductive capability is demonstrated via test grids. Furthermore, we apply transfer learning to improve the performance of a pre-trained GNN model on power grids not seen in the training process. The GNN model shows accuracy and balanced accuracy above 96% on selected test datasets not used in the training. Conversely, the FNN shows accuracy above 85% and balanced accuracy above 81% on test datasets unseen during training. Overall, the GNN model is successful in determining, if one or several simultaneous outages result in a critical grid state, under specific grid operating conditions.

Deep learning has led to major advances in fields like natural language processing, computer vision, and other Euclidean data domains. Yet, many important fields have data defined on irregular domains, requiring graphs to be explicitly modeled. One such application is drug discovery. Recently, research has found that using graph neural network (GNN) models, given enough data, can perform better than using human-engineered fingerprints or descriptors in predicting molecular properties of potential antibiotics.We explore these state-of-the-art AI models on predicting desirable molecular properties for drugs that can inhibit SARS-CoV-2. We build upon the GNN models with ideas from recent breakthroughs in geometric deep learning, inspired by the topologies of the molecules. In this poster paper, we present an overview of the drug discovery framework, drug-target interaction framework, and GNNs. Preliminary results on two COVID-19 related datasets are encouraging, achieving a ROC-AUC of 0.72 for FDA-approved chemical library screened against SARS-CoV-2 in vitro.

Speech is a powerful means of expressing thoughts, emotions, and perspectives. However, accurately determining the emotions conveyed through speech remains a challenging task. Existing manual methods for analyzing speech to recognize emotions are prone to errors, limiting our understanding and response to individuals’ emotional states. To address diverse accents, an automated system capable of real-time emotion prediction from human speech is needed. This paper introduces a speech emotion recognition (SER) system that leverages supervised learning techniques to tackle cross-accent diversity. Distinctively, the system extracts a comprehensive set of nine speech features—Zero Crossing Rate, Mel Spectrum, Pitch, Root Mean Square values, Mel Frequency Cepstral Coefficients, chroma-stft, and three spectral features (Centroid, Contrast, and Roll-off) for refined speech signal processing and recognition. Seven machine learning models are employed, encompassing Random Forest, Logistic Regression, Decision Tree, Support Vector Machines, Gaussian Naive Bayes, K-Nearest Neighbors, ensemble learning, and four individual, hybrid deep learning models including Long short-term memory (LSTM) and 1-Dimensional Convolutional Neural Network (1D-CNN) with stratified cross-validation. Audio samples from diverse English regions are combined to train the models. The performance evaluation results of conventional machine learning and deep learning models indicate that the Random Forest-based feature selection model achieves the highest accuracy of up to 76% among the conventional machine learning models. Simultaneously, the 1D-CNN model with stratified cross-validation reaches up to 99% accuracy. The proposed framework enhances the cross-accent emotion recognition accuracy up to 86.3%, 89.87%, 90.27%, and 84.96% by margins of 14.71%, 10.15%, 9.6%, and 16.52% respectively.

Recently, graph neural networks (GNNs) have revolutionized the field of chemical property prediction and achieved state-of-the-art results on benchmark data sets. Compared with the traditional descriptor- and fingerprint-based QSAR models, GNNs can learn task related representations, which completely gets rid of the rules defined by experts. However, due to the lack of useful prior knowledge, the prediction performance and interpretability of the GNNs may be affected. In this study, we introduced a new GNN model called RG-MPNN for chemical property prediction that integrated pharmacophore information hierarchically into message-passing neural network (MPNN) architecture, specifically, in the way of pharmacophore-based reduced-graph (RG) pooling. RG-MPNN absorbed not only the information of atoms and bonds from the atom-level message-passing phase, but also the information of pharmacophores from the RG-level message-passing phase. Our experimental results on eleven benchmark and ten kinase data sets showed that our model consistently matched or outperformed other existing GNN models. Furthermore, we demonstrated that applying pharmacophore-based RG pooling to MPNN architecture can generally help GNN models improve the predictive power. The cluster analysis of RG-MPNN representations and the importance analysis of pharmacophore nodes will help chemists gain insights for hit discovery and lead optimization.Graphical

Graph Neural Networks (GNNs) have become indispensable tools for learning from graph-structured data, catering to various applications such as social network analysis and fraud detection for financial services. At the heart of these networks are the edges, which are crucial in guiding GNN models' predictions. In many scenarios, these edges represent sensitive information, such as personal associations or financial dealings, which require privacy assurance. However, their contributions to GNN model predictions may, in turn, be exploited by the adversary to compromise their privacy. Motivated by these conflicting requirements, this paper investigates edge privacy in contexts where adversaries possess only black-box access to the target GNN model, restricted further by access controls, preventing direct insights into arbitrary node outputs. Moreover, we are the first to extensively examine situations where the target graph continuously evolves—a common trait of many real-world graphs. In this setting, we present a range of attacks that leverage the message-passing mechanism of GNNs. We evaluated the effectiveness of our attacks using nine real-world datasets, encompassing both static and dynamic graphs, across four different GNN architectures. The results demonstrate that our attack outperforms existing methods across various GNN architectures, consistently achieving an F1 score of at least 0.8 in static scenarios. Furthermore, our attack retains robustness in dynamic graph scenarios, maintaining F1 scores up to 0.8, unlike previous methods that only achieve F1 scores around 0.2.

Destructive cyber-attacks and cybercriminals are increasing with the increase in IoT (Internet of Things) devices globally. This ha/s led to the need for increase in security in IoT systems. Innovative and novel techniques are used by the intruders to accomplish malicious goals effectively through cyber-attacks. An Intrusion Detection System (IDS) is used for classification of attacks in IoT networks based on anomaly detection and machine learning techniques. Inefficiency is observed in the conventional machine learning models and intrusion detection techniques as the network technologies are unpredictable. Accurate identification of various anomalies is possible with deep learning models in several research segments. The input data along with its prominent characteristics may be categorized automatically for classification and anomaly detection using convolutional neural networks (CNN). Faster computations are enabled due to the performance efficiency of CNN. For IoT networks, an intrusion detection model based on anomaly detection is designed and developed in this paper. A multiclass classification framework is created initially using a CNN model. Further, 3D CNN is used for implementation of the proposed model. Various intrusion detection datasets from IoT networks are used for validation of the proposed CNN model. Pre-trained multiclass CNN model is used for implementation of multiclass and binary classification based on transfer learning. When compared to the conventional deep learning models, the proposed multiclass and binary classification framework has attained improved F1 score, recall, precision and accuracy.

Most distribution system operators may not accurately record or completely maintain the phase connections for the numerous LV customers. Different consumer phase identification (CPI) approaches based on voltages, powers or other measurements are proposed in the literature. Due to the technical challenges in collecting voltage measurements, power measurement based approaches are preferable. Hence, this paper proposes a novel power based CPI methodology applying Graph Neural Networks (GNNs). The CPI methodology generates synthetic transformer power profiles assuming random combinations of phases for the measured load profiles, which are used altogether to train the GNN model. The GNN model is then tested using measured transformer and load power profiles. The performance of the methodology is evaluated in a test low voltage grid of 55 loads under various conditions of Photovoltaic penetration, Photovoltaics under maintenance, measurement errors, unmetered consumption, uncertain grid asset parameters and inaccurate phase connections. Further tests on a real low voltage grid with 111 loads prove the scalability of the methodology. The attained results show that the GNN model can achieve accuracy above 90% in most cases, outperforming various state-of-the-art methods. • The Graph Neural Network identifies the phase labels with accuracy higher than 90%. • Under noisy or unmetered demand, the Graph Neural Network achieves accuracy over 90%. • Under various Photovoltaic cases, the Graph Neural Network yields accuracy over 90%. • The Graph Neural Network yields accuracy of 92% in case of multi-phase connections. • The grid losses and uncertain assets’ parameters have minor impact on model accuracy.

Drug response prediction is an essential part of precision medicine, which assesses the efficacy of drugs in specific cancer cell lines. Recently, numerous deep learning models have been developed for drug response prediction. The ability to effectively represent complicated interactions within a network makes Graph Neural Network (GNN) models a prominent area of exploration. To exploit the diverse capabilities of different GNN models, the proposed model uses an ensemble approach that combines results from Graph Convolutional Network (GCN), Graph SAmple and aggreGatE Network (GraphSAGE) and Graph Attention Network (GAT)-based models for drug response prediction. The proposed ensemble GNN model is used to predict the efficacy of drugs in treating leukemia. The model outperformed individual models when evaluated with the GDSC1000 dataset, with lower root mean squared error, higher Pearson’s correlation coefficient, higher coefficient of determination and higher Spearman’s correlation coefficient. Furthermore, most of the drugs identified by the model as potential candidates for the treatment of leukemia are currently in clinical trials or have supporting evidence in the PubMed literature database.

Graph Neural Networks (GNNs) are the current state-of-the-art models in learning node representations for many predictive tasks on graphs. Typically, GNNs reuses the same set of model parameters across all nodes in the graph to improve the training efficiency and exploit the translationally-invariant properties in many datasets. However, the parameter sharing scheme prevents GNNs from distinguishing two nodes that are isomorphic and that the translation invariance property may not exhibit in real-world graphs. In this paper, we present Graph Representation Learning with Adaptive Mixtures (GRAM), a novel approach for learning node representations in a graph by introducing multiple independent GNN models and a trainable mixture distribution for each node. GRAM is a general framework that can be readily embedded into existing models for enhancing representation learning. It allows for adaptive node clustering with mixtures and joint optimization of all GNN models simultaneously. To achieve this, we develop an Expectation-Maximization algorithm with stochastic gradient descent in a scalable manner. Specifically, in the E-Step, we approximate the posterior probability distribution of the latent cluster membership based on the k-th independent GNN models in the last iteration. In the M-step, we update all GNN models and the mixture parameters based on the refined latent cluster membership for downstream tasks, e.g., node classification. To fully exploit the graph structural information, we further implement a regularization term on unlabeled nodes, which encourages adjacent nodes to have the same mixture distribution. By jointly optimizing the model in an end-to-end manner, GRAM learns to capture different abstract patterns on graphs for different node clusters, which yields discriminative representation for each node. We evaluate GRAM on five benchmark datasets with extensive experiments. GRAM is demonstrated to consistently boost state-of-the-art GNN variants in node classification tasks.

Traditional Graph Neural Network (GNN) learning patterns can only achieve optimal performance under the assumption of independent and identically distributed data. However, in real-world scenarios, numerous unpredictable factors give rise to the Out-of-Distribution (OOD) problem. Moreover, this distribution discrepancy may lead to unreliable predictions by GNNs models within unknown OOD domains. Therefore, this paper proposes a machine learning method to enhance the stability of GNN cross-site classification from the perspective of stable learning. The aim is to extract genuine causal features while eliminating spurious causal features. By introducing a feature sample weighting decorrelation technique in the random Fourier transform space and combining it with a baseline GNN model, a Stable-GNN model and a constrained sampling weight gradient update algorithm are designed. Its theoretical proof indicates that this algorithm can ensure the decrease in the loss, thus remedying the shortcomings of the original algorithm in weight updating. While guaranteeing predictive performance on the training distribution data, it also reduces prediction bias on data from unseen test distributions. Compared with the GNN model and the Stable-GNN model (S-GNN), the experimental results of the S-GNN model demonstrate that it not only surpasses the current state-of-the-art GNN models, but also offers a flexible framework for strengthening existing GNNs.

Graph neural networks (GNNs) have made rapid developments in the recent years. Due to their great ability in modeling graph-structured data, GNNs are vastly used in various applications, including high-stakes scenarios such as financial analysis, traffic predictions, and drug discovery. Despite their great potential in benefiting humans in the real world, recent study shows that GNNs can leak private information, are vulnerable to adversarial attacks, can inherit and magnify societal bias from training data and lack interpretability, which have risk of causing unintentional harm to the users and society. For example, existing works demonstrate that attackers can fool the GNNs to give the outcome they desire with unnoticeable perturbation on training graph. GNNs trained on social networks may embed the discrimination in their decision process, strengthening the undesirable societal bias. Consequently, trust-worthy GNNs in various aspects are emerging to prevent the harm from GNN models and increase the users’ trust in GNNs. In this paper, we give a comprehensive survey of GNNs in the computational aspects of privacy, robustness, fairness, and explainability. For each aspect, we give the taxonomy of the related methods and formulate the general frameworks for the multiple categories of trustworthy GNNs. We also discuss the future research directions of each aspect and connections between these aspects to help achieve trustworthiness.

Optimizing graph neural network architectures for schizophrenia spectrum disorder prediction using evolutionary algorithms