Hierarchical Causal Graph Neural Networks With Cascading Failure Analysis for Complex Systems Root Cause Identification

Graph neural network (GNN) is popular now to solve the tasks in non-Euclidean space and most of them learn deep embeddings by aggregating the neighboring nodes. However, these methods are prone to some problems such as over-smoothing because of the single-scale perspective field and the nature of low-pass filter. To address these limitations, we introduce diffusion scattering network (DSN) to exploit high-order patterns. With observing the complementary relationship between multi-layer GNN and DSN, we propose Hierarchical Diffusion Scattering Graph Neural Network (HDS-GNN) to efficiently bridge DSN and GNN layer by layer to supplement GNN with multi-scale information and band-pass signals. Our model extracts node-level scattering representations by intercepting the low-pass filtering, and adaptively tunes the different scales to regularize multi-scale information. Then we apply hierarchical representation enhancement to improve GNN with the scattering features. We benchmark our model on nine real-world networks on the transductive semi-supervised node classification task. The experimental results demonstrate the effectiveness of our method.

Graph Neural Networks (GNNs) have become a prominent approach to machine learning with graphs and have been increasingly applied in a multitude of domains. Nevertheless, since most existing GNN models are based on flat message-passing mechanisms, two limitations need to be tackled: (i) they are costly in encoding long-range information spanning the graph structure; (ii) they are failing to encode features in the high-order neighbourhood in the graphs as they only perform information aggregation across the observed edges in the original graph. To deal with these two issues, we propose a novel Hierarchical Message-passing Graph Neural Networks framework. The key idea is generating a hierarchical structure that re-organises all nodes in a flat graph into multi-level super graphs, along with innovative intra- and inter-level propagation manners. The derived hierarchy creates shortcuts connecting far-away nodes so that informative long-range interactions can be efficiently accessed via message passing and incorporates meso- and macro-level semantics into the learned node representations. We present the first model to implement this framework, termed Hierarchical Community-aware Graph Neural Network (HC-GNN), with the assistance of a hierarchical community detection algorithm. The theoretical analysis illustrates HC-GNN’s remarkable capacity in capturing long-range information without introducing heavy additional computation complexity. Empirical experiments conducted on 9 datasets under transductive, inductive, and few-shot settings exhibit that HC-GNN can outperform state-of-the-art GNN models in network analysis tasks, including node classification, link prediction, and community detection. Moreover, the model analysis further demonstrates HC-GNN’s robustness facing graph sparsity and the flexibility in incorporating different GNN encoders.

Non-point source pollution from agricultural activities poses a significant threat to water quality by introducing excess nutrients like nitrogen into aquatic ecosystems, leading to issues such as eutrophication and groundwater contamination. In agricultural watersheds, nitrate transport involves intricate physical, chemical, and biological processes influenced by meteorological conditions, hydrological features, and spatial topologies, making accurate short-term predictions challenging. Traditional data-driven deep learning models often fail to incorporate physical constraints and complex spatiotemporal dynamics, limiting their interpretability and predictive accuracy. Here we show a hierarchical transformer and graph neural network model that accurately predicts watershed nitrate concentrations by integrating multi-source data and simulating pollutant migration. The model captures nonlinear multivariate temporal patterns through hierarchical transformers, fuses global meteorological and local hydrological features via neural networks, and models runoff topologies with physically constrained graph neural networks. For predicting the concentration changes of pollutants discharged from watersheds, it outperforms baselines like multi-layer perceptrons, recurrent neural networks, and long short-term memory networks, with state-of-the-art performance in root mean square error, mean absolute error, and R2. Ablation studies confirm the essential roles of multi-source data integration and watershed topological modeling in enhancing performance. This method of directly modeling physical processes by leveraging the characteristics of different neural network architectures opens up a new path for addressing the interpretability problem in neural earth system modeling, apart from the process-guided deep learning and differentiable modelling methods.

Inspired by recent improvements in point cloud processing for autonomous navigation, we focus on using hierarchical graph neural networks for processing and feature learning over large-scale outdoor LiDAR point clouds. We observe that existing GNN based methods fail to overcome challenges of scale and irregularity of points in outdoor datasets. Addressing the need to preserve structural details while learning over a larger volume efficiently, we propose Hierarchical Point Graph Neural Network (HPGNN). It learns node features at various levels of graph coarseness to extract information. This enables to learn over a large point cloud while retaining fine details that existing point-level graph networks struggle to achieve. Connections between multiple levels enable a point to learn features in multiple scales, in a few iterations. We design HPGNN as a purely GNN-based approach, so that it offers modular expandability as seen with other point-based and Graph network baselines. To illustrate the improved processing capability, we compare previous point based and GNN models for semantic segmentation with our HPGNN, achieving a significant improvement for GNNs (+36.7 mIoU) on the SemanticKITTI dataset.

Hierarchical graph representations in digital pathology.

Modern mechanical equipment increasingly relies on multi-sensor data for health monitoring, which generates complex, heterogeneous, non-Euclidean data that conventional methods struggle to process. While graph neural networks (GNNs) are promising for such data, existing GNN-based fault diagnosis methods often suffer from two key limitations: 1) neglecting morphological details when mapping time-series data to graph structures, resulting in the loss of critical information; and 2) The lack of domain knowledge regarding fault propagation to guide the graph construction process makes it difficult to capture complex correlations among heterogeneous sensor data. To overcome these challenges, this paper proposes a hierarchical graph neural network (HGNN) for the dynamic fusion of heterogeneous data. Specifically, our method first builds sensor-level graphs by extracting discriminative shapelets and applying a pattern-distance-based <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">k</i>-nearest neighbor algorithm to capture fine-grained local features. Subsequently, these individual graphs are dynamically interconnected into a comprehensive system-level graph using a dual-metric key-node screening method and a novel Feature-Type Dynamic Weighted Similarity (FTDWS) metric. This hierarchical approach preserves critical local fault signatures while effectively modeling system-wide fault propagation and sensor correlations across heterogeneous sensors. Finally, a graph attention network performs classification on this system-level representation. Extensive experiments on two public datasets and one real-world industrial coal mill dataset demonstrate the effectiveness and superiority of our proposed method.

Most sequential recommendation (SR) systems employing graph neural networks (GNNs) only model a user's interaction sequence as a flat graph without hierarchy, overlooking diverse factors in the user's preference. Moreover, the timespan between interacted items is not sufficiently utilized by previous models, restricting SR performance gains. To address these problems, we propose a novel SR system employing a hierarchical graph neural network (HGNN) to model factorial user preferences. Specifically, a timespan-aware sequence graph (TSG) for the target user is first constructed with the timespan among interacted items. Next, all original nodes in TSG are softly clustered into factor nodes, each of which represents a certain factor of the user's preference. At last, all factor nodes' representations are used together to predict SR results. Our extensive experiments upon two datasets justify that our HGNN-based factorial user modeling obtains better SR performance than the state-of-the-art SR models.

Recent years have witnessed the rapidly growing of the amount of semistructured documents in real-world applications. Due to the huge size of the real-world data, how to manage semistructured documents effectively is a big challenge for researchers. As a fundamental task in natural language processing field, document classification is a feasible way to handle the large-scale semistructured documents. However, existing methods fail to explicitly take advantage of the hierarchical semantics in semistructured documents. It's known that the contained semantics is beneficial for understanding the semistructured documents. Considering the hierarchical structure of a given semistructured document, we propose a semistructured document classification framework which explicitly utilizes the semantic hierarchical attention mechanism. More specifically, the hierarchical attention mechanism and graph neural network are employed to model semistructured documents, by which the multilevel semantic relationships and grammatical information are considered. Moreover, we propose an adaptive class cost learning method to treat the issue of data imbalance. Comprehensive experiments are conducted on two real-world data sets, and the results demonstrate that our framework performs better than selected baselines for semistructured document classification.

BackgroundComprehensively mapping the hierarchical structure of breast cancer protein communities and identifying potential biomarkers from them is a promising way for breast cancer research. Existing approaches are subjective and fail to take information from protein sequences into consideration. Deep learning can automatically learn features from protein sequences and protein–protein interactions for hierarchical clustering.ResultsUsing a large amount of publicly available proteomics data, we created a hierarchical tree for breast cancer protein communities using a novel hierarchical graph neural network, with the supervision of gene ontology terms and assistance of a pre-trained deep contextual language model. Then, a group-lasso algorithm was applied to identify protein communities that are under both mutation burden and survival burden, undergo significant alterations when targeted by specific drug molecules, and show cancer-dependent perturbations. The resulting hierarchical map of protein communities shows how gene-level mutations and survival information converge on protein communities at different scales. Internal validity of the model was established through the convergence on BRCA2 as a breast cancer hotspot. Further overlaps with breast cancer cell dependencies revealed SUPT6H and RAD21, along with their respective protein systems, HOST:37 and HOST:861, as potential biomarkers. Using gene-level perturbation data of the HOST:37 and HOST:861 gene sets, three FDA-approved drugs with high therapeutic value were selected as potential treatments to be further evaluated. These drugs include mercaptopurine, pioglitazone, and colchicine.ConclusionThe proposed graph neural network approach to analyzing breast cancer protein communities in a hierarchical structure provides a novel perspective on breast cancer prognosis and treatment. By targeting entire gene sets, we were able to evaluate the prognostic and therapeutic value of genes (or gene sets) at different levels, from gene-level to system-level biology. Cancer-specific gene dependencies provide additional context for pinpointing cancer-related systems and drug-induced alterations can highlight potential therapeutic targets. These identified protein communities, in conjunction with other protein communities under strong mutation and survival burdens, can potentially be used as clinical biomarkers for breast cancer.

In recent years, graph neural network is used to process graph data and has been successfully applied to graph node classification task. Due to the complexity of graph structure and the difficulty of obtaining node labels, node classification in datasets with fewer labels becomes a challenge. Existing node classification problems with few-label nodes datasets usually use neighbor aggregation schemes. However, these methods lack the “graph pooling mechanism,” which makes these models impossible to make full use of the global graph information. To solve the problem, we propose a Hierarchical Structure-Feature aware Graph Neural Network (HSFGNN) model. The model learns node features by a hierarchical mechanism that repeatedly utilizes coarsening and refining methods at different levels. Firstly, we use topological information, node characteristics and connectivity of the graph to construct an intuitive and effective coarsening part based on multivariable analysis technology. Secondly, we aggregate the unselected nodes’ features into the top-k nodes, so that the coarser nodes contain more abundant valid graph information. Finally, we propose an additional attention mechanism to obtain more accurate final representations of nodes. Experimental results show that the HSFGNN model achieves outstanding classification accuracy on the dataset with less labeled data.

HiSTGNN: Hierarchical spatio-temporal graph neural network for weather forecasting

Image representation as graphs can enhance the understanding of image semantics and facilitate multi-scale image representation. However, existing methods often overlook the significance of the relationship between elements at each scale or fail to encode the hierarchical relationship between graph elements. To cope with that, we introduce four novel approaches for graph construction from images. These approaches utilize hierarchical image segmentation techniques to generate segmentations at multiple scales, and one of them incorporates edges to encode the relationships at each scale. Leveraging these representations, we present two new models: the Hierarchical Graph Convolutional Network for Image Classification (HGCIC) and the Hierarchical Image Graph with Scale Importance (HIGSI). HGCIC uses an adaptive depth to capture significant features and patterns at different scales, while HIGSI employs a novel readout function that weighs the importance of each scale when generating a fixed-size graph representation. Experimental results with CIFAR-10 and STL-10 datasets show that the HIGSI model outperforms (or closely matches) state-of-the-art models. The model also utilizes smaller graphs, reaching the point of using graphs with 50% of the number of nodes compared to other approaches. Additionally, HIGSI outperforms models trained with only the base graph used to create the hierarchy, achieving up to 11.54% better performance while using fewer parameters.

The automatic identification of the topology of power networks is important for the data-driven and situation-aware operation of power grids. Traditional methods of topology identification lack a data-tolerant mechanism, and the accuracy of their performance in terms of identification is thus affected by the quality of data. Topology identification is related to the link prediction problem. The graph neural network can be used to predict the state of unlabeled nodes (lines) through training on features of labeled nodes (lines) with fault tolerance. Inspired by the characteristics of the graph neural network, we applied it to topology identification in this study. We propose a method to identify the topology of a power network based on a knowledge graph and the graph neural network. Traditional knowledge graphs can quickly mine possible connections between entities and generate graph structure data, but in the case of errors or informational conflicts in the data, they cannot accurately determine whether the relationships between the entities really exist. The graph neural network can use data mining to determine whether a connection obtained between entities is based on their eigenvalues, and has a fault tolerance mechanism to adapt to errors and informational conflicts in the graph data, but needs the graph data as database. The combination of the knowledge graph and the graph neural network can compensate for the deficiency of the single knowledge graph method. We tested the proposed method by using the IEEE 118-bus system and a provincial network system. The results showed that our approach is feasible and highly fault tolerant. It can accurately identify network topology even in the presence of conflicting and missing measurement-related information.

False data injection attacks (FDIA) are a main category of cyber-attacks threatening the security of power systems. Contrary to the detection of these attacks, less attention has been paid to identifying the attacked units of the grid. To this end, this work jointly studies detecting and localizing the stealth FDIA in power grids. Exploiting the inherent graph topology of power systems as well as the spatial correlations of measurement data, this paper proposes an approach based on the graph neural network (GNN) to identify the presence and location of the FDIA. The proposed approach leverages the auto-regressive moving average (ARMA) type graph filters (GFs) which can better adapt to sharp changes in the spectral domain due to their rational type filter composition compared to the polynomial type GFs such as Chebyshev. To the best of our knowledge, this is the first work based on GNN that automatically detects and localizes FDIA in power systems. Extensive simulations and visualizations show that the proposed approach outperforms the available methods in both detection and localization of FDIA for different IEEE test systems. Thus, the targeted areas can be identified and preventive actions can be taken before the attack impacts the grid.

The human nervous system is an ensemble of connected neuronal networks. Modeling and system identification of the human nervous system helps us understand how the brain processes sensory input and controls responses at the systems level. This study aims to propose an advanced approach based on a hierarchical neural network and non-linear system identification method to model neural activity in the nervous system in response to an external somatosensory input. The proposed approach incorporates basic concepts of Non-linear AutoRegressive Moving Average Model with eXogenous input (NARMAX) and neural network to acknowledge non-linear closed-loop neural interactions. Different from the commonly used polynomial NARMAX method, the proposed approach replaced the polynomial non-linear terms with a hierarchical neural network. The hierarchical neural network is built based on known neuroanatomical connections and corresponding transmission delays in neural pathways. The proposed method is applied to an experimental dataset, where cortical activities from ten young able-bodied individuals are extracted from electroencephalographic signals while applying mechanical perturbations to their wrist joint. The results yielded by the proposed method were compared with those obtained by the polynomial NARMAX and Volterra methods, evaluated by the variance accounted for (VAF). Both the proposed and polynomial NARMAX methods yielded much better modeling results than the Volterra model. Furthermore, the proposed method modeled cortical responded with a mean VAF of 69.35% for a three-step ahead prediction, which is significantly better than the VAF from a polynomial NARMAX model (mean VAF 47.09%). This study provides a novel approach for precise modeling of cortical responses to sensory input. The results indicate that the incorporation of knowledge of neuroanatomical connections in building a realistic model greatly improves the performance of system identification of the human nervous system.