Multi-Scale Dynamic Graph Learning for Time Series Anomaly Detection (Student Abstract)

Graph neural networks (GNNs) have recently achieved remarkable success on a variety of graph-related tasks, while such success relies heavily on a given graph structure that may not always be available in real-world applications. To address this problem, graph structure learning (GSL) is emerging as a promising research topic where task-specific graph structure and GNN parameters are jointly learned in an end-to-end unified framework. Despite their great progress, existing approaches mostly focus on the design of similarity metrics or graph construction, but directly default to adopting downstream objectives as supervision, which lacks deep insight into the power of supervision signals. More importantly, these approaches struggle to explain how GSL helps GNNs, and when and why this help fails. In this article, we conduct a systematic experimental evaluation to reveal that GSL and GNNs enjoy consistent optimization goals in terms of improving the graph homophily. Furthermore, we demonstrate theoretically and experimentally that task-specific downstream supervision may be insufficient to support the learning of both graph structure and GNN parameters, especially when the labeled data are extremely limited. Therefore, as a complement to downstream supervision, we propose homophily-enhanced self-supervision for GSL (HES-GSL), a method that provides more supervision for learning an underlying graph structure. A comprehensive experimental study demonstrates that HES-GSL scales well to various datasets and outperforms other leading methods. Our code will be available in https://github.com/LirongWu/Homophily-Enhanced-Self-supervision.

The transportation system is a complex dynamic giant system which integrates and intertwines the elements of people, vehicles, roads, and the environment. The city‐level traffic flow forecasting can effectively reflect the flow changes of the traffic system and provide practical guidance for the formulation of traffic rules. Recent city‐level traffic flow forecasting works rely on accurate prior knowledge of graphs (i.e., the spatial relationships between roads), which hinders their effectiveness and application in the real world. We propose a novel framework for urban traffic flow forecasting, which simultaneously infers and utilizes the relationship between time series. In our model, the graph structure learning module dynamically captures the correlation and causation between the different time series and infers a potentially fully connected graph. At the same time, the temporal convolution network captures the temporal correlation between a single time series. The graph neural network uses the graph for forecasting. Our model no longer relies on accurate graph priors and achieves better forecasting results than previous work. Experiments on two public datasets verify that the proposed model is very competitive.

Graph neural networks (GNNs) often struggle to learn discriminative node representations for heterophilic graphs, where connected nodes tend to have dissimilar labels and feature similarity provides weak structural cues. We propose frequency-guided graph structure learning (FgGSL), an end-to-end graph inference framework that jointly learns homophilic and heterophilic graph structures along with a spectral encoder. FgGSL employs a learnable, symmetric, feature-driven masking function to infer said complementary graphs, which are processed using pre-designed low- and high-pass graph filter banks. A label-based structural loss explicitly promotes the recovery of homophilic and heterophilic edges, enabling task-driven graph structure learning. We derive stability bounds for the structural loss and establish robustness guarantees for the filter banks under graph perturbations. Experiments on six heterophilic benchmarks demonstrate that FgGSL consistently outperforms state-of-the-art GNNs and graph rewiring methods, highlighting the benefits of combining frequency information with supervised topology inference.

Graph neural networks (GNNs) have achieved considerable success in dealing with graph-structured data by the message-passing mechanism. Actually, this mechanism relies on a fundamental assumption that the graph structure along which information propagates is perfect. However, the real-world graphs are inevitably incomplete or noisy, which violates the assumption, thus resulting in limited performance. Therefore, optimizing graph structure for GNNs is indispensable and important. Although current semi-supervised graph structure learning (GSL) methods have achieved a promising performance, the potential of labels and prior graph structure has not been fully exploited yet. Inspired by this, we examine GSL with dual reinforcement of label and prior structure in this article. Specifically, to enhance label utilization, we first propose to construct the prior label-constrained matrices to refine the graph structure by identifying label consistency. Second, to adequately leverage the prior structure to guide GSL, we develop spectral contrastive learning that extracts global properties embedded in the prior graph structure. Moreover, contrastive fusion with prior spatial structure is further adopted, which promotes the learned structure to integrate local spatial information from the prior graph. To extensively evaluate our proposal, we perform sufficient experiments on seven benchmark datasets, where experimental results confirm the effectiveness of our method and the rationality of the learned structure from various aspects.

In recent years, heterogeneous network/graph representation learning/embedding (HNE) has drawn tremendous attentions from research communities in multiple disciplines. HNE has shown its outstanding performances in various networked data analysis and mining tasks. In fact, most of real-world information networks in multiple fields can be modelled as the heterogeneous information networks (HIN). Thus, the HNE-based techniques can sufficiently capture rich-structured and semantic latent features from the given information network in order to facilitate for different task-driven learning tasks. This is considered as fundamental success of HNE-based approach in comparing with previous traditional homogeneous network/graph based embedding techniques. However, there are recent studies have also demonstrated that the heterogeneous network/graph modelling and embedding through graph neural network (GNN) is not usually reliable. This challenge is original come from the fact that most of real-world heterogeneous networks are considered as incomplete and normally contain a large number of feature noises. Therefore, multiple attempts have proposed recently to overcome this limitation. Within this approach, the meta-path-based heterogeneous graph-structured latent features and GNN-based parameters are jointly learnt and optimized during the embedding process. However, this integrated GNN and heterogeneous graph structure (HGS) learning approach still suffered a challenge of effectively parameterizing and fusing different graph-structured latent features from both GNN- and HGS-based sides into better task-driven friendly and noise-reduced embedding spaces. Therefore, in this paper we proposed a novel attention-supplemented heterogeneous graph structure embedding approach, called as: AGSE. Our proposed AGSE model supports to not only achieve the combined rich heterogeneous structural and GNN-based aggregated node representations but also transform achieved node embeddings into noise-reduced and task-driven friendly embedding space. Extensive experiments in benchmark heterogeneous networked datasets for node classification task showed the effectiveness of our proposed AGSE model in comparing with state-of-the-art network embedding baselines.

Graph Neural Networks (GNNs) have achieved remarkable success in graph-related tasks by combining node features and graph topology elegantly. Most GNNs assume that the networks are homophilous, which is not always true in the real world, i.e., structure noise or disassortative graphs. Only a few works focus on generalizing graph neural networks to heterophilous or low homophilous networks, where connected nodes may have different labels. In this paper, we design a simple and effective Graph Structure Learning strategy based on Feature and Label consistency (GSLFL) to increase the homophilous level of networks for generalizing any existing GNNs to heterophilous networks. Specifically, we first introduce a method to learn graph structure based on node features and then modify the graph structure based on label consistency. Further, we combine the GSLFL with three existing GNNs to learn node representations and graph structure together. And we design a self-training method to iteratively train models and modify graph structure with pseudo-labels. Finally, our empirical results on 6 public networks with homophily or heterophily, and structure attacks show that our methods outperform the state-of-the-art methods in most cases.

Due to the rapid adoption of 5G networks and the increasing number of devices and base stations (gNBs) connected to it, manually identifying malfunctioning machines or devices that cause a part of the networks to fail becomes more challenging. Furthermore, data collected from the networks are not always sufficient. To overcome these two issues, we proposed a novel root cause analysis (RCA) framework that integrates graph neural networks (GNNs) with graph structure learning (GSL) to infer hidden dependencies from available data. The learned dependencies are the graph structure utilized to predict the root cause machines or devices. We found that despite the fact that the data is often incomplete, the GSL model can infer fairly accurate hidden dependencies from data with a large number of nodes and generate informative graph representation for GNNs to identify the root cause. Our experimental results showed that higher accuracy of identifying a root cause and victim nodes can be achieved when the number of nodes in an environment is increased.

Extensive studies have been conducted to investigate Artificial Intelligence (AI) in the context of time series data. In this article, we investigate the complex domain of industrial time series, from the dimensions of classical machine learning (ML), deep neural networks (DNNs) and graph neural networks (GNNs). Current surveys often focus on a specific methodology or oversee the connection of diverse approaches; our article bridges this gap by providing an all-inclusive interpretation across numerous techniques. In addition, the aim of this article is to focus on the core areas of time series such as forecasting, classification, and anomaly detection. From traditional methodologies like Autoregressive Integrated Moving Average (ARIMA) and support vector machine (SVM) methods, the advancements of DNNs, for instance long-short-term memory (LSTMs), convolutional neural networks (CNNs), attention mechanisms, and transformers, describe how temporal information is used for forecasting, anomaly detection, and classification. Then the article discusses the advances and limitations in ML, DNN, and GNN in order to improve the different methods in either category. Lastly, we outline future directions and open research questions with the different methodologies used in time series.

At present, the graph neural network has achieved good results in the semisupervised classification of graph structure data. However, the classification effect is greatly limited in those data without graph structure, incomplete graph structure, or noise. It has no high prediction accuracy and cannot solve the problem of the missing graph structure. Therefore, in this paper, we propose a high-order graph learning attention neural network (HGLAT) for semisupervised classification. First, a graph learning module based on the improved variational graph autoencoder is proposed, which can learn and optimize graph structures for data sets without topological graph structure and data sets with missing topological structure and perform regular constraints on the generated graph structure to make the optimized graph structure more reasonable. Then, in view of the shortcomings of graph attention neural network (GAT) that cannot make full use of the graph high-order topology structure for node classification and graph structure learning, we propose a graph classification module that extends the attention mechanism to high-order neighbors, in which attention decays according to the increase of neighbor order. HGLAT performs joint optimization on the two modules of graph learning and graph classification and performs semisupervised node classification while optimizing the graph structure, which improves the classification performance. On 5 real data sets, by comparing 8 classification methods, the experiment shows that HGLAT has achieved good classification results on both a data set with graph structure and a data set without graph structure.

Graph Neural Networks (GNNs) are de facto solutions to structural data learning. However, it is susceptible to low-quality and unreliable structure, which has been a norm rather than an exception in real-world graphs. Existing graph structure learning (GSL) frameworks still lack robustness and interpretability. This paper proposes a general GSL framework, SE-GSL, through structural entropy and the graph hierarchy abstracted in the encoding tree. Particularly, we exploit the one-dimensional structural entropy to maximize embedded information content when auxiliary neighbourhood attributes are fused to enhance the original graph. A new scheme of constructing optimal encoding trees is proposed to minimize the uncertainty and noises in the graph whilst assuring proper community partition in hierarchical abstraction. We present a novel sample-based mechanism for restoring the graph structure via node structural entropy distribution. It increases the connectivity among nodes with larger uncertainty in lower-level communities. SE-GSL is compatible with various GNN models and enhances the robustness towards noisy and heterophily structures. Extensive experiments show significant improvements in the effectiveness and robustness of structure learning and node representation learning.

Graph neural networks (GNNs) have a powerful ability to capture long-range spatial correlations in hyperspectral images (HSIs). However, existing GNN-based HSI classification methods are vulnerable to hand-crafted graphs, as the manner in which these graphs are constructed are often inappropriate and are likely to violate intrinsic graph properties, such as sparsity and low-rank. More importantly, the goal of HSI classification is to categorize each individual pixel into a land-cover class, but existing methods usually overuse global dependencies and ignore the importance of individualized spectral characteristics. Therefore, this paper proposes a Global-to-Local decoupling Graph network (GoLoG) to conduct HSI classification in a global-to-local framework, which jointly optimizes the graph structure and network parameters guided by both intrinsic graph properties and classification loss. Specifically, a novel global-to-local network framework with successive global and local graph convolutional stages is constructed. By decoupling global and local stages, global contextual information can be exploited, and the individualized information of each hyperspectral pixel can be emphasized for HSI classification. Second, a sparse and low-rank graph structure learning model is proposed to refine and renovate the initial-construct graph. Finally, to unify graph structure learning and network training, a joint alternating update algorithm is introduced to jointly optimize the sparse and low-rank graph structure learning model and the global-to-local network framework. Extensive experiments demonstrate that the proposed GoLoG has obvious advantages compared with other state-of-the-art HSI classification methods.

GATE block setup. The transformation performed by the RNN is responsible for both incorporating into and translating the hidden representation in time to time . The updated latent representations at the target time are projected by a linear layer (LL) back into the original space. Data collection in many engineering fields involves multivariate time series gathered from a collection of sensors that operate independently of each other. These sensors often display various irregularities, such as different sampling rates and missing data. To manage these issues, complex preprocessing mechanisms are required, which become coupled with any statistical model trained with the transformed data. Modeling the motion of floating platforms anchored on seabeds from measurements acquired from sensors is a typical example. We propose and analyze a model in which each sensor is encoded using an independent time-informed recurrent neural network, information is propagated in a common latent space by a graph neural network, and a modified training method is used to induce temporal generalization of the model. Our method can generate forecasts in unseen frequencies, which provides empirical evidence that the model learns an implicit representation of the system’s time derivatives and is able to flexibly integrate the signal over the time domain. • A novel architecture is proposed to address irregularities in multivariate time series (MTS), including varying sampling rates, missing data, and time misalignments. • The model combines time-informed recurrent neural networks (RNNS) with graph neural networks (GNNs) for generalized temporal modeling. • A modified training method enables the model to generalize temporal patterns and generate forecasts at unseen frequencies. • Validation on real-world data from Floating Production, Storage, and Offloading units (FPSOs) demonstrates superior predictive accuracy and reduced preprocessing requirements.

Machine learning, with its advances in deep learning has shown great potential in analyzing time series. In many scenarios, however, additional information that can potentially improve the predictions is available. This is crucial for data that arise from e. g., sensor networks that contain information about sensor locations. Then, such spatial information can be exploited by modeling it via graph structures, along with the sequential (time series) information. Recent advances in adapting deep learning to graphs have shown potential in various tasks. However, these methods have not been adapted for time series tasks to a great extent. Most attempts have essentially consolidated around time series forecasting with small sequence lengths. Generally, these architectures are not well suited for regression or classification tasks where the value to be predicted is not strictly depending on the most recent values, but rather on the whole length of the time series. We propose TISER-GCN, a novel graph neural network architecture for processing, in particular, these long time series in a multivariate regression task. Our proposed model is tested on two seismic datasets containing earthquake waveforms, where the goal is to predict maximum intensity measurements of ground shaking at each seismic station. Our findings demonstrate promising results of our approach—with an average MSE reduction of 16.3%—compared to the best performing baselines. In addition, our approach matches the baseline scores by needing only half the input size. The results are discussed in depth with an additional ablation study.

Fund transfer fraud detection: Analyzing irregular transactions and customer relationships with self-attention and graph neural networks

Precise Global Navigation Satellite System (GNSS) time series have greatly advanced tectonic studies, particularly in detecting transient deformation signals like slow slip events (SSEs). However, GNSS position data can be noisy, impacting the accuracy of analyses. Traditional denoising methods often struggle with spatially heterogeneous and evolving networks. This study introduces a novel Graph Neural Network (GNN) approach to denoise GNSS time series, effectively managing network heterogeneity and varying station availability. GNNs are robust against temporal gaps, making them suitable for GNSS data. Applied to daily time series for the Cascadia Region processed by the University of Nevada Reno and Central Washington University, our method reduced common-mode noise by more than 70% and 30% on horizontal components, in the two datasets respectively, significantly enhancing surface displacement measurements and slow slip events (SSE) source property estimation. We compared the GNN approach with three simple stack filtering methods, which performed comparably in many situations but are more sensitive to parameter choices. For all methods, substantial noise reduction removes artifacts that could impact geophysical interpretations. Our findings suggest that GNN-based denoising offers a robust, adaptive solution for heterogeneous GNSS networks, enhancing accuracy in tectonic and volcanic process studies, but stack filtering approaches might still outperform the machine learning technique depending on the application.