Physics-Informed Graph Neural Networks for Water Distribution Systems

Video-based human action recognition is one of the most important and challenging areas of research in the field of computer vision. Human action recognition has found many pragmatic applications in video surveillance, human-computer interaction, entertainment, autonomous driving, etc. Owing to the recent development of deep learning methods for human action recognition, the performance of action recognition has significantly enhanced for challenging datasets. Deep learning techniques are mainly used for recognizing actions in images and videos comprising of Euclidean data. A recent development in deep learning methods is the extension of these techniques to non-Euclidean data or graph data with many nodes and edges. Human body skeleton resembles a graph, therefore, the graph convolutional network (GCN) is applicable to the non-Euclidean body skeleton. In the past few years, GCN has emerged as an important tool for skeleton-based action recognition. Therefore, we conduct a survey using GCN methods for action recognition. Herein, we present a comprehensive overview of recent GCN techniques for action recognition, propose a taxonomy for the categorization of GCN techniques for action recognition, carry out a detailed study of the benchmark datasets, enlist relevant resources and open-source codes, and finally provide an outline for future research directions and trends. To the best of authors' knowledge, this is the first survey for action recognition using GCN techniques.

Cross-subject emotion recognition in brain-computer interface based on frequency band attention graph convolutional adversarial neural networks

Messenger RNA (mRNA) has emerged as a critical global technology that requires global joint efforts from different entities to develop a COVID-19 vaccine. However, the chemical properties of RNA pose a challenge in utilizing mRNA as a vaccine candidate. For instance, the molecules are prone to degradation, which has a negative impact on the distribution of mRNA among patients. In addition, little is known of the degradation properties of individual RNA bases in a molecule. Therefore, this study aims to investigate whether a hybrid deep learning can predict RNA degradation from RNA sequences. Two deep hybrid neural network models were proposed, namely GCN_GRU and GCN_CNN. The first model is based on graph convolutional neural networks (GCNs) and gated recurrent unit (GRU). The second model is based on GCN and convolutional neural networks (CNNs). Both models were computed over the structural graph of the mRNA molecule. The experimental results showed that GCN_GRU hybrid model outperform GCN_CNN model by a large margin during the test time. Validation of proposed hybrid models is performed by well-known evaluation measures. Among different deep neural networks, GCN_GRU based model achieved best scores on both public and private MCRMSE test scores with 0.22614 and 0.34152, respectively. Finally, GCN_GRU pre-trained model has achieved the highest AuC score of 0.938. Such proven outperformance of GCNs indicates that modeling RNA molecules using graphs is critical in understanding molecule degradation mechanisms, which helps in minimizing the aforementioned issues. To show the importance of the proposed GCN_GRU hybrid model, in silico experiments has been contacted. The in-silico results showed that our model pays local attention when predicting a given position’s reactivity and exhibits interesting behavior on neighboring bases in the sequence.

Contemporary deep learning approaches exhibit state-of-the-art performance in various areas. In healthcare, the application of deep learning remains limited since deep learning methods are often considered as non-interpretable black-box models. However, the Machine Learning (ML) community made recent elaborations to develop the methods of eXplainable Artificial Intelligence (XAI). The explanation methods explain single decisions of an ML model when a single data point is fed into the model’s input. In a clinical setup, a data point can represent a single patient. Data point-specific explanations could possibly assist the need in personalized precision medicine decisions via explaining patient-specific predictions. Convolutional Neural Networks (CNNs) as deep learning methods have been already applied to classify transformed into images gene expression profiles of patients. Gene expression data can be structured by a prior knowledge molecular network (encoded as a graph) representing connections between genes. Each vertex of a molecular network is assigned a gene expression value as an attribute. The set of the attributes creates a graph signal representing a patient. Emerging field of geometric deep learning deals with methods applicable to graph structured data and extends CNNs as Graph Convolutional Neural Networks (GCNNs) classifying graph signals. Layer-wise Relevance Propagation (LRP) is a method to explain decisions of CNNs classifying image data. I extended the LRP method to make it available for GCNNs. Graph Layer-wise Relevance Propagation (GLRP) is presented as a new method to explain single decisions made by a GCNN model. In this thesis, I present a novel methodology generating patient-specific molecular subnetworks as explanations for classification decisions of an ML approach utilizing prior knowledge of molecular networks. GCNN serves as a ML approach, and its decisions are explained by developed GLRP. A sanity check of the developed GLRP method was demonstrated on a hand-written digits dataset. The biological validation was performed by applying the developed methodology to gene expression data from Human Umbilical Vein Endothelial Cells (HUVEC) treated or not treated with tumor necrosis factor alpha. To show the utility of introduced methodology in the scopes of precision medicine, it was applied to a large breast cancer dataset. The generated patient-specific subnetworks largely agree with clinical knowledge and could assist precision medicine approaches by identifying common as well as novel, and potentially druggable, drivers of tumor progression. Apart from generating patient-specific subnetworks, the developed methodology can be used as a general feature selection approach. The outcome of a feature selection approach is a subset of important for classification genes corresponding to a whole dataset. It is essential to sustain stability of selected feature subsets across different datasets with the same clinical endpoint since the selected genes are possible candidates for prognostic biomarkers. I analysed the stability of feature selection performed by GCNN+LRP. I have implemented a graph convolutional layer of GCNN as a Keras layer so that the SHapley Additive exPlanations (SHAP) method could be also applied to a Keras version of a GCNN model. The stability of feature selection performed by GCNN+LRP was compared to the stability of GCNN+SHAP and other ML-based feature selection approaches. The GCNN+LRP approach shows the highest stability. GCNN+LRP subnetworks were compared to GCNN+SHAP subnetworks in terms of connectivity and permutation feature importance. While GCNN+SHAP subnetworks demonstrate higher permutation importance than GCNN+LRP subnetworks, a GCNN+LRP subnetwork of an individual patient is on average substantially more connected and, therefore more interpretable in the context of prior knowledge than a GCNN+SHAP subnetwork which consists mainly of single vertices.

Deep learning (DL) models are increasingly used to forecast water quality variables for use in decision making. Ingesting recent observations of the forecasted variable has been shown to greatly increase model performance at monitored locations; however, observations are not collected at all locations, and methods are not yet well developed for DL models for optimally ingesting recent observations from other sites to inform focal sites. In this paper, we evaluate two different DL model structures, a long short-term memory neural network (LSTM) and a recurrent graph convolutional neural network (RGCN), both with and without data assimilation for forecasting daily maximum stream temperature 7 days into the future at monitored and unmonitored locations in a 70-segment stream network. All our DL models performed well when forecasting stream temperature as the root mean squared error (RMSE) across all models ranged from 2.03 to 2.11°C for 1-day lead times in the validation period, with substantially better performance at gaged locations (RMSE = 1.45–1.52°C) compared to ungaged locations (RMSE = 3.18–3.27°C). Forecast uncertainty characterization was near-perfect for gaged locations but all DL models were overconfident (i.e., uncertainty bounds too narrow) for ungaged locations. Our results show that the RGCN with data assimilation performed best for ungaged locations and especially at higher temperatures (>18°C) which is important for management decisions in our study location. This indicates that the networked model structure and data assimilation techniques may help borrow information from nearby monitored sites to improve forecasts at unmonitored locations. Results from this study can help guide DL modeling decisions when forecasting other important environmental variables.

Water distribution systems (WDS) face increasing challenges from climate change, urbanization, and growing populations, making efficient and accurate modeling crucial for their management. While traditional physics-based hydraulic models provide reliable results, they are computationally intensive, limiting their application in real-time decision-making and large-scale optimization. Our multi-year research journey demonstrates a progressive evolution in WDS modeling, successfully combining data-driven efficiency with fundamental physical principles.Our investigations evolve from conventional neural networks to increasingly sophisticated physically-informed approaches. We demonstrate how Graph Neural Networks can leverage network topology to improve prediction accuracy, but more importantly, how reformulating the problem in the edge space allows direct embedding of mass conservation principles. This novel Edge-Based Graph Neural Network (EGNN) architecture not only achieves superior performance but also demonstrates remarkable zero-shot generalization capabilities across different network configurations.Building on these insights, we further reformulate steady-state estimation as a diffusion process on graph edges, incorporating both mass and energy conservation laws. This physics-based reformulation enables direct GPU acceleration without relying on machine learning approximations, achieving near-perfect accuracy on multiple benchmarks while maintaining substantial computational speedups compared to traditional solvers.We then extend this approach to handle uncertainty by developing a topological Gaussian Process framework, where the covariance structure naturally encodes the physical conservation laws. This probabilistic extension enables rapid uncertainty quantification under variable demands, providing analytical uncertainty bounds without the computational burden of Monte Carlo sampling, while preserving the physical consistency guaranteed by our diffusion-based formulation.

DeepTrend 2.0: A light-weighted multi-scale traffic prediction model using detrending

The performance, vulnerability, and resilience of water distribution systems (WDS) are connected to its underlying topological structure (aka its shape). The literature mostly differentiates between two main shapes of networks - branched or looped. However, real networks come in various shapes and forms spanning between the two extremes of purely branched and looped types. Although these networks are globally topologically different, they may show high similarity on the local scale of a borough or a neighbourhood, or vice versa. Recent studies focused on describing WDS via graph theory representing pipes as edges and customers, tanks, and reservoirs as nodes, for example. The first attempt of graph theoretical applications showed promising results in estimating the global resilience of WDS, but there is a limited number of metrics that take the importance of local topology into consideration. Furthermore, iterative estimation of local vulnerability by simulating faults in each element of the system is prohibitively expensive from a computational point of view (i.e., various hydraulic simulations for assessing the vulnerability of each part of the system are needed).This research enters the new terrain of local WDS investigations through graphlet analysis. Graphlets are small connected subgraphs of a large network and have recently gathered much attention as a useful concept to describe local topology and uncover structural design principles of complex networks. Consequently, these novel analyses techniques can provide deep insights into how local WDS structures influence their overall performance.In this work, we investigate the influence of local network structures on the resilience of entire networks through graphlet analysis. First, we calculate local vulnerability of the network elements and global resilience indicators (i.e. Todini index, pipe and node criticality indices). We additionally simulate fault scenarios with EPANET and evaluate volumes of unsupplied demand on a multitude of real WDS. Second, we investigate the graphlet substructure of those WDSs to assess how much of the network’s vulnerability can be described by purely looking at the topology. First results show that graphlet representation of local neighbourhoods can serve as an efficient proxy metric capable of replacing computationally heavy performance analysis based on extensive hydraulic simulation. We additionally compare the influence of local changes on subgraphs to show how local changes in network design may grant improved robustness against such failures, ultimately increasing global resilience based on changing local topology. Urban water management can benefit from the proposed approach by not only identifying the most vulnerable elements of the critical infrastructure but providing insight into how to build globally more resilient WDS networks by enforcing small and therefore economical topological changes.

Geometric deep learning provides a principled and universal way for the integration of imaging and non-imaging modes in the medical field. Graph Convolutional Networks (GCNs) have been widely explored in a variety of problems, such as disease prediction, segmentation, and matrix completion. Using large, multi-modal data sets, graphs can capture the interaction of individual elements represented as nodes in the graphs. In particular, in medical applications, nodes can represent individuals (patients or healthy controls) in a potentially large population and are accompanied by a set of features, while the edges of the graph contain the associations between subjects in an intuitive way. This representation allows the inclusion of rich imaging and non-imaging information as well as individual subject characteristics in the task of disease classification. This article gives an overview of graph convolutional neural networks, focusing on the application of graph convolutional neural networks in disease prediction, and discusses the challenges and prospects of graph convolutional neural networks in disease prediction.

Water Distribution Networks (WDNs) are large-scale, spatially irregular systems composed of interconnected nodes and pipes shaped by urban infrastructure. Despite their critical role in public health and urban resilience, WDNs often lag behind other utilities in terms of digitization and integration into smart city frameworks. This work presents a generative, data-driven method for reconstructing the full hydraulic state—pressures at nodes and flows in pipes—using a feedforward neural network with radial basis function (RBF) activations. The model is trained on synthetic data generated via hydraulic simulations and uses sparse real-time measurements from a limited number of strategically placed pressure and flow sensors. Unlike classical RBF Neural Networks, which rely on fixed spatial centers and local interpolation, the proposed architecture enables full-field inference through a single forward pass. The RBF activations capture spatial dependencies while allowing the network to generalize across the entire topology of the WDN. The approach achieves high-precision reconstruction performance, with Mean Squared Errors on the order of $$10^{-13}$$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:msup> <mml:mn>10</mml:mn> <mml:mrow> <mml:mo>-</mml:mo> <mml:mn>13</mml:mn> </mml:mrow> </mml:msup> </mml:math> and Mean Absolute Errors around $$10^{-7}$$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:msup> <mml:mn>10</mml:mn> <mml:mrow> <mml:mo>-</mml:mo> <mml:mn>7</mml:mn> </mml:mrow> </mml:msup> </mml:math> , confirming the method’s accuracy and stability. So, RBF-NNs emerge as a particularly effective solution, as they efficiently capture local-to-global spatial dependencies inherent in the dynamic behavior of WDNs, contributing to the broader integration of AI-driven solutions in infrastructure management.

Artificial intelligence (AI) and deep learning (DL) have shown tremendous potential in driving sustainability across various sectors. This paper reviews recent advancements in AI and DL and explores their applications in achieving sustainable development goals (SDGs), renewable energy, environmental health, and smart building energy management. AI has the potential to contribute to 134 of the 169 targets across all SDGs, but the rapid development of these technologies necessitates comprehensive regulatory oversight to ensure transparency, safety, and ethical standards. In the renewable energy sector, AI and DL have been effectively utilized in optimizing energy management, fault detection, and power grid stability. They have also demonstrated promise in enhancing waste management and predictive analysis in photovoltaic power plants. In the field of environmental health, the integration of AI and DL has facilitated the analysis of complex spatial data, improving exposure modeling and disease prediction. However, challenges such as the explainability and transparency of AI and DL models, the scalability and high dimensionality of data, the integration with next-generation wireless networks, and ethics and privacy concerns need to be addressed. Future research should focus on enhancing the explainability and transparency of AI and DL models, developing scalable algorithms for processing large datasets, exploring the integration of AI with next-generation wireless networks, and addressing ethical and privacy considerations. Additionally, improving the energy efficiency of AI and DL models is crucial to ensure the sustainable use of these technologies. By addressing these challenges and fostering responsible and innovative use, AI and DL can significantly contribute to a more sustainable future.

COVID-19 is highly infectious and causes acute respiratory disease. Machine learning (ML) and deep learning (DL) models are vital in detecting disease from computerized chest tomography (CT) scans. The DL models outperformed the ML models. For COVID-19 detection from CT scan images, DL models are used as end-to-end models. Thus, the performance of the model is evaluated for the quality of the extracted feature and classification accuracy. There are four contributions included in this work. First, this research is motivated by studying the quality of the extracted feature from the DL by feeding these extracted to an ML model. In other words, we proposed comparing the end-to-end DL model performance against the approach of using DL for feature extraction and ML for the classification of COVID-19 CT scan images. Second, we proposed studying the effect of fusing extracted features from image descriptors, e.g., Scale-Invariant Feature Transform (SIFT), with extracted features from DL models. Third, we proposed a new Convolutional Neural Network (CNN) to be trained from scratch and then compared to the deep transfer learning on the same classification problem. Finally, we studied the performance gap between classic ML models against ensemble learning models. The proposed framework is evaluated using a CT dataset, where the obtained results are evaluated using five different metrics The obtained results revealed that using the proposed CNN model is better than using the well-known DL model for the purpose of feature extraction. Moreover, using a DL model for feature extraction and an ML model for the classification task achieved better results in comparison to using an end-to-end DL model for detecting COVID-19 CT scan images. Of note, the accuracy rate of the former method improved by using ensemble learning models instead of the classic ML models. The proposed method achieved the best accuracy rate of 99.39%.

PurposeTo conduct a head-to-head comparison between deep learning (DL) and radiomics models across institutions for predicting microvascular invasion (MVI) in hepatocellular carcinoma (HCC) and to investigate the model robustness and generalizability through rigorous internal and external validation.MethodsThis retrospective study included 2304 preoperative images of 576 HCC lesions from two centers, with MVI status determined by postoperative histopathology. We developed DL and radiomics models for predicting the presence of MVI using B-mode ultrasound, contrast-enhanced ultrasound (CEUS) at the arterial, portal, and delayed phases, and a combined modality (B + CEUS). For radiomics, we constructed models with enlarged vs. original regions of interest (ROIs). A cross-validation approach was performed by training models on one center’s dataset and validating the other, and vice versa. This allowed assessment of the validity of different ultrasound modalities and the cross-center robustness of the models. The optimal model combined with alpha-fetoprotein (AFP) was also validated. The head-to-head comparison was based on the area under the receiver operating characteristic curve (AUC).ResultsThirteen DL models and 25 radiomics models using different ultrasound modalities were constructed and compared. B + CEUS was the optimal modality for both DL and radiomics models. The DL model achieved AUCs of 0.802–0.818 internally and 0.667–0.688 externally across the two centers, whereas radiomics achieved AUCs of 0.749–0.869 internally and 0.646–0.697 externally. The radiomics models showed overall improvement with enlarged ROIs (P < 0.05 for both CEUS and B + CEUS modalities). The DL models showed good cross-institutional robustness (P > 0.05 for all modalities, 1.6–2.1% differences in AUC for the optimal modality), whereas the radiomics models had relatively limited robustness across the two centers (12% drop-off in AUC for the optimal modality). Adding AFP improved the DL models (P < 0.05 externally) and well maintained the robustness, but did not benefit the radiomics model (P > 0.05).ConclusionCross-institutional validation indicated that DL demonstrated better robustness than radiomics for preoperative MVI prediction in patients with HCC, representing a promising solution to non-standardized ultrasound examination procedures.

Crop price prediction is crucial in reducing market uncertainties and supporting knowledgeable decision-making among farmers, traders, and policymakers. Conventional techniques are relatively simple and easy to understand and implement; however, they provide low prediction performance for non-linear, non-smooth, and high-dimensional data, and they need more a priori knowledge and assumptions. Recently, the Machine Learning (ML) and Deep Learning (DL) models have helped model these non-linear patterns, providing highly reliable and robust crop price prediction. This research paper presents an Advanced Forecasting of Crop Prices through Graph Convolutional Neural Networks by Heuristic Search Optimisation (AFCP-GCNNHSO) approach. The main intention of the AFCP-GCNNHSO approach is to enable effective prediction of crop prices by the use of feature selection and a fine-tuned DL model. To handle high-dimensional data, the AFCP-GCNNHSO method utilizes the Aquila Optimiser (AO) technique for the optimal feature subset selection, which results in reduced complexity and enhanced prediction performance. For effective crop price prediction, the AFCP-GCNNHSO method employs an Enhanced Grasshopper Optimization Algorithm (EGOA) with a Gated Graph Convolution Neural Network (GGCN) technique. The EGOA based hyperparameter tuning is implemented to optimize the GGCN model by minimizing prediction error. The experimental analysis of the AFCP-GCNNHSO method is performed using the agricultural dataset from the Kaggle repository, and the comparison results highlight its supremacy with a minimal MSE of 0.00038.

Electricity load forecasting is a crucial undertaking within all the deregulated markets globally. Among the research challenges on a global scale, the investigation of deep transfer learning (DTL) in the field of electricity load forecasting represents a fundamental effort that can inform artificial intelligence applications in general. In this paper, a comprehensive study is reported regarding day-ahead electricity load forecasting. For this purpose, three sequence-to-sequence (Seq2seq) deep learning (DL) models are used, namely the multilayer perceptron (MLP), the convolutional neural network (CNN) and the ensemble learning model (ELM), which consists of the weighted combination of the outputs of MLP and CNN models. Also, the study focuses on the development of different forecasting strategies based on DTL, emphasizing the way the datasets are trained and fine-tuned for higher forecasting accuracy. In order to implement the forecasting strategies using deep learning models, load datasets from three Greek islands, Rhodes, Lesvos, and Chios, are used. The main purpose is to apply DTL for day-ahead predictions (1–24 h) for each month of the year for the Chios dataset after training and fine-tuning the models using the datasets of the three islands in various combinations. Four DTL strategies are illustrated. In the first strategy (DTL Case 1), each of the three DL models is trained using only the Lesvos dataset, while fine-tuning is performed on the dataset of Chios island, in order to create day-ahead predictions for the Chios load. In the second strategy (DTL Case 2), data from both Lesvos and Rhodes concurrently are used for the DL model training period, and fine-tuning is performed on the data from Chios. The third DTL strategy (DTL Case 3) involves the training of the DL models using the Lesvos dataset, and the testing period is performed directly on the Chios dataset without fine-tuning. The fourth strategy is a multi-task deep learning (MTDL) approach, which has been extensively studied in recent years. In MTDL, the three DL models are trained simultaneously on all three datasets and the final predictions are made on the unknown part of the dataset of Chios. The results obtained demonstrate that DTL can be applied with high efficiency for day-ahead load forecasting. Specifically, DTL Case 1 and 2 outperformed MTDL in terms of load prediction accuracy. Regarding the DL models, all three exhibit very high prediction accuracy, especially in the two cases with fine-tuning. The ELM excels compared to the single models. More specifically, for conducting day-ahead predictions, it is concluded that the MLP model presents the best monthly forecasts with MAPE values of 6.24% and 6.01% for the first two cases, the CNN model presents the best monthly forecasts with MAPE values of 5.57% and 5.60%, respectively, and the ELM model achieves the best monthly forecasts with MAPE values of 5.29% and 5.31%, respectively, indicating the very high accuracy it can achieve.