Compressing Time-Dependent Reservoir Simulations Using Graph-Convolutional Neural Network G-CNN

Messenger RNA (mRNA) has emerged as a critical global technology that requires global joint efforts from different entities to develop a COVID-19 vaccine. However, the chemical properties of RNA pose a challenge in utilizing mRNA as a vaccine candidate. For instance, the molecules are prone to degradation, which has a negative impact on the distribution of mRNA among patients. In addition, little is known of the degradation properties of individual RNA bases in a molecule. Therefore, this study aims to investigate whether a hybrid deep learning can predict RNA degradation from RNA sequences. Two deep hybrid neural network models were proposed, namely GCN_GRU and GCN_CNN. The first model is based on graph convolutional neural networks (GCNs) and gated recurrent unit (GRU). The second model is based on GCN and convolutional neural networks (CNNs). Both models were computed over the structural graph of the mRNA molecule. The experimental results showed that GCN_GRU hybrid model outperform GCN_CNN model by a large margin during the test time. Validation of proposed hybrid models is performed by well-known evaluation measures. Among different deep neural networks, GCN_GRU based model achieved best scores on both public and private MCRMSE test scores with 0.22614 and 0.34152, respectively. Finally, GCN_GRU pre-trained model has achieved the highest AuC score of 0.938. Such proven outperformance of GCNs indicates that modeling RNA molecules using graphs is critical in understanding molecule degradation mechanisms, which helps in minimizing the aforementioned issues. To show the importance of the proposed GCN_GRU hybrid model, in silico experiments has been contacted. The in-silico results showed that our model pays local attention when predicting a given position’s reactivity and exhibits interesting behavior on neighboring bases in the sequence.

Graph neural networks (GNNs) have gradually become an important research branch in graph learning since 2005, and the most active one is unquestionably graph convolutional neural networks (GCNs). Although convolutional neural networks have successfully learned for images, voices, and texts, over-smoothing remains a significant obstacle for non-grid graphs. In particular, because of the over-smoothing problem, most existing GCNs are only effective below four layers. This work proposes a novel GCN named DII-GCN that originally integrates Dropedge, Initial residual, and Identity mapping methods into traditional GCNs for mitigating over-smoothing. In the first step of the DII-GCN, the Dropedge increases the diversity of learning sample data and slows down the network’s learning speed to improve learning accuracy and reduce over-fitting. The initial residual is embedded into the convolutional learning units under the identity mapping in the second step, which extends the learning path and thus weakens the over-smoothing issue in the learning process. The experimental results show that the proposed DII-GCN achieves the purpose of constructing deep GCNs and obtains better accuracy than existing shallow networks. DII-GCN model has the highest 84.6% accuracy at 128 layers of the Cora dataset, highest 72.5% accuracy at 32 layers of the Citeseer dataset, highest 79.7% accuracy at 32 layers of the Pubmed dataset.

In recent years, there has been a significant increase in the application of graph neural networks on a wide range of different problems. A specially promising direction of research is on graph convolutional neural networks (GCN). This work focuses on the application of GCNs to graph combinatorial optimization problems (GCOPs), specifically on their decision versions. GCNs are applied on the family of GCOPs in which the objective function is directly related to the number of vertices in a graph. The selected problems are the minimum vertex cover, maximum clique, minimum dominating set and graph coloring. In this work, a framework is proposed for solving problems of this type. The performance of this approach is explored for standard multi-layer GCNs using different types of convolutional layers. On top of this, we propose a new structure that is based on graph isomorphism networks. Computational experiments show that this new structure is significantly more efficient than basic structures. To further improve the performance of this method, the use of GCN ensembles is also explored. When using GCNs to generate solution values for GCOPs, they often correspond to non-feasible solutions because they violate the problem boundaries. This issue is addressed by defining an asymmetric loss function that is used during the GCN training. The proposed method is evaluated on a large set of training data consisting of graph solution pairs that can also be accessed online.

Cross-subject emotion recognition in brain-computer interface based on frequency band attention graph convolutional adversarial neural networks

Automatic building extraction from optical imagery remains a challenge due to, for example, the complexity of building shapes. Semantic segmentation is an efficient approach for this task. The latest development in deep convolutional neural networks (DCNNs) has made accurate pixel-level classification tasks possible. Yet one central issue remains: the precise delineation of boundaries. Deep architectures generally fail to produce fine-grained segmentation with accurate boundaries due to their progressive down-sampling. Hence, we introduce a generic framework to overcome the issue, integrating the graph convolutional network (GCN) and deep structured feature embedding (DSFE) into an end-to-end workflow. Furthermore, instead of using a classic graph convolutional neural network, we propose a gated graph convolutional network, which enables the refinement of weak and coarse semantic predictions to generate sharp borders and fine-grained pixel-level classification. Taking the semantic segmentation of building footprints as a practical example, we compared different feature embedding architectures and graph neural networks. Our proposed framework with the new GCN architecture outperforms state-of-the-art approaches. Although our main task in this work is building footprint extraction, the proposed method can be generally applied to other binary or multi-label segmentation tasks.

Graph convolutional neural networks (GCNNs) have emerged as a promising tool in the deep learning community to learn complex hidden relationships of data generated from non-Euclidean domains and represented as graphs. GCNNs are formed by a cascade of layers of graph filters, which replace the classical convolution operation in convolutional neural networks. These graph filters, when operated over real networks, can be subject to random perturbations due to link losses that can be caused by noise, interference and adversarial attacks. In addition, these graph filters are executed by finite-precision processors, which generate numerical quantization errors that may affect their performance. Despite the research works studying the effect of either graph perturbations or quantization in GCNNs, their robustness against both of these problems jointly is still not well investigated and understood. In this paper, we propose a quantized GCNN architecture based on neighborhood graph filters under random graph perturbations. We investigate the stability of such architecture to both random graph perturbations and quantization errors. We prove that the expected error due to quantization and random graph perturbations at the GCNN output is upper-bounded and we show how this bound can be controlled. Numerical experiments are conducted to corroborate our theoretical findings.

Spectral graph convolutional neural networks (GCN) are proposed to incorporate important information contained in graphs such as gene networks. In a standard spectral GCN, there is only one gene network to describe the relationships among genes. However, for genomic applications, due to condition- or tissue-specific gene function and regulation, multiple gene networks may be available; it is unclear how to apply GCNs to disease classification with multiple networks. Besides, which gene networks may provide more effective prior information for a given learning task is unknown a priori and is not straightforward to discover in many cases. A deep multiple graph convolutional neural network is therefore developed here to meet the challenge. The new approach not only computes a feature of a gene as the weighted average of those of itself and its neighbors through spectral GCNs, but also extracts features from gene-specific expression (or other feature) profiles via a feed-forward neural networks (FNN). We also provide two measures, the importance of a given gene and the relative importance score of each gene network, for the genes' and gene networks' contributions, respectively, to the learning task. To evaluate the new method, we conduct real data analyses using several breast cancer and diffuse large B-cell lymphoma datasets and incorporating multiple gene networks obtained from "GIANT 2.0" Compared with the standard FNN, GCN, and random forest, the new method not only yields high classification accuracy but also prioritizes the most important genes confirmed to be highly associated with cancer, strongly suggesting the usefulness of the new method in incorporating multiple gene networks.

The era of new human–computer interaction has accelerated, and gesture recognition is one of the development trends of human–computer interaction system in the future. The emerging graph neural network can capture the interdependence between instances and infer the complete information of the image based on local features, which is helpful to the recognition of human gestures. Therefore, the combination of graph neural network and convolutional neural network (CNN) is applied to the research of gesture recognition and it has important research significance. This paper uses the Mixup method to perform data enhancement processing on the collected image data set, applies the pyramid pooling method to the EigenPooL model to achieve efficient capture of image features and selects the Sigmoid function and the PReLU function as the activation function of the network model to meet the model’s requirements, such as long time training and strong fitting ability. This paper introduces the structure and algorithm of EigenPooL model in detail. The algorithm uses hypergraph learning method instead of simple graph learning method. On the gesture picture test set, the average accuracy of the algorithm is 86.50%, the recall rate is 94.87% and the average detection time per frame is 421[Formula: see text]ms.

Contemporary deep learning approaches exhibit state-of-the-art performance in various areas. In healthcare, the application of deep learning remains limited since deep learning methods are often considered as non-interpretable black-box models. However, the Machine Learning (ML) community made recent elaborations to develop the methods of eXplainable Artificial Intelligence (XAI). The explanation methods explain single decisions of an ML model when a single data point is fed into the model’s input. In a clinical setup, a data point can represent a single patient. Data point-specific explanations could possibly assist the need in personalized precision medicine decisions via explaining patient-specific predictions. Convolutional Neural Networks (CNNs) as deep learning methods have been already applied to classify transformed into images gene expression profiles of patients. Gene expression data can be structured by a prior knowledge molecular network (encoded as a graph) representing connections between genes. Each vertex of a molecular network is assigned a gene expression value as an attribute. The set of the attributes creates a graph signal representing a patient. Emerging field of geometric deep learning deals with methods applicable to graph structured data and extends CNNs as Graph Convolutional Neural Networks (GCNNs) classifying graph signals. Layer-wise Relevance Propagation (LRP) is a method to explain decisions of CNNs classifying image data. I extended the LRP method to make it available for GCNNs. Graph Layer-wise Relevance Propagation (GLRP) is presented as a new method to explain single decisions made by a GCNN model. In this thesis, I present a novel methodology generating patient-specific molecular subnetworks as explanations for classification decisions of an ML approach utilizing prior knowledge of molecular networks. GCNN serves as a ML approach, and its decisions are explained by developed GLRP. A sanity check of the developed GLRP method was demonstrated on a hand-written digits dataset. The biological validation was performed by applying the developed methodology to gene expression data from Human Umbilical Vein Endothelial Cells (HUVEC) treated or not treated with tumor necrosis factor alpha. To show the utility of introduced methodology in the scopes of precision medicine, it was applied to a large breast cancer dataset. The generated patient-specific subnetworks largely agree with clinical knowledge and could assist precision medicine approaches by identifying common as well as novel, and potentially druggable, drivers of tumor progression. Apart from generating patient-specific subnetworks, the developed methodology can be used as a general feature selection approach. The outcome of a feature selection approach is a subset of important for classification genes corresponding to a whole dataset. It is essential to sustain stability of selected feature subsets across different datasets with the same clinical endpoint since the selected genes are possible candidates for prognostic biomarkers. I analysed the stability of feature selection performed by GCNN+LRP. I have implemented a graph convolutional layer of GCNN as a Keras layer so that the SHapley Additive exPlanations (SHAP) method could be also applied to a Keras version of a GCNN model. The stability of feature selection performed by GCNN+LRP was compared to the stability of GCNN+SHAP and other ML-based feature selection approaches. The GCNN+LRP approach shows the highest stability. GCNN+LRP subnetworks were compared to GCNN+SHAP subnetworks in terms of connectivity and permutation feature importance. While GCNN+SHAP subnetworks demonstrate higher permutation importance than GCNN+LRP subnetworks, a GCNN+LRP subnetwork of an individual patient is on average substantially more connected and, therefore more interpretable in the context of prior knowledge than a GCNN+SHAP subnetwork which consists mainly of single vertices.

Link prediction in complex networks aims to mine hidden or to-be-generated links between network nodes, which plays a significant role in fields such as the cold start of recommendation systems, knowledge graph completion and biomedical experiments. The existing link prediction models based on graph neural networks, such as graph convolution neural networks, often only learn the low-frequency information reflecting the common characteristics of nodes while ignoring the high-frequency information reflecting the differences between nodes when learning node representation, which makes the corresponding link prediction models show over smoothness and poor performance. Focusing on links in complex networks, this paper proposes an edge convolutional graph neural network EdgeConvHiF that fuses high-frequency node information to achieve the representation learning of links so that link prediction can be realized by implementing the classification of links. EdgeConvHiF can also be employed as a baseline, and extensive experiments on real-world benchmarks validate that EdgeConvHiF not only has high stability but also has more advantages than the existing representative baselines.

We describe a novel approach for up-scaling of geological models of heterogeneous reservoirs. The method, which is based on the use of wavelet transformations, incorporates all the relevant data on the permeability, porosity, and other important properties of a reservoir at all the important scales. It generates a nonuniform computational grid which preserves its resolved structure in the regions around the wells and in the high-permeability zones, but is scaled up in the rest of the geological model. To demonstrate the accuracy and efficiency of the method, we apply it to the study of three important problems, namely, the pressure transient analysis, unstable miscible displacements, and two-phase flows. In the study of multiphase flows, we apply a new and robust front detection method. The numerical dispersion is eliminated by a high-order total variation diminishing (TVD) method which yields results that are as accurate as those obtained by the dual-mesh methods, but with a greatly-reduced computational cost. The speed-up in the computations is up to three orders of magnitude. 1. Introduction Computer simulation of unstable miscible displacements and multiphase flows in petroleum reservoirs involves intensive computations1. Advances in measurement and estimation techniques, together with considerable progress in the development of methods for characterization of petroleum reservoirs, can provide significant amounts of data on various properties of the reservoirs. Using such data, modern geostatistical techniques generate highly resolved geological models of heterogeneous petroleum reservoirs. To carry out reservoir simulations, the governing flow and transport equations are discretized, and the flow domain is divided into grid blocks which are characterized by the associated rock properties. Due to computational limitations (mainly computation time) it is not feasible to carry out multiphase flow simulations with a computational grid that corresponds to the highly resolved geological model. It is, therefore, necessary to up-scale the properties of the grid blocks of the geological model in order to develop a coarsened grid that can be used in reservoir simulation with a reasonable amount of computation time. Up-scaling also plays a critical role in reservoir planning and management. In the past, it was common to forecast production based on reservoir simulations that used deterministic models of reservoirs - those that did not take into account the effect of the uncertainties in the distributions of the reservoirs' properties. However, deterministic models yield predictions that may be more harmful to reservoir planning and management than not having a model at all, since the appraisal of the potential production of a petroleum field must take into account the effect of the uncertainties in the properties of the field. Under such conditions, production forecast obtained by a stochastic model of a reservoir allows the quantification of the effect of the uncertainties on the reservoir performance by simulation of fluid flow in many possible realizations of the same reservoir, which are necessarily very time consuming. To reduce the simulation time, up scaling of the geological model is a natural and, perhaps, the only feasible way. For simulation of single-phase flows, the most important parameter is the absolute permeability, for which up-scaling methods are well-established. However, simulation of multiphase flows involves, in addition to computing the equivalent absolute permeabilities of the coarsened blocks, adjustments to the phase flow through the connected blocks of the coarsened grid. To accomplish this many methods have been proposed, including up-scaling techniques that employ pseudo-functions, the so-called dual-mesh approaches, and simpler non-uniform coarsening methods.

Short text classification task is a special kind of text classification task in that the text to be classified is generally short, typically generating a sparse text representation that lacks rich semantic information. Given this shortcoming, scholars worldwide have explored improved short text classification methods based on deep learning. However, existing methods cannot effectively use concept knowledge and long-distance word dependencies. Therefore, based on graph neural networks from the perspective of text composition, we propose concept and dependencies enhanced graph convolutional networks for short text classification. First, the co-occurrence relationship between words is obtained by sliding window, the inclusion relationship between documents and words is obtained by TF-IDF, long-distance word dependencies is obtained by Stanford CoreNLP, and the association relationship between concepts in the concept graph with entities in the text is obtained through Microsoft Concept Graph. Then, a text graph is constructed for an entire text corpus based on the four relationships. Finally, the text graph is input into graph convolutional neural networks, and the category of each document node is predicted after two layers of convolution. Experimental results demonstrate that our proposed method overall best on multiple classical English text classification datasets.

With the development of precision sensing instruments and data storage devices, the fusion of multi-sensor data in gearbox fault diagnosis has attracted much attention. However, existing methods have difficulty in capturing the local temporal dependencies of multi-sensor monitoring information, and the inescapable noise severely decreases the accuracy of multi-sensor information fusion diagnosis. To address these issues, this paper proposes a fault diagnosis method based on dynamic graph convolutional neural networks and hard threshold denoising. Firstly, considering that the relationships between monitoring data from different sensors change over time, a dynamic graph structure is adopted to model the temporal dependencies of multi-sensor data, and, further, a graph convolutional neural network is constructed to achieve the interaction and feature extraction of temporal information from multi-sensor data. Secondly, to avoid the influence of noise in practical engineering, a hard threshold denoising strategy is designed, and a learnable hard threshold denoising layer is embedded into the graph neural network. Experimental fault datasets from two typical gearbox fault test benches under environmental noise are used to verify the effectiveness of the proposed method in gearbox fault diagnosis. The experimental results show that the proposed DDGCN method achieves an average diagnostic accuracy of up to 99.7% under different levels of environmental noise, demonstrating good noise resistance.

A multi-agent artificial intelligence (AI) model is developed to automate the discovery of new metallic alloys, integrating multimodal data and external knowledge, including insights from physics via atomistic simulations. The system consists of (a) large language models (LLMs) for tasks such as reasoning and planning, (b) AI agents with distinct roles collaborating dynamically, and (c) a newly developed graph neural network (GNN) model for rapid retrieval of physical properties. We chose the ternary NbMoTa body-centered-cubic alloy as our model system and developed the GNN to predict two fundamental materials properties: the Peierls barrier and the solute/screw dislocation interaction energy. Our GNN model efficiently predicts these properties, reducing reliance on costly brute-force calculations and alleviating the computational demands on the multi-agent system. By combining the predictive capabilities of GNNs with the collaborative intelligence of LLM-driven reasoning agents, the system autonomously explores vast alloy design spaces, identifies trends in atomic-scale properties, and predicts macroscale mechanical strength, as demonstrated by several computational experiments. This synergistic approach accelerates the discovery of advanced alloys and holds promise for broader applications in other complex systems, marking a step forward in automated materials discovery and design. Impact statement Traditional deep learning models, such as graph neural networks and convolutional neural networks, operate within the confines of their training data sets, making single-step inferences for regression or classification. Our work introduces a multi-agent strategy that transcends these limitations by integrating deep learning with reasoning and decision-making capabilities. This intelligent system actively interprets results, determines subsequent actions, and iteratively refines predictions, accelerating the materials design process. We demonstrate its effectiveness in exploring the vast compositional space of a ternary alloy, where the model dynamically solicits data, analyzes trends, generates visualizations, and derives insights into materials behavior. By enabling accurate predictions of key alloy characteristics, our approach advances the discovery of novel metallic systems and underscores the critical role of solid-solution alloying. More broadly, it represents a major step toward integrating artificial intelligence with scientific reasoning, moving closer to artificial general intelligence in engineering. This paradigm shift has profound implications for materials science, enabling more efficient, autonomous, and intelligent exploration of complex materials spaces. Graphical Abstract

The center of human activity and a key point of contact for social interaction between city people and the built environment is the street. With more street-level photos becoming available, urban landscape studies have more chances to examine and evaluate streetscapes closely and from various angles. This manuscript present the graph neural networks (GNN) optimized with gazelle optimization algorithm (GOA) for urban plants cape design based on large-scale street view (UPLS-GNN-GOA). Initially data is taken from ADE20K dataset. Afterward the data is fed to General Robust Subband Adaptive Filtering (GRSAF) based pre-processing process. The pre-processing output is given to Discrete Fractional Fourier Transforms (DFrFT) to extract the street greenery and openness of street canyons. After that, the extracted features are fed to Graph Neural Networks (GNN) to classify the public health, urban microclimate, human perception. The weight parameters of the GNN are optimized using Gazelle Optimization Algorithm (GOA).The UPLS-GNN-GOA method is implemented in Python, and its efficiency is determined using a series of performance evaluation metrics analysis, including accuracy, precision, recall, f1-score, and sensitivity. The proposed UPLS-GNN-GOA method shows the highest accuracy of 98%, precision of 98%, and F1-score of 97% while comparing other existing methods such as urban plantscape design based on convolutional neural network (UPLS-CNN), urban plantscape design based on deep convolutional neural network (UPLS-DCNN) and urban plantscape design based on deep learning and convolutional neural network (UPLS-DL-CNN) respectively.