Enhancing ADMET property predictions using cross-aligned multimodal attention mechanisms.

AI-Driven molecule generation and bioactivity prediction: A multi-model approach combining VAE, graph and language-based neural networks.

The n-octanol/buffer solution distribution coefficient at pH = 7.4 (log D7.4) is an indicator of lipophilicity, and it influences a wide variety of absorption, distribution, metabolism, excretion, and toxicity (ADMET) properties and druggability of compounds. In log D7.4 prediction, graph neural networks (GNNs) can uncover subtle structure-property relationships (SPRs) by automatically extracting features from molecular graphs that facilitate the learning of SPRs, but their performances are often limited by the small size of available datasets. Herein, we present a transfer learning strategy called pretraining on computational data and then fine-tuning on experimental data (PCFE) to fully exploit the predictive potential of GNNs. PCFE works by pretraining a GNN model on 1.71 million computational log D data (low-fidelity data) and then fine-tuning it on 19,155 experimental log D7.4 data (high-fidelity data). The experiments for three GNN architectures (graph convolutional network (GCN), graph attention network (GAT), and Attentive FP) demonstrated the effectiveness of PCFE in improving GNNs for log D7.4 predictions. Moreover, the optimal PCFE-trained GNN model (cx-Attentive FP, Rtest2 = 0.909) outperformed four excellent descriptor-based models (random forest (RF), gradient boosting (GB), support vector machine (SVM), and extreme gradient boosting (XGBoost)). The robustness of the cx-Attentive FP model was also confirmed by evaluating the models with different training data sizes and dataset splitting strategies. Therefore, we developed a webserver and defined the applicability domain for this model. The webserver (http://tools.scbdd.com/chemlogd/) provides free log D7.4 prediction services. In addition, the important descriptors for log D7.4 were detected by the Shapley additive explanations (SHAP) method, and the most relevant substructures of log D7.4 were identified by the attention mechanism. Finally, the matched molecular pair analysis (MMPA) was performed to summarize the contributions of common chemical substituents to log D7.4, including a variety of hydrocarbon groups, halogen groups, heteroatoms, and polar groups. In conclusion, we believe that the cx-Attentive FP model can serve as a reliable tool to predict log D7.4 and hope that pretraining on low-fidelity data can help GNNs make accurate predictions of other endpoints in drug discovery.

MIGP: Metapath Integrated Graph Prompt Neural Network

Entity matching (EM) aims to identify equivalent entities across different data sources. Current EM assumes that these data are either homogeneous with aligned schema or heterogeneous but can be transformed into a unified modality. There is an urgent need to consider the entities with different modalities to support practical application scenarios over data lakes such as multi-modal data integration and recommendation system. It is impractical to unify their data modalities. To support EM on heterogeneous entity with different data formats and modalities, we propose cross-modal entity matching in this paper. Inspired by the promising performance achieved by recent pre-trained models, we perform cross-modal entity matching by prompt-tuning pre-trained multi-modal large models (MMLMs) in an unsupervised manner. However, the prompt-tuning faces three challenging issues: (i) objective gap between pre-training and tuning of MMLMs; (ii) data modality gap between the inputs of MMLMs and our matching task; (iii) prompt efficiency on large data. Therefore, we firstly propose a novel EM framework (namely, CrossEM) that addresses cross-modal EM as a matching probability problem with specific prompt-tuning. Secondly, two alternative prompt generation methods are designed to extract structural knowledge from heterogeneous data to overcome the data modality gap with pre-trained models. Thirdly, we present an improved matching framework (namely, CrossEM+) to boost the prompt efficiency on large heterogeneous data. Experimental evaluations verify that our methods significantly outperform the state-of-the-art approaches on three benchmarks. Furthermore, our case study highlights the considerable potential of cross-modal EM in improving the performance of downstream tasks, thereby benefitting a wider range of research areas.

Node representation learning on attributed graphs-whose nodes are associated with rich attributes (e.g., texts and protein sequences)-plays a crucial role in many important downstream tasks. To encode the attributes and graph structures simultaneously, recent studies integrate pre-trained models with graph neural networks (GNNs), where pre-trained models serve as node encoders (NEs) to encode the attributes. As jointly training large NEs and GNNs on large-scale graphs suffers from severe scalability issues, many methods propose to train NEs and GNNs separately. Consequently, they do not take feature convolutions in GNNs into consideration in the training phase of NEs, leading to a significant learning bias relative to the joint training. To address this challenge, we propose an efficient label regularization technique, namely Label Deconvolution (LD), to alleviate the learning bias by a novel and highly scalable approximation to the inverse mapping of GNNs. The inverse mapping leads to an objective function that is equivalent to that by the joint training, while it can effectively incorporate GNNs in the training phase of NEs against the learning bias. More importantly, we show that LD converges to the optimal objective function values by the joint training under mild assumptions. Experiments demonstrate LD significantly outperforms state-of-the-art methods on Open Graph Benchmark datasets.

The table base numerical question-answering task requires a mechanism to understand the relation between table content and numbers (present in table and question). It also needs an efficient method to address complex reasoning on table context. Most of the existing approaches in the natural language processing technology address the context-based questions on the table but fail to address the numerical reasoning part. They are also built on a large search database, which makes it challenging to use them in multiple domains. These approaches use pre-trained models like BERT to perform context encoding of a complete table. Hence these models fail when a large table is provided as input, as full table encoding is a very resource and time-consuming task. In this paper, a framework is proposed to answer questions on the table with numerical reasoning. This framework uses a context-snapshot mechanism to filter irrelevant table rows before tokenizing the table content. The filtered context and tokenized question are converted into vector representation using a pre-trained BERT model. This proposed model finds the correlation between the tokenized context-snapshot and numbers in question using graph neural networks. Further, it uses a feed-forward neural network to perform the numerical operation to compute the answer. The model is trained and evaluated on WikiTableQuestions datasets, shows a promising result.

Identifying pathogenic RNA methylation sites with a reasonable biological explanation has important implications for the treatment of diseases. Due to the limitations of in vitro experiments in identifying pathogenic RNA methylation sites, there is a growing need for computational workflows to enable accurate inference. Here, motivated by this profound meaning, we developed PathoRM, a biologically informed deep learning model, to infer associations between RNA methylation sites and diseases. PathoRM could provide convincing pathogenic RNA methylation sites and unravel the enigma of pathology in the epi-transcriptomic layer. PathoRM fuses RNA methylation host sequences and pathogenic descriptions as inputs, and subsequently employs large language models, multi-view learning algorithm, graph neural networks, an adversarial training approach, and “guilty-by-association”-derived negative sampling approach. PathoRM distils the semantically enriched feature embeddings, leading to more accurate and robust prediction performance across the metrics and datasets. Notably, incorporated with attention mechanism, PathoRM bestows itself biological interpretability through illuminating the dark matters in the host sequences of RNA methylation sites. This work is expected to assist in the discovery of pathogenic RNA methylation sites and conserved motifs, contributing to the advancement of genome research. Codes and pre-trained model are accessible at https://github.com/jianiM/PathoRM.

Graph neural networks (GNNs) have made rapid developments in the recent years. Due to their great ability in modeling graph-structured data, GNNs are vastly used in various applications, including high-stakes scenarios such as financial analysis, traffic predictions, and drug discovery. Despite their great potential in benefiting humans in the real world, recent study shows that GNNs can leak private information, are vulnerable to adversarial attacks, can inherit and magnify societal bias from training data and lack interpretability, which have risk of causing unintentional harm to the users and society. For example, existing works demonstrate that attackers can fool the GNNs to give the outcome they desire with unnoticeable perturbation on training graph. GNNs trained on social networks may embed the discrimination in their decision process, strengthening the undesirable societal bias. Consequently, trust-worthy GNNs in various aspects are emerging to prevent the harm from GNN models and increase the users’ trust in GNNs. In this paper, we give a comprehensive survey of GNNs in the computational aspects of privacy, robustness, fairness, and explainability. For each aspect, we give the taxonomy of the related methods and formulate the general frameworks for the multiple categories of trustworthy GNNs. We also discuss the future research directions of each aspect and connections between these aspects to help achieve trustworthiness.

The integration of artificial intelligence (AI) with bioinformatics has initiated a transformative shift in drug discovery, redefining how pharmaceutical research and development are conducted. This review examines both the current state and emerging prospects of AI-driven strategies across the drug discovery pipeline, from target identification and molecular design to clinical applications. Advances in machine learning, deep learning, graph neural networks, transformers, foundation models, and quantum computing have shown remarkable potential in overcoming long-standing bottlenecks of conventional drug development, which on average requires 12.5 years and over $2 billion to bring a single drug to market. The AI in pharmaceuticals market, valued at $1.8 billion in 2023, is projected to reach $13.1 billion by 2030, reflecting a compound annual growth rate of 18.8%. Landmark breakthroughs such as AlphaFold, which has generated over 200 million predicted protein structures and more than 20,000 citations, have established new paradigms in structural biology and drug design, while emerging tools like RFdiffusion and AlphaFold3 further extend capabilities into de novo protein design and multi-omics integration. AI-enabled workflows have demonstrated the ability to compress discovery timelines from five years to as little as 12–18 months and reduce costs by up to 40%. Despite these advances, significant challenges remain, including issues of data quality and bias, model transparency and interpretability, computational resource demands, and ethical concerns regarding privacy and algorithmic fairness. This review synthesizes findings from recent studies to provide a roadmap for the responsible and reproducible integration of AI into bioinformatics and drug discovery, while outlining critical gaps and future research priorities essential for clinical translation.

Cold-start problem is a fundamental challenge for recommendation tasks. Despite the recent advances on Graph Neural Networks (GNNs) incorporate the high-order collaborative signal to alleviate the problem, the embeddings of the cold-start users and items aren't explicitly optimized, and the cold-start neighbors are not dealt with during the graph convolution in GNNs. This paper proposes to pre-train a GNN model before applying it for recommendation. Unlike the goal of recommendation, the pre-training GNN simulates the cold-start scenarios from the users/items with sufficient interactions and takes the embedding reconstruction as the pretext task, such that it can directly improve the embedding quality and can be easily adapted to the new cold-start users/items. To further reduce the impact from the cold-start neighbors, we incorporate a self-attention-based meta aggregator to enhance the aggregation ability of each graph convolution step, and an adaptive neighbor sampler to select the effective neighbors according to the feedbacks from the pre-training GNN model. Experiments on three public recommendation datasets show the superiority of our pre-training GNN model against the original GNN models on user/item embedding inference and the recommendation task.

Past events have revealed that widespread blackouts are mostly a result of cascading failures in the power grid. Understanding the underlining mechanisms of cascading failures can help in developing strategies to minimize the risk of such events. Moreover, a real-time detection of precursors to cascading failures will help operators take measures to prevent their propagation. Currently, the well-established probabilistic and physics-based models of cascading failures offer low computational efficiency, hindering them to be used only as offline tools. In this work, we develop a data-driven methodology for online estimation of the risk of cascading failures. We utilize a physics-based cascading failure model to generate a cascading failure dataset considering different operating conditions and failure scenarios, thus obtaining a sample space covering a large set of power grid states that are labeled as safe or unsafe. We use the synthetic data to train deep learning architectures, namely Feed-forward Neural Networks (FNN) and Graph Neural Networks (GNN). With the development of GNNs, improved performance is achieved with graph-structured data, and GNNs can generalize to graphs of diverse sizes. A comparison between FNN and GNN is made and the GNNs inductive capability is demonstrated via test grids. Furthermore, we apply transfer learning to improve the performance of a pre-trained GNN model on power grids not seen in the training process. The GNN model shows accuracy and balanced accuracy above 96% on selected test datasets not used in the training. Conversely, the FNN shows accuracy above 85% and balanced accuracy above 81% on test datasets unseen during training. Overall, the GNN model is successful in determining, if one or several simultaneous outages result in a critical grid state, under specific grid operating conditions.

SGNN-T: Space graph neural network coupled transformer for molecular property prediction

Despite the demonstrated success of Pre-trained Language Models (PLM) in enhancing various recommendation tasks, their performance in friend recommendation systems, which are heavily influenced by complex social network dynamics, remains unproven. On the other hand, for social networks like Facebook, graph-based machine learning approaches in friend suggestion, like Graph Neural Networks (GNNs), struggle with the computational demands posed by social graphs. In is paper we investigate the application of PLM and GNN in the context of PYMK features within Facebook's social graph. We propose a representation learning scheme that captures both the local structure and semantic content within a second-degree connection space. Our approach leverages pre-trained transformer models with a feature aggregation scheme that enables efficient node representation learning in large social networks. We detail our model's architecture and discuss our training methods, accompanied by detailed experiments. Our model has improved the amount of friending on Facebook, facilitating new connections for users each day in online experiment.

This article introduces the Missingness-aware Self-Supervised Graph Neural Network (MissGNN), a novel model for tabular data predictions in scenarios with missing values. Addressing a significant challenge in fields like finance, healthcare, and environmental studies, MissGNN innovatively integrates feature imputation and label prediction within a unified framework. Utilizing a Tabular Bipartite Graph, it distinctively represents data points and features, enabling intricate modeling of their relationships. This approach is pivotal in contexts where missing data can significantly skew model training and predictions. MissGNN employs a Graph Neural Network to generate embeddings, capturing interactions between data points and features for accurate missing value imputation and label prediction. It features a dual-focus mechanism, comprising edge-wise feature imputation and node-wise label prediction. A pre-training phase for feature imputation enhances the model’s predictive ability for both features and labels. Additionally, MissGNN incorporates a multi-view learning strategy through feature subsetting, further enriching its representation learning. Experiments on 14 tabular datasets demonstrate MissGNN’s superiority over existing models in managing missing values across both regression and classification tasks. The results underscore its robustness and practical applicability, establishing it as a versatile tool for tackling tabular data prediction challenges with missing values. MissGNN’s contributions significantly redefine the approach to missing value imputation, aligning feature imputation and label prediction in a cohesive manner and underscoring the potential of Graph Neural Networks in processing tabular data.

To enhance security assessment and intelligence in penetration testing, this study employs a Graph Neural Network (GNN) to model network states in complex environments and dynamic decision-making scenarios. Integrating an improved Deep Q-Network (DQN), the intelligent agent’s threat path discovery is optimized and evaluated. Using the NASim platform to simulate network environments, multi-layer graph convolutional networks extract features, while pre-trained models and residual connections enhance performance. The model’s effectiveness is assessed based on cumulative reward values and operation steps, testing action space optimization and convergence. Experimental results show that the improved DQN significantly boosts efficiency, reducing steps per round from 35.5 to 12.75 after 800 training rounds. Decoupling techniques shrink the action space to 12.6% of its original size. In vulnerability detection, the method achieves 68.13% accuracy, a 6.05% improvement over conventional pre-trained models, and outperforms others in F1 score and recall. This study enables intelligent penetration detection and assessment by constructing a comprehensive network behavior graph, allowing the system to learn and recognize complex attack patterns effectively.