Identifying disease-related microbes based on multi-scale variational graph autoencoder embedding Wasserstein distance

In the ever-evolving landscape of data analysis, the need to efficiently and accurately interpret multimodal time series data has become paramount. Traditional methods often fall short in addressing the complex dependencies and dynamics inherent in such data, limiting their effectiveness in real-world applications. This work introduces a comprehensive approach that leverages Graph Attention Networks (GATs), Variational Graph Autoencoders (VGAEs), transfer learning with pretrained transformers, and Bayesian state-space models to overcome these limitations. GATs are selected for their ability to dynamically focus on relevant modalities through attention mechanisms, thereby capturing the intricate relationships between different data modalities. This method significantly enhances the model's ability to integrate multimodal information, leading to notable improvements in classification, prediction, and anomaly detection tasks. VGAEs are utilized to learn latent representations within a graph-based framework, promoting unsupervised learning while unveiling the underlying data structure. The resultant embeddings are pivotal for downstream tasks like clustering and visualization, encapsulating the interactions within multimodal time series data effectively. Furthermore, this work incorporates transfer learning with pretrained transformers to harness extensive knowledge from large datasets, adapting it to multimodal time series analysis. This strategy excels in capturing long-range dependencies, thereby augmenting generalization and performance in data-scarce scenarios. Bayesian state-space models are employed to elucidate the temporal dynamics and uncertainties of time series data, offering a robust framework for probabilistic inference and enhancing the interpretability and reliability of forecasting and anomaly detection. The efficacy of the proposed model is rigorously evaluated using diverse datasets, including the Yahoo! Stock Dataset, Forest Cover Dataset, and an empirical collection of 100k time series data samples. The results demonstrate a significant leap in performance metrics, including a 9.5% increase in precision, 8.5% boost in accuracy, 8.3% rise in recall, 10.4% reduction in delay, 9.4% enhancement in AUC, and a 5.9% improvement in specificity, alongside superior pre-emption capabilities compared to existing methods. This work not only addresses the pressing need for advanced multimodal time series analysis techniques but also sets a new benchmark for efficiency and accuracy. The integration of GATs, VGAEs, transfer learning with pretrained transformers, and Bayesian state-space models presents a formidable approach that significantly advances the field, offering profound impacts on a wide array of applications.

One of the main challenges that the industry faces when dealing with massive data for failure diagnosis is high dimensionality of such data. This can be tackled by dimensionality reduction method such as principal components analysis, which usually results in an improved fault diagnosis. Other available techniques include auto-encoders and its variants denoising auto-encoders and sparse auto-encoders. Most recently, variational auto-encoders are one of the most promising techniques for unsupervised learning with successful applications in image processing and speech recognition. Differently from other auto-encoder methods, variational auto-encoders use variational inference to generate a latent representation of the data and impose a distribution over the latent variables and the data itself. In this article, we propose a fully unsupervised deep variational auto-encoder-based approach for dimensionality reduction in fault diagnosis and explore the variational auto-encoder capabilities for such a task. This is achieved by comparing the latent representations provided by variational auto-encoders to the ones from principal components analysis as well as when no reduction is performed in ball bearings’ fault classification using vibration signals. To tackle massive sensor data, we propose different architectures for the variational auto-encoder’s encoder and decoder that are based on deep neural networks and deep convolutional neural networks. Experiments are also carried out by varying the data preprocessing methods for generating spectrograms and hand-engineering features as well as the use of raw vibration data, the architecture of the neural networks fault classifiers operating on the latent representations from variational auto-encoder and principal components analysis, the degree of data dimension reduction, and the size of the available labeled data used for training the fault classifiers. The results show that variational auto-encoders are a competent and promising tool for dimensionality reduction for use in fault diagnosis and worth further exploring their capabilities beyond vibration signals of ball bearing elements.

In this paper, we introduce Gene Knockout Inference (GenKI), a virtual knockout (KO) tool for gene function prediction using single-cell RNA sequencing (scRNA-seq) data in the absence of KO samples when only wild-type (WT) samples are available. Without using any information from real KO samples, GenKI is designed to capture shifting patterns in gene regulation caused by the KO perturbation in an unsupervised manner and provide a robust and scalable framework for gene function studies. To achieve this goal, GenKI adapts a variational graph autoencoder (VGAE) model to learn latent representations of genes and interactions between genes from the input WT scRNA-seq data and a derived single-cell gene regulatory network (scGRN). The virtual KO data is then generated by computationally removing all edges of the KO gene—the gene to be knocked out for functional study—from the scGRN. The differences between WT and virtual KO data are discerned by using their corresponding latent parameters derived from the trained VGAE model. Our simulations show that GenKI accurately approximates the perturbation profiles upon gene KO and outperforms the state-of-the-art under a series of evaluation conditions. Using publicly available scRNA-seq data sets, we demonstrate that GenKI recapitulates discoveries of real-animal KO experiments and accurately predicts cell type-specific functions of KO genes. Thus, GenKI provides an in-silico alternative to KO experiments that may partially replace the need for genetically modified animals or other genetically perturbed systems.

Posterior collapse is a pervasive issue in Variational Autoencoders (VAEs) that leads to the learned latent representations becoming trivial and devoid of meaningful information. To address this problem, this paper presents a novel β-VAE approach, which incorporates a hyperparameter β to strike an optimal balance between the reconstruction loss and the KL divergence loss. By conducting a comprehensive series of experiments and drawing comparisons with existing methods, robust evidence is provided that the proposed β-VAE method effectively mitigates posterior collapse and yields more expressive and informative latent representations.The experimental setup involves various architectures and datasets to demonstrate the versatility and efficacy of the β-VAE approach in diverse settings. Additionally, ablation studies are performed to investigate the impact of different β values on the model's performance, elucidating the role of this hyperparameter in controlling the trade-off between reconstruction quality and latent representation expressiveness. Furthermore, the disentanglement properties of the learned latent space are analyzed, which is a crucial aspect of VAEs, especially when applied to complex, real-world data.In-depth analysis of the results offers valuable insights into the underlying mechanisms of β-VAE, thereby contributing to a more profound understanding of VAEs and their inherent limitations. The findings not only establish the effectiveness of the β-VAE method in preventing posterior collapse but also pave the way for future research on improving VAEs' performance in various applications. Potential future work could explore alternative techniques for balancing the competing objectives of reconstruction and latent representation learning or delve into the theoretical properties of β-VAE, providing a more rigorous foundation for this approach.

Resting state Functional Magnetic Resonance Imaging fMRI (rs-fMRI) has been used extensively to study brain function in psychiatric disorders, yielding insights into brain organization. However, the high dimensionality of the rs-fMRI data presents significant challenges for data analysis. Variational autoencoders (VAEs), a type of neural network, have been instrumental in extracting low-dimensional latent representations of resting state functional connectivity (rsFC) patterns, thereby addressing the complex nonlinear structure of rs-fMRI data. Despite these advances, interpreting these latent representations remains a challenge. This paper aims to address this gap by developing explainable VAE models and testing their utility using rs-fMRI data in autism spectrum disorder (ASD). One-thousand one hundred and fifty participants (601 healthy controls [HC] and 549 patients with ASD) were included in the analysis. RsFC correlation matrices were extracted from the preprocessed rs-fMRI data using the Power atlas, which includes 264 regions of interest (ROIs). Then VAEs were trained in an unsupervised manner. Lastly, we introduce our latent contribution scores to explain the relationship between estimated representations and the original rs-fMRI brain measures. We quantified the latent contribution scores for both the ASD and HC groups at the network level. We found that both ASD and HC groups share the top network connectivitives contributing to all estimated latent components. For example, latent 0 was driven by rsFC within ventral attention network (VAN) in both the ASD and HC. However, we found significant differences in the latent contribution scores between the ASD and HC groups within the VAN for latent 0 and the sensory/somatomotor network for latent 2. This study introduced latent contribution scores to interpret nonlinear patterns identified by VAEs. These scores effectively capture changes in each observed rsFC feature as the estimated latent representation changes, enabling an explainable deep learning model that better understands the underlying neural mechanisms of ASD.

Attributed graph clustering aims to partition nodes of a graph structure into different groups. Recent works usually use variational graph autoencoder (VGAE) to make the node representations obey a specific distribution. Although they have shown promising results, how to introduce supervised information to guide the representation learning of graph nodes and improve clustering performance is still an open problem. In this article, we propose a Collaborative Decision-Reinforced Self-Supervision (CDRS) method to solve the problem, in which a pseudo node classification task collaborates with the clustering task to enhance the representation learning of graph nodes. First, a transformation module is used to enable end-to-end training of existing methods based on VGAE. Second, the pseudo node classification task is introduced into the network through multitask learning to make classification decisions for graph nodes. The graph nodes that have consistent decisions on clustering and pseudo node classification are added to a pseudo-label set, which can provide fruitful self-supervision for subsequent training. This pseudo-label set is gradually augmented during training, thus reinforcing the generalization capability of the network. Finally, we investigate different sorting strategies to further improve the quality of the pseudo-label set. Extensive experiments on multiple datasets show that the proposed method achieves outstanding performance compared with state-of-the-art methods. Our code is available at https://github.com/Jillian555/TNNLS_CDRS.

Investigating potential interactions between drugs and herbal medicines helps optimize combined treatment strategies and supports personalized and precision medicine. Deep learning-based methods have been successful in predicting drug-related interactions. However, these methods face challenges such as low data quality and uneven distribution. Large language models (LLMs) effectively address these challenges through their extensive knowledge bases. Motivated by this, we integrate LLMs, one-hot encoding, and variational graph autoencoders (VGAEs) to propose a herbal medicine-drug interaction (HDI) prediction model. First, LLMs are employed to extract features from drug SMILES, generating high-quality molecular representations. Second, one-hot encoding is applied to herbal medicines with multiple natural products to construct feature vectors and improve model interpretability. Finally, VGAEs are utilized to reconstruct herbal medicine-drug graphs and predict unknown HDIs. Additionally, we differentiate between herbal medicine-drug similarity and the degree of individual drug or herbal medicine nodes to mitigate the dominance of high-degree nodes in VGAE message flow. Multiple experiments were conducted to validate the significance of the proposed model and its key components. This method shows great potential for applications in traditional Chinese medicine formulation optimization, new drug development, and precision medicine.

Deep node clustering based on mutual information maximization

Background Pulmonary hypertension (PH) in newborns is a severe condition characterized by elevated pulmonary artery pressure, leading to right ventricular strain and impaired oxygenation, and significant morbidity and mortality. Right heart catheterization (RHC) is the gold standard for PH diagnosis but is invasive, costly, and associated with procedural risks. Instead, transthoracic echocardiography (ECHO) is widely used for non-invasive PH assessment, although its interpretation is highly operator-dependent. While automated machine learning methods have been proposed for PH detection, existing approaches largely focus on adults and often rely on single-view echocardiographic frames, limiting their generalizability to neonatal populations. Objective We aim to leverage a multi-view variational autoencoder (VAE) framework to predict PH severity in newborns using echocardiographic videos. We hypothesize that learning latent representations across multiple echocardiographic views improves classification performance and generalization compared to traditional models. Methods We analyzed a dataset of 936 transthoracic ECHO videos from 192 newborns (56±160 days old, 2.9±1.5Kg), with a held-out test set of 375 videos from 78 newborns (34±42 days old, 2.1±1.2Kg). Standard ECHO views included PLAX, A4C, and three parasternal short-axis views (PSAX-P, PSAX-S, PSAX-A). PH severity was categorized as none, mild, or moderate-to-severe based on septal flattening patterns. A multi-view VAE framework with a variational mixture-of-experts prior was trained to extract latent representations from ECHO videos, leveraging shared information across different views. Classification performance was compared against single-view and supervised learning from Ragnarsdottir et al. 24’ [1]. The study and method pipeline is illustrated in figure 1. Results Multi-view learning significantly improved PH classification accuracy compared to single-view approaches. Our VAE model achieved an AUROC of 0.83 ± 0.02 and a balanced accuracy of 0.74 ± 0.02 for binary PH detection, outperforming single-view VAEs and traditional supervised learning models. Importantly, our VAE exhibited greater robustness to unseen test data, demonstrating improved generalization across newborn populations. The model effectively distinguished PH severity levels, with the highest performance gains observed in the multi-class classification task (AUROC = 0.74 ± 0.01) –comprehensive results in figure 2. Conclusion Automated PH assessment using echocardiography is essential for early detection and intervention, particularly in neonates where timely diagnosis can impact survival. Our multi-view VAE-based framework enhances PH classification, reduces operator dependency, and improves diagnostic consistency. These improvements could enable more standardized and accessible screening, particularly in resource-limited settings where expert evaluation may not always be available.Study pipeline and method. Summary results (baseline [1]).

This paper proposes an effective IoT device classification method combining Variational Autoencoder (VAE) and an optimized XGBoost model to address challenges like large-scale data processing and sample imbalance. First, feature selection is improved by integrating mutual information analysis and domain knowledge. VAE is used for feature extraction, capturing latent representations from network traffic data. The XGBoost classifier is optimized using techniques like gradient clipping and model pruning to improve training efficiency and reduce computational overhead. This method boosts classification accuracy, handling data imbalance and noise effectively, with experimental results showing 90% accuracy and F1 score. The proposed approach outperforms traditional algorithms in large-scale IoT device classification tasks.

BackgroundProtein-protein interactions (PPIs) play a critical role in essential biological processes such as signal transduction, enzyme activity regulation, cytoskeletal structure, immune responses, and gene regulation. However, current methods mainly focus on extracting features from protein sequences and using graph neural network (GNN) to acquire interaction information from the PPI network graph. This limits the model’s ability to learn richer and more effective interaction information, thereby affecting prediction performance.ResultsIn this study, we propose a novel deep learning method, MESM, for effectively predicting PPI. The datasets used for the PPI prediction task were primarily constructed from the STRING database, including two Homo sapiens PPI datasets, SHS27k and SHS148k, and two Saccharomyces cerevisiae PPI datasets, SYS30k and SYS60k. MESM consists of three key modules, as follows: First, MESM extracts multimodal representations from protein sequence information, protein structure information, and point cloud features through Sequence Variational Autoencoder (SVAE), Variational Graph Autoencoder (VGAE), and PointNet Autoencoder (PAE). Then, Fusion Autoencoder (FAE) is used to integrate these multimodal features, generating rich and balanced protein representations. Next, MESM leverages GraphGPS to learn structural information from the PPI network graph structure and combines Graph Attention Network (GAT) to further capture protein interaction information. Finally, MESM uses Graph Convolutional Network (GCN) and SubgraphGCN to extract global and local features from the perspective of the overall graph and subgraphs. Moreover, we build seven independent graphs from the overall PPI network graph to specifically learn the features of each PPI type, thereby enhancing the model’s learning ability for different types of interactions.ConclusionsCompared to the state-of-the-art methods, MESM achieved improvements of 8.77%, 4.98%, 7.48%, and 6.08% on SHS27k, SHS148k, SYS30k, and SYS60k, respectively. The experimental results demonstrate that MESM exhibits significant improvements in PPI prediction performance.Supplementary InformationThe online version contains supplementary material available at 10.1186/s12915-025-02356-y.

Purpose:There has been burgeoning interest in applying machine learning methods for predicting radiotherapy outcomes. However, the imbalanced ratio of a large number of variables to a limited sample size in radiation oncology constitutes a major challenge. Therefore, dimensionality reduction methods can be a key to success. The study investigates and contrasts the application of traditional machine learning methods and deep learning approaches for outcome modeling in radiotherapy. In particular, new joint architectures based on variational autoencoder (VAE) for dimensionality reduction are presented and their application is demonstrated for the prediction of lung radiation pneumonitis (RP) from a large-scale heterogeneous dataset.Methods:A large-scale heterogeneous dataset containing a pool of 230 variables including clinical factors (e.g., dose, KPS, stage) and biomarkers (e.g., single nucleotide polymorphisms (SNPs), cytokines and micro-RNAs) in a population of 106 non-small cell lung cancer (NSCLC) patients who received radiotherapy was used for modeling RP. 22 patients had grade 2 or higher RP. Four methods were investigated, including feature selection (case A) and feature extraction (case B) with traditional machine learning methods, a VAE-MLP joint architecture (case C) with deep learning and lastly, the combination of feature selection and joint-architecture (case D). For feature selection, Random forest (RF), Support Vector Machine (SVM) and multi-layer perceptron (MLP) were implemented to select relevant features. Specifically, each method was run for multiple times to rank features within several cross-validated (CV) resampled sets. A collection of ranking lists were then aggregated by top 5% and Kemeny graph methods to identify the final ranking for prediction. A synthetic minority oversampling technique (SMOTE) was applied to correct for class imbalance during this process. For deep learning, a VAE-MLP joint architecture where a VAE aimed for dimensionality reduction and an MLP aimed for classification was developed. In this architecture, reconstruction loss and prediction loss were combined into a single loss function to realize simultaneous training and weights were assigned to different classes to mitigate class imbalance. To evaluate the prediction performance and conduct comparisons, the area under receiver operating characteristic (ROC) curves (AUCs) were performed for nested CVs for both handcrafted feature selections and the deep learning approach. The significance of differences in AUCs was assessed using the DeLong test of U-statistics.Results:An MLP-based method using weight pruning (WP) feature selection yielded the best performance among the different hand-crafted feature selection methods (case A), reaching an AUC of 0.804 (95% CI: 0.761–0.823) with 29 top features. A VAE-MLP joint architecture (case C) achieved a comparable but slightly lower AUC of 0.781 (95% CI: 0.737–0.808) with the size of latent dimension being 2. The combination of handcrafted features (case A) and latent representation (case D) achieved a significant AUC improvement of 0.831 (95% CI: 0.805–0.863) with 22 features (p-value = 0.000642 compared with handcrafted features only (Case A) and p-value=0.000453 compared to VAE alone (Case C)) with an MLP classifier.Conclusion:The potential for combination of traditional machine learning methods and deep learning VAE techniques has been demonstrated for dealing with limited datasets in modeling radiotherapy toxicities. Specifically, latent variables from a VAE-MLP joint architecture are able to complement handcrafted features for the prediction of RP and improve prediction over either method alone.

Variational Autoencoders (VAEs) have seen widespread use in learned image compression. They are used to learn expressive latent representations on which downstream compression methods can operate with high efficiency. Recently proposed 'bits-back' methods can indirectly encode the latent representation of images with codelength close to the relative entropy between the latent posterior and the prior. However, due to the underlying algorithm, these methods can only be used for lossless compression, and they only achieve their nominal efficiency when compressing multiple images simultaneously; they are inefficient for compressing single images. As an alternative, we propose a novel method, Relative Entropy Coding (REC), that can directly encode the latent representation with codelength close to the relative entropy for single images, supported by our empirical results obtained on the Cifar10, ImageNet32 and Kodak datasets. Moreover, unlike previous bits-back methods, REC is immediately applicable to lossy compression, where it is competitive with the state-of-the-art on the Kodak dataset.

US medical researchers brace themselves for budget cuts

Examining Drug-Drug Interactions (DDIs) is a pivotal element in the process of drug development. DDIs occur when one drug's properties are affected by the inclusion of other drugs. Detecting favorable DDIs has the potential to pave the way for creating and advancing innovative medications applicable in practical settings. However, existing DDI prediction models continue to face challenges related to generalization in extreme cases, robust feature extraction, and real-life application possibilities. We aim to address these challenges by leveraging the effectiveness of context-aware deep graph learning by introducing a novel framework named CADGL. Based on a customized variational graph autoencoder (VGAE), we capture critical structural and physio-chemical information using two context pre-processors for feature extraction from two different perspectives- local neighborhood and molecular context, in a heterogeneous graphical structure. Our customized VGAE consists of a graph encoder, a latent information encoder, and an MLP decoder. CADGL surpasses other state-of-the-art DDI prediction models, excelling in predicting clinically valuable novel DDIs, supported by rigorous case studies. Our code is available at: https://github.com/azminewasi/CADGL.