Memory Graph Networks for Explainable Memory-grounded Question Answering

We present an approach to automatically synthesise recursive predicates in Separation Logic (SL) from concrete data structure instances using Inductive Logic Programming (ILP) techniques. The main challenges to make such synthesis effective are (1) making it work without negative examples that are required in ILP but are difficult to construct for heap-based structures in an automated fashion, and (2) to be capable of summarising not just the shape of a heap (e.g., it is a linked list), but also the properties of the data it stores (e.g., it is a sorted linked list). We tackle these challenges with a new predicate learning algorithm. The key contributions of our work are (a) the formulation of ILP-based learning only using positive examples and (b) an algorithm that synthesises property-rich SL predicates from concrete memory graphs based on the positive-only learning. We show that our framework can efficiently and correctly synthesise SL predicates for structures that were beyond the reach of the state-of-the-art tools, including those featuring non-trivial payload constraints (e.g., binary search trees) and nested recursion (e.g., n -ary trees). We further extend the usability of our approach by a memory graph generator that produces positive heap examples from programs. Finally, we show how our approach facilitates deductive verification and synthesis of correct-by-construction code.

This study examined the types of questions caregivers use and their outcomes when conversing with their spouse with Alzheimer's disease (AD). Of particular interest was caregivers' use of yes-no and open-ended questions and the demands they make on the memory of the person with AD. It was hypothesized that communication between caregivers and their spouses would be more successful when caregivers used yes-no rather than open-ended questions; however, it was also predicted that a more positive communication outcome would occur when caregivers used open-ended questions that requested information from semantic rather than episodic memory. Eighteen caregivers and their spouses diagnosed with AD were audiotaped while they conversed for approximately 10 min on a topic of their choosing. The conversations were transcribed and coded according to the occurrence of questions, the type of question (yes-no, choice, or open-ended), the type of memory required to respond to a question (semantic or episodic), and the outcome of a response to a question (communication breakdown). The results indicated that caregivers used yes-no and open-ended questions to a similar extent, whereas episodic questions were used almost twice as frequently as semantic questions. Communication was more successful when caregivers used yes-no compared with open-ended questions and when questions placed demands on semantic rather than episodic memory. The findings from this study suggest that caregivers can reduce communication problems by avoiding the use of questions that depend on episodic memory. In addition, while yes-no questions were associated with more favorable outcomes than open-ended questions, the latter do not need to be avoided if they refer to information that draws only on semantic memory.

Purpose: The purpose of this study was to investigate the effects of asking open-ended episodic memory questions versus open-ended semantic memory questions on the conversational discourse of individuals with Alzheimer's disease (AD). Methods: Four females diagnosed with probable AD participated in the study. A within-subjects experimental design was employed to assess the effects of the different question types on participants’ spoken language. Transcripts were analyzed using specific discourse measures used in previous research involving individuals with AD. Results: Participants in this study produced more meaningful and relevant statements, as measured by ratios of on-topic utterances, when responding to the semantic memory questions as compared to episodic memory questions. Participants made few negative comments overall; however, more negative self-evaluative statements were made in the episodic memory condition. When considered in conjunction with previous research, the results support the use of multiple question types in conversation with individuals with mild and moderate AD. However, communication partners should limit their use of open-ended questions that primarily tax episodic memory.

Recommending medication combinations for patients is an essential part of artificial intelligence in the healthcare field. Existing approaches improve the effect of recommendations by considering how to make full use of patients’ electronic health records or by introducing additional external knowledge, but there is still room for improving the fusion of heterogeneous and diverse knowledge and the effect between accuracy and drug-drug interaction (DDI) rate. To fill this gap, we propose the Feature Fusion and Bipartite Decision Networks (FFBDNet) to leverage external knowledge and improve accuracy and DDI rate. FFBDNet is equipped with a patient feature encoder which extract useful information from current and historical visits of patient to supplement the patient’s health status, a medication feature encoder which can easily fuse the heterogeneous and diverse external knowledge of medications as feature, and a bipartite decision module to give medication recommendation results. FFBDNet also has a greedy loss function to improve accuracy and DDI rate. We demonstrate the effectiveness of FFBDNet by comparing with several state-of-the-art methods on a benchmark dataset. FFBDNet outperformed all baselines in all effective measures, reduced relatively the DDI rate by 97.65% from existing EHR data, and also is shown to improve 1.02% on Jaccard similarity.

Frontiers Events is a rapidly growing calendar management system dedicated to the scheduling of academic events. This includes announcements and invitations, participant listings and search functionality, abstract handling and publication, related events and post-event exchanges. Whether an organizer or participant, make your event a Frontiers Event!

Multimodal aspect-based sentiment analysis (MABSA) is an essential task in the field of sentiment analysis, which still confronts several critical challenges. The first challenge is how to effectively capture key information within both image and text features to enhance the recognition and understanding of complex sentiment expressions. The second challenge is how to achieve cross-modal alignment of multi-granularity text features and image features. The third challenge is how to narrow the semantic gap between image modality and text modality through effective cross-modal feature fusion. To address these issues, a framework that leverages external knowledge and multi-granularity image and text features (EKMG) is proposed. Firstly, an external knowledge enhanced semantic extraction module is introduced to fuse external knowledge with image features and text features, thereby capturing the key information from texts and images. Secondly, we design a multi-granularity image-text contrastive learning module. This module initially introduces a graph attention network and a novel cross-modal fusion mechanism to align image features and text features at multiple granularities. Additionally, the module employs an image-text contrastive learning strategy to narrow the semantic gap between different modalities. Experimental results on two public benchmark datasets demonstrate that EKMG achieves significant performance improvements compared to state-of-the-art baseline models.

A system for efficient keyword search in graphs is demonstrated. The system has two components, a search through only the nodes containing the input keywords for a set of nodes that are close to each other and together cover the input keywords and an exploration for finding how these nodes are related to each other. The system generates all or top-k answers in polynomial delay. Answers are presented to the user according to a ranking criterion so that the answers with nodes closer to each other are presented before the ones with nodes farther away from each other. In addition, the set of answers produced by our system is duplication free. The system uses two methods for presenting the final answer to the user. The presentation methods reveal relationships among the nodes in an answer through a tree or a multi-center graph. We will show that each method has its own advantages and disadvantages. The system is demonstrated using two challenging datasets, very large DBLP and highly cyclic Mondial. Challenges and difficulties in implementing an efficient keyword search system are also demonstrated.

Bridging-Inferences and Enthymemes

Approximate K-Nearest Neighbor Search (AKNNS) has now become ubiquitous in modern applications, such as a fast search procedure with two-tower deep learning models. Graph-based methods for AKNNS in particular have received great attention due to their superior performance. These methods rely on greedy graph search to traverse the data points as embedding vectors in a database. Under this greedy search scheme, we make a key observation: many distance computations do not influence search updates so that these computations can be approximated without hurting performance. As a result, we propose FINGER, a fast inference method for efficient graph search in AKNNS. FINGER approximates the distance function by estimating angles between neighboring residual vectors. The approximated distance can be used to bypass unnecessary computations for faster searches. Empirically, when it comes to speeding up the inference of HNSW, which is one of the most popular graph-based AKNNS methods, FINGER significantly outperforms existing acceleration approaches and conventional libraries by 20 to 60 across different benchmark datasets.

Human actions can be realized by observing the trajectories of skeleton joints. In this paper, we propose an unsupervised episodic memory learning model for skeleton based action learning and recognition. The proposed model, Multi-channel Episodic Memory Adaptive Resonance Theory (McEM-ART), consists of three layers: short term memory, working memory and Episodic memory. The short term memory layer is formed by multiple ART networks to obtain sensory data and cluster them into neurons in working memory layer. Instead of obtaining the whole skeleton as an input, we divide the human skeleton into three parts, upper part body, main body and lower part body. Each of them is then feed to McEM-ART short term memory layer for learning. Episodic memory layer extracts novel events and encodes spatio-temporal connection between them as episodes by generating cognitive neurons incrementally for action recognition. Comparing with previous works, McEM-ART further integrates a novel memory anticipation functions for encoding crucial events and episodes and recalling them using partial and inexact cues. Experimental results demonstrate that McEM-ART is capable of clustering human skeleton data into event neurons, encoding sequence of activation events as episode neurons for action recalling and recognition.

ALSC (Aspect-level Sentiment Classification) is a fine-grained task in the field of NLP (Natural Language Processing) which aims to identify the sentiment toward a given aspect. In addition to exploiting the sentence semantics and syntax, current ALSC methods focus on introducing external knowledge as a supplementary to the sentence information. However, the integration of the three categories of information is still challenging. In this paper, a novel method is devised to effectively combine sufficient semantic and syntactic information as well as use of external knowledge. The proposed model contains a sentence encoder, a semantic learning module, a syntax learning module, a knowledge enhancement module, an information fusion module and a sentiment classifier. The semantic information and syntactic information are respectively extracted via a self-attention network and a graphical convolutional network. Specifically, the KGE (Knowledge Graph Embedding) is employed to enhance the feature representation of the aspect. Then, the attention-based gate mechanism is taken to fuse three types of information. We evaluated the proposed model on three benchmark datasets and the experimental results establish strong evidence of high accuracy.

A Heterogeneous Directed Graph Attention Network for inductive text classification using multilevel semantic embeddings

This study introduces GAT-CAMDA, a novel framework for the structural health monitoring (SHM) of composite materials under temperature-induced variability, leveraging the powerful feature extraction capabilities of Graph Attention Networks (GATs) and advanced domain adaptation (DA) techniques. By combining Maximum Mean Discrepancy (MMD) and Correlation Alignment (CORAL) losses with a domain-discriminative adversarial model, the framework achieves scalable alignment of feature distributions across temperature domains, ensuring robust damage detection. A simple yet at the same time efficient data augmentation process extrapolates damage behaviour across unmeasured temperature conditions, addressing the scarcity of damaged-state observations. Hyperparameter optimisation via Optuna not only identifies the optimal settings to enhance model performance, achieving a classification accuracy of 95.83% on a benchmark dataset, but also illustrates hyperparameter significance for explainability. Additionally, the GAT architecture’s attention demonstrates the importance of various sensors, enhancing transparency and reliability in damage detection. The dual use of Optuna serves to refine model accuracy and elucidate parameter impacts, while GAT-CAMDA represents a significant advancement in SHM, enabling precise, interpretable, and scalable diagnostics across complex operational environments.

Identifying the frequencies of the drug-side effects is a very important issue in pharmacological studies and drug risk-benefit. However, designing clinical trials to determine the frequencies is usually time consuming and expensive, and most existing methods can only predict the drug-side effect existence or associations, not their frequencies. Inspired by the recent progress of graph neural networks in the recommended system, we develop a novel prediction model for drug-side effect frequencies, using a graph attention network to integrate three different types of features, including the similarity information, known drug-side effect frequency information and word embeddings. In comparison, the few available studies focusing on frequency prediction use only the known drug-side effect frequency scores. One novel approach used in this work first decomposes the feature types in drug-side effect graph to extract different view representation vectors based on three different type features, and then recombines these latent view vectors automatically to obtain unified embeddings for prediction. The proposed method demonstrates high effectiveness in 10-fold cross-validation. The computational results show that the proposed method achieves the best performance in the benchmark dataset, outperforming the state-of-the-art matrix decomposition model. In addition, some ablation experiments and visual analyses are also supplied to illustrate the usefulness of our method for the prediction of the drug-side effect frequencies. The codes of MGPred are available at https://github.com/zhc940702/MGPred and https://zenodo.org/record/4449613.

This article presents MST, a communication-efficient message library for fast graph traversal on exascale clusters. The key idea is to follow the multi-level network topology to perform topology-aware message aggregation, where small messages are gathered and scattered at each level of domain. To facilitate message aggregation, we equip MST with flexible buffer management including active buffer switching and dynamic buffer expansion. We implement MST on the newest-generation Tianhe supercomputer and evaluated its performance using various traversal-centric algorithms on both synthetic trillion-scale graphs and real-world big graphs. The results show that MST-based graph traversal is orders of magnitude faster than that based on Active Messages Library (AML). For the Graph500-BFS benchmark, MST-based Tianhe (with 77.2 K nodes) outperforms the Fugaku supercomputer (with 148.5 K nodes) by 18.53%, while Fugaku is ranked No. 1 in the latest Graph500-BFS ranking (June 2023). MST also greatly improves graph processing performance on other commercial large-scale computing systems at the National Supercomputing Center in Changsha (NSCC) and WuzhenLight.