MOOC Learner Profiling Using Relation-Aware Heterogeneous Graph Neural Networks

A relation-aware heterogeneous graph convolutional network for relationship prediction

Many complex systems derived from nature and society consist of multiple types of entities and heterogeneous interactions, which can be effectively modeled as heterogeneous information network (HIN). Structural analysis of heterogeneous networks is of great significance by leveraging the rich semantic information of objects and links in the heterogeneous networks. And, clustering heterogeneous networks aims to group vertices into classes, which sheds light on revealing the structure–function relations of the underlying systems. The current algorithms independently perform the feature extraction and clustering, which are criticized for not fully characterizing the structure of clusters. In this study, we propose a learning model by joint <underline>G</underline>raph <underline>E</underline>mbedding and <underline>N</underline>onnegative <underline>M</underline>atrix <underline>F</underline>actorization (aka GEjNMF ), where feature extraction and clustering are simultaneously learned by exploiting the graph embedding and latent structure of networks. We formulate the objective function of GEjNMF and transform the heterogeneous network clustering problem into a constrained optimization problem, which is effectively solved by l 0 -norm optimization. The advantage of GEjNMF is that features are selected under the guidance of clustering, which improves the performance and saves the running time of algorithms at the same time. The experimental results on three benchmark heterogeneous networks demonstrate that GEjNMF achieves the best performance with the least running time compared with the best state-of-the-art methods. Furthermore, the proposed algorithm is robust across heterogeneous networks from various fields. The proposed model and method provide an effective alternative for heterogeneous network clustering.

To address the challenges faced by existing artificial intelligence methods in modeling complex heterogeneous biological networks, particularly their limitations in capturing collaborative relationships between nodes and in extracting high-order topological semantic features, we propose a novel drug repositioning prediction method based on dynamic representation learning on heterogeneous graphs. A heterogeneous biological graph that integrates drugs, diseases, and their interaction relationships was constructed, based on which a dynamic gated attention module was designed to extract discriminative topological features of drugs and diseases by incorporating a dynamic graph attention mechanism. A gated residual feature fusion mechanism was developed to precisely integrate structural and semantic information from multiple similarity networks to reduce feature redundancy and information loss, thereby enabling accurate prediction of drug-disease associations. Experiments and case studies conducted on multiple drug datasets related to complex diseases demonstrated that the proposed method outperformed existing mainstream models in drug repositioning prediction. The proposed method can effectively model complex associations in heterogeneous biological networks, enhance the accuracy of drug repositioning prediction, and provide important technical support for precision treatment of complex diseases and development of medical artificial intelligence.

This study aims to guide the management and service of highways towards a more efficient and intelligent direction, and also provides intelligent and green data support for achieving sustainable development goals. The forecasting of traffic flow at highway stations serves as the cornerstone for spatiotemporal analysis and is vital for effective highway management and control. Despite considerable advancements in data-driven traffic flow prediction, the majority of existing models fail to differentiate between directions. Specifically, entrance flow prediction has applications in dynamic route guidance, disseminating real-time traffic conditions, and offering optimal entrance selection suggestions. Meanwhile, exit flow prediction is instrumental for congestion and accident alerts, as well as for road network optimization decisions. In light of these needs, this study introduces an enhanced heterogeneous spatiotemporal graph network model tailored for predicting highway station traffic flow. To accurately capture the dynamic impact of upstream toll stations on the target station’s flow, we devise an influence probability matrix. This matrix, in conjunction with the covariance matrix across toll stations, updated graph structure data, and integrated external weather conditions, allows the attention mechanism to assign varied combination weights to the target toll station from temporal, spatial, and external standpoints, thereby augmenting prediction accuracy. We undertook a case study utilizing traffic flow data from the Chengdu-Chengyu station on the Sichuan Highway to gauge the efficacy of our proposed model. The experimental outcomes indicate that our model surpasses other baseline models in performance metrics. This study provides valuable insights for highway management and control, as well as for reducing traffic congestion. Furthermore, this research highlights the importance of using data-driven approaches to reduce carbon emissions associated with transportation, enhance resource allocation at toll plazas, and promote sustainable highway transportation systems.

Network embedding (graph embedding) has become the focus of studying graph structure in recent years. In addition to the research on homogeneous networks and heterogeneous networks, there are also some methods to attempt to solve the problem of dynamic network embedding. However, in dynamic networks, there is no research method specifically for heterogeneous networks. Therefore, this paper proposes a heterogeneous dynamic graph attention network (HDGAN), which attempts to use the attention mechanism to take the heterogeneity and dynamics of the network into account at the same time, so as to better learn network embedding. Our method is based on three levels of attention, namely structural-level attention, semantic-level attention and time-level attention. Structural-level attention pays attention to the network structure itself, and obtains the representation of structural-level nodes by learning the attention coefficients of neighbor nodes. Semantic-level attention integrates semantic information into the representation of nodes by learning the optimal weighted combination of different meta-paths. Time-level attention is based on the time decay effect, and the time feature is introduced into the node representation by neighborhood formation sequence. Through the above three levels of attention mechanism, the final network embedding can be obtained.Through experiments on two real-world heterogeneous dynamic networks, our models have the best results, proving the effectiveness of the HDGAN model.

Aspect-based sentiment analysis aims to predict sentiment polarities of given aspects in text. Most current approaches employ attention-based neural methods to capture semantic relationships between aspects and words in one sentence. However, these methods ignore the fact that sentences with the same aspect and sentiment polarity often share the structure and semantic information in a domain, which leads to lower model performance. To mitigate this problem, we propose a heterogeneous aspect graph neural network (HAGNN) to learn the structure and semantic knowledge from intersentence relationships. Our model is a heterogeneous graph neural network since it contains three different kinds of nodes: word nodes, aspect nodes, and sentence nodes. These nodes can pass structure and semantic information between each other and update their embeddings to improve the performance of our model. To the best of our knowledge, we are the first to use a heterogeneous graph to capture relationships between sentences and aspects. The experimental results on five public datasets show the effectiveness of our model outperforming some state-of-the-art models.

RA-HGNN: Attribute completion of heterogeneous graph neural networks based on residual attention mechanism

Heterogeneous graphs are an essential structure that models real-world data through different types of nodes and relationships between them, including multimodality, which comprises different types of data such as text, image, and audio. Graph Neural Networks (GNNs) are a prominent graph representation learning method that takes advantage of the graph structure and its attributes that, when applied to the multimodal heterogeneous graph, learn a unique semantic space for the different modalities. Consequently, it allows multimodal fusion through simple operators such as sum, average, or multiplication, generating unified representations considering the supplementary and complementarity relationships between the modalities. In multimodal heterogeneous graphs, the labeling process tends to be even more costly due to the multiple modalities analyzed, in addition to the imbalance of classes inherent to some applications. In order to overcome these problems in applications that comprise a class of interest, One-Class Learning (OCL) is used. Given the lack of studies on multimodal early fusion in heterogeneous graphs for OCL tasks, we proposed a method based on unsupervised GNN for heterogeneous graphs and evaluated different early fusion operators. In this paper, we extend another work by evaluating the behavior of the main GNN convolutions in the method. We highlight that using operators such as average, addition, and subtraction were the best early fusion operators. In addition, GNN layers that do not use an attention mechanism performed better. In this way, we argue for heterogeneous graph neural networks in multimodal using early fusion simple operators instead of well-often-used concatenation and less complex convolutions.

Heterogeneous networks refer to the networks comprising multiple types of entities as well as their interaction relationships. They arise in a great variety of domains, for example, event-based social networks Meetup and Plancast, and DBLP. Recommendation is a useful task in these heterogeneous network systems. Although many recommendation algorithms are proposed for heterogeneous data, none of them is able to explicitly model the influence strength between different types of entities, which is useful not only for achieving higher recommendation accuracy but also better understanding the role of each entity type in recommendation problems. Moreover, many of those algorithms are designed for a particular task, and hence it is challenging to apply them in other problems. In this paper, we propose a graph-based model, called HeteRS, which can solve general recommendation problems on heterogeneous networks. Our method models the rich information with a heterogeneous graph and considers the recommendation problem as a query-dependent node proximity problem. To address the challenging issue of weighting the influences between different types of entities, we propose a learning scheme to set the influence weights between different types of entities in recommendation. Experimental results on real-world datasets demonstrate that our proposed method significantly outperforms the baseline methods in our experiments for all the recommendation tasks, and the learned influence weights help understanding user behaviors.

Predicting the next interaction of a short-term interaction session is a challenging task in session-based recommendation. Almost all existing works rely on item transition patterns, and neglect the impact of user historical sessions while modeling user preference, which often leads to non-personalized recommendation. Additionally, existing personalized session-based recommenders capture user preference only based on the sessions of the current user, but ignore the useful item-transition patterns from other user's historical sessions. To address these issues, we propose a novel Heterogeneous Global Graph Neural Networks (HG-GNN) to exploit the item transitions over all sessions in a subtle manner for better inferring user preference from the current and historical sessions. To effectively exploit the item transitions over all sessions from users, we propose a novel heterogeneous global graph that contains item transitions of sessions, user-item interactions and global co-occurrence items. Moreover, to capture user preference from sessions comprehensively, we propose to learn two levels of user representations from the global graph via two graph augmented preference encoders. Specifically, we design a novel heterogeneous graph neural network (HGNN) on the heterogeneous global graph to learn the long-term user preference and item representations with rich semantics. Based on the HGNN, we propose the Current Preference Encoder and the Historical Preference Encoder to capture the different levels of user preference from the current and historical sessions, respectively. To achieve personalized recommendation, we integrate the representations of the user current preference and historical interests to generate the final user preference representation. Extensive experimental results on three real-world datasets show that our model outperforms other state-of-the-art methods.

Real-world graphs or networks are usually heterogeneous, involving multiple types of nodes and relationships. Heterogeneous graph neural networks (HGNNs) can effectively handle these diverse nodes and edges, capturing heterogeneous information within the graph, thus exhibiting outstanding performance. However, most methods of HGNNs usually involve complex structural designs, leading to problems such as high memory usage, long inference time, and extensive consumption of computing resources. These limitations pose certain challenges for the practical application of HGNNs, especially for resource-constrained devices. To mitigate this issue, we propose the Spiking Heterogeneous Graph Attention Networks (SpikingHAN), which incorporates the brain-inspired and energy-saving properties of Spiking Neural Networks (SNNs) into heterogeneous graph learning to reduce the computing cost without compromising the performance. Specifically, SpikingHAN aggregates metapath-based neighbor information using a single-layer graph convolution with shared parameters. It then employs a semantic-level attention mechanism to capture the importance of different meta-paths and performs semantic aggregation. Finally, it encodes the heterogeneous information into a spike sequence through SNNs, simulating bioinformatic processing to derive a binarized 1-bit representation of the heterogeneous graph. Comprehensive experimental results from three real-world heterogeneous graph datasets show that SpikingHAN delivers competitive node classification performance. It achieves this with fewer parameters, quicker inference, reduced memory usage, and lower energy consumption.

In recent years, heterogeneous network/graph representation learning/embedding (HNE) has drawn tremendous attentions from research communities in multiple disciplines. HNE has shown its outstanding performances in various networked data analysis and mining tasks. In fact, most of real-world information networks in multiple fields can be modelled as the heterogeneous information networks (HIN). Thus, the HNE-based techniques can sufficiently capture rich-structured and semantic latent features from the given information network in order to facilitate for different task-driven learning tasks. This is considered as fundamental success of HNE-based approach in comparing with previous traditional homogeneous network/graph based embedding techniques. However, there are recent studies have also demonstrated that the heterogeneous network/graph modelling and embedding through graph neural network (GNN) is not usually reliable. This challenge is original come from the fact that most of real-world heterogeneous networks are considered as incomplete and normally contain a large number of feature noises. Therefore, multiple attempts have proposed recently to overcome this limitation. Within this approach, the meta-path-based heterogeneous graph-structured latent features and GNN-based parameters are jointly learnt and optimized during the embedding process. However, this integrated GNN and heterogeneous graph structure (HGS) learning approach still suffered a challenge of effectively parameterizing and fusing different graph-structured latent features from both GNN- and HGS-based sides into better task-driven friendly and noise-reduced embedding spaces. Therefore, in this paper we proposed a novel attention-supplemented heterogeneous graph structure embedding approach, called as: AGSE. Our proposed AGSE model supports to not only achieve the combined rich heterogeneous structural and GNN-based aggregated node representations but also transform achieved node embeddings into noise-reduced and task-driven friendly embedding space. Extensive experiments in benchmark heterogeneous networked datasets for node classification task showed the effectiveness of our proposed AGSE model in comparing with state-of-the-art network embedding baselines.

Dynamic heterogeneous graph representation learning with neighborhood type modeling

Heterogeneous networks, consisting of multi-type objects coupled with various relations, are ubiquitous in the real world. Most previous work on clustering heterogeneous networks either converts them into homogeneous networks or simplifies the modeling of the heterogeneity in terms of specific objects, structures or assumptions. However, few studies consider all relevant objects and relations, and trade-off between integrating relevant objects and reducing the noises caused by relations across objects. In this paper, we propose a general probabilistic graphical model for clustering heterogeneous networks. First, we present a novel graphical representation based on our basic assumptions: different relation types produce different weight distributions to specify intra-cluster probability between two objects, and clusters are formed around cluster cores. Then, we derive an efficient algorithm called PROCESS, standing for PRObabilistic Clustering modEl for heterogeneouS networkS. PROCESS employs a balance-controlled message passing algorithm and mathematical programming for inference and estimation. Experimental results show that our approach is effective and significantly outperforms the state-of-the-art algorithms on both synthetic and real data from heterogeneous networks.

Heterogeneous graph neural networks (HGNNs) have the powerful capability to embed rich structural and semantic information of a heterogeneous graph into node representations. Existing HGNNs inherit many mechanisms from graph neural networks (GNNs) designed for homogeneous graphs, especially the attention mechanism and the multi-layer structure. These mechanisms bring excessive complexity, but seldom work studies whether they are really effective on heterogeneous graphs. In this paper, we conduct an in-depth and detailed study of these mechanisms and propose the Simple and Efficient Heterogeneous Graph Neural Network (SeHGNN). To easily capture structural information, SeHGNN pre-computes the neighbor aggregation using a light-weight mean aggregator, which reduces complexity by removing overused neighbor attention and avoiding repeated neighbor aggregation in every training epoch. To better utilize semantic information, SeHGNN adopts the single-layer structure with long metapaths to extend the receptive field, as well as a transformer-based semantic fusion module to fuse features from different metapaths. As a result, SeHGNN exhibits the characteristics of a simple network structure, high prediction accuracy, and fast training speed. Extensive experiments on five real-world heterogeneous graphs demonstrate the superiority of SeHGNN over the state-of-the-arts on both accuracy and training speed.