Deep Semantic Network Representation

Semantic annotation for Web services represents an important approach that transforms Web services into Semantic Web services. This paper analyzes the distribution of implicit semantic information to enhance the understanding about Web services. After quantifying implicit semantic information in Web services, the PCA methodology is utilized for obtaining principal components of implicit semantic information. The experiment conducted on 11836 WSDL documentations shows that almost 97 percent of implicit semantic information is distributed in inputs and outputs. Thus, the evaluation of how much implicit semantic information can be described by semantic annotation is offered.

Cross-scale contrastive triplet networks for graph representation learning

Network embedding (NE) maps a network into a low-dimensional space while preserving intrinsic features of the network. Variational Auto-Encoder (VAE) has been actively studied for NE. These VAE-based methods typically utilize both network topologies and node semantics and treat these two types of data in the same way. However, the information of network topology and information of node semantics are orthogonal and are often from different sources; the former quantifies coupling relationships among nodes, whereas the latter represents node specific properties. Ignoring this difference affects NE. To address this issue, we develop a network-specific VAE for NE, named as NetVAE. In the encoding phase of our new approach, compression of network structures and compression of node attributes share the same encoder in order to perform co-training to achieve transfer learning and information integration. In the decoding phase, a dual decoder is introduced to reconstruct network topologies and node attributes separately. Specifically, as a part of the dual decoder, we develop a novel method based on a Gaussian mixture model and the block model to reconstruct network structures. Extensive experiments on large real-world networks demonstrate a superior performance of the new approach over the state-of-the-art methods.

In this work, we focus on the problem of news citation recommendation. The task aims to recommend news citations for both authors and readers to create and search news references. Due to the sparsity issue of news citations and the engineering difficulty in obtaining information on authors, we focus on content similarity-based methods instead of collaborative filtering-based approaches. In this paper, we explore word embedding (i.e., implicit semantics) and grounded entities (i.e., explicit semantics) to address the variety and ambiguity issues of language. We formulate the problem as a reranking task and integrate different similarity measures under the learning to rank framework. We evaluate our approach on a real-world dataset. The experimental results show the efficacy of our method.

This article examined the integration of semantic and morphosyntactic information by Korean learners of English as a second language (L2). In Experiment 1, L2 learners listened to English active or passive sentences that were either plausible or implausible and translated them into Korean. A significant number of Korean translations maintained the original passive/active structure, but switched the thematic roles of the actors in the sentences. In Experiment 2, the direction of translation was reversed and participants made very few translation errors, showing that the errors in Experiment 1 were not due to participants’ lack of control over the English passive morphosyntax. The results are strikingly similar to previous results in the first language (L1) psycholinguistics literature, and support a view of L2 processing (like L1 processing) that is ‘good enough’ in nature: misinterpretations arise from only a ‘good enough’ integration of semantic and morphosyntactic information in the input.

In the traditional task of aspect sentiment triplet extraction, existing approaches typically focus on either syntactic or semantic features independently, failing to leverage the complementary integration of these two types of information. Although graph convolutional network-based approaches have demonstrated impressive performance in triplet extraction tasks, they often ignore distance features and semantic information when capturing sentence information. As a result, the integration of syntactic and semantic information remains suboptimal, negatively impacting sentiment analysis performance. To address this limitation, we propose a novel Syntax-Semantics Graph Convolutional Network for aspect sentiment triplet extraction. Our method first extracts syntactic structural information using the probability matrix of dependency trees, from which a mask matrix is constructed based on the varying distances between words. Next, semantic information is captured via a self-attention mechanism and an aspect-attention mechanism, utilizing an attention score matrix. Finally, an interaction module is introduced to effectively integrate syntactic and semantic features. Extensive experiments on several benchmark datasets demonstrate that our approach significantly outperforms existing baselines, achieving an average F1-score improvement of at least 1.083%.

With the rapid increase of scientific papers, it is difficult for researchers to obtain appropriate references. Due to the black-box property of neural network, the existing content-based citation recommendation methods are mainly embedding text into a low-dimensional space, and citation links are not interpretable (e.g., paper A cites paper B because of topic X). To solve this problem, we propose a Weighted Citation Network with Citation Links Explanation (WCN-CLE) algorithm. Because citation links mainly depends on the content, we need to explore textual reasons for citation links. Firstly, we extract word fragments through citation network information for each citation link as its explanation. Secondly, we construct a weighted HIN network with two kinds of vertex (paper and author). The weight of each edge is calculated by its word fragments. Then the query document is linked to the HIN network through author information and word fragments similarity with candidate papers. Finally, each vertex’s feature representation is obtained by network representation learning method, and recommend papers through vector similarity. Experimental results on two real-world datasets show that WCN-CLE has better performance due to its integration of structural information and semantic information.

Anomalous behaviors in social networks can lead to privacy leaks and the spread of false information. In this paper, we propose an anomalous behavior detection method based on optimized graph embedding representation. Specifically, the user behavior logs are first extracted into a social network user behavior temporal knowledge graph, based on which the graph embedding representation method is used to transform the network topology and temporal information in the user behavior knowledge graph into structural embedding vectors and temporal information embedding vectors, and the hybrid attention mechanism is used to merge the two types of vectors to obtain the final entity embedding to complete the prediction and complementation of the temporal knowledge graph of user behavior. We use graph neural networks, which use the temporal information of user behaviors as a time constraint and capture both user behavioral and semantic information. It converts the two parts of information into vectors for concatenation and linear transformation to obtain a comprehensive representation vector of the whole subgraph, as well as joint deep learning models to evaluate abnormal behavior. Finally, we perform experiments on the Yelp dataset to validate that our method achieves a 9.56% improvement in the F1-score.

Network embedding, aiming to learn the low-dimensional representations of nodes in networks, is of paramount importance in many real applications. One basic requirement of network embedding is to preserve the structure and inherent properties of the networks. While previous network embedding methods primarily preserve the microscopic structure, such as the first- and second-order proximities of nodes, the mesoscopic community structure, which is one of the most prominent feature of networks, is largely ignored. In this paper, we propose a novel Modularized Nonnegative Matrix Factorization (M-NMF) model to incorporate the community structure into network embedding. We exploit the consensus relationship between the representations of nodes and community structure, and then jointly optimize NMF based representation learning model and modularity based community detection model in a unified framework, which enables the learned representations of nodes to preserve both of the microscopic and community structures. We also provide efficient updating rules to infer the parameters of our model, together with the correctness and convergence guarantees. Extensive experimental results on a variety of real-world networks show the superior performance of the proposed method over the state-of-the-arts.

Attributed network embedding aims to learn low dimensional node representations by combining both the network's topological structure and node attributes. Most of the existing methods either propagate the attributes over the network structure or learn the node representations by an encoder-decoder framework. However, propagation based methods tend to prefer network structure to node attributes, whereas encoder-decoder methods tend to ignore the longer connections beyond the immediate neighbors. In order to address these limitations while enjoying the best of the two worlds, we design cross fusion layers for unsupervised attributed network embedding. Specifically, we first construct two separate views to handle network structure and node attributes, and then design cross fusion layers to allow flexible information exchange and integration between the two views. The key design goals of the cross fusion layers are three-fold: 1) allowing critical information to be propagated along the network structure, 2) encoding the heterogeneity in the local neighborhood of each node during propagation, and 3) incorporating an additional node attribute channel so that the attribute information will not be overshadowed by the structure view. Extensive experiments on three datasets and three downstream tasks demonstrate the effectiveness of the proposed method.

Network embedding (graph embedding) has become the focus of studying graph structure in recent years. In addition to the research on homogeneous networks and heterogeneous networks, there are also some methods to attempt to solve the problem of dynamic network embedding. However, in dynamic networks, there is no research method specifically for heterogeneous networks. Therefore, this paper proposes a heterogeneous dynamic graph attention network (HDGAN), which attempts to use the attention mechanism to take the heterogeneity and dynamics of the network into account at the same time, so as to better learn network embedding. Our method is based on three levels of attention, namely structural-level attention, semantic-level attention and time-level attention. Structural-level attention pays attention to the network structure itself, and obtains the representation of structural-level nodes by learning the attention coefficients of neighbor nodes. Semantic-level attention integrates semantic information into the representation of nodes by learning the optimal weighted combination of different meta-paths. Time-level attention is based on the time decay effect, and the time feature is introduced into the node representation by neighborhood formation sequence. Through the above three levels of attention mechanism, the final network embedding can be obtained.Through experiments on two real-world heterogeneous dynamic networks, our models have the best results, proving the effectiveness of the HDGAN model.

Multi-level attentive deep user-item representation learning for recommendation system

Colon polyps represent a common gastrointestinal form. In order to effectively treat and prevent complications arising from colon polyps, colon polypectomy has become a commonly used therapeutic approach. Accurately segmenting polyps from colonoscopy images can provide valuable information for early diagnosis and treatment. Due to challenges posed by illumination and contrast variations, noise and artifacts, as well as variations in polyp size and blurred boundaries in polyp images, the robustness of segmentation algorithms is a significant concern. To address these issues, this paper proposes a Double Loss Guided Residual Attention and Feature Enhancement Network (DLGRAFE-Net) for polyp segmentation. Firstly, a newly designed Semantic and Spatial Information Aggregation (SSIA) module is used to extract and fuse edge information from low-level feature graphs and semantic information from high-level feature graphs, generating local loss-guided training for the segmentation network. Secondly, newly designed Deep Supervision Feature Fusion (DSFF) modules are utilized to fuse local loss feature graphs with multi-level features from the encoder, addressing the negative impact of background imbalance caused by varying polyp sizes. Finally, Efficient Feature Extraction (EFE) decoding modules are used to extract spatial information at different scales, establishing longer-distance spatial channel dependencies to enhance the overall network performance. Extensive experiments conducted on the CVC-ClinicDB and Kvasir-SEG datasets demonstrate that the proposed network outperforms all mainstream networks and state-of-the-art networks, exhibiting superior performance and stronger generalization capabilities.

Heterogeneous graph neural networks (HGNNs) have the powerful capability to embed rich structural and semantic information of a heterogeneous graph into node representations. Existing HGNNs inherit many mechanisms from graph neural networks (GNNs) designed for homogeneous graphs, especially the attention mechanism and the multi-layer structure. These mechanisms bring excessive complexity, but seldom work studies whether they are really effective on heterogeneous graphs. In this paper, we conduct an in-depth and detailed study of these mechanisms and propose the Simple and Efficient Heterogeneous Graph Neural Network (SeHGNN). To easily capture structural information, SeHGNN pre-computes the neighbor aggregation using a light-weight mean aggregator, which reduces complexity by removing overused neighbor attention and avoiding repeated neighbor aggregation in every training epoch. To better utilize semantic information, SeHGNN adopts the single-layer structure with long metapaths to extend the receptive field, as well as a transformer-based semantic fusion module to fuse features from different metapaths. As a result, SeHGNN exhibits the characteristics of a simple network structure, high prediction accuracy, and fast training speed. Extensive experiments on five real-world heterogeneous graphs demonstrate the superiority of SeHGNN over the state-of-the-arts on both accuracy and training speed.

Low-dimensional vector representations of network nodes have proven successful to feed graph data to machine learning algorithms and to improve performance across diverse tasks. Most of the embedding techniques, however, have been developed with the goal of achieving dense, low-dimensional encoding of network structure and patterns. Here, we present a node embedding technique aimed at providing low-dimensional feature vectors that are informative of dynamical processes occurring over temporal networks – rather than of the network structure itself – with the goal of enabling prediction tasks related to the evolution and outcome of these processes. We achieve this by using a lossless modified supra-adjacency representation of temporal networks and building on standard embedding techniques for static graphs based on random walks. We show that the resulting embedding vectors are useful for prediction tasks related to paradigmatic dynamical processes, namely epidemic spreading over empirical temporal networks. In particular, we illustrate the performance of our approach for the prediction of nodes’ epidemic states in single instances of a spreading process. We show how framing this task as a supervised multi-label classification task on the embedding vectors allows us to estimate the temporal evolution of the entire system from a partial sampling of nodes at random times, with potential impact for nowcasting infectious disease dynamics.