A Heterogeneous Directed Graph Attention Network for inductive text classification using multilevel semantic embeddings

Text classification is an important and classic task of natural language processing. Deep neural networks are becoming more and more popular in text classification due to their expressive power and low requirements for feature engineering. However, the more flexible graph convolutional neural network is rarely used for this task. This paper proposes a text classification model (GENET) based on graph convolution word embedding and attention mechanism. The model can better combine the semantic and lexical information of the text with the discontinuous global word co-occurrence information and long-distance semantic information in the corpus. It breaks the shortcomings of traditional neural networks that have limited structure and can only learn local information. Experimental results show that our proposed text classification method out performs other models on multiple datasets. At the same time, it is proved through experiments that the word global information obtained by GCN is an important supplement to the word embedding representation.

Aiming at the problems of short text length, less information contained in text and poor classification performance caused by easy interference of deep learning model, combined with the advantages of CNN and BiLSTM, a text classification method based on fusion semantics, attention mechanism and BiLSTM-CNN model is proposed. Firstly, the Skip-gram model in word2vec is used to map the words in the data and convert them into vectors with fixed dimensions as the input of the model. The local features of the text vector are extracted by CNN and the global features related to the text context are extracted by BiLSTM. The features extracted by the two complementary models are fused, It solves the problem that the single convolution neural network model ignores the semantic and grammatical information of words in the context, and effectively avoids the gradient disappearance or gradient dispersion of the traditional cyclic neural network. This paper makes an experimental analysis on R8 and Ohsumed datasets. The results show that the proposed feature fusion model effectively improves the accuracy of text classification.

The dominant paradigm in Natural Language Processing (NLP) has traditionally relied on sequential modelling, often overlooking the rich, non-linear structural dependencies inherent in linguistic data. Graph Neural Networks (GNNs) offer a powerful alternative by modelling text as structured graphs, yet a systematic evaluation of their comparative efficacy across tasks of varying structural complexity remains under-explored. This paper presents a comprehensive comparative study of three prominent GNN architectures- Graph Convolutional Networks (GCN), Graph Attention Networks (GAT), and Graph SAGE- applied to three distinct NLP tasks: Text Classification, Semantic Role Labelling (SRL), and Abstract Meaning Representation (AMR) Parsing. We construct task-specific graph topologies, utilizing heterogeneous corpus graphs for document classification and syntactic dependency trees for SRL and AMR. Our empirical results, derived from benchmarks on the 20 Newsgroups, CoNLL-2009, and AMR 2.0 datasets, reveal a clear correlation between architectural properties and task complexity. While GCNs provide a robust and efficient baseline for global text classification (Accuracy: 96.6%), they struggle with the fine-grained structural modelling required for semantic tasks. Conversely, GATs demonstrate superior performance on syntax-heavy tasks, achieving a Labelled F1 score of 85.8% on SRL and a Smatch score of 74.5 on AMR parsing, significantly outperforming the GCN baseline (82.5% and 71.2%, respectively). These findings suggest that the anisotropic aggregation capability of attention mechanisms is critical for capturing the nuanced, long-range dependencies in natural language, establishing GAT as the preferred architecture for structure-dependent NLP applications.

Word vector is an important tool for natural language processing (NLP) tasks such as text classification. However, existing static language models such as Word2vec cannot solve the polysemy problem, leading to a decline in text classification performance. To solve this problem, this paper proposes a method for making Chinese word vector dynamic (MCWVD). The part of speech (POS) is used to solve the ambiguity problem caused by different POS. The POS structure graph is constructed and the syntactic structure information of POS features is extracted by GCN (Graph Convolutional Network). POS vector and word vector are concatenated into PW (POS-Word) vector. Parametric matrix is added to improve the fusion effect of POS and word features. Multilayer attention is used to distinguish the importance of different features and further update the vector expression of word vectors about the current context. Experiments on Chinese datasets THUCNews and SogouNews show that MCWVD effectively improves the accuracy of text classification and achieves better performance than CoVe (Context Vectors) and ELMo (Embeddings from Language Models). MCWVD also achieves similar performance to BERT and GPT-1 (Generative Pre-Training), but with a much lower computational cost and only 4% of BERT parameters.

Text classification is a popular research topic in the natural language processing. Recently solving text classification problems with graph neural network (GNN) has received increasing attention. However, current graph-based studies ignore the hidden information in text syntax and sequence structure, and it is difficult to use the model directly for processing new documents because the text graph is built based on the whole corpus including the test set. To address the above problems, we propose a text classification model based on long short-term memory network (LSTM) and graph attention network (GAT). The model builds a separate graph based on the syntactic structure of each document, generates word embeddings with contextual information using LSTM, then learns the inductive representation of words by GAT, and finally fuses all the nodes in the graph together into the document embedding. Experimental results on four datasets show that our model outperforms existing text classification methods with faster convergence and less memory consumption than other graph-based methods. In addition, our model shows a more notable improvement when using less training data. Our model proves the importance of text syntax and sequence information for classification results.

The set of features which the traditional feature selection algorithm of chi-square selected is not complete. This causes the low performance for the final text classification. Therefore, this paper proposes a method. The method utilizes word2vec to generate word vector to improve feature selection algorithm of the chi square. The algorithm applies the word vector generated by word2vec to the process of the traditional feature selection and uses these words to supplement the set of features as appropriate. Ultimately, the set of features obtained by this method has better discriminatory power. Because, the feature words with the better discriminatory power has the strong ability of distinguishing categories as its semantically similar words. On this base, multiple experiments have been carried out in this paper. The experimental results show that the performance of text classification can increase after extension of feature words.

Aspect-based sentiment analysis is a fine-grained sentiment analysis that focuses on the sentiment polarity of different aspects of text, and most current research methods use a combination of dependent syntactic analysis and graphical neural networks. In this paper, a graph attention network aspect-based sentiment analysis model based on the weighting of dependencies (WGAT) is designed to address the problem in that traditional models do not sufficiently analyse the types of syntactic dependencies; in the proposed model, graph attention networks can be weighted and averaged according to the importance of different nodes when aggregating information. The model first transforms the input text into a low-dimensional word vector through pretraining, while generating a dependency syntax graph by analysing the dependency syntax of the input text and constructing a dependency weighted adjacency matrix according to the importance of different dependencies in the graph. The word vector and the dependency weighted adjacency matrix are then fed into a graph attention network for feature extraction, and sentiment polarity is predicted through the classification layer. The model can focus on syntactic dependencies that are more important for sentiment classification during training, and the results of the comparison experiments on the Semeval-2014 laptop and restaurant datasets and the ACL-14 Twitter social comment dataset show that the WGAT model has significantly improved accuracy and F1 values compared to other baseline models, validating its effectiveness in aspect-level sentiment analysis tasks.

Graph Neural Networks (GNNs) have achieved notable success in graph classification tasks, such as chemical molecular graph classification in cheminformatics and drug design, owing to their ability to effectively capture structural features of molecular graphs. However, conventional GNNs still face challenges in comprehensively characterizing complex node interactions, particularly in discerning distinct node roles and their intricate interplay. To address this limitation, we propose a novel Hyperlink-Induced Topic Search (HITS)-inspired Interactive Graph ATtention network (HITS-InterGAT) for graph classification. This model employs an interactive attention mechanism to capture nuanced node interactions within molecular graphs. Specifically, we integrate the HITS algorithm to compute node authority and hub features, followed by an interaction graph attention mechanism to model feature-level relationships. Extensive experiments on multiple chemical and molecular datasets demonstrate that HITS-InterGAT significantly outperforms existing baselines in both classification accuracy and robustness. Our findings underscore the substantial potential of combining interactive attention mechanisms with the HITS algorithm for advancing chemical molecular graph classification.

Text document clustering is a task that seeks to cluster together related documents based on content similarity. Clustering helps to improve information diversity retrieval, topic spotting, and sentiment analysis by ordering unstructured text and dividing it into information structures. To address the issue of text-based work and its optimization through more efficient and accurate text classification and clustering, this paper offers a more advanced text classification and clustering framework that incorporates the Adaptive Quantum Bacterial Colony Optimization (AQBCO) algorithm and deep word embedding models, such as Word2Vec, TF-IDF, and FastText. AQBCO presents two innovations: adaptive chemotaxis, which produces the movement of bacteria dynamically controlled by the optimal solution, and quantum-inspired position updating, which increases the ability to search the globe and eliminates premature convergence. The framework was tested using benchmark datasets- BBC, SMS, and DMOZ, and was compared with the available optimization-based learning methods, including Improved BCO (IBCO), standard BCO, GQPSO, IPSO, PSO, GA, and k-means. According to the experimental findings, AQBCO always performs better on Accuracy, Precision, Recall, and F-Measure as compared to other models. The highest accuracy of 97.38% on BBC, 96.79% on SMS, and 93.68% on DMOZ datasets allowed AQBCO to use FastText embeddings. The suggested adaptive and quantum systems preserve exploration-exploitation balance, enhancing the speed of convergence and resiliency. These findings make AQBCO a scalable, stable, and efficient method of text classification as well as document clustering.

Graph Neural Networks (GNNs) are deep learning models specifically designed for analyzing graph-structured data, capturing complex relationships and structures to improve analysis and prediction. A common task in GNNs is node classification, where each node in the graph is assigned a predefined category. The Graph Attention Network (GAT) is a popular variant of GNNs known for its ability to capture complex dependencies by assigning importance weights to nodes during information aggregation. However, the GAT’s reliance on local attention mechanisms limits its effectiveness in capturing global information and long-range dependencies. To address this limitation, we propose a new attention mechanism called Global-Local Graph Attention (GLGA). Our mechanism enables the GAT to capture long-range dependencies and global graph structures while maintaining its ability to focus on local interactions. We evaluate our algorithm on three citation datasets (Cora, Citeseer, and Pubmed) using multiple metrics, demonstrating its superiority over other baseline models. The proposed GLGA mechanism has been proven to be an effective solution for improving node classification tasks.

In advanced technology nodes, aging effects like negative and positive bias temperature instability (NBTI and PBTI) become increasingly significant, making timing closure and optimization more challenging. Unfortunately, conventional critical path (CP) selection tools used in reliability-aware design flow cannot accurately identify CPs under different aging conditions. To address this issue, we propose an aging-aware CP selection flow comprising two parts: 1) critical cell detection and 2) path criticality (PC) computation. We employ graph-attention (GAT) networks to predict the critical cells in the aged circuits, and a PC computation algorithm that takes into account circuit-level and transistor-level parameters to generate PC rank lists. Our experimental results demonstrate that our GAT model outperforms classical machine learning models in detecting critical cells. Additionally, compared with the commercial tool, our aging-aware flow achieves an average accuracy of 99.52%, 98.69%, and 97.20% for top-10%, top-5%, and top-1% path sets, respectively, in five industrial designs subjected to different aging conditions and workloads.

Text classification is a basic but important task in many natural language processing tasks. Nowadays, the mainstream classification methods mostly use deep learning technology, which shows better accuracy and stability in English text classification. Different from English text, Chinese text classification task involves the granularity of feature description in text decomposition. The two commonly used feature granularity are word-level feature and character-level feature. The former will bring semantic loss in the process of word segmentation, while the latter can't use the advanced semantic feature in the pre-trained word vector. We propose a method to fuse the word-level and the character-level information with attention mechanism. We train the CWC-Net, which combines the features to make the embedded information of characters and words complementary, so as to improve the semantic understanding ability of the network for Chinese text and reduce semantic loss. The comparative experiments on four Chinese text datasets, which involving topic classification and emotion analysis show that our model is more accurate than the traditional model which only relies on word-level features or character-level features. That verifies the effectiveness of the fusion of word-level features and character-level features on the improvement of model capability.

TreeLSTM with tag-aware hypernetwork for sentence representation

Text classification is a fundamental task in many applications such as topic labeling, sentiment analysis, and spam detection. The text syntactic relationship and word sequence are important and useful for text classification. How to model and incorporate them to improve performance is one key challenge. Inspired by human behavior in understanding text. In this paper, we combine the syntactic relationship, sequence structure, and semantics for text representation, and propose an attention-enhanced capsule network-based text classification model. Specifically, we use graph convolutional neural networks to encode syntactic dependency trees, build multi-head attention to encode dependencies relationship in text sequence, merge with semantic information by capsule network at last. Extensive experiments on five datasets demonstrate that our approach can effectively improve the performance of text classification compared with state-of-the-art methods. The result also shows capsule network, graph convolutional neural network, and multi-headed attention has integration effects on text classification tasks.

In the product reviews of online shopping platforms, the star rating and text comments given by the same user often appear. These data are processed by the unreasonable scoring system to give the merchants a false rating and mislead consumers. In order to improve the scoring system, an improved feature weighting method combining text review content and star rating is proposed. Firstly, the weighting rules were defined. Secondly, according to the part-of-speech evaluation function in the short text, the feature word selection was combined with the star rating and the text comment content. The CBOW model was used to train the word vector. Finally, the text classification method SVM was used to obtain the final result. This model was applied to two text datasets and compared with the traditional TF-IDF feature selection. The results show that the algorithm can effectively improve the accuracy and F1 value of text classification.