Research on text classification technology based on natural language processing

Semantic text classification: A survey of past and recent advances

The dominant paradigm in Natural Language Processing (NLP) has traditionally relied on sequential modelling, often overlooking the rich, non-linear structural dependencies inherent in linguistic data. Graph Neural Networks (GNNs) offer a powerful alternative by modelling text as structured graphs, yet a systematic evaluation of their comparative efficacy across tasks of varying structural complexity remains under-explored. This paper presents a comprehensive comparative study of three prominent GNN architectures- Graph Convolutional Networks (GCN), Graph Attention Networks (GAT), and Graph SAGE- applied to three distinct NLP tasks: Text Classification, Semantic Role Labelling (SRL), and Abstract Meaning Representation (AMR) Parsing. We construct task-specific graph topologies, utilizing heterogeneous corpus graphs for document classification and syntactic dependency trees for SRL and AMR. Our empirical results, derived from benchmarks on the 20 Newsgroups, CoNLL-2009, and AMR 2.0 datasets, reveal a clear correlation between architectural properties and task complexity. While GCNs provide a robust and efficient baseline for global text classification (Accuracy: 96.6%), they struggle with the fine-grained structural modelling required for semantic tasks. Conversely, GATs demonstrate superior performance on syntax-heavy tasks, achieving a Labelled F1 score of 85.8% on SRL and a Smatch score of 74.5 on AMR parsing, significantly outperforming the GCN baseline (82.5% and 71.2%, respectively). These findings suggest that the anisotropic aggregation capability of attention mechanisms is critical for capturing the nuanced, long-range dependencies in natural language, establishing GAT as the preferred architecture for structure-dependent NLP applications.

Text classification is a key task of the Natural Language Processing (NLP) field that aims at assigning predefined categories to textual documents. Performing text classification requires features that effectively represent the content and the meaning of textual documents. Selecting a suitable method for term weighting is of central importance and can improve the quality of the classification method. In this paper, we propose to a new text classification solution to perform Category-based Feature Augmentation (CFA) on the document representation. First, a term-category feature matrix is derived from a modified version of the supervised Term-Frequency Inverse-Category-Frequency (TF-ICF) weighting model. This is done by embedding the TF-ICF matrix in a one-layer feed-forward neural network. The latter is trained using the gradient descent algorithm allowing to iteratively update the term-category matrix until reaching convergence. The model produces category-based feature vector representations that are used to augment the document representations and perform the classification task. Experimental results on four benchmark datasets show that our lean model approach improves text classification accuracy and is significantly more efficient compared with its deep model alternatives.

One of the most under-researched area in the field of natural language processing (NLP) has been the text classification. Text classification of languages like Urdu is more deprived of research since it is not among the top-most spoken language of the world. Despite that, almost 230 million people around the world speak Urdu. Researchers have worked with various machine learning and deep learning models for the identification of handwritten text of Urdu language. This paper is the comparative analysis of the various algorithms that have been used on the offline as well as online text recognition of Urdu language. From our analysis, CNN is the most widely adopted model for recognition of Urdu handwritten text due to its excellent properties of image classification as well as feature extraction. This paper gives a brief of various machine learning algorithms used for recognition of Urdu text and their comparative analysis.

Graph theory and the fields of natural language processing and information retrieval are well-studied disciplines. Traditionally, these areas have been perceived as distinct, with different algorithms, different applications and different potential end-users. However, recent research has shown that these disciplines are intimately connected, with a large variety of natural language processing and information retrieval applications finding efficient solutions within graph-theoretical frameworks. This book extensively covers the use of graph-based algorithms for natural language processing and information retrieval. It brings together topics as diverse as lexical semantics, text summarization, text mining, ontology construction, text classification and information retrieval, which are connected by the common underlying theme of the use of graph-theoretical methods for text and information processing tasks. Readers will come away with a firm understanding of the major methods and applications in natural language processing and information retrieval that rely on graph-based representations and algorithms.

Sequence Classification is one of the on-demand research projects in the field of Natural Language Processing (NLP). Classifying a set of images or text into an appropriate category or class is a complex task that a lot of Machine Learning (ML) models fail to accomplish accurately and end up under-fitting the given dataset. Some of the ML algorithms used in text classification are KNN, Naïve Bayes, Support Vector Machines, Convolutional Neural Networks (CNNs), Recursive CNNs, Recurrent Neural Networks (RNNs), Long Short Term Memory (LSTM), etc. For this experimental study, LSTM and a few other algorithms were chosen for a more comparative study. The dataset used is the SMS Spam Collection Dataset from Kaggle and 150 more entries were additionally added from different sources. Two possible class labels for the data points are spam and ham. Each entry consists of the class label, a few sentences of text followed by a few useless features that are eliminated. After converting the text to the required format, the models are run and then evaluated using various metrics. In experimental studies, the LSTM gives much better classification accuracy than the other machine learning models. F1-Scores in the high nineties were achieved using LSTM for classifying the text. The other models showed very low F1-Scores and Cosine Similarities indicating that they had underperformed on the dataset. Another interesting observation is that the LSTM had reduced the number of false positives and false negatives than any other model.

With the rapid development of the Internet and social media, a large amount of text data is constantly generated, and text classification has become an important task. Text classification is an important branch in the field of natural language processing and has attracted the attention of many researchers in recent years. Due to the ease of editing text data, most texts in the network are manually constructed and uploaded by users. Therefore, the standardization of online texts and the classification of texts with different granularity are of great significance in the field of information retrieval. This article introduces the basic concepts of text classification and three text classification methods: knowledge engineering (rule) based text classification methods, traditional machine learning-based text classification methods, and deep neural network-based text classification methods. This article also analyzes the advantages and disadvantages of the three methods. Subsequently, the basic concepts and common models of graph neural networks were explored. An application case of graph neural networks in text classification is provided. This article provides effective suggestions for text classification applications in the field of natural language processing.

This study aims to demonstrate the methods for detecting negations in a sentence by uniquely evaluating the lexical structure of the text via word sense disambiguation. Additionally, the proposed method examined all the unique features of the related expressions within a text to resolve the contextual usage of the sentence and the effect of negation on sentiment analysis. The application of popular expression detectors skips this important step, thereby neglecting the root words caught in the web of negation, and making text classification difficult for machine learning and sentiment analysis. This study adopts the Natural Language Processing (NLP) approach to discover and antonimize words that were negated for better accuracy in text classification. This method acts as a lens that reads through a given word sequence using a knowledge base provided by an NLP library called WordHoard in order to detect negation signals. Early results show that our initial analysis improved traditional sentiment analysis that sometimes neglects word negations or assigns an inverse polarity score. The SentiWordNet analyzer was improved by 35%, the Vader analyzer by 20% and the TextBlob analyzer by 6%.

Introduction: Natural Language Processing (NLP) has experienced significant advancements to address the growing demand for efficient and accurate text classification. Despite numerous methodologies, achieving a balance between high accuracy and model stability remains a critical challenge. This research aims to explore the implementation of a hybrid architecture integrating Convolutional Neural Networks (CNN) and Bidirectional Long Short-Term Memory (BiLSTM) with FastText embeddings, targeting effective text classification.Methods: The proposed hybrid architecture combines the CNN's ability to capture local patterns and BiLSTM's temporal feature extraction capabilities, enhanced by FastText embeddings for richer word representation. Regulatory mechanisms such as Dropout and Early Stopping were employed to mitigate overfitting. Comparative experiments were conducted to evaluate the performance of the model with and without Early Stopping.Results: The experimental findings reveal that the model without Early Stopping achieved a remarkable accuracy of 99%, albeit with a higher susceptibility to overfitting. Conversely, the implementation of Early Stopping resulted in a stable accuracy of 73%, demonstrating enhanced generalization capabilities while preventing overfitting. The inclusion of Dropout further contributed to model regularization and stability.Conclusions: This study underscores the significance of balancing accuracy and stability in deep learning models for text classification. The proposed hybrid architecture effectively combines the strengths of CNN, BiLSTM, and FastText embeddings, providing valuable insights into the trade-offs between achieving high accuracy and ensuring robust generalization. Future work could further explore optimization techniques and datasets for broader applicability.

Text classification is one of the most popular tasks in natural language processing, especially in the context of sentiment classification. Insufficient training data poses a significant challenge in many text classification studies. This research focuses on optimizing classification performance using the Passive Aggressive (PA) algorithm, leveraging limited training data. It compares conventional text representation methods like TF-IDF with modern approaches employing word embeddings such as FastText and BERT. The primary dataset encompasses sentiment issues related to Kaesang Pangarep's appointment as the chairman of PSI, gathered through Twitter crawling, and classified into positive, negative, and neutral sentiment labels. Two versions of the training data, each containing only 300 balanced tweets for positive, negative, and neutral classes, were used. The data was split 80% for training and 20% for validation in the search for an optimal model. External data with different issues and pre-existing sentiment labels was used to augment the training data. Experimental results demonstrated that the BERT language model, used as input features for the Passive Aggressive method with hyperparameter tuning, outperformed TF-IDF features. Evaluation on the test data revealed that BERT features with Passive Aggressive achieved an F1-score of 0.52, surpassing the conventional TF-IDF representation with an F1-score of 0.42. The utilization of the BERT language model significantly contributed to improving text classification performance in the field of natural language processing, particularly for the Passive Aggressive method.

SRL-ACO: A text augmentation framework based on semantic role labeling and ant colony optimization

Text classification is a classic task in the field of natural language processing. however, the existing methods of text classification tasks still need to be improved because of the complex abstraction of text semantic information and the strong relecvance of context. In this paper, we combine the advantages of two traditional neural network model, Long Short-Term Memory(LSTM) and Convolutional Neural Network(CNN). LSTM can effectively preserve the characteristics of historical information in long text sequences, and extract local features of text by using the structure of CNN. We proposes a hybrid model of LSTM and CNN, construct CNN model on the top of LSTM, the text feature vector output from LSTM is further extracted by CNN structure. The performance of the hybrid model is compared with that of other models in the experiment. The experimental results show that the hybrid model can effectively improve the accuracy of text classification.

Natural Language Processing (NLP) has witnessed significant advancements in recent years, driven by the emergence of deep learning techniques.Transformers, introduced in 2017, have revolutionized the field of NLP, demonstrating exceptional performance in various tasks.This study aims to explore the potential of Transformers in text classification, a fundamental task in NLP.We conduct a comprehensive evaluation of three pre-trained Transformer models -BERT, RoBERTa, and XLNet -on three benchmark datasets -20 Newsgroups, IMDB, and Stanford Sentiment Treebank.Our results show that Transformers achieve state-of-the-art performance in text classification, outperforming traditional machine learning approaches.We also analyze the strengths and limitations of each model, highlighting their ability to capture long-range dependencies and contextual relationships in text data.Our findings suggest that Transformers are robust and effective models for text classification, with applications in sentiment analysis, spam detection, and information retrieval.We also discuss the potential of Transformers in other NLP tasks, such as question answering, machine translation, and text generation.This study provides a comprehensive overview of the capabilities of Transformers in text classification, highlighting their potential as a powerful tool for NLP tasks.Our results and analysis provide insights for researchers and practitioners working in the field of NLP, highlighting the potential of Transformers for a wide range of applications.

Analytics of machine learning-based algorithms for text classification

In the past fifteen years, the grow of using Bag of Words (BoW) method in the field of computer vision is visibly observed. In addition,-for the text classification and texture recognition, it can also be used in classification of images, videos, robot localization, etc. It is one of the most common methods for the categorization of text and objects. In text classification, the BoW method records the number of occurrences of each bag that is created for each instance type or word disregarding the order of the words or the grammar. And in visual scene classification it is based on clusters of local descriptors which are taken from the images disregarding the order of the clusters. The key idea is to generate a histogram for the words in the documents or the features in the images to represent the specified document or image. The BoW method is computationally and even conceptually is simpler than many other classification methods. For that reason, BoW based systems could record new and higher performance scores on common used benchmarks of text and image classification algorithms. This paper presents an overview of BoW, importance of BoW, how does it work, applications and challenges of using BoW. This study is useful in terms of introducing the BoW method to the new researchers and providing a good background with associated related works to the researchers that are working on the model.