A Chinese Character-Level and Word-Level Complementary Text Classification Method

Text classification is a task to assign text documents according to its content to one or more classes automatically. Recently character-level models using deep neural networks have been developed to do classification text. Moreover, in some cases, character-level models have outperformed word-level models and traditional models, especially on user-generated dataset. The topologies that have been used for the character-level models are convolutional neural networks (CNN) and bidirectional recurrent neural networks (Bi-RNN), with its variants; long short-term memory (LSTM) and gated recurrent units (GRU). In this paper, CNN, Bi-RNN, and the combination of both are tested with character-level features and word-level features for text classification on English and Indonesian social media datasets. On small size datasets, word-level model outperformed character-level models. However, on dataset with millions of data, character-level model outperformed word-level model. Further analysis on the evaluation of word-level and character-level models is also discussed in this paper.

Long Short-Term Memory (LSTM) networks are unique to exercise data in its memory cell with long-term memory as Natural Language Processing (NLP) tasks have inklings of intensive time and computational power due to their complex structures like magnitude language model Transformer required to pre-train and learn billions of data performing different NLP tasks. In this paper, a dynamic chaotic model is proposed for the objective of transforming neurons states in network with neural dynamic characteristics by restructuring LSTM as Chaotic Neural Oscillatory-Long-Short Term Memory (CNO-LSTM), where neurons in LSTM memory cells are weighed in substitutes by oscillatory neurons to speed up computational training of language model and improve text classification accuracy for real-world applications. From the implementation perspective, five popular datasets of general text classification including binary, multi classification and multi-label classification are used to compare with mainstream baseline models on NLP tasks. Results showed that the performance of CNO-LSTM, a simplified model structure and oscillatory neurons state in exercising different types of text classification tasks are above baseline models in terms of evaluation index such as Accuracy, Precision, Recall and F1. The main contributions are time reduction and improved accuracy. It achieved approximately 46.76% of the highest reduction training time and 2.55% accuracy compared with vanilla LSTM model. Further, it achieved approximately 35.86% in time reduction compared with attention model without oscillatory indicating that the model restructure has reduced GPU dependency to improve training accuracy.

The existing methods for text classification fail to achieve high accuracy in processing Chinese texts, for that the basic unit of Chinese texts is not hanzis but Chinese phrases, and there is no natural delimiter in Chinese texts to separate the phrases. Things go even worse in the case of processing large number of Chinese Web texts, for these texts often lack of enough context, because most of these text are often short, irregular and sparse. In this paper, a new classification method is proposed for Chinese texts based on apparent semantics and latent aspects (ASLA). First, the apparent semantics of Chinese text are extracted as features instead of hanzis by BaiduBaike; Second, pLSA is applied for mining the latent aspects of these apparent semantics. Third, the relevant degree of a document to a category is calculated according to the apparent semantics and latent aspects. Finally, the category of a document is determined by the relevant degree. The proposed method is able to process Chinese web short text well with mini train data. Our experiments showed that the proposed method is promising, and it outperforms pLSA,SVM, KNN and CRF in the case of training data is not enough and the text is irregular.

How to use advanced deep learning technology to build an effective and powerful text classification model, extract text semantic attributes and achieve good classification results on large-scale test datasets has always been the main problem and severe challenge of text classification tasks. The purpose of this paper is to study the application optimization of NLP system in text semantics and text classification based on deep learning technology. A WC-GCN classification algorithm combining GCN and Attention mechanism is proposed. The related tasks of text classification are discussed, including three typical text classification tasks, including sentiment analysis, topic analysis, and news classification. The algorithm model proposed in this paper is verified from the experimental point of view. Compared with other deep learning-based text classification methods, the proposed algorithm model performs better on the public datasets of three text classification tasks. The news text also achieved a high accuracy rate.

Text classification is one of the basic tasks of natural language processing. In recent years, deep learning has been widely used in text classification tasks. The representative one is the convolutional neural network. The convolutional neural network(CNN) is limited by the size of the local window and can only extract local features of the text. For long texts like news, CNN cannot learn the longterm dependence of the long text. Another model of deep learning recurrent neural networks based on long short-term memory (LSTM) can learn the long-term dependence of text. Therefore, in the work of this paper, combining the advantages of CNN and LSTM, a LSTM_CNN Hybrid model is constructed for Chinese news text classification tasks. We first use LSTM to learn the longterm dependence of text, then we design a shallow convolution structure to further extract the semantic features of the text, and finally use the max-pooling operation to filter to obtain important features for classification. The model we proposed has achieved good results on the News dataset.

Text classification is an important and classic task of natural language processing. Deep neural networks are becoming more and more popular in text classification due to their expressive power and low requirements for feature engineering. However, the more flexible graph convolutional neural network is rarely used for this task. This paper proposes a text classification model (GENET) based on graph convolution word embedding and attention mechanism. The model can better combine the semantic and lexical information of the text with the discontinuous global word co-occurrence information and long-distance semantic information in the corpus. It breaks the shortcomings of traditional neural networks that have limited structure and can only learn local information. Experimental results show that our proposed text classification method out performs other models on multiple datasets. At the same time, it is proved through experiments that the word global information obtained by GCN is an important supplement to the word embedding representation.

This paper proposes an ontology learning method which is used to generate a graphical ontology structure called ontology graph. The ontology graph defines the ontology and knowledge conceptualization model, and the ontology learning process defines the method of semiautomatic learning and generates ontology graphs from Chinese texts of different domains, the so-called domain ontology graph (DOG). Meanwhile, we also define two other ontological operations—document ontology graph generation and ontology graph-based text classification, which can be carried out with the generated DOG. This research focuses on Chinese text data, and furthermore, we conduct two experiments: the DOG generation and ontology graph-based text classification, with Chinese texts as the experimental data. The first experiment generates ten DOGs as the ontology graph instances to represent ten different domains of knowledge. The generated DOGs are then further used for the second experiment to provide performance evaluation. The ontology graph-based approach is able to achieve high text classification accuracy (with 92.3 % in f-measure) over other text classification approaches (such as 86.8 % in f-measure for tf–idf approach). The better performance in the comparative experiments reveals that the proposed ontology graph knowledge model, the ontology learning and generation process, and the ontological operations are feasible and effective.

Text classification is a crucial task in natural language processing. Due to the characteristics of text structure, achieving the best result remains an ongoing challenge. In this paper, we propose an ensemble model which outperforms the state-of-the-art. We first utilize rule-based n-gram approach to extend corpus. Then two different features, global dependencies of word and local semantic feature, are extracted by gated recurrent unit and global average pooling model respectively. In order to take advantage of the complementarity of the global and local features, a decision-level fusion is applied to fuse those different kinds of features. We evaluate the quality of our model on various public datasets, including sentiment analysis, ontology classification and text categorization. Experimental results show that our model can effectively learn representations for language modeling, and achieves the best accuracy of text categorization.

With the development of Internet technology, more and more Chinese platforms are creating massive Chinese texts. At present, obtaining data samples for training is no longer a problem, and more and more researchers are beginning to devote themselves to obtaining great value from mining text information. Chinese text classification is mainly used for user sentiment analysis, personalized recommendation, topic tracking, and public opinion monitoring. However, Chinese texts naturally have many difficulties, such as many ambiguities, difficult word segmentation, fewer words, and sparse features. Traditional machine learning has a poor realization effect on Chinese texts. Deep neural networks are gradually becoming a new trend in Chinese text classification.

Text classification is a fundamental task in natural language processing (NLP). Deep-learning-based text classification methods usually have two stages: training and inference. However, the training dataset is only used in the training stage. To make full use of the training dataset in the inference stage in order to improve model performance, we propose a k-nearest neighbors retrieval augmented method (KRA) for deep-learning-based text classification models. KRA works by first constructing a storage system that stores the embeddings of the training samples during the training stage. During the inference stage, the model retrieves the top k-nearest neighbors of the testing text from the storage. Then, we use text augmentation methods to expand the retrieved neighbors, including traditional augmentation methods and a large language model (LLM)-based method. Next, the method weights the augmented neighbors based on their distances from the target text and incorporates their labels into the inference of the final results accordingly. We evaluate our KRA method on six benchmark datasets using four commonly used deep learning models: CNN, LSTM, BERT, and RoBERTa. The results demonstrate that KRA significantly improves the classification performance of these models, with an average accuracy improvement of 0.3% for BERT and up to 0.4% for RoBERTa. These improvements highlight the effectiveness and generalizability of KRA across different models and datasets, making it a valuable enhancement for a wide range of text classification tasks.

Text FCG: Fusing Contextual Information via Graph Learning for text classification

Text extraction is an important part of natural language processing (NLP) tasks. Most NLP tasks like text classification, machine translation, text-to-speech, text-based language identification, text summarization, and named-entity recognition involve the use of textual data. Such data is limited for low-resourced languages making it difficult to experiment advanced NLP techniques on these languages. This paper presents a Python-based toolkit for text analysis and text extraction from different types of images, documents, and audio files. The toolkit is built as a library that has functions that can be imported and utilized for text extraction.

Sequence labeling is a basic task in natural language processing, which is of great help to processing text information. Conventional sequence labeling approaches heavily rely on hand-crafted or language-specific features, which requires a lot of time. Therefore, most of the existing methods are based on the BiLSTM-CRF model, but how to use a neural network to extract useful information for each unit or segment in the input sequence becomes the main factor limiting the efficiency. Several BiLSTM-CRF based models for sequence labeling have been presented, but the major limitation is how to use neural networks for extracting useful representations for each unit or segment in the input sequence. In response to this problem, this paper proposes a sequence labeling algorithm based on hierarchical features and attention mechanism, which uses a hierarchical structure to integrate character-level and word-level information, and applies different attention mechanisms to these two layers of information. According to the structural characteristics of different levels, excavate more potential information. Finally, the previously captured and guided features are used for sequence tag prediction using CRF. Finally, the proposed model is subjected to comparative experiments and the results obtained are analyzed.

Incorporating context-relevant concepts into convolutional neural networks for short text classification

In recent years, with the rapid development of the Internet and multimedia technology, English translation text classification has played an important role in various industries. However, English translation remains a complex and difficult problem. Seeking an efficient and accurate English translation method has become an urgent problem to be solved. The study first elucidated the possibility of the development of transfer learning technology in multimedia environments, which was recognized. Then, previous research on this issue, as well as the Bidirectional Encoder Representations from Transformers (BERT) model, the attention mechanism and bidirectional long short-term memory (Att-BILSTM) model, and the transfer learning based cross domain model (TLCM) and their theoretical foundations, were comprehensively explained. Through the application of transfer learning in multimedia network technology, we deconstructed and integrated these methods. A new text classification technology fusion model, the BATCL transfer learning model, has been established. We analyzed its requirements and label classification methods, proposed a data preprocessing method, and completed experiments to analyze different influencing factors. The research results indicate that the classification system obtained from the study has a similar trend to the BERT model at the macro level, and the classification method proposed in this study can surpass the BERT model by up to 28%. The classification accuracy of the Att-BILSTM model improves over time, but it does not exceed the classification accuracy of the method proposed in this study. This study not only helps to improve the accuracy of English translation, but also enhances the efficiency of machine learning algorithms, providing a new approach for solving English translation problems.