A Feature Extraction Method Based on Feature Fusion and its Application in the Text-Driven Failure Diagnosis Field.

The goal of text classification is to assign labels to text units accurately, which is a basic task in natural language processing. This technology has shown great value in many practical application scenarios, covering spam detection, emotional tendency analysis, user intent recognition, and many other aspects. In recent years, because of the excellent performance of graph convolutional neural networks (GCNs) in processing non-European spatial data, it has become a research hotspot in text classification and has been widely adopted. This paper first introduces the background knowledge and working principle of graph convolutional neural networks. Then the text classification method based on GCNs is described. The last thing discussed is the limitations of graph convolutional neural networks, as well as the difficulties and future development directions in this field. This paper serves as a guide for researchers and practitioners in relevant fields to fully comprehend the latest advancements in text classification methods using graph convolutional neural networks, and by revealing its limitations and challenges, it also provides guidance for future research and development work, intending to promote technical progress and application expansion in this field.

Long Short-Term Memory (LSTM) networks are unique to exercise data in its memory cell with long-term memory as Natural Language Processing (NLP) tasks have inklings of intensive time and computational power due to their complex structures like magnitude language model Transformer required to pre-train and learn billions of data performing different NLP tasks. In this paper, a dynamic chaotic model is proposed for the objective of transforming neurons states in network with neural dynamic characteristics by restructuring LSTM as Chaotic Neural Oscillatory-Long-Short Term Memory (CNO-LSTM), where neurons in LSTM memory cells are weighed in substitutes by oscillatory neurons to speed up computational training of language model and improve text classification accuracy for real-world applications. From the implementation perspective, five popular datasets of general text classification including binary, multi classification and multi-label classification are used to compare with mainstream baseline models on NLP tasks. Results showed that the performance of CNO-LSTM, a simplified model structure and oscillatory neurons state in exercising different types of text classification tasks are above baseline models in terms of evaluation index such as Accuracy, Precision, Recall and F1. The main contributions are time reduction and improved accuracy. It achieved approximately 46.76% of the highest reduction training time and 2.55% accuracy compared with vanilla LSTM model. Further, it achieved approximately 35.86% in time reduction compared with attention model without oscillatory indicating that the model restructure has reduced GPU dependency to improve training accuracy.

Text classification is the automated technique used to classify text into a predefined category that is more related to text. Most studies and researches have focused on classification texts written in English rather than Arabic because of the Arabic nature and difficulty of its structures. The difficult nature of Arabic makes it more complex and difficult to deal with because of its many rules and characteristics that are unique to it, but it has become necessary to deal with this language because of its widespread use on the internet. Feature representation and extraction, especially for text, have attracted considerable attention in recent years. The main object of this process is to convert text into a numerical representation. All approaches and systems depend on the frequency of words within text, but these approaches are few. This paper presents a new feature extraction method aimed at furthering natural language processing applications in any language, especially Arabic. It is based on transforming a vector of representation in the time domain into the frequency domain. Fourier feature extraction is a powerful and versatile technique. The primary goal of this approach is to extract salient features from raw text, and then a filter is used to remove noise from the extracted features. Its scalability and efficiency make it suitable to be used on large datasets, making it a widely adopted feature extraction technique. For classification, we used logistic regression, which is a powerful tool for classifying text data and offers significant advantages such as speed and accuracy. We used three world datasets (CNN, OSAC, and SANAD), and the results showed a good performance. The proposed method outperformed recent approaches, in which accuracy reached 98%.

Feature extraction is an important factor to improve the recognition rate of coal and gangue. The existing feature extraction methods have some shortcomings in coal and gangue recognition, such as unsatisfactory recognition rate for actual images. Therefore, aiming at the shortcomings of existing coal and gangue feature extraction methods in coal and gangue recognition, the contour feature extraction method is researched based on images analysis to coal and gangue recognition. The center normalization detrended fluctuation analysis (CNDFA) feature extraction algorithm of contour is proposed based on the process of contour feature extraction for coal and gangue. Based on the analysis of the representative features of coal and gangue, the extraction process of target contour features is established based on hardness difference. Combined with the overall features of contour curve after detrending, a center normalized CNDFA feature extraction algorithm is proposed. First, the detrended analysis of contour curve is realized by least square optimal fitting, and then the detrended data are normalized. Finally, the contour features are described quantitatively by multifractal (MF) spectrum to form the geometric features of the target contour curve, which is used to train the support vector machine classifier. The experiment is carried out on the basis of image preprocessing, and the CNDFA method and other feature extraction methods, such as wavelet, gray level co-occurrence matrix, gray level difference statistics, auto-correlation function, and MF, are applied to the contour feature extraction of coal and gangue. Through the comprehensive comparison of the results after different methods recognition in confusion matrix, accuracy, and coal cleanliness, it is concluded that the overall effect of the CNDFA method is better than other methods, and the accuracy is improved by 5% to 25%. The results show that the CNDFA method has better performance. Compared with the other methods, it can better extract the contour features to improve the recognition rate of coal and gangue.

Text extraction is an important part of natural language processing (NLP) tasks. Most NLP tasks like text classification, machine translation, text-to-speech, text-based language identification, text summarization, and named-entity recognition involve the use of textual data. Such data is limited for low-resourced languages making it difficult to experiment advanced NLP techniques on these languages. This paper presents a Python-based toolkit for text analysis and text extraction from different types of images, documents, and audio files. The toolkit is built as a library that has functions that can be imported and utilized for text extraction.

This paper presents a comprehensive investigation into the classification of Sundanese news articles, focusing on the evaluation of various classification models and feature extraction methods. Using a dataset obtained from Sundanese news websites, this study conducts a systematic comparison of Naive Bayes and Logistic Regression classifiers combined with TF-IDF and Bag-of-Words feature extraction methods. The research process involves critical steps such as data preprocessing, model training, hyperparameter optimization, and performance assessment based on standard metrics, including accuracy, precision, recall, and F1-score. Results demonstrate high accuracy across all combinations, with the Logistic Regression model using Bag-of-Words feature extraction achieving the highest accuracy of 98.20%. Beyond model evaluation, the research delves into qualitative data analysis. Word clouds and TF-IDF weighting are employed to uncover prominent themes and topics within the news articles, highlighting recurring patterns in the Sundanese language. The study identifies key challenges, including the scarcity of annotated datasets for low-resource languages like Sundanese and the limitations of traditional models in capturing complex linguistic structures. Future opportunities are highlighted, such as leveraging deep learning models, including transformers, to enhance classification performance and address current limitations. Additionally, ensemble methods and domain-specific adaptations could further improve accuracy. Overall, this research contributes to advancing Sundanese language processing and provides a roadmap for future innovations in text classification and natural language processing applications.

Text classification is a basic task in natural language processing, but the small character perturbations in words can greatly decrease the effectiveness of text classification models, which is called character-level adversarial example attack. There are two main challenges in character-level adversarial examples defense, which are out-of-vocabulary words in word embedding model and the distribution difference between training and inference. Both of these two challenges make the character-level adversarial examples difficult to defend. In this paper, we propose a framework which jointly uses the character embedding and the adversarial stability training to overcome these two challenges. Our experimental results on five text classification data sets show that the models based on our framework can effectively defend character-level adversarial examples, and our models can defend 93.19% gradient-based adversarial examples and 94.83% natural adversarial examples, which outperforms the state-of-the-art defense models.

In this paper, we present the preliminary results on the analysis of deep learning terms used for natural language processing (NLP) tasks. We propose a statistical analysis of papers published from 2012 to 2015 in the main ACL conferences. Our aim is investigating which DL term, and consequently which DL method, is mostly used for each specific NLP task, since its introduction in the field. In order to do this, our first contribution is the development of two terminological lists, referring respectively to DL methods for text analysis and NLP tasks. The list of deep learning terms contains 41 terms and acronyms, as well as a NLP term list contains 145 terms and acronyms. From our corpus, the frequencies of various terms have been extracted with respect to the ACL conference and the publication year. After the preliminary data analysis, we decided to restrict the extraction process to abstract texts. We applied multivariate techniques called correspondence analysis in order to visualize and evaluate the joint behavior of our variables.

Text data is one of the dominating data types in Big Data driven services and applications. The performance of text classification largely depends on the quality of feature extraction over the text corpus. For supervised learning over text documents, the TF-IDF (Term Frequency-Inverse Document Frequency) weighting factor is one of the most frequently used features in text classification. In this paper, we address two known limitations of TF-IDF based feature extraction method: First, the conventional TF-IDF weighting factor lacks of consideration about the synonymous relationship between feature terms. Second, for big corpus with large number of text documents and large number of feature terms, the computational complexity of text classification increases with the dimensionality of the feature space. We address these problems by introducing an optimization technique based on the Inter-Category Distributions (ICD) of terms and the Inter-Category Distributions of documents. We call this new weighting factor TF-IDF-ICD, namely TF-IDF with Inter-Category Distributions. To further enhance the effectiveness of our TF-IDF-ICD method, we describe a TF-IDF-ICD threshold based Dimensionality Reduction (DR) optimization. We test the text classifier with a corpus of 10, 000 articles. The evaluation results show that the proposed TF-IDF-ICD based text classification method outperforms the conventional TF-IDF based classification solution by \(7.84\%\) at only about \(43.19\%\) of the training time used by the conventional TF-IDF based text classification methods.

Background: Text classification techniques are increasingly important with the exponential growth of textual data on the internet. Term Frequency-Inverse Document Frequency (TF-IDF) and Count Vectorizer(CV) are commonly used methods for feature extraction. TF-IDF assigning weights to terms based on their frequency. CV simply counts the occurrences of terms. The performance of CV as well as TF-IDF are evaluated and compared with KNN and DT classifiers across text datasets. Methodology: The investigation begins with preprocessing. The feature vectors are created using both TF-IDF and CV. Feature vectors are passed into the KNN and DT classifiers at in training stage. Experiments are executed the usage of Kaggle's public database Ukraine 10K tweets sentiment_analysis dataset and the Womens ecommerce clothing reviews dataset. Findings: The average of precision, recall, f1 score and accuracy of KNN with TF-IDF were 84.5%, 87%, 83%, 87% respectively and KNN with CV were 83.5%, 87%, 83.5%, 87% respectively. Similarly, average of precision, recall, f1 score and accuracy of DT with TF-IDF were 89%, 89%, 89%, 89% respectively and DT with CV were 89%, 89.5%, 89.5%, 89.5% respectively. The results obtained in this research is consistent with previous similar research result. Conclusions: The performance of TF-IDF is almost similar as CV for a particular dataset and a particular classifier in this study. Novelty: The experiment performed using these classifiers and feature extraction methods on the datasets is a novelty and contribution of this research.

Paraphrase detection allows determining how original and suspect documents convey the same meaning. It has attracted attention from researchers in many Natural Language Processing (NLP) tasks such as plagiarism detection, question answering, information retrieval, etc., Traditional methods (e.g., Term Frequency-Inverse Document Frequency (TF-IDF), Latent Dirichlet Allocation (LDA), and Latent Semantic Analysis (LSA)) cannot capture efficiently hidden semantic relations when sentences may not contain any common words or the co-occurrence of words is rarely present. Therefore, we proposed a deep learning model based on Global Word embedding (GloVe) and Recurrent Convolutional Neural Network (RCNN). It was efficient for capturing more contextual dependencies between words vectors with precise semantic meanings. Seeing the lack of resources in Arabic language publicly available, we developed a paraphrased corpus automatically. It preserved syntactic and semantic structures of Arabic sentences using word2vec model and Part-Of-Speech (POS) annotation. Overall experiments shown that our proposed model outperformed the state-of-the-art methods in terms of precision and recall

This study documents our efforts to provide computer support for the diagnosis of congestive heart failure (CHF). That computer support takes the form of an index value. A high index value indicates a low probability of CHF, and an index value below a threshold of 25.6 suggests a high probability of CHF. To create that index, we have designed a sophisticated algorithm chain which takes electrocardiogram signals as input. The signals are pre-processed before they are sent to a range of nonlinear feature extraction algorithms. The top 10 feature extraction methods were used to create the CHF index. By using objective feature extraction algorithms, we avoid the problem of inter- and intra-observer variability. We observed that the nonlinear feature extraction methods reflect the nature of the human heart very well. That observation is based on the fact that the nonlinear features achieved low [Formula: see text]-values and high feature ranking criterion scores.

The natural language processing (NLP) area has been magically transformed by the Deep Neural Network (DNN). The two variations of Neural Networks, Convolutional Neural Network (CNN) and Recurrent Neural Network (RNN) can handle the different NLP tasks effectively. CNN is based on feature extraction using N-grams at higher levels. The RNN can be effectively used to model the sequential information. Choosing a neural technique for the various NLP tasks is always a challenge. The paper focuses on the evaluating the performance of the two DNN techniques CNN and RNN for the various NLP tasks, so that appropriate technique can be selected.

Text classification is a basic but important task in many natural language processing tasks. Nowadays, the mainstream classification methods mostly use deep learning technology, which shows better accuracy and stability in English text classification. Different from English text, Chinese text classification task involves the granularity of feature description in text decomposition. The two commonly used feature granularity are word-level feature and character-level feature. The former will bring semantic loss in the process of word segmentation, while the latter can't use the advanced semantic feature in the pre-trained word vector. We propose a method to fuse the word-level and the character-level information with attention mechanism. We train the CWC-Net, which combines the features to make the embedded information of characters and words complementary, so as to improve the semantic understanding ability of the network for Chinese text and reduce semantic loss. The comparative experiments on four Chinese text datasets, which involving topic classification and emotion analysis show that our model is more accurate than the traditional model which only relies on word-level features or character-level features. That verifies the effectiveness of the fusion of word-level features and character-level features on the improvement of model capability.

The advances in deep learning have led to great achievements in many Natural Language Processing (NLP) tasks. With the nature of language, i.e., sequential data, most NLP tasks can be framed into the sequence learning framework, such as text generation. As one of the most important foundations for modern NLP techniques, autoregressive generation models have achieved dominant performance in a great deal of NLP tasks. Therefore, this thesis emphasizes improving the autoregressive generation model for different NLP tasks. While many tasks can naturally fit into the sequence learning framework, some of them, e.g., building discourse parsing tree, require sophisticated designs to fit into neural models. Therefore, this thesis firstly emphasizes a novel unified framework for discourse parsing, which builds a discourse tree in a top-down depth-first manner, and it frames the task as an autoregressive generation task with the goal of each step being the prediction of the node position given a piece of text. The proposed approach is proven effective with extensive empirical experiments. In addition, I extend the above framework by proposing a hierarchical decoder, which leverages the information from parents and siblings of the nodes that are currently processed. The proposed decoder utilizes the nature of the tree structure and further improves the experiment performance on both discourse parsing and dependency parsing tasks. On the other hand, the de facto strategies, i.e., cross entropy loss and teacher forcing, for training the autoregressive generation models have been shown problematic in certain aspects. For example, cross entropy loss, which is one of the widely leveraged training objective functions, often leads to text degeneration in text generation, and teacher forcing suffers from the exposure bias problem, where there exists a mismatch between the training and testing setup. For text degeneration, I introduce a class of diminishing attentions, which enforces the submodularity of the coverage calculated by cross attention in the sequence-to-sequence model. The proposed diminishing attentions achieve notable improvement on several neural text generation tasks, including text summarization, machine translation, and image paragraph generation. Further, I propose a novel training objective, ScaleGrad, to replace cross entropy, which significantly reduces the degeneration problem in different text generation tasks. In fact, ScaleGrad can be extended to problems beyond text degeneration. It provides wide flexibility to inject different inductive biases into the text generation model by directly modifying the gradient information in the output layer. Next, for the exposure bias problem, this thesis introduces a novel type of scheduled sampling based on training accuracy, which requires only minimal hyper-parameter tuning compared to existing scheduled sampling methods. Additionally, a novel imitation loss is proposed to further enforce the model’s generative behavior to match the teacher-forced behavior. Moreover, this thesis demonstrates that reducing exposure bias can improve the robustness of language models against repetition and toxic errors.