Improving Probabilistic Models In Text Classification Via Active Learning

Semantic text classification: A survey of past and recent advances

Natural Language Processing (NLP) is a science that integrates computer knowledge, mathematical knowledge and linguistic knowledge, and text classification and recognition is considered to be an important research field and direction of natural language processing. This paper mainly studies the realization of text classification model through the processing method of text data in natural language processing and the theoretical knowledge and technical means of machine learning. We summarize the existing text classification algorithms, analyze their applicable scenarios, and optimize on the basis of these algorithm models. The paper proposes a Chinese text classification algorithm based on weight preprocessing. Algorithm based on the weight preprocessing link, so that the optimized classifier model can improve the accuracy of the existing text classifier. In this paper, the English Newsgroups corpus is used for experimental verification. The experimental results show that the classification accuracy and accuracy of the improved algorithm are better than those of the traditional text classification algorithm, thereby improving the accuracy of English text classification.

In the past fifteen years, the grow of using Bag of Words (BoW) method in the field of computer vision is visibly observed. In addition,-for the text classification and texture recognition, it can also be used in classification of images, videos, robot localization, etc. It is one of the most common methods for the categorization of text and objects. In text classification, the BoW method records the number of occurrences of each bag that is created for each instance type or word disregarding the order of the words or the grammar. And in visual scene classification it is based on clusters of local descriptors which are taken from the images disregarding the order of the clusters. The key idea is to generate a histogram for the words in the documents or the features in the images to represent the specified document or image. The BoW method is computationally and even conceptually is simpler than many other classification methods. For that reason, BoW based systems could record new and higher performance scores on common used benchmarks of text and image classification algorithms. This paper presents an overview of BoW, importance of BoW, how does it work, applications and challenges of using BoW. This study is useful in terms of introducing the BoW method to the new researchers and providing a good background with associated related works to the researchers that are working on the model.

The traditional methods of English text classification have two disadvantages. One is that they cannot fully represent the semantic information of the text. The other is that they cannot fully extract and integrate the global and local information of the text. Therefore, we propose a multi-feature fusion model based on long and short term memory network and improved artificial bee colony algorithm for English text classification. In this method, the character-level vector and word-level vector representations of English text are calculated using a pre-training model to obtain a more comprehensive text feature vector representation. Then the multi-head attention mechanism is used to capture the dependencies in the text sequence to improve the semantic understanding of the text. Through feature fusion, the channel features are optimized and the spatial features and time series features are combined to improve the classification performance of the hybrid model. In the stage of network training, the weighted linear combination of maximum Shannon entropy and minimum cross entropy is used as the return degree evaluation function of the bee colony algorithm, and the scale factor is introduced to adjust the solution search strategy of leading bees and following bees, and the improved artificial bee colony algorithm is combined with the classification network to realize the automatic optimization and adjustment of network parameters. Experiments are carried out on public data set. Compared with traditional convolutional neural networks, the classification accuracy of the new model increases by 2% on average, and the accuracy of data set increases by 2.4% at the highest.

Texts classification is an important field that can be used in data mining, information retrieval, machine learning. Documents classification now widely used in different domains, such as mail spam filtering, article indexing, Web searching, and Web page categorization. There are many researches in documents classification for English language, but a few research in Arabic language, while there are large community in the world that uses this language. This paper analyzes the effect of preprocessing, such as tokenization, normalization and removing stop words, stemming using Khoja stemmer, stemming using light stemmer, stemming using Khoja stemmer with tokenization, normalization and removing stop words, and stemming using light stemmer with tokenization, normalization and removing stop words on documents classification. This study uses three classification algorithms, Naïve Bayes (NB), K-Nearest Neighbor (KNN), and Support Vector Machine (SVM), it is applied on online Arabic corpus prepared by Diab Abuaiadh. Rapid Miner tool is used to apply the three classification models. Whereas the condition of the documents before preprocessing is compared with their condition after preprocessing to determine the extent of the effect of preprocessing on documents classification. the results demonstrate variation between document classification before preprocessing and after preprocessing, and difference between the three algorithms in terms of Accuracy, Precision, Recall, and F1-Score, whereas it will be discussed later.

Active Learning is a machine learning and data mining technique that selects the most informative samples for labeling and uses them as training data; it is especially useful when there are large amount of unlabeled data and labeling them is expensive. Recently, batch-mode active learning, where a set of samples are selected concurrently for labeling, based on their collective merit, has attracted a lot of attention. The objective of batch-mode active learning is to select a set of informative samples so that a classifier learned on these samples has good generalization performance on the unlabeled data. Most of the existing batch-mode active learning methodologies try to achieve this by selecting samples based on certain criteria. In this article we propose a novel criterion which achieves good generalization performance of a classifier by specifically selecting a set of query samples that minimize the difference in distribution between the labeled and the unlabeled data, after annotation. We explicitly measure this difference based on all candidate subsets of the unlabeled data and select the best subset. The proposed objective is an NP-hard integer programming optimization problem. We provide two optimization techniques to solve this problem. In the first one, the problem is transformed into a convex quadratic programming problem and in the second method the problem is transformed into a linear programming problem. Our empirical studies using publicly available UCI datasets and two biomedical image databases demonstrate the effectiveness of the proposed approach in comparison with the state-of-the-art batch-mode active learning methods. We also present two extensions of the proposed approach, which incorporate uncertainty of the predicted labels of the unlabeled data and transfer learning in the proposed formulation. In addition, we present a joint optimization framework for performing both transfer and active learning simultaneously unlike the existing approaches of learning in two separate stages, that is, typically, transfer learning followed by active learning. We specifically minimize a common objective of reducing distribution difference between the domain adapted source, the queried and labeled samples and the rest of the unlabeled target domain data. Our empirical studies on two biomedical image databases and on a publicly available 20 Newsgroups dataset show that incorporation of uncertainty information and transfer learning further improves the performance of the proposed active learning based classifier. Our empirical studies also show that the proposed transfer-active method based on the joint optimization framework performs significantly better than a framework which implements transfer and active learning in two separate stages.

Prediction of cause of death from forensic autopsy reports using text classification techniques: A comparative study

Due to the numerous issues or challenges that aren't always within the company's control.Customers became unhappy.Customer complaint is the method by which they convey their dissatisfaction.Due to the rapid advancement of technology and the various convenient channels available for customers to voice their complaints, including email, web, and chatbots, online complaints have experienced exponential growth.As a result, classifying these complaints under the pertinent issue in time became a difficult task.Selecting the appropriate classification model and Fitting it with the proper training and testing ratios is a crucial topic that always faces researchers.This paper implements and compares the performance of six text classification machine learning algorithms used in multiclassification (SVM, KNN, NB, DT, RF, and GB) under two types of sampling (random and stratified) with the use of various data splitting ratios 50:50,80:20, 60:40, 70:30, and 90:10 on a Complaint Dataset.This paper aims to provide a roadmap for researchers working in the text classification field that helps them select the optimum classification model and splitting ratio.The results demonstrate that DT with an accuracy of 99%, F1-measure of 99%, and runtime of 1 second outperformed all other algorithms.And that the most suitable splitting ratio that fits most algorithms and acts as a secure base to work with is 80:20.It also indicates that using stratified sampling in multi-class text classification produces better results than random sampling.

The considerable time and expense required for labeling data has prompted the development of algorithms which maximize the classification accuracy for a given amount of labeling effort. On the one hand, the effort has been to develop the so-called "active learning" algorithms which sequentially choose the patterns to be explicitly labeled so as to realize the maximum information gain from each labeling. On the other hand, the effort has been to develop algorithms that can learn from labeled as well as the more abundant unlabeled data. Proposed in this paper is an algorithm that integrates the benefits of active learning with the benefits of learning from labeled and unlabeled data. Our approach is based on reversing the roles of the labeled and unlabeled data. Specifically, we use a Genetic Algorithm (GA) to iteratively refine the class membership of the unlabeled patterns so that the maximum a posteriori (MAP) based predicted labels of the patterns in the labeled dataset are in agreement with the known labels. This reversal of the role of labeled and unlabeled patterns leads to an implicit class assignment of the unlabeled patterns. For active learning, we use a subset of the GA population to construct multiple MAP classifiers. Points in the input space where there is maximal disagreement amongst these classifiers are then selected for explicit labeling. The learning from labeled and unlabeled data and active learning phases are interlaced and together provide accurate classification while minimizing the labeling effort.

In many human activity recognition systems the size of the unlabeled training data may be significantly large due to expensive human effort required for data annotation. Moreover, the insufficient data collection process from heterogenous sources may cause dissimilarities between training and testing data. To address these limitations, a novel probabilistic approach that combines learning using privileged information (LUPI) and active learning is proposed. A pool-based privileged active learning approach is presented for semi-supervising learning of human activities from multimodal labeled and unlabeled data. Both uncertainty and distance from the decision boundary are used as query inference strategies for selecting an unlabeled observation and querying its label. Experimental results in four publicly available datasets demonstrate that the proposed method can identify complex human activities with high accuracy.

Text classification is a powerful technique for automating assignment of documents to topic hierarchies. Although there are a number of text classification algorithms, most of them are either inefficient or too complex. We present a linear text classification algorithm called SRFW, which is fast, effective and easily used. SRFW obtains relevance factors. For new unlabelled documents, SRFW adopts sum of weights based on relevance factors to obtain the probability that these documents belong to each category and assigns them to categories that have the biggest probability. We have evaluated our algorithm on a subset of Reuters-21578 and 20-newsgroups text collections and compared it against k-NN and SVM. Experimental results show that SRFW is competitive with k nearest neighbor (k-NN) and support vector machines (SVM), while SRFW is much simpler and faster than them.

The advent of internet has lead to colossal development of e-learning frameworks. The efficiency of such systems however relies on the effectiveness and fast content based retrieval approaches. This paper presents a methodology for efficient search and retrieval of lecture videos based on Machine Learning (ML) text classification algorithm. The text transcript is generated exclusively from the audio content extracted from the video lectures. This content is utilized for the summary and keyword extraction which is used for training the ML text classification model. An optimized search is achieved based on the trained ML model. The performance of the system is compared by training the system using Naive Bayes, Support Vector Machine and Logistic Regression algorithms. Performance evaluation was done by precision, recall, F-score and accuracy of the search for each of the classifiers. It is observed that the system trained on Naive Bayes classification algorithm achieved better performance both in terms of time and also with respect to relevancy of the search results.

The paper presents an online system for clustering and classification of texts in the Polish language. It allows running complex workflows of language and machine learning tools. A high throughput and low latency was achieved by an asynchronous style of programming and a usage of message oriented middleware—RabbitMQ. Authors discuss the architecture assumptions, the language processing modelling notation for a workflow definition and the system architecture. Moreover, a sample Single Page Application is presented that clusters uploaded corpora and shows results online.

An enormous amount of clinical free-text information, such as pathology reports, progress reports, clinical notes and discharge summaries have been collected at hospitals and medical care clinics. These data provide an opportunity of developing many useful machine learning applications if the data could be transferred into a learn-able structure with appropriate labels for supervised learning. The annotation of this data has to be performed by qualified clinical experts, hence, limiting the use of this data due to the high cost of annotation. An underutilised technique of machine learning that can label new data called active learning (AL) is a promising candidate to address the high cost of the label the data. AL has been successfully applied to labelling speech recognition and text classification, however, there is a lack of literature investigating its use for clinical purposes. We performed a comparative investigation of various AL techniques using ML and deep learning (DL)-based strategies on three unique biomedical datasets. We investigated random sampling (RS), least confidence (LC), informative diversity and density (IDD), margin and maximum representativeness-diversity (MRD) AL query strategies. Our experiments show that AL has the potential to significantly reducing the cost of manual labelling. Furthermore, pre-labelling performed using AL expediates the labelling process by reducing the time required for labelling.

Active semi-supervised learning with multiple complementary information