An improved term weighting scheme for sentiment classification

The performance of text classification can be affected by the choice of appropriate term weighting scheme as well as other parameters. The terminology supervised term weighting scheme has become popular in recent years, as it may provide discriminative representation in vector space for text documents belonging to different classes. A term weighting scheme generally consists of three factors, namely term frequency factor, collection frequency factor, and length normalization factor. The researchers mostly have been focused on developing new collection frequency factors in term weighting studies. However, the term frequency factor has an important role, especially in supervised term weighting. In this study, we extensively analyzed the effects of using different term frequency factors on seven supervised term weighting schemes. While six of these supervised term weighting schemes were applied in the previous studies in the literature, we derived one of them from an existing feature selection method and it was not used as a weighting method before. This analysis is performed using SVM and Roccio classifiers on two widely known benchmark datasets with different characteristics. Experimental results showed that modification of term frequency factor in supervised term weighting schemes increased the performance of almost all weighting schemes. Also, term weighting schemes using square root function-based term frequency factor (SQRT_TF) are more successful than the ones using term frequency (TF) and logarithmic function-based term frequency (LOG_TF) factors. TF term frequency factor seems as the least effective one among three different term frequency factors according to the experimental results and statistical analysis.

Sentiment classification (SC) is an ongoing field of research, which involves computing opinions, sentiments, and the subjectivity of a text. It has recently been proven that imbalanced classification is challenging for the SC research community. Most existing studies assume that the balance between negative and positive samples may not be true in reality. This work describes a method to improve the problem of imbalanced sentiment classification using supervised term weighting schemes and shows how these weighting schemes can improve the performance of sentiment classification with imbalanced data, especially in the domain of multi-class classification. Nonetheless, to obtain the most appropriate term weighting schemes, five term weighting schemes are comparatively studied, namely tf-idf, tf-idf-icf, tf-rf, tf-igm, and sqrt_tf-igm. In addition to comparing several term weightings schemes, this work also compares four supervised machine learning algorithms to obtain an appropriate algorithm, including k-Nearest Neighbor (k-NN), Multinomial Naïve Bayes (MNB), Support Vector Machines (SVM) with linear, and SVM with RBF. After evaluating by F1, the performance of sqrt_tf-igm was superior to all other weighting schemes. Since the overall picture of sqrt_tf-igm returned better results than the tf-idf, tf-idf-icf, and tf-rf methods, with improved scores of F1 at 10.94%. Meanwhile, the result of sqrt_tf-igm was slightly better than tf-igm.

The term weighting scheme, which is used to convert documents into vectors in the term spaces, is a vital step in automatic text categorization. The previous studies showed that term weighting schemes dominate the performance rather than the kernel functions of SVMs for the text categorization task. In this paper, we conducted experiments to compare various term weighting schemes with SVM on two widely-used benchmark data sets. We also presented a new term weighting scheme tf.rf for text categorization. The cross-scheme comparison was performed by using McNemar's tests. The controlled experimental results showed that the newly proposed tf.rf scheme is significantly better than other term weighting schemes. Compared with schemes related with tf factor alone, the idf factor does not improve or even decrease the term's discriminating power for text categorization. The binary and tf.chi representations significantly underperform the other term weighting schemes.

The research on text sentiment classification has been very popular. The main methods include the methods based on rules and lexicons, machine statistical learning, and neural network methods. These methods have achieved great success in text sentiment classification, but there are still many defects. For traditional methods, its versatility is poor, once the text style or format changes significantly, the rules need to be re-established. Although the neural network method is generally applicable, it is not sufficient for the application of language knowledge. In order to solve the above problems, we propose a new model combining the traditional method and neural network method. It combines transition lexicon, stacked BILSTM and attention mechanism, and uses word2vec to initialize word vectors. For sentiment classification, semantic features are necessary, stacked BILSTM can capture contextual semantics multiple times to obtain richer contextual information representation. At the same time, we consider that the polarity of the clauses before and after the transition words may be completely opposite, which will also have a significant impact on the classification results. Therefore, we propose two strategies for transition words. One is to use the transition lexicon to supervise the attention mechanism and make the attention model sensitive to the transition words. The other is to cut the sentence according to the transition words to eliminate the noise of irrelevant items in the sentence. Our method combines the advantages of traditional methods and neural network methods, and makes up for their respective shortcomings. We use the proposed method to conduct a sentimental binary classification experiment on three data sets. The experimental results show that our method is superior than the previous methods.

ABSTRACTWith diverse online media emerging, there is a growing concern of sentiment classification problem. At present, text sentiment classification mainly utilizes supervised machine learning methods, which feature certain domain dependency. On the basis of Markov logic networks (MLNs), this study proposed a cross-domain multi-task text sentiment classification method rooted in transfer learning. Through many-to-one knowledge transfer, labeled text sentiment classification, knowledge was successfully transferred into other domains, and the precision of the sentiment classification analysis in the text tendency domain was improved. The experimental results revealed the following: (1) the model based on a MLN demonstrated higher precision than the single individual learning plan model. (2) Multi-task transfer learning based on Markov logical networks could acquire more knowledge than self-domain learning. The cross-domain text sentiment classification model could significantly improve the precision and efficiency of text sentiment classification.

A novel term weighting scheme for text classification: TF-MONO

A study of supervised term weighting scheme for sentiment analysis

Term weighting strategy plays an essential role in the areas related to text processing such as text categorization and information retrieval. In such systems, term frequency, inverse document frequency, and document length normalization are important factors to be considered when a term weighting strategy is developed. Term length normalization is proposed to give equal opportunities to retrieve both lengthy documents and shorter ones. However, terms in very short documents that may be useless for users, especially in the scenario of Web information retrieval, could be assigned very high weights, resulting in a situation where shorter documents are ranked higher than lengthy documents that are more relevant to users information needs. In this research, a new R-tfidf term weighting strategy is proposed to alleviate the side effects of document length normalization. Experimental results demonstrate the proposed approach can to some extent improve the performance of text categorization.

Nowadays, an abundance of short text is being generated that uses nonstandard writing styles influenced by regional languages. Such informal and code-switched content are under-resourced in terms of labeled datasets and language models even for popular tasks like sentiment classification. In this work, we (1) present a labeled dataset called MultiSenti for sentiment classification of code-switched informal short text, (2) explore the feasibility of adapting resources from a resource-rich language for an informal one, and (3) propose a deep learning-based model for sentiment classification of code-switched informal short text. We aim to achieve this without any lexical normalization, language translation, or code-switching indication. The performance of the proposed models is compared with three existing multilingual sentiment classification models. The results show that the proposed model performs better in general and adapting character-based embeddings yield equivalent performance while being computationally more efficient than training word-based domain-specific embeddings.

With the rapid increase of the number of Internet users and the amount of online comment data, a large number of referable information samples are provided for data mining technology. As a technical application of data mining, text sentiment classification can be widely used in public opinion management, marketing, and other fields. In this study, a combination approach to SVM (support vector machine) and IPSO (improved particle swarm optimization) is proposed to classify sentiment by using text data. First, the text data of 30,000 goods reviews and corresponding ratings are collected through the web crawler. Then, TFIDF (term frequency-inverse document frequency) and Word2vec are used to vectorize the goods review text data. Next, the proposed classification model is trained by the SVM, and the initial parameters of the SVM are optimized by the IPSO. Finally, we applied the trained SVM-IPSO model to the test set and evaluated the performance by several measures. Our experiment results indicate that the proposed model performed the best for text data sentiment classification. Additionally, the traditional machine learning model SVM becomes very effective after parameter optimization, which demonstrates that the parameters’ optimization by IPSO has successfully improved the classification accuracy. Furthermore, our proposed model SVM-IPSO significantly outperforms other benchmark models, indicating that it could be applied to improve the accuracy and efficiency for text data sentiment classification.

Text sentiment classification is an important domain in NLP, and the related technical research has been mature. The sentiment classification of text with the but contrastive marker is a challenging problem. In this paper, a semi-supervised framework based on conditional variational autoencoder using attention, called CVAE-Attention, is proposed for sentiment classification. In the CVAE-Attention framework, the attention mechanism is introduced to cope with the contrastive structure. The latent semantic information of the clause after but (but-clause) is extracted through the attention model, and is incorporated into the generative model to enlarge the effect of the but-clause. Experiments show that the proposed method is effective compared with other state-of-the-art semi-supervised methods.

With the development of modern information technology, social networks have become an im-portant platform for people to express, and at the same time, a large number of texts have been produced. However, the comment text has the characteristics of randomness and colloquialism in the way of expression, and also contains a lot of non-text data. Therefore, manually analyzing the emotional information in the text will consume a lot of time and the accuracy will be limited. To solve emotion classification, this study proposes a knowledge enhanced double loop emotion classification neural network model with attention mechanism. The study first preprocesses text data using a full sentence vector word vector model, then uses convolutional neural networks to recognize emotions in emoticons and emoticons in the text. Finally, the classification results are integrated using algorithms such as a dual loop sentiment classification neural network with pool-ing layers and attention mechanisms, K-nearest neighbors, and decision trees. The final compre-hensive expression recognition and text recognition results are used to obtain the text sentiment classification results. The experimental data shows that the model proposed in this study has an accuracy of 0.947 in the training set test, which is significantly better than other models. In da-tasets A and B, the accuracy of the research design model was 0.958 and 0.924, and the recall was 0.964 and 0.986, respectively. Compared to the baseline method or existing research models, the values of each indicator were significantly higher. The recall rate is the proportion of instances correctly identified as positive by the model to all actual positive instances, which can reflect the emotional classification performance of the research and design model. The higher the value, the better the performance of the model. In practical applications, the positive review rate of this model is above 0.9, which has obvious advantages compared to other models. This study utilizes deep learning techniques to classify sentiment in comment texts, providing reference for the field of text sentiment classification. In the e-commerce industry, it is possible to identify the emotions in user comments on products, further understand the product situation on the platform, and make targeted planning for product reserves, specifications, and so on.

The category imbalance of data in text sentiment classification is a widely existent phenomenon, and it is a serious challenge for designing an effective classifier. In this paper, we propose a two-stage data balancing scheme for text sentiment classification, which not only can make the data boundary clear, but also can balance the class distribution of training data set. The core algorithm LDMRC of the scheme is proposed based on the shortest distance from a point to a straight line, to remove some majority class texts in the neighborhood of a minority class text for balancing the class distribution of data in the local dense mixed region. The second stage employs SS or RS as a data rebalancing strategy to globally balance the training dataset after local dense mixed region cutting. The proposed two-stage data balancing scheme is used by situating at the front of a learning algorithm such as SVM. Using the machine learning algorithm SVM on eight imbalanced data sets including Book_c, Hotel, Jadeite, Insurance in Chinese, and DVD, Book_e, Electronics, Kitchen in English, we verify the effectiveness of the proposed method. The experimental results show that LDMRC is superior to the best existing cutting algorithm BRC for Acc, RN and FN. Furthermore, LDMRC+SS and LDMRC+RS are superior to the corresponding method LDMRC on Chinese datasets. This indicates that alone use of local boundary cutting cannot obtain the best effect, and data rebalancing strategies are necessary for text sentiment classification.

Traditional word vector generation model cannot solve the polysemy problem of word representation in Twitter text, so, a method is presented here, firstly, by using Bidirectional Encoder Representations from Transformers (BERT), the semantic feature vector of the text can be obtained, and then, the feature vector is inputted into the Softmax classifier to implement the sentiment classification of Twitter text. The experimental datasets are source from passengers’ Twitter comments of USA six airlines, the sentiment classification model based on Embeddings from Language Model (ELMo) as the experimental control group. The experimental results indicate that the proposed model is advantage over the experimental control group by using F1-score as the evaluation index.

Sentiment analysis mainly studies the emotional tendencies of texts from grammar, semantic rules and other aspects. The texts from social network are characterized by less words, irregular grammar, data noise and so on, which have increased the difficulty of emotion analysis. In order to improve the performance of machine learning in sentiment analysis, this study proposed the Majority Decision Algorithm to classify the emotional tendentious of the text in WeChat, combined the characteristics of five classifiers and integrated the classification results of five classifiers, eventually the text can be classified in WeChat. Firstly, this study utilized the BlueStacks to crawl the cache of WeChat Moment developed by Tencent company. Secondly, the cache was processed by Python to get the WeChat dataset. After the Chinese word segmentation, data cleaning and segmentation, the sentiment classification experiment were carried out using different classifiers. Finally, a Majority Decision Algorithm composed of five classifiers was established. It included, Naive Bayes (sklearn), Naive Bayes (SnowNLP), SVM (linear), SVM (RBF) and SGD. Then, the comparison was carried out between the performance of the algorithm and the five classifiers. Results show that the precision rates of the five classifiers are 0.8598, 0.8154, 0.8511, 0.8739 and 0.8678; the recall rates are 0.8544, 0.8482, 0.9380, 0.9226 and 0.9349; F1 scores are 0.8571, 0.8315, 0.8924, 0.8975 and 0.9001, respectively. The algorithm of the Precision rate, Recall rate and F1 score were 0.8804, 0.9349 and 0.9069, respectively, indicating that algorithm in current study significantly improved the performance, which can be effectively applied into the new text form of WeChat Moment. The study can provide theoretical reference for sentiment classification of Chinese text based on machine learning.