BiLSTM-CNN Text Emotion Analysis Based on Self Attention Mechanism and Dense Connection

A hybrid machine learning model for sentiment analysis and satisfaction assessment with Turkish universities using Twitter data

We present a case study as part of a work-in-progress project about multimodal sentiment analysis on historic German plays, taking Emilia Galotti by G. E. Lessing as our initial use case. We analyze the textual version and an audio version (audio book). We focus on ready-to-use sentiment analysis methods: For the textual component, we implement a naive lexicon-based approach and another approach that enhances the lexicon by means of several NLP methods. For the audio analysis, we use the free version of the Vokaturi tool. We compare the results of all approaches and evaluate them against the annotations of a human expert, which serves as a gold standard. For our use case, we can show that audio and text sentiment analysis behave very differently: textual sentiment analysis tends to predict sentiment as rather negative and audio sentiment as rather positive. Compared to the gold standard, the textual sentiment analysis achieves accuracies of 56% while the accuracy for audio sentiment analysis is only 32%. We discuss possible reasons for these mediocre results and give an outlook on further steps we want to pursue in the context of multimodal sentiment analysis on historic plays.

Dense connected convolutional neural network (Dense Net) is a new architecture of deep convolutional neural network, which ensures the maximum information transmission between network layers by establishing the connection relationship between different layers. In the task of text remote supervised relation extraction, aiming at the limitation of existing neural network methods using shallow network to extract features, a deep convolution neural network model with dense connection based on attention mechanism is designed. In this model, the dense connection module and maximum pooling layer composed of 5 layer convolutional neural network are used as the sentence encoder. By combining the lexical, syntactic and semantic features of different levels, it helps the network to learn the features, so as to obtain more abundant semantic information of the input sentence. At the same time, it reduces the gradient vanishing phenomenon of deep neural network, and makes the network more capable of representing natural language Strong. Secondly, the word level attention is introduced to calculate the relevance between two entities and context words, so as to fully capture the semantic information of the entity context in the sentence; then the sentence level attention is constructed on multiple instances to reduce the problem of label error labeling; finally, the dependency inclusion relationship between different relationships is automatically learned through the attention of the relationship level. The average accuracy of the model on the NYT freebase dataset is 83.5%. Experimental results show that the model can effectively use features and improve the accuracy of remote supervised relationship extraction.

With the rapid development of textual information on the web, sentiment analysis is changing to an essential analytic tool rather than an academic endeavor and numerous studies have been carried out in recent years to address this issue. By the emergence of deep learning, deep neural networks have attracted a lot of attention and become mainstream in this field. Despite the remarkable success of deep learning models for sentiment analysis of text, they are in the early steps of development and their potential is yet to be fully explored. Convolutional neural network is one of the deep learning methods that has been surpassed for sentiment analysis but is confronted with some limitations. Firstly, convolutional neural network requires a large number of training data. Secondly, it assumes that all words in a sentence have an equal contribution to the polarity of a sentence. To fill these lacunas, a convolutional neural network equipped with the attention mechanism is proposed in this paper which not only takes advantage of the attention mechanism but also utilizes transfer learning to boost the performance of sentiment analysis. According to the empirical results, our proposed model achieved comparable or even better classification accuracy than the state-of-the-art methods.

Aspect based sentiment analysis (ABSA) is a fine-grained sentiment analysis task, whose main goal is to identify the sentiment polarity of an aspect in a sentence. A sentence may contain many different aspects, each of which may have different sentiment polarities. Based on the current researches in this area, ABSA can be divided into two subtasks: aspect-category sentiment analysis (ACSA) and aspect-term sentiment analysis (ATSA). In the past, more commonly used method is to adopt the time serial algorithm such as Long Short-Term Memory (LSTM) or Recurrent Neural Network (RNN), which usually needs more training time and has complex structures. Moreover, many previous models lack the abilities to effectively learn the internal structure features of sentences. For ABSA, sometimes the sentence structure may significantly affect the final classification results. However, we found the excellent performance of self-attention algorithm and gating mechanism in some other related researches. Therefore, to solve the problems above, we build a new model based on gating mechanism, combined with convolutional neural networks (CNN) and self-attention mechanism. First, we use self-attention to extract the structural feature of the input, and integrate it with the features of the original sentence extracted by CNN. On such basis, we further combine the aspect-category or aspect-term of the input sentence to form the final sentiment feature. Experiments on SemEval datasets show the performance of our models and the effectiveness of the model is proved.

The sentiment analysis of microblog text has always been a challenging research field due to the limited and complex contextual information. However, most of the existing sentiment analysis methods for microblogs focus on classifying the polarity of emotional keywords while ignoring the transition or progressive impact of words in different positions in the Chinese syntactic structure on global sentiment, as well as the utilization of emojis. To this end, we propose the emotion-semantic-enhanced bidirectional long short-term memory (BiLSTM) network with the multi-head attention mechanism model (EBILSTM-MH) for sentiment analysis. This model uses BiLSTM to learn feature representation of input texts, given the word embedding. Subsequently, the attention mechanism is used to assign the attentive weights of each words to the sentiment analysis based on the impact of emojis. The attentive weights can be combined with the output of the hidden layer to obtain the feature representation of posts. Finally, the sentiment polarity of microblog can be obtained through the dense connection layer. The experimental results show the feasibility of our proposed model on microblog sentiment analysis when compared with other baseline models.

Text sentiment analysis is one of the most important tasks in the field of public opinion monitoring, service evaluation and satisfaction analysis under network environments. Compared with the traditional Natural Language Processing analysis tools, convolution neural networks can automatically learn useful features from sentences and improve the performance of the affective analysis model. However, the original convolution neural network model ignores sentence structure information which is very important for text sentiment analysis. In this paper, we add piece-wise pooling to the convolution neural network, which allows the model to obtain the sentence structure. And the main features of different sentences are extracted to analyze the emotional tendencies of the text. At the same time, the user’s feedback involves many different fields, and there is less labeled data. In order to alleviate the sparsity of the data, this paper also uses the generative adversarial network to make common feature extractions, so that the model can obtain the common features associated with emotions in different fields, and improves the model’s Generalization ability with less training data. Experiments on different datasets demonstrate the effectiveness of this method.

Sentiment analysis also referred to as opinion mining, plays a significant role in automating the identification of negative, positive, or neutral sentiments expressed in textual data. The proliferation of social networks, review sites, and blogs has rendered these platforms valuable resources for mining opinions. Sentiment analysis finds applications in various domains and languages, including English and Arabic. However, Arabic presents unique challenges due to its complex morphology characterized by inflectional and derivation patterns. To effectively analyze sentiment in Arabic text, sentiment analysis techniques must account for this intricacy. This paper proposes a model designed using the transformer model and deep learning (DL) techniques. The word embedding is represented by Transformer-based Model for Arabic Language Understanding (ArabBert), and then passed to the AraBERT model. The output of AraBERT is subsequently fed into a Long Short-Term Memory (LSTM) model, followed by feedforward neural networks and an output layer. AraBERT is used to capture rich contextual information and LSTM to enhance sequence modeling and retain long-term dependencies within the text data. We compared the proposed model with machine learning (ML) algorithms and DL algorithms, as well as different vectorization techniques: term frequency-inverse document frequency (TF-IDF), ArabBert, Continuous Bag-of-Words (CBOW), and skipGrams using four Arabic benchmark datasets. Through extensive experimentation and evaluation of Arabic sentiment analysis datasets, we showcase the effectiveness of our approach. The results underscore significant improvements in sentiment analysis accuracy, highlighting the potential of leveraging transformer models for Arabic Sentiment Analysis. The outcomes of this research contribute to advancing Arabic sentiment analysis, enabling more accurate and reliable sentiment analysis in Arabic text. The findings reveal that the proposed framework exhibits exceptional performance in sentiment classification, achieving an impressive accuracy rate of over 97%.

Rain streaks degrade the quality of optical images and seriously affect the effectiveness of subsequent vision-based algorithms. Although the applications of a convolutional neural network (CNN) and self-attention mechanism (SA) in single image deraining have shown great success, there are still unresolved issues regarding the deraining performance and the large computational load. The work in this paper fully coordinates and utilizes the advantages between CNN and SA and proposes a hybrid content-aware deraining network (CAD) to reduce complexity and generate high-quality results. Specifically, we construct the CADBlock, including the content-aware convolution and attention mixer module (CAMM) and the multi-scale double-gated feed-forward module (MDFM). In CAMM, the attention mechanism is used for intricate windows to generate abundant features and simple convolution is used for plain windows to reduce computational costs. In MDFM, multi-scale spatial features are double-gated fused to preserve local detail features and enhance image restoration capabilities. Furthermore, a four-token contextual attention module (FTCA) is introduced to explore the content information among neighbor keys to improve the representation ability. Both qualitative and quantitative validations on synthetic and real-world rain images demonstrate that the proposed CAD can achieve a competitive deraining performance.

Implicit sentiment analysis is an important part of sentiment computing, especially sentiment analysis based on deep learning has become a research hotspot in recent years. In this paper, convolutional neural networks are used to extract text features, combined with long-term and short-term memory network (LSTM) structure to extract context information, and add attention mechanism to the network to construct a new hybrid neural network model to achieve implicit emotion for text analysis. The hybrid neural network model extracts more meaningful sentence semantic and structural hidden features from the word-level and sentence-level hierarchies, respectively, and pays attention to the features with a large emotional contribution rate through the attention mechanism. The classification accuracy rate of the model on the public implicit sentiment dataset has reached 77%. The research of implicit sentiment analysis can improve the effect of text sentiment analysis more comprehensively, and further promote the application of text sentiment analysis in the fields of knowledge embedding, text representation learning, user modeling and natural language.

The media communication effect refers to the influence that various forms of media, such as television, social media, radio, and digital platforms, have on individuals’ thoughts, emotions, behaviors, and attitudes. Understanding how the media affects public opinion, information dissemination, and social norms is essential for those studying communication studies, marketing, politics, and psychology, among other disciplines. This study uses an innovative approach that combines algorithm fusion and multi-model integration to enhance the evaluation process of media communication effects. However, constructing an opinion model effectively is severely restricted by the lack of labeled data. To address this issue, this study proposed a novel approach for public opinion sentiment analysis using multi-model fusion with the moth flame optimizer with refined long short-term memory (MFO-RLSTM) model. To improve the accuracy of sentiment identification, the attention mechanism is also included to concentrate on important emotional features. Sentiment analysis of public opinion is conducted using data from social media platforms like Twitter, Facebook, and Instagram, which collect user-generated content through images, text, and audio. This method improves sentiment feature learning efficiency by utilizing the benefits of multi-models, even while labeled data are scarce. The data were preprocessed using Noise removal and tokenization for obtained data. A convolutional neural network (CNN) is used to extract both local and global features. This fusion of local and global characteristics offers a thorough comprehension of public sentiment. Performance is assessed using F1-score, recall, accuracy, and precision. The findings show that the suggested approach significantly enhances sentiment analysis when applied to media communication.

Aiming at the problem that normal vector representation method cannot fully represent the emotional semantic information contained in the text when dealing with Chinese text sentiment analysis task, a multi-granularity convolutional capsule neural network model is constructed. The input of sentiment analysis model is vector representation of the texts trained through the language model. As there exists the problem that a single language model is not enough to abstract text features, a multi-granularity text vector representation method based on BERT is proposed. The text vector representations with different granularities are input into the improved multi-channel convolutional capsule neural network. The capsule layer can associate the low-level and high-level text features, and it will extract information selectively through the dynamic routing algorithm to construct emotional and semantic features of the whole text. Multiple comparative experiments confirm that the method proposed in this paper will efficiently improve the accuracy of Chinese sentiment analysis.

In the semantic segmentation of remote sensing images, methods based on Convolutional Neural Networks (CNN) and Transformers have been extensively studied. Nevertheless, CNN struggles to capture global context due to its local feature extraction, while Transformer is constrained by the complexity of quadratic calculations. Recently, there has been a great deal of interest in Mamba-based state space models. However, the existing Mamba-based methods do not adequately consider the significance of local information in remote sensing image segmentation tasks. In this paper, a codec style network UMFormer is constructed for the semantic segmentation of remote sensing image. Specifically, UMFormer employs the ResNet18 as the encoder, with the objective of performing a preliminary image feature extraction. Subsequently, a self-attention mechanism is optimized to extract the global information pertaining to the objects of disparate sizes within the context of a multi-scale condition. For fusing the codec feature map information, another attention structure is built to reconstruct the space information and to capture the relative position relationship. Finally, a decoder based on Mamba is designed to effectively model both global and local information. Concurrently, a feature fusion mechanism utilizing feature similarity is devised with the objective of embedding local information into global ones. Numerous experiments on UAVid, Vaihingen and Potsdam datasets have demonstrated that the proposed UMFormer exhibits enhanced accuracy while maintaining an efficient running speed. The code will be freely available at https://github.com/takeyoutime/UMFormer.

Semantics perception and refinement network for aspect-based sentiment analysis

Although deep learning models are widely used in text sentiment analysis, it is a challenging task to extract richer semantic features to improve model performance in corpora with weak label characteristics. This study crawls the agricultural product review of Jingdong e-commerce as a corpus, and proposes a deep learning method based on the characteristics of the corpus for sentiment analysis. The method first uses frequent item mining to construct a sentiment dictionary, and converts weakly labeled data into high-quality corpus through sentiment value calculation. Secondly, Convolutional Neural Network (CNN) and Bidirectional Long Short Term Memory (BiLSTM) are combined in the sentiment analysis model, and the word vectors trained by Glove and Word2vec are imported into the multi-channel neural network, so that the model can learn local and global semantic features in parallel, and embed the attention mechanism in the channel. The experimental results show that the performance of the model considering the characteristics of the corpus is significantly improved, and the MAtt-CNN-BiLSTM model constructed in this paper has the best performance in the experiments under the three datasets.