Aspect Level Sentiment Classification with Memory Network Using Word Sentiment Vectors and a New Attention Mechanism AM-PPOSC

Aspect level sentiment classification aims to recognize the sentiment polarity of each aspect term in a sentence. However, most of the existing methods usually applied the attention mechanism over position-weighted memory and did not consider inter-aspect information. To address these issues, we propose a novel framework for aspect level sentiment classification, Deep Selective Memory Network (DSMN), which selects the context memory dynamically for better guiding the multi-hop attention mechanism and integrates inter-aspect information with deep memory network. By designing a selective attention mechanism based on the distance information between an aspect and its context, DSMN focuses on different parts of the context memory in different memory network layers to capture abundant aspect-aware context information. Besides, to make full use of the inter-aspect information, we also design effective inter-aspect modeling modules to generate both semantic and relation information of the nearby aspects for the desired aspect. We evaluate the advantages of our framework on three benchmark datasets, and experiment results show that our framework achieves state-of-the-art performance.

Aspect-level sentiment classification aims to determine sentiment polarities of various aspects in reviews, where each review typically contains multiple aspects, that may correspond to different polarities. Aspect-level sentiment classification, unlike document-level sentiment classification, requires different context representations for different aspects. Existing methods normally use Long Short-Term Memory (LSTM) network to model aspects and contexts separately, and they combine attention mechanisms to extract features of a specific aspect in its context. Attention mechanisms are not used for sequence modeling, so aspects are not considered when generating context sequence representations. This study proposes a novel aspect-context interactive representation structure that only relies on an attention mechanism for generating sequence-to-sequence representations in both context and aspect. It is capable of extracting features related to the specific aspect during the process of its context sequence modeling, and generating a high quality aspect representation simultaneously. We have conducted comprehensive experiments to compare with thirteen existing methods. Our experimental results show that the proposed model is able to achieve significantly better performance on Restaurant dataset, as well as very competitive results on Laptop and Twitter datasets.

Aspect-level sentiment classification is a fine-grained sentiment analysis task, which aims to predict the sentiment of a text in different aspects. One key point of this task is to allocate the appropriate sentiment words for the given aspect.Recent work exploits attention neural networks to allocate sentiment words and achieves the state-of-the-art performance. However, the prior work only attends to the sentiment information and ignores the aspect-related information in the text, which may cause mismatching between the sentiment words and the aspects when an unrelated sentiment word is semantically meaningful for the given aspect. To solve this problem, we propose a HiErarchical ATtention (HEAT) network for aspect-level sentiment classification. The HEAT network contains a hierarchical attention module, consisting of aspect attention and sentiment attention. The aspect attention extracts the aspect-related information to guide the sentiment attention to better allocate aspect-specific sentiment words of the text. Moreover, the HEAT network supports to extract the aspect terms together with aspect-level sentiment classification by introducing the Bernoulli attention mechanism. To verify the proposed method, we conduct experiments on restaurant and laptop review data sets from SemEval at both the sentence level and the review level. The experimental results show that our model better allocates appropriate sentiment expressions for a given aspect benefiting from the guidance of aspect terms. Moreover, our method achieves better performance on aspect-level sentiment classification than state-of-the-art models.

An algorithm for automatic speech prominence detection is reported in this paper. We describe a comparative analysis on various acoustic features for word prominence detection and report results using a spoken dialog corpus with manually assigned prominence labels. The focus is on features such as spectral intensity and speech rate that are directly extracted from speech based on a correlation-based approach without requiring explicit linguistic or phonetic knowledge. Additionally, various pitch-based measures are studied with respect to their discriminating ability for prominence detection. A parametric scheme for modeling pitch plateau is proposed and this feature alone is found to outperform the traditional local pitch statistics. Two sets of experiments are used to explore the usefulness of the acoustic score generated using these features. The first set focuses on a more traditional way of word prominence detection based on a manually-tagged corpus. A 76.8% classification accuracy was achieved on a corpus of role-playing spoken dialogs. Due to difficulties in manually tagging speech prominence into discrete levels (categories), the second set of experiments focuses on evaluating the score indirectly. Specifically, through experiments on the Switchboard corpus, it is shown that the proposed acoustic score can discriminate between content word and function words in a statistically significant way. The relation between speech prominence and content/function words is also explored. Since prominent words tend to be predominantly content words, and since content words can be automatically marked from text-derived part of speech (POS) information, it is shown that the proposed acoustic score can be indirectly cross-validated through POS information.

The domain of Aspect Level Sentiment Classification, in which the sentiment toward a given aspect is analyzed, attracts much attention in NLP. Recently, the state-of-the-art Aspect Level Sentiment Classification methods are devised by using the Graph Convolutional Networks to deal with both the semantics and the syntax of the sentence. Generally, the parsing of syntactic structure inevitably incorporates irrelevant information toward the aspect. Besides, the syntactic and semantic alignment and uniformity that contribute to the sentiment delivery is currently neglected during processing. In this work, a Triplet Contrastive Learning Network is developed to coordinate the syntactic information and the semantic information. To start with, the aspect-oriented sub-tree is constructed to replace the syntactic adjacency matrix. Further, a sentence-level contrastive learning scheme is proposed to highlight the features of sentiment words. Based on The Triple Contrastive Learning, the syntactic information and the semantic information are thoroughly interacted and coordinated whilst the global semantics and syntax can be exploited. Extensive experiments are performed on three benchmark datasets and achieve accuracies (BERT-based) of 87.40, 82.80, 77.55 on Rest14, Lap14, and Twitter datasets, which demonstrate that our approach achieves state-of-the-art results in Aspect Level Sentiment Classification task.

Aspect-based sentiment analysis (ABSA) aims to analyze the sentiment polarity of an input sentence in a certain aspect. Many existing methods of ABSA employ long short-term memory (LSTM) networks and attention mechanism. However, the attention mechanism only models the local certain dependencies of the input information, which fails to capture the global dependence of the inputs. Simply improving the attention mechanism fails to solve the issue of target-sensitive sentiment expression, which has been proven to degrade the prediction effectiveness. In this work, we propose the multi-head self-attention transformation (MSAT) networks for ABSA tasks, which conducts more effective sentiment analysis with target specific self-attention and dynamic target representation. Given a set of review sentences, MSAT applies multi-head target specific self-attention to better capture the global dependence and introduces target-sensitive transformation to effectively tackle the problem of target-sensitive sentiment at first. Second, the part-of-speech (POS) features are integrated into MSAT to capture the grammatical features of sentences. A series of experiments carried on the SemEval 2014 and Twitter datasets show that the proposed model achieves better effectiveness compared with several state-of-the-art methods.

Multimodal architecture for video captioning with memory networks and an attention mechanism

Aspect-level sentiment classification (ASC) is a fine-grained sentiment analysis task that involves detecting the sentiment polarity of a specific opinion target in a given sentence. Despite the popularity of deep learning methods, the limited availability of ASC training data has resulted in the suboptimal performance of neural network models. To address this issue, researchers have proposed transferring resource-rich document-level sentiment knowledge to ASC tasks using model transfer and shared parameters. However, these methods have neglected the impact of differences in the training data for different tasks on the shared layer parameters. Therefore, this paper proposes a transfer method that automatically adapts to task and domain differences—Auto-adaptive Model Transfer (AAMT). This method considers adding specific disturbance variables on shared layer parameters to subdivide task differences to further obtain more accurate and effective transfer knowledge. In addition, on this basis, we introduce an attention mechanism to combine aspect and sentence features to better capture the sentiment vocabulary associated with a given target. A series of experiments on three public datasets show that our proposed method can solve the problem of insufficient training data for ASC and achieve good results.

The research on part of speech (POS) tagging has been widely applied and used through a variety of approaches, particularly for European languages. But it is more challenging for Asian languages, especially Malay as it has some element of modification from other languages such as English and Arabic. Among the issues that often occur in POS tagging are the existence of ambiguous words and unknown words. Meanwhile, the lack of rules in the existing work has become a major problem in Malay POS tagging. Therefore, this research aims to develop new rules for Malay POS tagging and to compare the performance of this new development with the existing gold standard. This process begins with the collection and selection of the corpus using secondary data, obtained from online daily news which covers several domains. Next, the corpus has gone through the process of pre-processing in raw text of article form which include sentence splitter and tokenization process to generate an unlabeled corpus. POS tag dictionary also has been constructed to form a lexicon that only consists of root words. The rule development process involves detailing every type of POS tag to its suitable rules and get the best rules ordering for each type of this POS. A total of 30 rules including affixation rules and 16 word type relations have been developed in this process. The evaluation process is used to test the precision of the developed POS tagger and to get the best rules ordering. The POS tagging result is compared with existing gold standard. Overall, the test showed good result with an accuracy of 93.06% compared to the gold standard performance of 77.17%. Hence, this research showed better accuracy compared with the gold standard and at the same time, it proves that the addition of a new rules and rules ordering among the factors that contributed to the higher precision in tagging Malay corpus. As an improvement in future studies, the use of compound words should be taken into account because most of these words are used in most news sources. In addition, corpus from social media sources can be used because the content of information disseminated through social media is fast and up-to-date even though the language used for this resource is mostly informal and confronts with noise data issues.

Accurately predicting the remaining useful life (RUL) of lithium-ion batteries is crucial for ensuring the safe and reliable operation of battery systems. However, due to the highly nonlinear degradation behavior of lithium-ion batteries under complex working conditions, traditional prediction methods often fail to achieve satisfactory accuracy. This study proposes a hybrid prediction model that integrates an attention (ATT) mechanism, a convolutional neural network (CNN), and a bidirectional long short-term memory (BiLSTM) network, termed ATT-CNN-BiLSTM. First, a one-dimensional CNN is employed to extract local degradation patterns and inter-channel correlated features from multi-dimensional sensor time-series data. Then, the BiLSTM network captures long-term dependencies within the degradation process. Furthermore, an attention mechanism is introduced to dynamically adjust the contribution of key time-step features and suppress noise interference. Experimental results based on the NASA battery aging datasets demonstrate that the proposed ATT-CNN-BiLSTM model reduces RMSE and MAE by 49.08% and 54.67%, respectively, compared with the LSTM model, and by 10.89% and 17.85%, respectively, compared with the ACNN-LSTM-MMD model. Ablation experiments also confirm that further confirm that the ATT-CNN-BiLSTM model decreases RMSE and MAE by 26.45% and 33.33%, respectively, compared with CNN-BiLSTM, and by 21.85% and 23.36%, respectively, compared with ATT-BiLSTM.

The Impact of Arabic Part of Speech Tagging on Sentiment Analysis: A New Corpus and Deep Learning Approach

To solve the problem that offshore wind power generation is susceptible to extreme weather conditions that result in low accuracy of monitoring data, this work proposes a method for predicting offshore wind power generation output by combining convolution and attention mechanisms. The method combines bidirectional long and short-term memory (BiLSTM) network with attention mechanism (AM). The input data are first weighted with the AM for reducing the predictive weight of the interference data, and then the attention mechanism-weighted data are fed into the BiLSTM network. The bi-directional propagation neural network can effectively utilize all the input information, resulting in higher prediction accuracy. The method of combining BiLSTM network with AM is compared with the method employing long and short-term memory network, gated recurrent unit, and bidirectional long and short-term memory network alone through simulation. The mean square error (MSE) of the BiLSTM combined with AM method is 41.28% smaller than the MSE of the best LSTM method among the compared methods and 58.43% smaller than the average of the compared methods. The R2-R-Square is 14.21% larger than the R2-R-Square of the LSTM and 32.91% larger than the average of the compared methods. The results show that the proposed method of combining convolution and attention mechanism for offshore wind power generation output prediction has higher prediction accuracy.

After an examination of a Chinese-English bilingual corpus with 2239 sentence pairs, a new definition of Chinese noun phrase (NP), quasi-equivalent noun phrase (equNP), is proposed with a goal of translation from Chinese NPs to English NPs. Firstly, all the equNPs in the corpus are tagged manually according to the definition in this paper. A set of part of speech (POS) templates for equNP is automatically acquired. Secondly, all the possible equNPs in a sentence are identified using the templates. These equNPs are the candidates for equNP identification. Finally, a classification process and a chunking process are carried out. In classification process, the correct equNPs are chosen from the candidates set using a maximum entropy classifier which combined POS, syntactic and semantic information. In chunking process, the equNPs in the sentence are finally chosen. On open test set, the precision is 83.75% and recall is 86.50%.

Current temperature prediction methods often focus on time-series information while neglecting the contributions of different meteorological factors and the context of varying time steps. Accordingly, this study developed a Dual-Attention-BiLSTM (a bidirectional long short-term memory network with dual attention mechanisms) network model, which integrates a bidirectional long short-term memory (BiLSTM) network model with random forest-based feature selection and two self-designed attention mechanisms. A sensitivity analysis was conducted to evaluate the influence of the attention mechanisms. This study focuses on Shijiazhuang City, China, which has a temperate continental monsoon climate with significant seasonal and daily variations. The data were sourced from ERA5-Land, comprising hourly near-surface temperature and related meteorological variables for the year of 2022. The results indicate that integrating the two attention mechanisms significantly improves the model’s prediction performance compared to using BiLSTM alone. The mean absolute error between simulation results ranges from 0.80 °C to 1.08 °C, with a reduction of 0.17 °C to 0.39 °C, and the root mean square error ranges from 1.17 °C to 1.37 °C, with a reduction of 0.12 °C to 0.22 °C.

BackgroundAntimicrobial peptides (AMPs) are short peptides with diverse biological activities and playing a crucial role in various biological processes. Due to the widespread misuse of traditional antibiotics and the increasing resistance of microorganisms to these drugs, AMPs have emerged as a promising alternative. Consequently, the identification of AMPs has garnered significant research interest. Numerous computational methods based on machine learning algorithms have been developed to facilitate AMP recognition. However, some existing AMPs recognition models only focus on binary classification tasks or only identify the functional activity of a limited number of AMPs categories in multi-class classification tasks. To address this limitation, this study proposes a two-stage AMPs recognition model, iAMP-SeE.MethodsThe iAMP-SeE model extracts features from protein sequences using ESM2, employs a Convolutional Neural Network (CNN) module to capture local patterns from ESM features and utilizes a Bidirectional Long Short-Term Memory (BiLSTM) network to capture long-term dependencies. Furthermore, it incorporates Squeeze-and-Excitation (SE) and Efficient Channel Attention (ECA) mechanisms, which focus on global and local channel relationships, respectively. These two attention mechanisms are complementary, as they enhance features across various dimensions and granularities while simultaneously suppressing irrelevant or redundant features, thereby boosting the model’s performance. Additionally, to address the issue of imbalanced datasets, the Synthetic Minority Over-sampling Technique (SMOTE) is incorporated into the multi-classification task. This method balances the number of AMP categories and ensures that minority classes are not overlooked during model training.ResultsEvaluation across a range of classification thresholds demonstrated the stability of the model’s performance metrics in both binary and multi-class tasks. Furthermore, comparative experiments with existing AMP recognition models confirmed the superior performance of iAMP-SeE.ConclusionsRigorous experimental comparisons and ablation studies demonstrate the effectiveness of iAMP-SeE for both binary and multi-class AMP classification tasks. The source code is publicly available at: https://github.com/cqw0715/iAMP-SeE.git.