ECHANT: An efficient context hierarchical attention network for named entity recognition

ST-SHAP: A hierarchical and explainable attention network for emotional EEG representation learning and decoding

This paper presents our system submitted to the EmotionX challenge. It is an emotion detection task on dialogues in the EmotionLines dataset. We formulate this as a hierarchical network where network learns data representation at both utterance level and dialogue level. Our model is inspired by Hierarchical Attention network (HAN) and uses pre-trained word embeddings as features. We formulate emotion detection in dialogues as a sequence labeling problem to capture the dependencies among labels. We report the performance accuracy for four emotions (anger, joy, neutral and sadness). The model achieved unweighted accuracy of 55.38% on Friends test dataset and 56.73% on EmotionPush test dataset. We report an improvement of 22.51% in Friends dataset and 36.04% in EmotionPush dataset over baseline results.

The authors present a novel approach to the design of a class of hierarchical interconnection networks for multiprocessor systems. This approach, based on an architecture providing separate networks for each level, gives a general and flexible way to construct efficient hierarchical networks. The performance and cost-effectiveness of the resulting networks are analyzed and compared in detail, using both unbuffered and buffered network models. It is shown that, if the design parameters are determined based on the degree of locality, the cost-effectiveness of a hierarchical network can be significantly improved. In addition, the authors investigate how to construct a cost-effectiveness hierarchical network by determining appropriate design parameters. Two associated algorithms are developed for this purpose.

Big data and its classification have been the recent challenge in the evolving world. Data evolving needs to be classified in an effective way. For the classifying process, deep learning and machine learning models are evolved. Hierarchical Attention Network (HAN) is one of the most dominant neural network structures in classification. The major demerits which the HAN is facing are, high computation time and numerous layers. The drawback of HAN is vanquished by a new idea arrived from the mining methods that yield mixed attention network for android data classification. By this flow it could handle more complex request apart from the concept identified. The EHAN (Enhanced Hierarchical Attention Network) has two prototypes. The first one is the attention model to distinguish the features and the second one is the self-attention model to identify worldwide facts. By the demonstrated outcome, it is inferring that the partitioning of task is constructive and therefore the EHAN features shows a significant growth on the news database. In addition to this paper we could add other subnetworks subsequently to assist this ability further.

Aspect-level sentiment classification is a fine-grained sentiment analysis task, which aims to predict the sentiment of a text in different aspects. One key point of this task is to allocate the appropriate sentiment words for the given aspect.Recent work exploits attention neural networks to allocate sentiment words and achieves the state-of-the-art performance. However, the prior work only attends to the sentiment information and ignores the aspect-related information in the text, which may cause mismatching between the sentiment words and the aspects when an unrelated sentiment word is semantically meaningful for the given aspect. To solve this problem, we propose a HiErarchical ATtention (HEAT) network for aspect-level sentiment classification. The HEAT network contains a hierarchical attention module, consisting of aspect attention and sentiment attention. The aspect attention extracts the aspect-related information to guide the sentiment attention to better allocate aspect-specific sentiment words of the text. Moreover, the HEAT network supports to extract the aspect terms together with aspect-level sentiment classification by introducing the Bernoulli attention mechanism. To verify the proposed method, we conduct experiments on restaurant and laptop review data sets from SemEval at both the sentence level and the review level. The experimental results show that our model better allocates appropriate sentiment expressions for a given aspect benefiting from the guidance of aspect terms. Moreover, our method achieves better performance on aspect-level sentiment classification than state-of-the-art models.

Recipe recommendation systems play an important role in helping people find recipes that are of their interest and fit their eating habits. Unlike what has been developed for recommending recipes using content-based or collaborative filtering approaches, the relational information among users, recipes, and food items is less explored. In this paper, we leverage the relational information into recipe recommendation and propose a graph learning approach to solve it. In particular, we propose HGAT, a novel hierarchical graph attention network for recipe recommendation. The proposed model can capture user history behavior, recipe content, and relational information through several neural network modules, including type-specific transformation, node-level attention, and relation-level attention. We further introduce a ranking-based objective function to optimize the model. Thorough experiments demonstrate that HGAT outperforms numerous baseline methods.

Legal documents are often lengthy and complex, making it challenging and time-consuming for experts to accurately predict case outcomes. Older methods are not well-suited to the structure and language used in legal texts. This paper proposes a model called LegalHANOpt (Legal Hierarchical Attention Network with Optimized Parameters) to make accurate predictions about legal case decisions and explain how those decisions are made. LegalHANOpt utilizes a Hierarchical Attention Network (HAN) that analyzes legal documents by examining individual words and sentences, much like lawyers typically do. To improve the model's performance, Bayesian Optimization with Hyperband (BOHB) is utilized. This sophisticated method automatically determines the optimal settings for training the model, such as learning rate and dropout. The LegalHANOpt is trained on an extensive collection of past legal cases, including relevant facts, laws, and decisions. Results show that LegalHANOpt gives more accurate predictions than older methods achieving superior performance with an accuracy of 0.91%, macro F1-score of 0.83%, and AUC-ROC of 0.92%. It also highlights essential parts of the text, helping users understand why the model made a particular decision. In short, LegalHANOpt is a valuable and explainable tool to support legal experts in making better and faster decisions.

Recently the diagnosis of respiratory diseases based on respiratory-sound data has drawn much attention. However, no quantitative assessment method exists to detect abnormal respiratory sounds when making a diagnosis. Although existing studies have attempted to support doctors by providing them with machine learning results, there were limitations on explaining causal symptoms and establishing quantitative assessment. Therefore, a reliable method that can diagnose and explain the causal symptoms based on respiratory sound data is required. In this study, we propose using a hierarchical attention network for detecting abnormal respiratory sounds. The hierarchical attention network reflects hierarchical patterns of respiratory sounds and allows us to interpret the important feature of respiratory sounds. The experimental results showed that hierarchical attention network performed better than other existing methods in terms of sensitivity (ability of correctly detecting abnormal respiratory sounds) and explainability. We believe that the framework presented in this study can not only serve as a “second opinion” that can help doctors diagnose existing respiratory diseases, but also help doctors’ future research on unidentified respiratory diseases.

Hierarchical Multi-scale Attention Networks for action recognition

Document classification is one of the most critical steps in the document analysis pipeline. There are two types of approaches for document classification, known as image-based and multimodal approaches. Image-based document classification approaches are solely based on the inherent visual cues of the document images. In contrast, the multimodal approach co-learns the visual and textual features, and it has proved to be more effective. Nonetheless, these approaches require a huge amount of data. This paper presents a novel approach for document classification that works with a small amount of data and outperforms other approaches. The proposed approach incorporates a hierarchical attention network (HAN) for the textual stream and the EfficientNet-B0 for the image stream. The hierarchical attention network in the textual stream uses dynamic word embedding through fine-tuned BERT. HAN incorporates both the word level and sentence level features. While earlier approaches rely on training on a large corpus (RVL-CDIP), we show that our approach works with a small amount of data (Tobacco-3482). To this end, we trained the neural network at Tobacco-3482 from scratch. Therefore, we outperform the state-of-the-art by obtaining an accuracy of 90.3%. This results in a relative error reduction rate of 7.9%.

Hierarchical Attentional Fusion Graph Attention Network for Marine Diesel Engines Based on Imbalanced Datasets

Prediction of antimicrobial resistance based on whole-genome sequencing data has attracted greater attention due to its rapidity and convenience. Numerous machine learning-based studies have used genetic variants to predict drug resistance in Mycobacterium tuberculosis (MTB), assuming that variants are homogeneous, and most of these studies, however, have ignored the essential correlation between variants and corresponding genes when encoding variants, and used a limited number of variants as prediction input. In this study, taking advantage of genome-wide variants for drug-resistance prediction and inspired by natural language processing, we summarize drug resistance prediction into document classification, in which variants are considered as words, mutated genes in an isolate as sentences, and an isolate as a document. We propose a novel hierarchical attentive neural network model (HANN) that helps discover drug resistance-related genes and variants and acquire more interpretable biological results. It captures the interaction among variants in a mutated gene as well as among mutated genes in an isolate. Our results show that for the four first-line drugs of isoniazid (INH), rifampicin (RIF), ethambutol (EMB) and pyrazinamide (PZA), the HANN achieves the optimal area under the ROC curve of 97.90, 99.05, 96.44 and 95.14% and the optimal sensitivity of 94.63, 96.31, 92.56 and 87.05%, respectively. In addition, without any domain knowledge, the model identifies drug resistance-related genes and variants consistent with those confirmed by previous studies, and more importantly, it discovers one more potential drug-resistance-related gene.

Periodontal inflammation, a chronic condition affecting teeth and supporting structures, is linked to cardiovascular disease. The renin-angiotensin system plays a crucial role in inflammation and oxidative stress, with AT1 and AT2 receptors affecting vascular functions and inflammatory responses. This study aims to utilize angiotensin receptors in periodontal inflammation through enhanced hysterical attention network-based drug-gene association. Data preprocessing is crucial for ensuring the quality and reliability of drug and gene data, particularly in research involving angiotensin receptors, by identifying duplicates and correcting inconsistent formats. Cytoscape was utilized to import drugs and genes linked to angiotensin receptors, thereby constructing and analyzing a network. The Hierarchical Attention Network is an innovative framework for processing structured data with hierarchical relationships. It is suitable for tasks where features can be organized into multi-level structures. The network, with 1,172 nodes and 4,315 edges, has efficient communication, low density, significant connectivity variance, moderate centralization, and four connected components, with a 1.531-second analysis time. The model's R² score is 0.3631, indicating that the features can explain 36.31% of the target variable's variance. However, the model's predictions are about 0.8013 units away from actual values, suggesting room for improvement. Integrating hierarchical attention networks in deep learning models is promising for predicting drug and gene interactions in angiotensin receptors in periodontal inflammation.

This paper describes our models for the Moldavian vs. Romanian Cross-Topic Identification (MRC) evaluation campaign, part of the VarDial 2019 workshop. We focus on the three subtasks for MRC: binary classification between the Moldavian (MD) and the Romanian (RO) dialects and two cross-dialect multi-class classification between six news topics, MD to RO and RO to MD. We propose several deep learning models based on long short-term memory cells, Bidirectional Gated Recurrent Unit (BiGRU) and Hierarchical Attention Networks (HAN). We also employ three word embedding models to represent the text as a low dimensional vector. Our official submission includes two runs of the BiGRU and HAN models for each of the three subtasks. The best submitted model obtained the following macro-averaged F1 scores: 0.708 for subtask 1, 0.481 for subtask 2 and 0.480 for the last one. Due to a read error caused by the quoting behaviour over the test file, our final submissions contained a smaller number of items than expected. More than 50% of the submission files were corrupted. Thus, we also present the results obtained with the corrected labels for which the HAN model achieves the following results: 0.930 for subtask 1, 0.590 for subtask 2 and 0.687 for the third one.

Recently, with the rise of deep learning, text classification techniques have developed rapidly. However, the existing work usually takes the entire text as the modeling object and pays less attention to the hierarchical structure within the text, ignoring the internal connection between the upper and lower sentences. To address these issues, this paper proposes a Bert-based hierarchical graph attention network model (BHGAttN) based on a large-scale pretrained model and graph attention network to model the hierarchical relationship of texts. During modeling, the semantic features are enhanced by the output of the intermediate layer of BERT, and the multilevel hierarchical graph network corresponding to each layer of BERT is constructed by using the dependencies between the whole sentence and the subsentence. This model pays attention to the layer-by-layer semantic information and the hierarchical relationship within the text. The experimental results show that the BHGAttN model exhibits significant competitive advantages compared with the current state-of-the-art baseline models.