Data-Mining Based Novel Neural-Networks-Hierarchical Attention Structures for Obtaining an Optimal Efficiency

ST-SHAP: A hierarchical and explainable attention network for emotional EEG representation learning and decoding

COVID-19 is highly infectious and causes acute respiratory disease. Machine learning (ML) and deep learning (DL) models are vital in detecting disease from computerized chest tomography (CT) scans. The DL models outperformed the ML models. For COVID-19 detection from CT scan images, DL models are used as end-to-end models. Thus, the performance of the model is evaluated for the quality of the extracted feature and classification accuracy. There are four contributions included in this work. First, this research is motivated by studying the quality of the extracted feature from the DL by feeding these extracted to an ML model. In other words, we proposed comparing the end-to-end DL model performance against the approach of using DL for feature extraction and ML for the classification of COVID-19 CT scan images. Second, we proposed studying the effect of fusing extracted features from image descriptors, e.g., Scale-Invariant Feature Transform (SIFT), with extracted features from DL models. Third, we proposed a new Convolutional Neural Network (CNN) to be trained from scratch and then compared to the deep transfer learning on the same classification problem. Finally, we studied the performance gap between classic ML models against ensemble learning models. The proposed framework is evaluated using a CT dataset, where the obtained results are evaluated using five different metrics The obtained results revealed that using the proposed CNN model is better than using the well-known DL model for the purpose of feature extraction. Moreover, using a DL model for feature extraction and an ML model for the classification task achieved better results in comparison to using an end-to-end DL model for detecting COVID-19 CT scan images. Of note, the accuracy rate of the former method improved by using ensemble learning models instead of the classic ML models. The proposed method achieved the best accuracy rate of 99.39%.

Aspect-level sentiment classification is a fine-grained sentiment analysis task, which aims to predict the sentiment of a text in different aspects. One key point of this task is to allocate the appropriate sentiment words for the given aspect.Recent work exploits attention neural networks to allocate sentiment words and achieves the state-of-the-art performance. However, the prior work only attends to the sentiment information and ignores the aspect-related information in the text, which may cause mismatching between the sentiment words and the aspects when an unrelated sentiment word is semantically meaningful for the given aspect. To solve this problem, we propose a HiErarchical ATtention (HEAT) network for aspect-level sentiment classification. The HEAT network contains a hierarchical attention module, consisting of aspect attention and sentiment attention. The aspect attention extracts the aspect-related information to guide the sentiment attention to better allocate aspect-specific sentiment words of the text. Moreover, the HEAT network supports to extract the aspect terms together with aspect-level sentiment classification by introducing the Bernoulli attention mechanism. To verify the proposed method, we conduct experiments on restaurant and laptop review data sets from SemEval at both the sentence level and the review level. The experimental results show that our model better allocates appropriate sentiment expressions for a given aspect benefiting from the guidance of aspect terms. Moreover, our method achieves better performance on aspect-level sentiment classification than state-of-the-art models.

Social media platforms provide valuable insights into mental health trends by capturing user-generated discussions on conditions such as depression, anxiety, and suicidal ideation. Machine learning (ML) and deep learning (DL) models have been increasingly applied to classify mental health conditions from textual data, but selecting the most effective model involves trade-offs in accuracy, interpretability, and computational efficiency. This study evaluates multiple ML models, including logistic regression, random forest, and LightGBM, alongside DL architectures such as ALBERT and Gated Recurrent Units (GRUs), for both binary and multi-class classification of mental health conditions. Our findings indicate that ML and DL models achieve comparable classification performance on medium-sized datasets, with ML models offering greater interpretability through variable importance scores, while DL models are more robust to complex linguistic patterns. Additionally, ML models require explicit feature engineering, whereas DL models learn hierarchical representations directly from text. Logistic regression provides the advantage of capturing both positive and negative associations between features and mental health conditions, whereas tree-based models prioritize decision-making power through split-based feature selection. This study offers empirical insights into the advantages and limitations of different modeling approaches and provides recommendations for selecting appropriate methods based on dataset size, interpretability needs, and computational constraints.

Metabolic dysfunction-associated steatotic liver disease (MASLD) can progress to metabolic dysfunction-associated steatohepatitis (MASH) and liver fibrosis, contributing to a heavier global health burden. Non-invasive diagnostic tools developed using machine learning (ML) and deep learning (DL), two representative artificial intelligence algorithms, are increasingly being explored for MASH and its related fibrosis assessment. This study aimed to compare the diagnostic performance of different ML and DL models and identify the top-performing models for diagnosing MASH and associated liver fibrosis. A systematic review and meta-analysis were conducted across PubMed, Web of Science, Embase and Cochrane Library from inception to May 18, 2025. Pooled area under the receiver operator characteristic curve (AUROC) values with 95 % confidence interval (CI) were calculated. Accuracy, specificity, sensitivity, positive predictive values, and negative predictive values were also recorded. Of 4,314 studies initially identified, 106 met the inclusion criteria, with 35 studies (ML: n=28; DL: n=7) providing data for analysis. Logistic Regression and Neural Network are the most commonly algorithms applied in ML and DL, respectively. The pooled AUROCs for diagnosing MASH were 0.833 (95 %CI: 0.806-0.860) for ML models and 0.841 (95 %CI: 0.782-0.900) for DL models. Light Gradient Boosting Machine (LightGBM) and ResNet50 were the best-performing models for diagnosing MASH within ML and DL algorithms, respectively, achieving corresponding AUROCs of 0.920 (95 %CI: 0.916-0.924) and 0.960 (95 %CI: 0.951-0.969). For fibrosis diagnosis, ML models had a pooled AUROC of 0.826 (95 %CI: 0.792-0.860), with Categorical Boosting (CatBoost) achieving the highest AUROC of 0.960 (95 %CI: 0.950-0.970). DL models yielded the pooled AUROC of 0.875 (95 %CI: 0.816-0.934) for fibrosis diagnosis. Both ML and DL models demonstrated strong diagnostic performance for MASH and liver fibrosis, with DL achieving marginally higher AUROCs. AI-driven approaches show promise in MASLD management.

The tremendous development and rapid evolution in computing advancements has urged a lot of organizations to expand their data as well as computational needs. Such type of services offers security concepts like confidentiality, integrity, and availability. Thus, a highly secured domain is the fundamental need of cloud environments. In addition, security breaches are also growing equally in the cloud because of the sophisticated services of the cloud, which cannot be mitigated efficiently through firewall rules and packet filtering methods. In order to mitigate the malicious attacks and to detect the malicious behavior with high detection accuracy, an effective strategy named Multiverse Fractional Calculus (MFC) based hybrid deep learning approach is proposed. Here, two network classifiers namely Hierarchical Attention Network (HAN) and Random Multimodel Deep Learning (RMDL) are employed to detect the presence of malicious behavior. The network classifier is trained by exploiting proposed MFC, which is an integration of multi-verse optimizer and fractional calculus. The proposed MFC-based hybrid deep learning approach has attained superior results with utmost testing sensitivity, accuracy, and specificity of 0.949, 0.939, and 0.947.

This paper describes our models for the Moldavian vs. Romanian Cross-Topic Identification (MRC) evaluation campaign, part of the VarDial 2019 workshop. We focus on the three subtasks for MRC: binary classification between the Moldavian (MD) and the Romanian (RO) dialects and two cross-dialect multi-class classification between six news topics, MD to RO and RO to MD. We propose several deep learning models based on long short-term memory cells, Bidirectional Gated Recurrent Unit (BiGRU) and Hierarchical Attention Networks (HAN). We also employ three word embedding models to represent the text as a low dimensional vector. Our official submission includes two runs of the BiGRU and HAN models for each of the three subtasks. The best submitted model obtained the following macro-averaged F1 scores: 0.708 for subtask 1, 0.481 for subtask 2 and 0.480 for the last one. Due to a read error caused by the quoting behaviour over the test file, our final submissions contained a smaller number of items than expected. More than 50% of the submission files were corrupted. Thus, we also present the results obtained with the corrected labels for which the HAN model achieves the following results: 0.930 for subtask 1, 0.590 for subtask 2 and 0.687 for the third one.

Antisocial Online Behavior Detection Using Deep Learning

In social media, code-mixed language questions(combination of two distinct languages) is turning into the favored method of expression and communication. In twitter people can go over mixed language where individuals frequently utilize their mother tongue along with English. But question answering systems do not support mixed language. They only work on a single language such as English, French and German. This paper presents an online question answering framework for mixed languages. This system analyzes the user's mixed language query and gives the answer and also presents a contrivance that facilitates decode of English mixed with 3 different Indian languages: Hindi, Telugu and Tamil. The code mixed words will be translated into English to reduce data complexity. Later, featurization is performed on them. Deep learning algorithms such as Recurrent Neural Network (RNN), Hierarchical Attention Network (HAN) are used for question classification. Confusion matrix is used as evaluation measure of RNN, HAN. This system is facilitated on the web and in future, it can be used for collecting more code mixed questions and answers data for further improvement.

PurposeThe first transmission of coronavirus to humans started in Wuhan city of China, took the shape of a pandemic called Corona Virus Disease 2019 (COVID-19), and posed a principal threat to the entire world. The researchers are trying to inculcate artificial intelligence (Machine learning or deep learning models) for the efficient detection of COVID-19. This research explores all the existing machine learning (ML) or deep learning (DL) models, used for COVID-19 detection which may help the researcher to explore in different directions. The main purpose of this review article is to present a compact overview of the application of artificial intelligence to the research experts, helping them to explore the future scopes of improvement.MethodsThe researchers have used various machine learning, deep learning, and a combination of machine and deep learning models for extracting significant features and classifying various health conditions in COVID-19 patients. For this purpose, the researchers have utilized different image modalities such as CT-Scan, X-Ray, etc. This study has collected over 200 research papers from various repositories like Google Scholar, PubMed, Web of Science, etc. These research papers were passed through various levels of scrutiny and finally, 50 research articles were selected.ResultsIn those listed articles, the ML / DL models showed an accuracy of 99% and above while performing the classification of COVID-19. This study has also presented various clinical applications of various research. This study specifies the importance of various machine and deep learning models in the field of medical diagnosis and research.ConclusionIn conclusion, it is evident that ML/DL models have made significant progress in recent years, but there are still limitations that need to be addressed. Overfitting is one such limitation that can lead to incorrect predictions and overburdening of the models. The research community must continue to work towards finding ways to overcome these limitations and make machine and deep learning models even more effective and efficient. Through this ongoing research and development, we can expect even greater advances in the future.

Legal documents are often lengthy and complex, making it challenging and time-consuming for experts to accurately predict case outcomes. Older methods are not well-suited to the structure and language used in legal texts. This paper proposes a model called LegalHANOpt (Legal Hierarchical Attention Network with Optimized Parameters) to make accurate predictions about legal case decisions and explain how those decisions are made. LegalHANOpt utilizes a Hierarchical Attention Network (HAN) that analyzes legal documents by examining individual words and sentences, much like lawyers typically do. To improve the model's performance, Bayesian Optimization with Hyperband (BOHB) is utilized. This sophisticated method automatically determines the optimal settings for training the model, such as learning rate and dropout. The LegalHANOpt is trained on an extensive collection of past legal cases, including relevant facts, laws, and decisions. Results show that LegalHANOpt gives more accurate predictions than older methods achieving superior performance with an accuracy of 0.91%, macro F1-score of 0.83%, and AUC-ROC of 0.92%. It also highlights essential parts of the text, helping users understand why the model made a particular decision. In short, LegalHANOpt is a valuable and explainable tool to support legal experts in making better and faster decisions.

Recently the diagnosis of respiratory diseases based on respiratory-sound data has drawn much attention. However, no quantitative assessment method exists to detect abnormal respiratory sounds when making a diagnosis. Although existing studies have attempted to support doctors by providing them with machine learning results, there were limitations on explaining causal symptoms and establishing quantitative assessment. Therefore, a reliable method that can diagnose and explain the causal symptoms based on respiratory sound data is required. In this study, we propose using a hierarchical attention network for detecting abnormal respiratory sounds. The hierarchical attention network reflects hierarchical patterns of respiratory sounds and allows us to interpret the important feature of respiratory sounds. The experimental results showed that hierarchical attention network performed better than other existing methods in terms of sensitivity (ability of correctly detecting abnormal respiratory sounds) and explainability. We believe that the framework presented in this study can not only serve as a “second opinion” that can help doctors diagnose existing respiratory diseases, but also help doctors’ future research on unidentified respiratory diseases.

Prediction of antimicrobial resistance based on whole-genome sequencing data has attracted greater attention due to its rapidity and convenience. Numerous machine learning-based studies have used genetic variants to predict drug resistance in Mycobacterium tuberculosis (MTB), assuming that variants are homogeneous, and most of these studies, however, have ignored the essential correlation between variants and corresponding genes when encoding variants, and used a limited number of variants as prediction input. In this study, taking advantage of genome-wide variants for drug-resistance prediction and inspired by natural language processing, we summarize drug resistance prediction into document classification, in which variants are considered as words, mutated genes in an isolate as sentences, and an isolate as a document. We propose a novel hierarchical attentive neural network model (HANN) that helps discover drug resistance-related genes and variants and acquire more interpretable biological results. It captures the interaction among variants in a mutated gene as well as among mutated genes in an isolate. Our results show that for the four first-line drugs of isoniazid (INH), rifampicin (RIF), ethambutol (EMB) and pyrazinamide (PZA), the HANN achieves the optimal area under the ROC curve of 97.90, 99.05, 96.44 and 95.14% and the optimal sensitivity of 94.63, 96.31, 92.56 and 87.05%, respectively. In addition, without any domain knowledge, the model identifies drug resistance-related genes and variants consistent with those confirmed by previous studies, and more importantly, it discovers one more potential drug-resistance-related gene.

Periodontal inflammation, a chronic condition affecting teeth and supporting structures, is linked to cardiovascular disease. The renin-angiotensin system plays a crucial role in inflammation and oxidative stress, with AT1 and AT2 receptors affecting vascular functions and inflammatory responses. This study aims to utilize angiotensin receptors in periodontal inflammation through enhanced hysterical attention network-based drug-gene association. Data preprocessing is crucial for ensuring the quality and reliability of drug and gene data, particularly in research involving angiotensin receptors, by identifying duplicates and correcting inconsistent formats. Cytoscape was utilized to import drugs and genes linked to angiotensin receptors, thereby constructing and analyzing a network. The Hierarchical Attention Network is an innovative framework for processing structured data with hierarchical relationships. It is suitable for tasks where features can be organized into multi-level structures. The network, with 1,172 nodes and 4,315 edges, has efficient communication, low density, significant connectivity variance, moderate centralization, and four connected components, with a 1.531-second analysis time. The model's R² score is 0.3631, indicating that the features can explain 36.31% of the target variable's variance. However, the model's predictions are about 0.8013 units away from actual values, suggesting room for improvement. Integrating hierarchical attention networks in deep learning models is promising for predicting drug and gene interactions in angiotensin receptors in periodontal inflammation.

Accurate prediction of urban residential rents is of great importance for landlords, tenants, and investors. However, existing rents prediction models face challenges in meeting practical demands due to their limited perspectives and inadequate prediction performance. The existing individual prediction models often lack satisfactory accuracy, while ensemble learning models that combine multiple individual models to improve prediction results often overlook the impact of spatial heterogeneity on residential rents. To address these issues, this paper proposes a novel prediction model called GERPM, which stands for Geographically Weighted Stacking Ensemble Learning-Based Urban Residential Rents Prediction Model. GERPM comprehensively analyzes the influencing factors of residential rents from multiple perspectives and leverages a geographically weighted stacking ensemble learning approach. The model combines multiple machine learning and deep learning models, optimizes parameters to achieve optimal predictions, and incorporates the geographically weighted regression (GWR) model to consider spatial heterogeneity. By combining the strengths of deep learning and machine learning models and taking into account geographical factors, GERPM aims to improve prediction accuracy and provide robust predictions for urban residential rents. The model is evaluated using housing data from Nanjing, a major city in China, and compared with representative individual prediction models, the equal weight combination model, and the ensemble learning model. The experimental results demonstrate that GERPM outperforms other models in terms of prediction performance. Furthermore, the model’s effectiveness and robustness are validated by applying it to other major cities in China, such as Shanghai and Hangzhou. Overall, GERPM shows promising potential in accurately predicting urban residential rents and contributing to the advancement of the rental market.