Reading Between the Lines: Machine Learning Ensemble and Deep Learning for Implied Threat Detection in Textual Data

Parts of Speech (pos) tagging corresponds to techniques of tagging accurate pos tags of each word of a sentence. It is a pivotal and indispensable part of sentiment analysis, synthesizing text to speech, analyzing textual data and many more Natural Language Processing (NLP) tasks. Though there are many researches done in the field of languages like English, Chinese etc., it is still comparatively unexplored for Bangla. An excellent pos tagger in Bangla language can accelerate NLP of Bangla textual data. In this study, we have developed a novel hybrid Deep Learning (DL) model with two parallel input networks with Convolutional Neural Network (CNN), and Bidirectional Long Short Term Memory (BiLSTM), and an output network of CNN. CNN works well regarding the local dependencies of words in sentences, and BiLSTM works well with the global dependencies of words regarding the context in a sentence. No other work has been done with local features of textual data along with long term word dependencies in Bangla pos tagging studies. The model can work with any external Word Embeddings. In this study, it is implemented with both Embedding Layer and Word2Vec layer. In the evaluation process, Mathews Correlation Coefficient (MCC), Precision-Recall curve are used along with accuracy and F1-score due to the dataset's imbalanced characteristic. Our proposed model accompanied by Word2Vec layer has outperformed Vanilla Recurrent Neural Network (RNN), Gated Recurrent Units (GRU), Long Short Term Memory (LSTM), BiLSTM networks, and also one recent study using the same dataset with the highest accuracy of 0.974, F1-score of 0.883, and MCC value of 0.936.

Happiness is a state of contentment, joy, and fulfillment, arising from relationships, accomplishments, and inner peace, leading to well-being and positivity. The greatest happiness principle posits that morality is determined by pleasure, aiming for a society where individuals are content and free from suffering. While happiness factors vary, some are universally recognized. The World Happiness Report (WHR), published annually, includes data on ‘GDP per capita’, ‘social support’, ‘life expectancy’, ‘freedom to make life choices’, ‘generosity’, and ‘perceptions of corruption’. This paper predicts happiness scores using Machine Learning (ML), Deep Learning (DL), and ensemble ML and DL algorithms and examines the impact of individual variables on the happiness index. We also show the impact of COVID-19 pandemic on the happiness features. We design two ensemble ML and DL models using blending and stacking ensemble techniques, namely, Blending RGMLL, which combines Ridge Regression (RR), Gradient Boosting (GB), Multilayer Perceptron (MLP), Long Short-Term Memory (LSTM), and Linear Regression (LR), and Stacking LRGR, which combines LR, Random Forest (RF), GB, and RR. Among the trained models, Blending RGMLL demonstrates the highest predictive accuracy with R2 of 85%, MSE of 0.15, and RMSE of 0.38. We employ Explainable Artificial Intelligence (XAI) techniques to uncover changes in happiness indices, variable importance, and the impact of the COVID-19 pandemic on happiness. The study utilizes an open dataset from the WHR, covering 156 countries from 2018 to 2023. Our findings indicate that ‘GDP per capita’ is the most critical indicator of happiness score (HS), while ‘social support’ and ‘healthy life expectancy’ are also important features before and after the pandemic. However, during the pandemic, ‘social support’ emerged as the most important indicator, followed by ‘healthy life expectancy’ and ‘GDP per capita’, because social support is the prime necessity in the pandemic situation. The outcome of this research helps people understand the impact of these features on increasing the HS and provides guidelines on how happiness can be maintain during unwanted situations. Future research will explore advanced methods and include other related features with real-time monitoring for more comprehensive insights.

In today's digital world, app stores have become an essential part of software distribution, providing customers with a wide range of applications and opportunities for software developers to showcase their work. This study elaborates on the importance of end-user feedback for software evolution. However, in the literature, more emphasis has been given to high-rating & popular software apps while ignoring comparatively low-rating apps. Therefore, the proposed approach focuses on end-user reviews collected from 64 low-rated apps representing 14 categories in the Amazon App Store. We critically analyze feedback from low-rating apps and developed a grounded theory to identify various concepts important for software evolution and improving its quality including user interface (UI) and user experience (UX), functionality and features, compatibility and device-specific, performance and stability, customer support and responsiveness and security and privacy issues. Then, using a grounded theory and content analysis approach, a novel research dataset is curated to evaluate the performance of baseline machine learning (ML), and state-of-the-art deep learning (DL) algorithms in automatically classifying end-user feedback into frequently occurring issues. Various natural language processing and feature engineering techniques are utilized for improving and optimizing the performance of ML and DL classifiers. Also, an experimental study comparing various ML and DL algorithms, including multinomial naive Bayes (MNB), logistic regression (LR), random forest (RF), multi-layer perception (MLP), k-nearest neighbors (KNN), AdaBoost, Voting, convolutional neural network (CNN), long short-term memory (LSTM), bidirectional long short term memory (BiLSTM), gated recurrent unit (GRU), bidirectional gated recurrent unit (BiGRU), and recurrent neural network (RNN) classifiers, achieved satisfactory results in classifying end-user feedback to commonly occurring issues. Whereas, MLP, RF, BiGRU, GRU, CNN, LSTM, and Classifiers achieved average accuracies of 94%, 94%, 92%, 91%, 90%, 89%, and 89%, respectively. We employed the SHAP approach to identify the critical features associated with each issue type to enhance the explainability of the classifiers. This research sheds light on areas needing improvement in low-rated apps and opens up new avenues for developers to improve software quality based on user feedback.

Sentiment Analysis of Self Driving Car Dataset: A comparative study of Deep Learning approaches

Evaluation of multispectral imaging for freeze damage assessment in strawberries using AI-based computer vision technology

This article investigates the identification of hate speech on social media using machine learning and deep learning techniques. The research uses metrics such as F-measure, AUC-ROC, precision, accuracy, and recall assessing the effectiveness of various tactics. The findings indicate that deep learning models, particularly the bidirectional long short-term memory (BiLSTM) architecture, consistently outperform other methods in categorization tasks. The research emphasizes the importance of sophisticated neural network designs in identifying the intricacies of hostile and offensive content online. The study offers insights for promoting early identification and prevention of cyberbullying, improving secure and inclusive online environments. Future research may explore real-time detection systems, hybrid approaches, or the integration of complementary components to enhance and improve innovative technology in tackling this significant social issue. A sample tweet was annotated by specialists who categorize tweets as hate speech, offensive language, or neutral. The researchers applied shallow learning methodologies and integrated word embeddings like Word2Vec and GloVe to enhance the efficacy of deep learning models. The results indicate that BiLSTM surpasses shallow learning methods in detecting hate speech on Twitter, highlighting the efficacy of deep learning approaches in recognizing and tracking hate speech on social media platforms. When comparing different deep learning and machine learning models on different datasets, the results reveal that deep learning techniques are usually more effective. A reasonably high level of accuracy is achieved by KNN and SVM among classical algorithms, whereas Naïve Bayes performs the poorest. While deep learning approaches provide better results, tree-based models such as Random Forest and Decision Trees offer more consistent accuracy. Models based on neural networks, such as LSTM, CNN, and BI-LSTM, perform well, with LSTM-based methods excelling in particular. The most successful strategy for classification problems is the model presented, which obtains the greatest accuracy, precision, recall, F1-score of 95%. The research aids in the development of advanced tools and methodologies to mitigate hate speech on social media and foster positive online interactions. Future research may investigate alternative deep learning architectures, such as transformers, to enhance hate speech detection efficacy. The advancement of interpretable AI methodologies for identifying hate speech and delivering transparent forecasts might enhance user confidence and facilitate better content moderation decisions.

Small and Medium-sized Enterprises (SMEs) in Sri Lanka are increasingly turning to social media to gain insights into consumer preferences; however, they frequently lack sophisticated analytical tools to derive actionable insights. This research introduces a hybrid deep learning framework aimed at sentiment and emotion analysis specifically for the Sri Lankan cake sector. Customer reviews and social media comments were gathered and processed through various methods including text cleaning, emoji translation, stopword elimination, tokenization, and sequence padding. Several deep learning models— Convolutional Neural Networks (CNN), Bidirectional Long Short-Term Memory (BiLSTM), and Gated Recurrent Units (GRU) were assessed, along with hybrid models that combined CNN and BiLSTM with GPT-2 embeddings. The CNN+GPT-2 model achieved the highest accuracy in sentiment classification at 96.14%, while BiLSTM performed best in emotion classification with an accuracy of 91.57%. In terms of trend forecasting, Long Short-Term Memory (LSTM) outshone both SARIMA and Prophet, yielding the lowest error metrics and the most reliable sentiment predictions. These results underscore the potential of hybrid deep learning and transformer-based models to effectively capture complex consumer sentiments and provide SMEs with practical, AI-driven insights. This study is among the first to implement such hybrid NLP models within a localized Sri Lankan context, presenting a scalable approach for business intelligence and informed decision-making.

The classification of genomic sequences is a crucial area of research in the field of virology. This is due to the increasing number of outbreaks we have faced in recent times. We have a vast repository of genomic sequences from various species, including humans, animals, plants, bacteria, and viruses, which tend to mutate and form new variants or strains. In the realm of machine learning, several models are employed for genome sequence classification. Among these are traditional algorithms such as Random Forest (RF), K-nearest neighbors (KNNs), Decision Tree (DT), and Naive Bayes (NB), each offering unique advantages in handling genetic data. Additionally, deep learning models like Convolutional Neural Networks (CNNs), Long Short-Term Memory (LSTM) networks, and Bi-Directional LSTM networks are utilized for their robust capabilities in capturing complex patterns and dependencies within genomic sequences. In this study, we explored the application of Natural Language Processing (NLP) techniques to classify the genomic sequences. The focus of our research involves utilizing advanced large language models (LLMs) such as DNABERT, DNAGPT, and GENA LM, which are fine-tuned explicitly on the language of DNA. In this research, after a detailed analysis, we found that DNAGPT achieved an accuracy of 96%, which exceeds the performance of state-of-the-art machine learning and deep learning models.

Emotion recognition (ER) is crucial for understanding human behaviours, social interactions, and psychological well-being. Electroencephalography (EEG) has emerged as a promising tool for capturing the neural correlates of emotions. This work is a systematic review of articles in ER using EEG signals. A total of 120 articles from 1041 articles were selected based on PRISMA guidelines using defined inclusion and exclusion criteria, published between 2018 and 2024. This article aims to provide an in-depth understanding of the current landscape of ER from EEG signals utilizing deep learning (DL). This review offers valuable guidance for researchers and practitioners seeking more refined and reliable emotion classification systems. To explore the effectiveness of DL models in EEG-based ER, several potential DL models, such as convolutional neural network, long short-term memory (LSTM), gated recurrent unit (GRU), hybrid bidirectional LSTM (BiLSTM), bidirectional GRU, and advanced DL models such as convolutional recurrent neural network and EEG-Conformer models are applied to two popular datasets, SEED and GAMEEMO, respectively, to depict the full process of ER. Additionally, the performance of DL models is also compared with the performance of basic machine learning (ML) models such as SVM, k-nearest neighbors, logistic regression, and boosting algorithms such as AdaBoost, XGBoost and LightGBM. Through extensive experiments and performance evaluations, the performance of different models when applied to the datasets mentioned above is compared. The accuracy, precision, recall, and F1-scores are analysed to determine the most effective model for EEG-based ER. The findings of this study demonstrate that the performance of hybrid DL models is more efficacious than that of ML models. The best-performing model (BiLSTM) classified the emotions, with an accuracy of 90.54% when applied to the GAMEEMO dataset. This research contributes to the growing body of literature on ER and provides insights into the feasibility of using EEG signals to understand emotional states, and presents a structured roadmap for future exploration. The findings can aid in the development of more accurate and reliable ER systems, which can have wide-ranging applications in psychology, social sciences, and human-computer interactions.

In cybersecurity, the rise of fileless malware poses a significant challenge to endpoint security. Traditional detection methods often fail against these sophisticated attacks, necessitating advanced techniques like deep learning models. This study highlights the limitations of Bi-Directional Long Short-Term Memory (BLSTM) models in dynamic malware analysis and proposes enhancements through Convolutional Long Short-Term Memory (ConvLSTM) architecture. BLSTM models process input sequences in forward and backward directions, combining the results into one output. While this dual-layer approach improves analysis, it is time-consuming, potentially increasing the risk of fileless malware attacks. A key limitation of BLSTM is the lack of parameter sharing between forward and backward directions. This reduces its ability to capture spatial and temporal features simultaneously, hindering effectiveness in detecting fileless malware. To address this, the ConvLSTM model consolidates feature extraction within a single LSTM cell layer. ConvLSTM breaks down samples into subsequence and uses timesteps for additional feature extraction, enabling spatial-temporal data analysis and improving malware prediction accuracy. The model was tested using a dynamic malware dataset. Unlike traditional LSTM, ConvLSTM integrates convolutional layers, allowing parameter sharing across both spatial and temporal dimensions. This reduces computational complexity and improves model performance in handling multidimensional data. The research re-simulated prior work with BLSTM using the same malware dataset. The Spyder app ran the event simulator and the ConvLSTM model's results replaced BLSTM's using identical parameters. Time, accuracy and loss were the main performance metrics. ConvLSTM outperformed BLSTM, achieving 98% detection accuracy compared to BLSTM's 90%. It also significantly reduced processing time, averaging 10 seconds, while BLSTM took 22 seconds. ConvLSTM experienced lower losses, averaging 10% per epoch versus BLSTM's 20%. In conclusion, ConvLSTM offers superior performance over BLSTM in fileless malware detection. Its enhanced computational efficiency and ability to quickly mitigate threats make it a robust solution for fortifying endpoint security against evolving cyber threats. ConvLSTM holds potential in strengthening defence mechanisms against sophisticated malware attacks, providing a proactive approach to safeguarding networks and data.

Particulate matter (PM) in the air can cause various health problems and diseases in humans. In particular, the smaller size of PM2.5 enable them to penetrate deep into the lungs, causing severe health impacts. Exposure to PM2.5 can result in respiratory, cardiovascular, and allergic diseases, and prolonged exposure has also been linked to an increased risk of cancer, including lung cancer. Therefore, forecasting the PM2.5 concentration in the surrounding is crucial for preventing these adverse health effects. This paper proposes a method for forecasting the PM2.5 concentration after 1 h using bidirectional long short-term memory (Bi-LSTM). The proposed method involves selecting input variables based on the feature importance calculated by random forest, classifying the data to assign weight variables to reduce bias, and forecasting the PM2.5 concentration using Bi-LSTM. To compare the performance of the proposed method, two case studies were conducted. First, a comparison of forecasting performance according to preprocessing. Second, forecasting performance between deep learning (long short-term memory, gated recurrent unit, and Bi-LSTM) and conventional machine learning models (multi-layer perceptron, support vector machine, decision tree, and random forest). In case study 1, The proposed method shows that the performance indices (RMSE: 3.98%p, MAE: 5.87%p, RRMSE: 3.96%p, and R2:0.72%p) are improved because weights are given according to the input variables before the forecasting is performed. In case study 2, we show that Bi-LSTM, which considers both directions (forward and backward), can effectively forecast when compared to conventional models (RMSE: 2.70, MAE: 0.84, RRMSE: 1.97, R2: 0.16). Therefore, it is shown that the proposed method can effectively forecast PM2.5 even if the data in the high-concentration section is insufficient.

This study explores Human Activity Recognition (HAR) using smartphone sensors to address the challenges posed by position-dependent datasets. We propose a position-independent system that leverages data from accelerometers, gyroscopes, linear accelerometers, and gravity sensors collected from smartphones placed either on the chest or in the left/right leg pocket. The performance of traditional machine learning algorithms (Decision Trees (DT), K-Nearest Neighbors (KNN), Random Forest (RF), Support Vector Classifier (SVC), and XGBoost) is compared against deep learning models (Gated Recurrent Unit (GRU), Long Short-Term Memory (LSTM), Temporal Convolutional Networks (TCN), and Transformer models) under two sensor configurations. Our findings highlight that the Temporal Convolutional Network (TCN) model consistently outperforms other models, particularly in the four-sensor non-overlapping configuration, achieving the highest accuracy of 97.70%. Deep learning models such as LSTM, GRU, and Transformer also demonstrate strong performance, showcasing their effectiveness in capturing temporal dependencies in HAR tasks. Traditional machine learning models, including RF and XGBoost, provide reasonable performance but do not match the accuracy of deep learning models. Additionally, incorporating data from linear accelerometers and gravity sensors led to slight improvements over using accelerometer and gyroscope data alone. This research enhances the recognition of passenger behaviors for intelligent transportation systems, contributing to more efficient congestion management and emergency response strategies.

Emotion detection plays a vital role in understanding human sentiments and behaviors across various applications, including customer feedback analysis and mental health monitoring. This research assesses the efficiency of different algorithms for machine learning in detecting emotions in text data. A meticulously curated dataset is utilized for the study. The research compares conventional models like Logistic Regression (LR), Random Forest (RF), Support Vector Machines (SVM), and Naive Bayes (NB) with deep learning models like Long Short-Term Memory (LSTM), Convolutional Neural Networks (CNN), and Bidirectional Encoder Representations from Transformers (BERT). The performance of each algorithm is assessed using accuracy, precision, recall, and F1 score. BERT exhibits superiority over other models, achieving the maximum accuracy of 0.8867 and F1 Score of 0.8871. CNN and SVM also display commendable performance. While the traditional models perform adequately, they are surpassed by deep learning models, with Naive Bayes showing the lowest metrics. This study underscores the significance of selecting models based on specific application requirements, taking into account factors like interpretability and efficiency. Future research endeavors may explore multimodal approaches, model interpretability, bias reduction, and real-time applications, thereby contributing to the advancement of emotion detection in text.

Emergence in the social network leads to the extensive and faster diffusion of news than conventional news channels. Verification of data is challenging due to massive information on a social network. Unverified information can be a rumor or fake news that causes damage to an individuals and organizations, revealing the harmful impact on humanity. Therefore, it is vital to combat rumor diffusion to minimize the adverse effects on society. Despite vigorous efforts to deal with this issue, researchers mainly focussed on temporal dynamics of posts and other features like a user, network, content-based, which demonstrate a moderate accuracy. The time series features are associated with an event that suppresses the other quality features related to each post. There is a scope for improvement in the accuracy, so this paper focuses on post-wise features such as user-based, content-based and lexical-based features along with post sequences. We proposed a framework that uses various essential features and combines two deep learning models. Word embedding is utilized with bidirectional long short-term memory (BiLSTM) and combined with post-wise features using a multilayer perceptron (MLP), which improves accuracy. The experiments on the real-world dataset of Twitter demonstrate a notable improvement in accuracy compared to state-of-the-art approaches.

Phishing scams are internet frauds that target people by sending them harmful links. Many victims ranging from individuals to big companies, have suffered numerous losses due to phishing, highlighting the increasing need to effectively detect and prevent a phishing attack as soon as it is received. This paper applied machine and deep learning models to detect phishing attacks by natural language processing of Uniform Resource Locators. Machine learning algorithms such as Logistic Regression and Multi-Naive Bayes were used Uniform Resource Locators for classification of legitimate and phishing. Additionally, Long Term Short Memory, Gated Recurrent Units, and Bidirectional Recurrent Neural Networks were used as Deep Learning models. Two of the used models are Long Term Short Memory and Gated Recurrent Units models possess significantly high training and validation scores with an overall accuracy of 95%. The Bidirectional Recurrent Neural Net using Gated Recurrent Units and Bidirectional Recurrent Neural Net using LTSM shows 97% accuracy. Therefore, using multiple deep learning models to predict whether URLs are phishing or legitimate is a significant assistance in reviewing websites. Further research for other parameters aside from using Uniform Resource Locators with different deep learning models can be used to improve the accuracy of phishing detection.