An Extended Benchmark System of Word Embedding Methods for Vulnerability Detection

In today's digital world, app stores have become an essential part of software distribution, providing customers with a wide range of applications and opportunities for software developers to showcase their work. This study elaborates on the importance of end-user feedback for software evolution. However, in the literature, more emphasis has been given to high-rating & popular software apps while ignoring comparatively low-rating apps. Therefore, the proposed approach focuses on end-user reviews collected from 64 low-rated apps representing 14 categories in the Amazon App Store. We critically analyze feedback from low-rating apps and developed a grounded theory to identify various concepts important for software evolution and improving its quality including user interface (UI) and user experience (UX), functionality and features, compatibility and device-specific, performance and stability, customer support and responsiveness and security and privacy issues. Then, using a grounded theory and content analysis approach, a novel research dataset is curated to evaluate the performance of baseline machine learning (ML), and state-of-the-art deep learning (DL) algorithms in automatically classifying end-user feedback into frequently occurring issues. Various natural language processing and feature engineering techniques are utilized for improving and optimizing the performance of ML and DL classifiers. Also, an experimental study comparing various ML and DL algorithms, including multinomial naive Bayes (MNB), logistic regression (LR), random forest (RF), multi-layer perception (MLP), k-nearest neighbors (KNN), AdaBoost, Voting, convolutional neural network (CNN), long short-term memory (LSTM), bidirectional long short term memory (BiLSTM), gated recurrent unit (GRU), bidirectional gated recurrent unit (BiGRU), and recurrent neural network (RNN) classifiers, achieved satisfactory results in classifying end-user feedback to commonly occurring issues. Whereas, MLP, RF, BiGRU, GRU, CNN, LSTM, and Classifiers achieved average accuracies of 94%, 94%, 92%, 91%, 90%, 89%, and 89%, respectively. We employed the SHAP approach to identify the critical features associated with each issue type to enhance the explainability of the classifiers. This research sheds light on areas needing improvement in low-rated apps and opens up new avenues for developers to improve software quality based on user feedback.

Parts of Speech (pos) tagging corresponds to techniques of tagging accurate pos tags of each word of a sentence. It is a pivotal and indispensable part of sentiment analysis, synthesizing text to speech, analyzing textual data and many more Natural Language Processing (NLP) tasks. Though there are many researches done in the field of languages like English, Chinese etc., it is still comparatively unexplored for Bangla. An excellent pos tagger in Bangla language can accelerate NLP of Bangla textual data. In this study, we have developed a novel hybrid Deep Learning (DL) model with two parallel input networks with Convolutional Neural Network (CNN), and Bidirectional Long Short Term Memory (BiLSTM), and an output network of CNN. CNN works well regarding the local dependencies of words in sentences, and BiLSTM works well with the global dependencies of words regarding the context in a sentence. No other work has been done with local features of textual data along with long term word dependencies in Bangla pos tagging studies. The model can work with any external Word Embeddings. In this study, it is implemented with both Embedding Layer and Word2Vec layer. In the evaluation process, Mathews Correlation Coefficient (MCC), Precision-Recall curve are used along with accuracy and F1-score due to the dataset's imbalanced characteristic. Our proposed model accompanied by Word2Vec layer has outperformed Vanilla Recurrent Neural Network (RNN), Gated Recurrent Units (GRU), Long Short Term Memory (LSTM), BiLSTM networks, and also one recent study using the same dataset with the highest accuracy of 0.974, F1-score of 0.883, and MCC value of 0.936.

Nowadays, most smartphones feature a number of strong sensors, such as direction, network, location, and motion sensors. Motion and inertial sensors (i.e., accelerometer, gyroscope, etc.) are particularly popular in human activity recognition (HAR) for detecting human physical activity which helps in many research areas of the Internet of healthcare things (IoHT) including several patient’s diseases such as Parkinson, obesity, cardiovascular, and diabetes. Deep learning (DL) techniques have been increasingly popular over the years as a result of their significant outcomes. The use of deep learning techniques to recognize human physical activity in wearable and mobile sensor situations has also gotten a lot of attention from around the world. In this paper, a deep neural network (DNN) combining bidirectional long short-term memory (Bi-LSTM) and convolutional neural network (CNN) is proposed. The performance of the model has been evaluated on two publicly available datasets: WISDM and UCI-HAR. The model has achieved 97.96 and 97.15% accuracy for the WISDM and UCI-HAR, respectively. Moreover, the simulation results show the effectiveness of the proposed work compared to other state-of-the-art methods.KeywordsHuman activity recognitionDeep learningIoHTConvolutional neural networks (CNN)Bidirectional long short-term memory networks (Bi-LSTM)

This study investigates the application of advanced deep learning models to forecast fossil energy prices, a critical factor influencing global economic stability. Unlike previous research, this study conducts a comparative analysis of Gated Recurrent Unit (GRU), Recurrent Neural Network (RNN), Bidirectional Long Short-Term Memory (Bi-LSTM), Long Short-Term Memory (LSTM), Convolutional Neural Network (CNN), and Deep Neural Network (DNN) models. The evaluation metrics employed include Root Mean Squared Error (RMSE) and Mean Absolute Percentage Error (MAPE). The results reveal that recurrent architectures, particularly GRU, LSTM, and Bi-LSTM, consistently outperform feedforward and convolutional models, demonstrating superior ability to capture temporal dependencies and nonlinear dynamics in energy markets. In contrast, the RNN and DNN show relatively weaker generalization capabilities. Additionally, visualizations of actual versus predicted prices for each model further emphasize superior forecasting accuracy of recurrent models. The results highlight the potential of deep learning in enhancing investment and policy decisions. Additionally, the results provide significant implications for policymakers and investors by emphasizing the value of accurate energy price forecasting in mitigating market volatility, improving portfolio management, and supporting evidence-based energy policies.

A mobile ad hoc network (MANET) is a collection of mobile devices attached without infrastructure or central management. Network size increases rapidly, resulting in congestion, network delay, data packet loss, and a drop in throughput, resulting in poor energy efficiency. Data should be mitigated based on the prediction of congestion. To resolve the problem of congestion, a novel dragonfly optimized deep learning for congestion elimination (DODGE) technique was proposed, which predicts the congested node effectively. Initially, the Transmission Control Protocol (TCP), and User Datagram Protocol (UDP) packets from the MANET environment were pre-processed and the features were selected using dragonfly optimization (DFO). The features that are selected from the DFO model were provided to the stacked convolutional neural network combined with bidirectional long short-term memory (SCNN-BiLSTM). The deep learning network will predict the congested node and if congestion is found, then the message will be displayed. The DODGE is simulated by using Network simulator2 (NS2) and a comparison is made between proposed DODGE and traditional approaches such as hybrid gravitational fuzzy neural network (HGFNN), quality of service-aware distributed congestion control (QoS-ADCC), and improved priority aware ad hoc on-demand distance vector (IPA-AODV) in terms of packet delivery ratio (PDR), delay (DE), throughput (TP), energy consumption (EC), latency (L), detection rate (DR), and network lifetime (NL). The proposed SCNN-BiLSTM improves the overall accuracy better than 10.05%, 6.59%, and 3.26% bidirectional long short-term memory (BiLSTM), deep neural network (DNN), convolutional neural network (CNN) for predicting the congested node in the shortest time.

Cyber-physical system (CPS) security has become more important in the age of Industry 4.0 because of the quick integration of automation and the Internet of Things. The goal of this project is to create a strong intrusion detection and control system that can recognize and lessen security risks in CPS settings. The suggested approach makes use of deep learning (DL) and reinforcement learning (RL) techniques. To guarantee data consistency, pre-processing procedures such as mean-based imputation and min–max scaling come after data collection. ADASYN data augmentation is used to address class imbalance, while entropy analysis and statistical techniques are used to extract key features. The intrusion detection phase uses a combination of deep convolutional neural networks (DCNN) and bidirectional long short-term memory (BI-LSTM) networks to capture both spatial and temporal relationships in the data, while a hybrid feature selection technique improves the model’s performance. A deep Q-network (DQN) handles attack mitigation and uses reinforcement learning to adjust to new threats. Detecting attack patterns with high sensitivity (0.984), specificity (0.983), and accuracy (0.991626) for dataset 1, the accuracy of dataset 2 is 0.985 for 70% of training and 0.988 for 80% of training, and the Proposed-DBID-Net architecture enhances CPS security in Industry 4.0. The evaluation phase emphasizes how crucial feature selection is to maximize the model’s accuracy. In conclusion, this study offers a thorough and flexible method for protecting CPS in Industry 4.0, guaranteeing accuracy and scalability across changing cyber threats.

Many researchers in various disciplines have focused on extracting meaningful information from social media platforms in recent years. Identification of behaviors and emotions from user posts is examined under the heading of sentiment analysis (SA) studies using the natural language processing (NLP) techniques. In this study, a novel TCNN-Bi-LSTM model using the two-stage convolutional neural network (TCNN) and bidirectional long short-term memory (Bi-LSTM) architectures was proposed. While TCNN layers enable the extraction of strong local features, the output of these layers feeds the Bi-LSTM model that remembers forward-looking information and capture long-term dependencies. In this study, first, preprocessing steps were applied to the raw dataset. Thus, strong features were extracted from the obtained quality dataset using the FastText word embedding technique that pre-trained with location-based and sub-word information features. The experimental results of the proposed method are promising compared to the baseline deep learning and machine learning models. Also, experimental results show that while the FastText data embedding technique achieves the best performance compared to other word embedding techniques in all deep learning classification models, it has not had the same outstanding success in machine learning models. This study aims to investigate the sentiments of tweets about the COVID-19 vaccines and comments on these tweets among Twitter users by using the power of Twitter data. A new dataset collected from Twitter was constructed to be used in experimental results. This study will facilitate detecting inappropriate, incomplete, and erroneous information about vaccination. The results of this study will enable society to broaden its perspective on the administered vaccines. It can also assist the government and healthcare agencies in planning and implementing the vaccination's promotion on time to achieve the herd immunity provided by the vaccination.

Liquefaction susceptibility using machine learning based on SPT data

Text classification is one of the important applications of Natural Language Processing (NLP) in which text is classified into pre-defined categories or classes. Several state-of-the-arts have already been conducted on text classification in different Indian languages, however, limited researches on text classification performed in Odia. Moreover, the news headlines classification is a text classification technique which is performed in our study mainly focusing on Odia language. The Convolutional Neural Network (CNN) based Deep Learning technique is used to classify the Odia news headlines dataset available at Kaggle repository. The dataset consisting of approx. 15,200 Odia news headlines is split into training and testing sets of 80:20 ratio and is categorized into different classes, such as sports, business, and entertainment by using CNN. The efficacy and efficiency of the proposed CNN-based Odia news headlines classification is evaluated in contrary to other existing machine learning and deep learning models, such as Neural Networks, LSTM (Long Short-Term Memory), and Bi-LSTM (Bidirectional Long Short-Term Memory) with respect to different performance metrics like precision, accuracy, recall, F1-score, training time, and AUC-ROC (Area Under the Receiver Operating Characteristic Curve). The simulation results revealed that the proposed CNN model outperformed with respect to other models.

Prediction of PM2.5 concentration in urban agglomeration of China by hybrid network model

This paper proposes a multi-layered anomaly detection scheme to train feature extraction and to test anomaly prediction by using Convolutional Neural Networks (CNNs) layer, Bidirectional and Unidirectional Long Short-Term Memory (LSTM) Recurrent Neural Networks (RNNs), which is one of a novel deep architecture named stacked convolutional bidirectional LSTM network (SCB-LSTM). In the proposed model, the stacked CNNs perform feature extraction of vibration sensor signal patterns, and the result is used to feature learning with the stacked bidirectional LSTMs (SB-LSTMs). After this procedure, the stacked unidirectional LSTMs (SU-LSTMs) enhance the feature learning, and a regression layer finally predicts anomaly detections. The experimental results of bearing data not only show the accuracy of the proposed model in anomaly detection for rotating machinery diagnostics, but also suggest the better performance than other state-of-the-art algorithms such as a plain uni-LSTM or Bi-LSTM.

The precise prediction of the remaining useful life (RUL) of lithium-ion batteries is of great significance for improving energy management efficiency and extending battery lifespan, and it is widely applied in the fields of new energy and electric vehicles. However, accurate RUL prediction still faces significant challenges. Although various methods based on deep learning have been proposed, the performance of their neural networks is strongly correlated with the hyperparameters. To overcome this limitation, this study proposes an innovative approach that combines the Alpha evolutionary (AE) algorithm with a deep learning model. Specifically, this hybrid deep learning architecture consists of convolutional neural network (CNN), time convolutional network (TCN), bidirectional long short-term memory (BiLSTM) and multi-scale attention mechanism, which extracts the spatial features, long-term temporal dependencies, and key degradation information of battery data, respectively. To optimize the model performance, the AE algorithm is introduced to automatically optimize the hyperparameters of the hybrid model, including the number and size of convolutional kernels in CNN, the dilation rate in TCN, the number of units in BiLSTM, and the parameters of the fusion layer in the attention mechanism. Experimental results demonstrate that our method significantly enhances prediction accuracy and model robustness compared to conventional deep learning techniques. This approach not only improves the accuracy and robustness of battery RUL prediction but also provides new ideas for solving the parameter tuning problem of neural networks.

Deep learning (also known as deep structured learning, hierarchical learning, or deep machine learning) is a branch of machine learning based on a set of algorithms that attempt to model high-level abstractions in data by using a deep graph with multiple processing layers, composed of multiple linear and nonlinear transformations. Deep learning has been characterized as a class of machine learning algorithms with the following characteristics [257]: • They use a cascade of many layers of nonlinear processing units for feature extraction and transformation. Each successive layer uses the output from the previous layer as input. The algorithms may be supervised or unsupervised and applications include pattern analysis (unsupervised) and classification (supervised). • They are based on the (unsupervised) learning of multiple levels of features or representations of the data. Higher-level features are derived from lower-level features to form a hierarchical representation. • They are part of the broader machine learning field of learning representations of data. • They learn multiple levels of representations that correspond to different levels of abstraction; the levels form a hierarchy of concepts. These definitions have in common: multiple layers of nonlinear processing units and the supervised or unsupervised learning of feature representations in each layer, with the layers forming a hierarchy from low-level to high-level features. Various deep learning architectures such as deep neural networks, convolutional deep neural networks, deep belief networks (DBN), and recurrent neural networks have been applied to fields like computer vision, automatic speech recognition, natural language processing, audio recognition, and bioinformatics where they have been shown to produce state-of-the-art results on various tasks. In this chapter we start in Section 7.1 with an introduction, giving a brief history of this field, the relevant literature, and its applications. Then we study some basic concepts of deep learning such as convolutional neural networks, recurrent neural networks, backpropagation algorithm, restricted Boltzmann machines, and deep learning networks in Section 7.2. Then we illustrate three examples for Apache Spark implementation for mobile big data (MBD), user moving pattern extraction, and combination with nonparametric Bayesian learning, respectively, in Sections 7.3 through 7.5. Finally, we have summary in Section 7.6.

<p>Deep neural networks have achieved tremendous success in many domains (e.g., computer vision~\cite{Alexnet12,vggnet15,fastrcnn15}, speech recognition~\cite{hinton2012deep,dahl2012context}, natural language processing~\cite{dahl2012context,collobert2011natural}, games~\cite{silver2017mastering,silver2016mastering}), however, there are still many challenges in deep learning comunity such as how to speed up training large deep neural networks, how to compress large nerual networks for mobile/embed device without performance loss, how to automatically design the optimal network structures for a certain task, and how to further design the optimal networks with improved performance and certain model size with reduced computation cost.</p> <p>To speed up training large neural networks, we propose to use multinomial sampling for dropout, i.e., sampling features or neurons according to a multinomial distribution with different probabilities for different features/neurons. To exhibit the optimal dropout probabilities, we analyze the shallow learning with multinomial dropout and establish the risk bound for stochastic optimization. By minimizing a sampling dependent factor in the risk bound, we obtain a distribution-dependent dropout with sampling probabilities dependent on the second order statistics of the data distribution. To tackle the issue of evolving distribution of neurons in deep learning, we propose an efficient adaptive dropout (named evolutional dropout) that computes the sampling probabilities on-the-fly from a mini-batch of examples.</p> <p>To compress large neural network structures, we propose a simple yet powerful method for compressing the size of deep Convolutional Neural Networks (CNNs) based on parameter binarization. The striking difference from most previous work on parameter binarization/quantization lies at different treatments of $1\times 1$ convolutions and $k\times k$ convolutions ($k>1$), where we only binarize $k\times k$ convolutions into binary patterns. By doing this, we show that previous deep CNNs such as GoogLeNet and Inception-type Nets can be compressed dramatically with marginal drop in performance. Second, in light of the different functionalities of $1\times 1$ (data projection/transformation) and $k\times k$ convolutions (pattern extraction), we propose a new block structure codenamed the pattern residual block that adds transformed feature maps generated by $1\times 1$ convolutions to the pattern feature maps generated by $k\times k$ convolutions, based on which we design a small network with $\sim 1$ million parameters. Combining with our parameter binarization, we achieve better performance on ImageNet than using similar sized networks including recently released Google MobileNets.</p> <p>To automatically design neural networks, we study how to design a genetic programming approach for optimizing the structure of a CNN for a given task under limited computational resources yet without imposing strong restrictions on the search space. To reduce the computational costs, we propose two general strategies that are observed to be helpful: (i) aggressively selecting strongest individuals for survival and reproduction, and killing weaker individuals at a very early age; (ii) increasing mutation frequency to encourage diversity and faster evolution. The combined strategy with additional optimization techniques allows us to explore a large search space but with affordable computational costs.</p> <p>To further design the optimal networks with improved performance and certain model size under reduced computation cost, we propose an ecologically inspired genetic approach for neural network structure search , that includes two types of succession: primary and secondary succession as well as accelerated extinction. Specifically, we first use primary succession to rapidly evolve a community of poor initialized neural network structures into a more diverse community, followed by a secondary succession stage for fine-grained searching based on the networks from the primary succession. Accelerated extinction is applied in both stages to reduce computational cost. In addition, we also introduce the gene duplication to further utilize the novel block of layers that appeared in the discovered network structure.</p>

Emotion recognition (ER) is crucial for understanding human behaviours, social interactions, and psychological well-being. Electroencephalography (EEG) has emerged as a promising tool for capturing the neural correlates of emotions. This work is a systematic review of articles in ER using EEG signals. A total of 120 articles from 1041 articles were selected based on PRISMA guidelines using defined inclusion and exclusion criteria, published between 2018 and 2024. This article aims to provide an in-depth understanding of the current landscape of ER from EEG signals utilizing deep learning (DL). This review offers valuable guidance for researchers and practitioners seeking more refined and reliable emotion classification systems. To explore the effectiveness of DL models in EEG-based ER, several potential DL models, such as convolutional neural network, long short-term memory (LSTM), gated recurrent unit (GRU), hybrid bidirectional LSTM (BiLSTM), bidirectional GRU, and advanced DL models such as convolutional recurrent neural network and EEG-Conformer models are applied to two popular datasets, SEED and GAMEEMO, respectively, to depict the full process of ER. Additionally, the performance of DL models is also compared with the performance of basic machine learning (ML) models such as SVM, k-nearest neighbors, logistic regression, and boosting algorithms such as AdaBoost, XGBoost and LightGBM. Through extensive experiments and performance evaluations, the performance of different models when applied to the datasets mentioned above is compared. The accuracy, precision, recall, and F1-scores are analysed to determine the most effective model for EEG-based ER. The findings of this study demonstrate that the performance of hybrid DL models is more efficacious than that of ML models. The best-performing model (BiLSTM) classified the emotions, with an accuracy of 90.54% when applied to the GAMEEMO dataset. This research contributes to the growing body of literature on ER and provides insights into the feasibility of using EEG signals to understand emotional states, and presents a structured roadmap for future exploration. The findings can aid in the development of more accurate and reliable ER systems, which can have wide-ranging applications in psychology, social sciences, and human-computer interactions.