Global-Local Self-Attention-Based Long Short-Term Memory with Optimization Algorithm for Speaker Identification

As the need for high-quality video increases it has become necessary to develop smarter compression methods. Artificial neural networks (ANNs) have emerged as an effective tool in this field enabling them to extract the most important features from video and minimize temporal and spatial redundancy resulting in an efficient and high-quality compressed representation. Some of the most prominent models used are: Hybrid neural networks (CNN+RNN(LSTM)) Hybrid neural networks combine convolutional neural networks (CNNs), recurrent neural networks (RNNs), and especially long short-term memory (LSTM) units to extract spatial and temporal features from data, GAN, GANs compress video files while maintaining high perceptual quality. They encode each frame into a compressed latent representation, allowing for realistic reconstruction. This method is efficient, making it ideal for video transmission applications where perceptual accuracy is more important than perfect pixel accuracy. These techniques enable efficient compression without significant quality loss, representing a paradigm shift in modern video compression techniques. After analyzing the results the highest compression ratio (94.75) was achieved by compressing using the GAN algorithm. The highest PSNR (36.08) was achieved by CNN+RNN(LSTM) compression.

A multi-modal Parkinson’s disease diagnosis system from EEG signals and online handwritten tasks using grey wolf optimization based deep learning model

Auto-grading of short answer questions is considered a challenging problem in the processing of natural language. It requires a system to comprehend the free text answers to automatically assign a grade for a student answer compared to one or more model answers. This paper suggests an optimized deep learning model for grading short-answer questions automatically by using various sizes of datasets collected in the Science subject for students in seventh grade in Egypt. The proposed system is a hybrid approach that optimizes a deep learning technique called LSTM (Long Short Term Memory) with a recent optimization algorithm called a Grey Wolf Optimizer (GWO). The GWO is employed to optimize the LSTM by selecting the best dropout and recurrent dropout rates of LSTM hyperparameters rather than manual choice. Using GWO makes the LSTM model more generalized and can also avoid the problem of overfitting in forecasting the students’ scores to improve the learning process and save instructors’ time and effort. The model’s performance is measured in terms of the Root Mean Squared Error (RMSE), the Pearson correlation coefficient, and R-Square. According to the simulation results, the hybrid GWO with the LSTM model ensured the best performance and outperformed the classical LSTM model and other compared models such that it had the highest Pearson correlation coefficient value, the lowest RMSE value, and the best R square value in all experiments, but higher training time than the traditional deep learning model.

Recently, there has been a remarkable growth in Artificial Intelligence (AI) with the development of efficient AI models and high-power computational resources for processing complex datasets. There has been a growing number of applications of machine learning in satellite remote sensing image data processing. In this work, machine learning methods were applied for crop classification of temporal multi- spectral satellite image to achieve better prediction of crop-wise area statistics. In India, agriculture has a huge impact on the national economy and most of the critical decisions are dependent on agricultural statistics. Sentinel-2 satellite image data for the Guntur district region of Andhra Pradesh, India has been used as the study area. The main reason for selecting this region is the diversity of agricultural crops and the availability of ground truth. As a baseline, the performance of machine learning models like Support Vector Machine (SVM), Random Forest (RF), Decision Tree (DT), and K-Nearest Neighbor (KNN) on crop classification was evaluated on 2016 single time multi-spectral satellite image. SVM performed well with an F1 score of 0.94. Further, the performance of SVM, RF, 1D and 2D Convolutional Neural Network (CNN), Recurrent Neural Network (RNN) with Long Short-Term Memory (LSTM) and RNN with Gated Recurrent Unit (GRU) on 2018 Kharif season temporal multi-spectral satellite image has been evaluated for estimation of crop areas. The results show that SVM has the best F1 score of 0.99 and achieved a 95.9% agreement with the ground surveyed crop areas.

Optimised autoencoder-based ensemble deep learning approaches for cyber-physical event classification utilizing synchrophasor PMU data

Accurate locational marginal price forecasting (LMPF) is crucial for the efficient allocation of resources. Nevertheless, the sudden changes in LMP make it inadequate for many existing long short‐term memory (LSTM) network‐based prediction models to achieve the required accuracy for practical applications. This study adopts a hierarchical method of three layers based on double quantum‐inspired grey wolf optimisation (QGWO) to improve the LSTM model (HD‐QGWO‐LSTM) for a one‐step LMPF. The top layer completes the data processing. The middle layer is a QGWO‐optimised support vector machine (SVM) for classifing whether LMPs are price spikes. The bottom laver is a double QGWO‐improved LSTM (QGWO‐LSTM) model for a real LMPF, where one QGWO‐LSTM is for the spike LMPF and the other is for the non‐spike LMPF. To address the issue of excessively long training times during the design of the LSTM network structure and parameter selection, a QGWO algorithm is proposed and used to optimise four LSTM parameters. The simulation results on the New England electricity market show that the HD‐QGWO‐LSTM method achieves similar prediction accuracy to other four LSTM‐based methods. The results also validate that the QGWO algorithm significantly reduces time consumption while ensuring optimisation effectiveness when optimising SVM and LSTM.

Speech signals carry various bits of information relevant to the speaker such as age, gender, accent, language, health, and emotions. Emotions are conveyed through modulations of facial and vocal expressions. This paper conducts an empirical comparison of performances between the classical classifiers: Gaussian Mixture Model (GMM), Support Vector Machine (SVM), K-Nearest Neighbors (KNN), Artificial neural networks (ANN); and the deep learning classifiers, i.e., Long Short-Term Memory (LSTM), Convolutional Neural Network (CNN), and Gated Recurrent Unit (GRU) in addition to the ivector approach for a text-independent speaker verification task in neutral and emotional talking environments. The deep models undergo hyperparameter tuning using the Grid Search optimization algorithm. The models are trained and tested using a private Arabic Emirati Speech Database, Ryerson Audio–Visual Database of Emotional Speech and Song dataset (RAVDESS) database, and a public Crowd-Sourced Emotional Multimodal Actors (CREMA) database. Experimental results illustrate that deep architectures do not necessarily outperform classical classifiers. In fact, evaluation was carried out through Equal Error Rate (EER) along with Area Under the Curve (AUC) scores. The findings reveal that the GMM model yields the lowest EER values and the best AUC scores across all datasets, amongst classical classifiers. In addition, the ivector model surpasses all the fine-tuned deep models (CNN, LSTM, and GRU) based on both evaluation metrics in the neutral, as well as the emotional speech. In addition, the GMM outperforms the ivector using the Emirati and RAVDESS databases.

This study aims to detect faults in wheelset bearings by analyzing vibration-sound fusion data, proposing a novel method based on Grey Wolf Optimizer (GWO) and Support Vector Machine (SVM). Wheelset bearings play a vital role in transportation. However, malfunctions in the bearing might result in extensive periods of inactivity and maintenance, disrupting supply chains, increasing operational costs, and causing delays that affect both businesses and consumers. Fast fault identification is crucial for minimizing maintenance expenses. In this paper, we proposed a new integration of GWO for optimizing SVM hyperparameters, specifically tailored for handling sound-vibration signals in fault detection. We have developed a new fault detection method that efficiently processes fusion data and performs rapid analysis and prediction within 0.0027 milliseconds per data segment, achieving a test accuracy of 98.3%. Compared to the SVM and neural network models built in MATLAB, the proposed method demonstrates superior detection performance. Overall, the GWO-SVM-based method proposed in this study shows significant advantages in fault detection of wheelset bearing vibrations, providing an efficient and reliable solution that is expected to reduce maintenance costs and improve the operational efficiency and reliability of equipment.

Platform X is widely used by students to express emotions related to academic activities, yet it has not been optimally analyzed. This study aims to classify student emotions and compare the performance of Support Vector Machine (SVM) and Long Short-Term Memory (LSTM). The method used is a computational experiment involving data crawling, preprocessing, lexicon-based labeling, and modeling using TF-IDF for SVM and word embedding for LSTM. The dataset consists of 2,914 data points. The results show that SVM achieved an accuracy of 71.18%, outperforming LSTM at 68.95%. This indicates that SVM is more stable on limited datasets, while LSTM tends to overfit. The study concludes that algorithm selection should consider data characteristics and contributes to the comparison of machine learning and deep learning methods. Keywords: LSTM, Sentiment Analysis, Social Media X, Student Emotion, SVM, TF-IDF

Speaker recognition using high-level features has been a successful area of exploration. Features obtained from many different levels?phones, words, prosodic events, etc.?are used to characterize the speaker. A good modeling technique for these features is the support vector machine (SVM). SVMs model the n-gram frequencies from speaker utterances in a high-dimensional SVM feature space and have shown excellent performance over a wide variety of high-level features. A complimentary method of recent exploration in SVM speaker recognition is the use of nuisance attribute projection (NAP). NAP removes directions from SVM feature space that are superfluous to the task of speaker recognition?channel information, session variability, etc. In this paper, we consider the application of NAP to high-level speaker recognition. We describe the difficulties in applying this method and propose solutions. We also conduct experiments showing that NAP can reduce variability in SVM feature space leading to improved performance.

In the process of air combat intention identification, expert experience and traditional algorithm are relied on to analyze enemy aircraft combat intention in a single moment, but the identification time and accuracy are not excellent. In this paper, from the dynamic attributes of an airspace fighter air combat target and the dynamic and time series changing characteristics of the battlefield environment, we introduce the bidirectional long short-term memory neural network (BiLSTM + Attention) intention identification method based on the attention mechanism for air combat intention identification. In this method, five kinds of state parameters, including target maneuver type, distance, flight velocity, altitude and heading angle, were taken as datasets. The BiLSTM + Attention was used to extract enemy aircraft intention features. By introducing attention mechanism, the weight coefficients of characteristic states corresponding to air combat victories were corrected. Finally, it was input into the SoftMax function to obtain the category of the enemy’s intention. Experimental results showed that the proposed method can effectively identify enemy aircraft in the case of high complexity, multidimensional and large amount of data. Compared with bidirectional long short-term memory (BiLSTM), long short-term memory (LSTM), long short-term memory based on attention mechanisms (LSTM + Attention) and support vector machine (SVM) classification, the proposed method had higher accuracy and lower loss value.

Analyzing natural language-based Customer Satisfaction (CS) is a tedious process. This issue is practically true if one is to manually categorize large datasets. Fortunately, the advent of supervised machine learning techniques has paved the way toward the design of efficient categorization systems used for CS. This paper presents the feasibility of designing a text categorization model using two popular and robust algorithms – the Support Vector Machine (SVM) and Long Short-Term Memory (LSTM) Neural Network, in order to automatically categorize complaints, suggestions, feedbacks, and commendations. The study found that, in terms of training accuracy, SVM has best rating of 98.63% while LSTM has best rating of 99.32%. Such results mean that both SVM and LSTM algorithms are at par with each other in terms of training accuracy, but SVM is significantly faster than LSTM by approximately 35.47s. The training performance results of both algorithms are attributed on the limitations of the dataset size, high-dimensionality of both English and Tagalog languages, and applicability of the feature engineering techniques used. Interestingly, based on the results of actual implementation, both algorithms are found to be 100% effective in accurately predicting the correct CS categories. Hence, the extent of preference between the two algorithms boils down on the available dataset and the skill in optimizing these algorithms through feature engineering techniques and in implementing them toward actual text categorization applications.

Long short-term memory (LSTM) is widely applied in both academic and industrial fields. However, there is no reliable criterion on selecting hyperparameters of LSTM. Currently, although some widely used classic methods such as random search and grid search have obtained success to some extent, the problems in local optimum and convergence still exist. In this research, we propose to use grey wolf optimizer (GWO) to search for the hyperparameters of LSTM. Through the method, the superiority of metaheuristic in global optimization and the strength of LSTM in predicting are combined. In this model, number of hidden layer nodes and learning rate of LSTM are set as preys, and grey wolf pack has a simple but efficient mechanism to search for the optimal hyperparameters. The benchmark tests on several basic functions were utilized, and the results were verified by a comparative study with random search, support vector regression and several other regression methods. Specifically, we applied this algorithm in predicting the degradation trend of the airborne fuel pump. As a result, the ergodicity and convergence of the algorithm are proved mathematically based on Markov processes theory. The benchmark tests show that the GWO-LSTM model holds for predicting data with low overall slope and high partial fluctuation. The application in airborne fuel pump shows that, trained by dataset with 5700 points, the proposed model could predict sequence of 300 points with root mean square error 0.617 after 30 iterations of optimizing, which is 2.512 previously. The result further demonstrates that the proposed algorithm is applicable to make prediction with high accuracy. Overall, the effectiveness of GWO-LSTM model is verified from theoretical proof to benchmark tests and then to actual product application.

Sequence Classification is one of the on-demand research projects in the field of Natural Language Processing (NLP). Classifying a set of images or text into an appropriate category or class is a complex task that a lot of Machine Learning (ML) models fail to accomplish accurately and end up under-fitting the given dataset. Some of the ML algorithms used in text classification are KNN, Naïve Bayes, Support Vector Machines, Convolutional Neural Networks (CNNs), Recursive CNNs, Recurrent Neural Networks (RNNs), Long Short Term Memory (LSTM), etc. For this experimental study, LSTM and a few other algorithms were chosen for a more comparative study. The dataset used is the SMS Spam Collection Dataset from Kaggle and 150 more entries were additionally added from different sources. Two possible class labels for the data points are spam and ham. Each entry consists of the class label, a few sentences of text followed by a few useless features that are eliminated. After converting the text to the required format, the models are run and then evaluated using various metrics. In experimental studies, the LSTM gives much better classification accuracy than the other machine learning models. F1-Scores in the high nineties were achieved using LSTM for classifying the text. The other models showed very low F1-Scores and Cosine Similarities indicating that they had underperformed on the dataset. Another interesting observation is that the LSTM had reduced the number of false positives and false negatives than any other model.

Carbon/epoxy specimens were made and stretched to fracture. In the process, acoustic emission (AE) signals were collected and their parameters were set as the input parameters of the neural network. Results show that using support vector machine (SVM) network can recognize the difference of AE sources more accurately than using the BP neural network. In addition, the accuracy of the SVM increases when the number of the training set increases. It is proved that using AE signal parameters and SVM network can recognize the AE sources’ pattern well.