A study of the performance of embedding methods for Arabic short-text sentiment analysis using deep learning approaches

Artificial Intelligence (AI) has been used widely to extract people's opinions from social media websites. However, most of the existing works focus on eliciting the features from English text. In this paper, we describe an Arabic text sentiment analysis approach using a Deep Neural network, namely Long Short-Term Memory Recurrent Neural Network (LSTM-RNN). In this research, we investigate how the different pre-trained Word Embedding (WE) models affect our model's accuracy. The dataset includes Arabic corpus collected from Twitter. The results show significant improvement in Arabic text classification.

Companies that deliver food (food delivery services, or FDS) try to use customer feedback to identify aspects where the customer experience could be improved. Consumer feedback on purchasing and receiving goods via online platforms is a crucial tool for learning about a company's performance. Many English-language studies have been conducted on sentiment analysis (SA). Arabic is becoming one of the most extensively written languages on the World Wide Web, but because of its morphological and grammatical difficulty as well as the lack of openly accessible resources for Arabic SA, like as dictionaries and datasets, there has not been much research done on the language. Using a manually annotated FDS dataset, the current study conducts extensive sentiment analysis using reviews related to FDS that include Modern Standard Arabic and dialectal Arabic. It does this by utilizing word embedding models, deep learning techniques, and natural language processing to extract subjective opinions, determine polarity, and recognize customers' feelings in the FDS domain. Convolutional neural network (CNN), bidirectional long short-term memory recurrent neural network (BiLSTM), and an LSTM-CNN hybrid model were among the deep learning approaches to classification that we evaluated. In addition, the article investigated different effective approaches for word embedding and stemming techniques. Using a dataset of Modern Standard Arabic and dialectal Arabic corpus gathered from Talabat.com, we trained and evaluated our suggested models. Our best accuracy was approximately 84% for multiclass classification and 92.5% for binary classification on the FDS. To verify that the proposed approach is suitable for analyzing human perceptions in diversified domains, we designed and carried out excessive experiments on other existing Arabic datasets. The highest obtained multi-classification accuracy is 88.9% on the Hotels Arabic-Reviews Dataset (HARD) dataset, and the highest obtained binary classification accuracy is 97.2% on the same dataset.

The study presents the results of the development and testing of deep learning neural network architectures, which demonstrate high accuracy rates in classifying neurophysiological data, in particular hemodynamic brain activation patterns obtained by functional near-infrared spectroscopy, during solving mathematical problems on spatial-numerical associations. The analyzed signal represents a multidimensional time series of oxyhemoglobin and deoxyhemoglobin dynamics. Taking the specificity of the fNIRS signal into account, a comparative analysis of 2 types of neural network architectures was carried out: (1) architectures based on recurrent neural networks: recurrent neural network with long short-term memory, recurrent neural network with long short-term memory with fully connected layers, bidirectional recurrent neural network with long short-term memory, convolutional recurrent neural network with long short-term memory; (2) architectures based on convolutional neural networks with 1D convolutions: convolutional neural network, fully convolutional neural network, residual neural network. Trained long short-term memory recurrent neural network architectures showed worse results in accuracy in comparison with 1D convolutional neural network architectures. Residual neural network (model_Resnet) demonstrated the highest accuracy rates in three experimental conditions more than 88% in detecting age-related differences in brain activation during spatial-numerical association tasks considering the individual characteristics of the respondents’ signal.

Sentiment analysis also referred to as opinion mining, plays a significant role in automating the identification of negative, positive, or neutral sentiments expressed in textual data. The proliferation of social networks, review sites, and blogs has rendered these platforms valuable resources for mining opinions. Sentiment analysis finds applications in various domains and languages, including English and Arabic. However, Arabic presents unique challenges due to its complex morphology characterized by inflectional and derivation patterns. To effectively analyze sentiment in Arabic text, sentiment analysis techniques must account for this intricacy. This paper proposes a model designed using the transformer model and deep learning (DL) techniques. The word embedding is represented by Transformer-based Model for Arabic Language Understanding (ArabBert), and then passed to the AraBERT model. The output of AraBERT is subsequently fed into a Long Short-Term Memory (LSTM) model, followed by feedforward neural networks and an output layer. AraBERT is used to capture rich contextual information and LSTM to enhance sequence modeling and retain long-term dependencies within the text data. We compared the proposed model with machine learning (ML) algorithms and DL algorithms, as well as different vectorization techniques: term frequency-inverse document frequency (TF-IDF), ArabBert, Continuous Bag-of-Words (CBOW), and skipGrams using four Arabic benchmark datasets. Through extensive experimentation and evaluation of Arabic sentiment analysis datasets, we showcase the effectiveness of our approach. The results underscore significant improvements in sentiment analysis accuracy, highlighting the potential of leveraging transformer models for Arabic Sentiment Analysis. The outcomes of this research contribute to advancing Arabic sentiment analysis, enabling more accurate and reliable sentiment analysis in Arabic text. The findings reveal that the proposed framework exhibits exceptional performance in sentiment classification, achieving an impressive accuracy rate of over 97%.

The present study aimed to describe and evaluate the current assessment practices prevalent in the different translation courses offered at the College of Languages and Translation (COLT). A sample of specialized translation final exams in 18 translation subject areas was collected. Each final exam was analyzed in terms of the following: (1) # of English and Arabic source texts included on each final exam for each course (2) readability and difficulty level of texts included in the translation exams, (3) # of exams with a terminology subtest, (4) English and Arabic text length in words, (5) reliability, validity and discriminating power of final exams. Data analysis showed that 50% of the exams included one English text, 32% included 2 texts and 18% included 3 texts. Arabic texts were included in 73% of the exams. However, 59% of the exams included one Arabic text, 9% included two texts and 5% included 3 texts. In Addition to English and Arabic texts, 56% of the exams included a vocabulary subtest. 41% do not have any Arabic texts. The English text length ranged between 66-430 words with a median length of 181 words. The Arabic text length ranged between 26-180 with a median length of 97. The typical Flesch Reading Ease of English texts was 40 and the typical Flesch-Kincaid Grade level score was 11. There were no significant differences among the different college levels nor different subject areas in text length or text difficulty level. Translation exams currently used at COLT lack validity, reliability, and discriminating power. Some reasons for lack of reliability and validity are given. Students’ views on translation exams are also reported. A model for more valid, reliable, and discriminating translation exams is given with students views of it as well.

Deep learning analysis of mobile physiological, environmental and location sensor data for emotion detection

Communication is essential to humans because it allows the dissemination of knowledge and the formation of interpersonal connections. We communicate through speaking, facial expressions, hand gestures, reading, writing, and sketching, among other things. However, speaking is the most often utilized means of communication. People having speech and hearing disabilities can only communicate using hand gestures, making them extremely reliant on nonverbal modes of communication. Hearing-impaired persons can communicate via sign language. Globally, around 1 percent(5 million) of the Indian population falls into this group. ISL is a complete language with its own vocabulary, semantics, lexicon, and a variety of other distinctive linguistic features. In our work, we present the methods for Indian sign language recognition at the character and word levels. The Bag of Visual Words(BoVW) technique recognizes ISL at character level(A-Z, 0-9) with an accuracy of 99 percent. Indian Lexicon Sign Language Dataset - INCLUDE-50 dataset is used for word-level sign language recognition. Inception model, a deep Convolutional Neural Network(CNN) is used to train the spatial features and LSTM RNN(Recurrent Neural Network) is used to train the temporal features of the video. Using CNN predictions as input to RNN, we achieved an accuracy of 86.7 %. In order to optimize the training process, only 60 % of the dataset is trained using the Meta-Learning model along with LSTM RNN and obtained an accuracy of 84.4 %, thus reducing the training time by 70 % and reaching nearly as close accuracy as the previous pre-trained model.

Social networks have changed the communication patterns significantly. Information available from different social networking sites can be well utilized for the analysis of users opinion. Hence, the organizations would benefit through the development of a platform, which can analyze public sentiments in the social media about their products and services to provide a value addition in their business process. Over the last few years, deep learning is very popular in the areas of image classification, speech recognition, etc. However, research on the use of deep learning method in sentiment analysis is limited. It has been observed that in some cases the existing machine learning methods for sentiment analysis fail to extract some implicit aspects and might not be very useful. Therefore, we propose a deep learning approach for aspect extraction from text and analysis of users sentiment corresponding to the aspect. A seven layer deep convolutional neural network (CNN) is used to tag each aspect in the opinionated sentences. We have combined deep learning approach with a set of rule-based approach to improve the performance of aspect extraction method as well as sentiment scoring method. We have also tried to improve the existing rule-based approach of aspect extraction by aspect categorization with a predefined set of aspect categories using clustering method and compared our proposed method with some of the state-of-the-art methods. It has been observed that the overall accuracy of our proposed method is 0.87 while that of the other state-of-the-art methods like modified rule-based method and CNN are 0.75 and 0.80 respectively. The overall accuracy of our proposed method shows an increment of 7–12% from that of the state-of-the-art methods.

In recent years, domain generation algorithms (DGAs) are the foundational mechanisms for many malware families. Mainly, due to the fact that DGA can generate immense number of pseudo random domain names to associate to a command and control (C2) infrastructures. This paper focuses on to detect and classify the pseudo random domain names without relying on the feature engineering or any other linguistic, contextual or semantics and statistical information by adopting deep learning approaches. A deep learning approach is a complex model of traditional machine learning mechanism that has received renewed interest by solving the long-standing tasks in artificial intelligence (AI) related to the field of natural language processing, image recognition, speech processing and many others. They have immense capability to extract optimal feature representations by taking input as in the form of raw input texts. To leverage this and to transfer the performance enhancement in aforementioned areas towards characterize, detect and classify the DGA generated domain names to a specific malware family, this paper adopts deep learning mechanisms with a known one million benign domain names from Alexa, OpenDNS and a corpus of malicious domain names generated from 17 DGA malware families in real time for training in character and bigram level and a trained model has been evaluated on the OSNIT data set in real-time. Specifically, to understand the effectiveness of various deep learning mechanisms, we used recurrent neural network (RNN), identity-recurrent neural network (I-RNN), long short-term memory (LSTM), convolution neural network (CNN), and convolutional neural network-long short-term memory (CNN-LSTM) architectures. Additionally, to find out an optimal architecture, experiments are done with various configurations of network parameters and network structures. All experiments run up to 1000 epochs with a learning rate set in the range [0.01-0.5]. Overall, deep learning approaches, particularly family of recurrent neural network and a hybrid network (where the first layer is CNN and a subsequent layer is LSTM) have showed significant performance with a highest detection rate 0.9945 and 0.9879 respectively. The main reason is deep learning approaches have inherent mechanisms to capture hierarchical feature extraction and long range-dependencies in sequence inputs.

Traditional machine learning techniques, including support vector machine (SVM), random walk, and so on, have been applied in various tasks of text sentiment analysis, which makes poor generalization ability in terms of complex classification problem. In recent years, deep learning has made a breakthrough in the research of Natural Language Processing. Convolutional neural network (CNN) and recurrent neural networks (RNNs) are two mainstream methods of deep learning in document and sentence modeling. In this paper, a model of capturing deep sentiment representation based on CNN and long short-term memory recurrent neural network (LSTM) is proposed. The model uses the pre-trained word vectors as input and employs CNN to gain significant local features of the text, then features are fed to two-layer LSTMs, which can extract context-dependent features and generate sentence representation for sentiment classification. We evaluate the proposed model by conducting a series of experiments on dataset. The experimental results show that the model we designed outperforms existing CNN, LSTM, CNN-LSTM (our implement of one-layer LSTM directly stacked on one-layer CNN) and SVM (support vector machine).

Recently, the deep learning based methods, especially the ones based on convolutional neural network (CNN), achieved remarkable progresses in sentiment analysis. However, the CNN based methods do not take the latent topic in text into consideration. In this paper, we propose a CNN based Diversified Restrict Boltzmann Machine (RBM) method to model the sequence level latent topics in the sentences for sentiment analysis. The basic idea is to use CNN mapping the sentences into a sequence level feature space, and then utilize the Diversified RBM to model the latent topics in text. The obtained latent topic features are embedding to improve the performance of sentiment analysis. The evaluations on COAE 2014 and NLPCC 2014 datasets show that our proposed method outperforms the state-of-the-art methods in sentiment analysis. Furthermore, based on our knowledge, this is the first attempt to employ the Diversified RBM in sentiment analysis.

Maxout neurons for deep convolutional and LSTM neural networks in speech recognition

This study focuses on the short-term prediction of traffic delays, for passenger cars, at the three Niagara Frontier Border Crossings, namely the Peace Bridge, the Lewiston–Queenston Bridge, and the Rainbow Bridge. Predictions are made for up to 60 min into the future, using a delay dataset, collected by Bluetooth readers recently installed at these three border crossings. The delay data were first analyzed to identify the factors affecting traffic delay. Next, future delays were predicted using different deep learning techniques and statistical modeling approaches, including (1) multilayer perceptron (MLP); (2) convolutional neural network (CNN); (3) long short-term memory recurrent neural networks (LSTM RNN); (4) gated recurrent unit recurrent neural network (GRU RNN); and (5) the statistical technique known as the auto-regressive integrated moving average (ARIMA) method. A comparative analysis of the prediction accuracy of the results from the different techniques revealed that the deep learning techniques were capable of predicting border traffic delays with high accuracy, resulting in a value of the mean absolute error (MAEs) of less than 3.5 min, even when predicting delays for up to 60 min into the future. The models developed in this study can serve as a part of a traveler information system that guide travelers to the crossing with the least delay, resulting in more efficient border crossing operations.

Sentiment analysis is crucial, in the field of natural language processing (NLP). Has applications in different areas. This study focuses on analyzing sentiments in massive open online course (MOOC) forums highlighting its importance in understanding how users interact and shaping educational strategies. The study presents a novel hybrid neural network model specifically tailored for sentiment analysis in MOOC forums. This innovative model combines word level and character level embeddings to handle the linguistic expressions commonly found in this context. The model architecture integrates bidirectional long short-term memory (BiLSTM) layers for word level embeddings and convolutional neural networks (CNNs) for character level embeddings aiming to harness the strengths of both types of embeddings for a view of the linguistic used in MOOC forum posts. Notably this model achieves an accuracy rate of 93.11% showcasing its effectiveness, in sentiment analysis within MOOC forums. This research contributes to sentiment analysis within the context of online education.

Multi-label text classification is a natural extension of text classification in which each document can be assigned with a possible widespread set of labels. Natural Language Processing (NLP) helps to understand and manipulate text in natural language by using the computer. Arabic Text Classification is challenging recently because the Arabic language is under-resourced although it has many users. The aim of this paper is to build a model to classify Arabic news and help users get and display the most relevant news to their interests. In this paper, we demonstrate the efficiency of using deep learning models in solving Arabic multi-label text classification problem. Multilayer Perceptron (MLP) and Recurrent Neural Network (RNN) with Long Short-Term Memory (LSTM) have been used; we build two models via python. All data has been cleaned to improve the quality of experimental data. The result of test data in LSTM was 82.03 whereas in the MLP model was 80.37, and both models were evaluated using F1 score.