Urdu Sentiment Analysis

Urdu language written in English alphabets for communication is known as Roman Urdu. In pronunciation, both are the same but different in spelling and have different shapes of the alphabet. A survey acknowledges that 300 million people are speaking Urdu and about 11 million speakers in Pakistan from which maximum users prefer Roman Urdu for the textual communication. Today most of the modern technologies like computers and mobile phones using English script, due to this local Urdu user has to use English letters to type Urdu script that is Roman Urdu. In this research, Roman Urdu to Urdu Translator (RUTUT) is proposed that consists of preprocessing methods, rule-based character substitution and Unicode based character mapping techniques. It can transliterate the messages or descriptions from the Roman Urdu script to Urdu script which may help the Urdu speaker to elaborate their message in efficient manners. The focus of this research is to analyze the issues related to the Roman Urdu script to Urdu script transliteration and develop a translator based on the concepts of transliteration. This research analyzed Roman Urdu data and identified different rules-based character substitution techniques that transform the Roman Urdu into Urdu script at fundamental levels. This research is carried out using a python programming language in programming tool Anaconda in Jupiter notebook and user-friendly Graphical User Interface (GUI) created by using Tkinter library. To evaluate the RUTUT, different translational tests are performed and compare those results with famous Google online translator and ijunoon online transliteration. The analyses of results show that the proposed RUTUT approach translates accurately than Google online translator and ijunoon online transliteration.

The majority of online comments/opinions are written in text-free format. Sentiment Analysis can be used as a measure to express the polarity (positive/negative) of comments/opinions. These comments/ opinions can be in different languages i.e. English, Urdu, Roman Urdu, Hindi, Arabic etc. Mostly, people have worked on the sentiment analysis of the English language. Very limited research work has been done in Urdu or Roman Urdu languages. Whereas, Hindi/Urdu is the third largest language in the world. In this paper, we focus on the sentiment analysis of comments/opinions in Roman Urdu. There is no publicly available Roman Urdu public opinion dataset. We prepare a dataset by taking comments/opinions of people in Roman Urdu from different websites. Three supervised machine learning algorithms namely NB (Naive Bayes), LRSGD (Logistic Regression with Stochastic Gradient Descent) and SVM (Support Vector Machine) have been applied on this dataset. From results of experiments, it can be concluded that SVM performs better than NB and LRSGD in terms of accuracy. In case of SVM, an accuracy of 87.22% is achieved.

Deep sentiments in Roman Urdu text using Recurrent Convolutional Neural Network model

Emotion detection (ED) and sentiment analysis (SA) play a vital role in identifying an individual’s level of interest in any given field. Humans use facial expressions, voice pitch, gestures, and words to convey their emotions. Emotion detection and sentiment analysis in English and Chinese have received much attention in the last decade. Still, poor-resource languages such as Urdu have been mostly disregarded, which is the primary focus of this research. Roman Urdu should also be investigated like other languages because social media platforms are frequently used for communication. Roman Urdu faces a significant challenge in the absence of corpus for emotion detection and sentiment analysis because linguistic resources are vital for natural language processing. In this study, we create a corpus of 1021 sentences for emotion detection and 20,251 sentences for sentiment analysis, both obtained from various areas, and annotate it with the aid of human annotators from six and three classes, respectively. In order to train large-scale unlabeled data, the bag-of-word, term frequency-inverse document frequency, and Skip-gram models are employed, and the learned word vector is then fed into the CNN-LSTM model. In addition to our proposed approach, we also use other fundamental algorithms, including a convolutional neural network, long short-term memory, artificial neural networks, and recurrent neural networks for comparison. The result indicates that the CNN-LSTM proposed method paired with Word2Vec is more effective than other approaches regarding emotion detection and evaluating sentiment analysis in Roman Urdu. Furthermore, we compare our based model with some previous work. Both emotion detection and sentiment analysis have seen significant improvements, jumping from an accuracy of 85% to 95% and from 89% to 93.3%, respectively.

Sentiment analysis is a dynamic field focused on understanding and predicting emotional sentiments in text or images. With the prevalence of smartphones, e-commerce and social networks, individuals readily express opinions, aiding businesses, political analysts and organizations in decision-making. Despite extensive research in sentiment analysis for various languages, challenges persist in low-resource languages like Roman Urdu. Roman Urdu, the use of Roman script to write Urdu, has gained popularity, yet limited linguistic resources hinder sentiment analysis research. This study addresses this gap by developing a bidirectional long short-term memory network with FastText embeddings and additional layers. A large Roman Urdu corpus for sentiment analysis, consisting of over 51 000 reviews, is crated and the proposed model is trained and compared with 14 other models, demonstrating an accuracy of 0.854 and an F1-score of 0.84.

Sentiment analysis is the computational study of re-views, emotions, and sentiments expressed in the text. In the past several years, sentimental analysis has attracted many concerns from industry and academia. Deep neural networks have achieved significant results in sentiment analysis. Current methods mainly focus on the English language, but for minority languages, such as Roman Urdu that has more complex syntax and numerous lexical variations, few research is carried out on it. In this paper, for sentiment analysis of Roman Urdu, the novel “Self-attention Bidirectional LSTM (SA-BiLSTM)” network is proposed to deal with the sentence structure and inconsistent manner of text representation. This network addresses the limitation of the unidirectional nature of the conventional architecture. In SA-BiLSTM, Self-Attention takes charge of the complex formation by correlating the whole sentence, and BiLSTM extracts context rep-resentations to tackle the lexical variation of attended embedding in preceding and succeeding directions. Besides, to measure and compare the performance of SA-BiLSTM model, we preprocessed and normalized the Roman Urdu sentences. Due to the efficient design of SA-BiLSTM, it can use fewer computation resources and yield a high accuracy of 68.4% and 69.3% on preprocessed and normalized datasets, respectively, which indicate that SA-BiLSTM can achieve better efficiency as compared with other state-of-the-art deep architectures.

Sentiment analysis is the computational study of attitudes, opinions, and sentiments towards certain issues, products, individuals, and organizations. Companies and customers are making decisions by seeking opinions from social media text. Sentiment analysis is getting intelligent with the advancement of artificial intelligence and natural language processing. With a stunning increase in the use of social media, a huge volume of text available on these platforms is in imperfect and informal languages like Roman Urdu mixed with the English language. Present sentiment analysis techniques do not perform precisely on these code-mixed imperfect, informal, and poorly resourced languages. A promising solution is the use of deep learning models on these code-mixed Roman Urdu and English text. Therefore, the objective of this paper is to perform a sentiment analysis of code-mixed Roman Urdu and English social media text using state-of-the-art deep learning models. Our work is independent of lexical normalization, language dictionary, and code transfer indication. We perform sentiment analysis using Multilingual BERT (mBERT) and XLM-RoBERTa (XLM-R) models. The results reveal that performance of XLM-R model with tuned hyperparameters for code-mixed Roman Urdu and English social media text is better than the mBERT model with F1 score of 71%.

Over the last two decades, social media platforms have grown dramatically. Twitter and Facebook are the two most popular social media platforms, with millions of active users posting billions of messages daily. These platforms allow users to have freedom of expression. However, some users exploit this facility by disseminating hate speeches. Manual detection and censorship of such hate speeches are impractical; thus, an automatic detection mechanism is required to detect and counter hate speeches in a real-time environment. Most research in hate speech detection has been carried out in the English language. Still, minimal work has been explored in other languages, mainly Urdu written in Roman Urdu script. A few research have attempted machine learning, and deep learning models for Roman Urdu hate speech detection; however, due to a scarcity of Roman Urdu resources, and a large corpus with defined annotation rules, a robust hate speech detection model is still required. With this motivation, this study contributes in the following manner: we developed annotation guidelines for Roman Urdu Hate Speech. Second, we constructed a new Roman Urdu Hate Speech Dataset (RU-HSD-30K) that was annotated by a team of experts using the annotation rules. To the best of our knowledge, the Bi-LSTM model with an attention layer for Roman-Urdu Hate Speech Detection has not been explored. Therefore, we developed a context-aware Roman Urdu Hate Speech detection model based on Bi-LSTM with an attention layer and used custom word2vec for word embeddings. Finally, we examined the effect of lexical normalization of Roman Urdu words on the performance of the proposed model. Different traditional as well as deep learning models, including LSTM and CNN models, were used as baseline models. The performance of the models was assessed in terms of evaluation matrices like accuracy, precision, recall, and F1-score. The generalization of each model is also evaluated on a cross-domain dataset. Experimental results revealed that Bi-LSTM with attention outperformed the traditional machine learning models and other deep learning models with an accuracy score of 0.875 and an F-Score of 0.885. In addition, the results demonstrated that our suggested model (Bi-LSTM with Attention Layer) is more general than previous models when applied to unseen data. The results confirmed that lexical normalization of Roman Urdu words enhanced the performance of the suggested model.

Social media have become a very viable medium for communication, collaboration, exchange of information, knowledge, and ideas. However, due to anonymity preservation, the incidents of hate speech and cyberbullying have been diversified across the globe. This intimidating problem has recently sought the attention of researchers and scholars worldwide and studies have been undertaken to formulate solution strategies for automatic detection of cyberaggression and hate speech, varying from machine learning models with vast features to more complex deep neural network models and different SN platforms. However, the existing research is directed towards mature languages and highlights a huge gap in newly embraced resource poor languages. One such language that has been recently adopted worldwide and more specifically by south Asian countries for communication on social media is Roman Urdu i-e Urdu language written using Roman scripting. To address this research gap, we have performed extensive preprocessing on Roman Urdu microtext. This typically involves formation of Roman Urdu slang- phrase dictionary and mapping slangs after tokenization. We have also eliminated cyberbullying domain specific stop words for dimensionality reduction of corpus. The unstructured data were further processed to handle encoded text formats and metadata/non-linguistic features. Furthermore, we performed extensive experiments by implementing RNN-LSTM, RNN-BiLSTM and CNN models varying epochs executions, model layers and tuning hyperparameters to analyze and uncover cyberbullying textual patterns in Roman Urdu. The efficiency and performance of models were evaluated using different metrics to present the comparative analysis. Results highlight that RNN-LSTM and RNN-BiLSTM performed best and achieved validation accuracy of 85.5 and 85% whereas F1 score was 0.7 and 0.67 respectively over aggression class.

Sentiment analysis plays a pivotal role in understanding the sentiments and opinions expressed in textual data, offering valuable insights into various domains, including tourism. In this study, we present a comprehensive review of sentiment analysis techniques applied to tourism reviews using machine learning algorithms. The abundance of user-generated content on tourism platforms has made sentiment analysis an indispensable tool for businesses and researchers alike. By leveraging machine learning algorithms, researchers can extract sentiments from vast amounts of textual data efficiently and accurately. This review outlines the key methodologies and approaches utilized in sentiment analysis of tourism reviews. It discusses preprocessing techniques such as text tokenization, stop-word removal, and stemming, which are crucial for preparing textual data for analysis. Furthermore, it examines various machine learning algorithms employed for sentiment classification, including Naive Bayes, Support Vector Machines, and Recurrent Neural Networks. Additionally, the review delves into feature extraction methods such as bag-of-words, TF-IDF, and word embeddings, highlighting their impact on sentiment analysis accuracy. Moreover, it explores the challenges and limitations associated with sentiment analysis in the tourism domain, such as sarcasm detection and language nuances.

Urdu is widely spoken and a national language of Pakistan. The language covers huge variety from others languages as well therefore known as “Lashkari Zuban” (a mixture of different languages. We have performed experiments on Urdu Named Entity Recognition (NER) using model-based approach. NER is the task for the identification and classification of named entities from the given text therefore; name, place, organization, time/date, etc. The task has an important role for automated systems, information extraction, machine learning and artificial intelligence. A lot of work has been done for European languages but the task for South Asian languages is its development stage. We chose Urdu language as it is our national language but still there are a lot of challenges in Urdu language as the language as very limited resources and it is also free structured language. Our research has been conducted on IJCNLP-08 dataset which is IOB (inside Outside Beginning) tagged using maximum entropy model. Precision, Recall and F-measure are used to evaluate the accuracy of the model.

Though Urdu language has no religious background, it is inspired by Arabic a mother-tongue of Islam. However, after its development, Urdu was exclusively attributed to be the language of Muslims of the Sub-continent. They faced many challenges for the survival of Urdu during the Movement of Independence. For the same reason, after independence, Urdu was declared to be the National Language of Pakistan. Afterward, certain bodies were established for its proper implementation as official language of the country. This aspect is recognized as constitutional obligation in term of Article 251 of Constitution, 1973. Though, the government was given sufficient time for the compliance of the same, this task could not have been accomplished despite of lapse of almost fifty years. Even, the judicial verdicts could not make the concerned institutions mindful. Instead, English is the official language of Pakistan since independence. Of course, there is difference of opinion on the matter of Official Language. Among others, the medium of education and legislation including policy papers are the major issues which are on English pattern. However, if these issues are addressed, the constitutional obligation of implementing Urdu as official language may be ensured.

The analysis of the opinions and likelihood as well as emotions in any form either in the form of text is called as sentiment analysis. Opinion mining is termed as sentiment analysis. The analysis of the data is very helpful in expressing the likelihood of the options for group of persons or individuals. With the advancement of the Internet, a huge collection data is being generated. Facebook, Twitter, YouTube, LinkedIn, Instagram and other social sites are gaining a lot of popularity as they allow users from different parts of the world to share their views upon various topics through comments, posts, tweets and tags. This paper provides a survey of existing technique for sentimental analysis like machine learning-based approaches like Naive Bayes, logistic regression and SVM. Among them, decision tree recorded the best performance. Decision tree generated a classification accuracy of 91.6% and a minimum execution time of 1.12 s. Hence, it can be concluded that sentiment analysis with decision tree algorithm is an optimal concept for efficient movie review analysis.

Nowadays, people find it easier to express opinions via social media-formally known as Web 2.0. Sentiment analysis is an essential field under natural language processing in Computer Science that deals with analyzing people's opinions on the subject matter and discovering the polarity they contain. These opinions could be processed in collective form (as a document) or segments or units as sentences or phrases. Sentiment analysis can be applied in education, research optimization, politics, business, education, health, science and so on, thus forming massive data that requires efficient tools and techniques for analysis. Furthermore, the standard tools currently used for data collection, such as online surveys, interviews, and student evaluation of teachers, limit respondents in expressing opinions to the researcher's surveys and could not generate huge data as Web 2.0 becomes bigger. Sentiment analysis techniques are classified into three (3): Machine learning algorithms, lexicon and hybrid. This study explores sentiment analysis of Web 2.0 for novice researchers to promote collaboration and suggest the best tools for sentiment data analysis and result efficiency. Studies show that machine learning approaches result in large data sets on document-level sentiment classification. In some studies, hybrid techniques that combine machine learning and lexicon-based performance are better than lexicon. Python and R programming are commonly used tools for sentiment analysis implementation, but SentimentAnalyzer and SentiWordnet are recommended for the novice.
 Keywords: Sentiment Analysis; Web 2.0; Applications; Tools; Novice

A subfield of artificial intelligence is that of natural language processing and sentiment analysis. The interest in sentiment analysis has increased with the development of the internet and web 2.0. With sentiment analysis it is possible to analyse the sentiment or emotions of written communication through dictionary-based sentiment analysis or machine learning algorithms. However, sentiment analysis also holds the potential of supporting people with disadvantages in interpreting the nuances in written communication. One such group is autistic people. The aim of the study is to examine autistic peoples’ perceptions of important design factors and functionality for an application with sentiment analysis to support emotional navigation in written communication.
 This study has been conducted with the first steps of design science to outline the requirements of a potential application that can support autistic people to navigate the emotions in written communication with sentiment analysis. The problem to be addressed was identified through related research and one of authors’ own experience of navigating written communications with autism. The requirements for the application were the main focus for this study and has been the primarily concern for data collection. Data have been collected through semi-structured interviews with autistic people and analysed with thematic analysis.
 Results of the study provide several important recommendations for the design of applications with sentiment analysis to support autistic people navigate the emotions in written communication. The study further provides an understanding of autistic peoples’ needs when navigating written communication. These findings can be used by researchers and developers to design support-applications with autistic peoples’ needs in the centre. An interesting next step of research would be to develop a prototype with the findings of this study addressed in the application’s functionality and design, which could then be evaluated on a larger scale.