خصائص اللهجة الكويتية المكتوبة واستخدامها في إنشاء موارد للتحليل الصرفي الآلي.

Modern Standard Arabic (MSA) is the formal language in most Arabic countries. Arabic Dialects (AD) or daily language differs from MSA especially in social media communication. However, most Arabic social media texts have mixed forms and many variations especially between MSA and AD. This paper aims to bridge the gap between MSA and AD by providing a framework for the translation of texts of social media. More precisely, this paper focuses on the Tunisian Dialect of Arabic (TAD) with an application on automatic machine translation for a social media text into MSA and any other target language. Linguistic tools such as a bilingual TAD-MSA lexicon and a set of grammatical mapping rules are collaboratively constructed and exploited in addition to a language model to produce MSA sentences of Tunisian dialectal sentences. This work is a first-step towards collaboratively constructed semantic and lexical resources for Arabic Social Media within the ASMAT (Arabic Social Media Analysis Tools) project.

Arabic language has very rich vocabulary. It is manifested in different forms. The formal, Modern Standard Arabic (MSA), and the informal, colloquial or dialects. Dialectical languages become important as a result of the proliferation of social networks which resulted in the vast unstructured dialectical text available on the web. Unique properties of modern standard Arabic and dialects present major challenges to build sentiment analysis systems by adopting similar models designed for the English language. In this paper, we present a supervised Arabic sentiment analysis using a bag-of-words feature. We further examine using a set of key words (lexicon) for better polarity classification. The testing of the system is carried out on the freely-available Arabic books' reviews (LABR) dataset. LABR includes both modern standard Arabic and Egyptian dialectal reviews. We used both balanced and unbalanced datasets. Clearly, the balanced data set is small in size and, henceforth, a large-scale balanced dataset is required for training of the classifier model. Further, we compared the computed predicted sentiments against the actual reviews for a specific book. Findings, by annotators, had indicated ambiguity between a review and its rating when verified alongside the predicted sentiment, which provided a more reasonable result. Moreover, working with dialects and sarcasm is exceedingly exciting. Experimental results on the adopted logistic classifier model and LABR are encouraging and promising. However, a key prerequisite is the availability of rich and well represented datasets in order to develop robust and efficient Arabic sentiment analyzers.

Subject Verb Agreement in Modern Standard Arabic (MSA) and Saudi Dialect of Arabic (SA): A New Minimalist Account Khalaf M. J. AlShammiry Abstract Using Chomsky’s theory of syntax, the Minimalist Program (MP) (1993, 1995, 2000), this paper presents a new analysis for the ongoing debatable word order variation in both Modern Standard Arabic (MSA) and the Saudi dialect of Arabic (SA). It has been noticed that in SVO order, the subject agrees partially with the verb; whereas in VSO order, the subject fully agrees with the verb. In SA, the subject always fully agrees with the verb in both word orders. Previous studies have debated whether the preverbal subject is a subject or a topic or a focus. I argue that the noun in the MSA and SA is a subject which is either base generated in the case of the MSA or moved from inside the clause in the case of SA; making use of the distinction between al-jumlaalismiyya “ the nominal sentence” and al-jumlaal-fi?liyya “the verbal sentence” in Arabic, Abd Al-bidee (1996) and among others, I ague that there is a Head Phrase (HP) at the left periphery of the clause where the subject surfaces to check the nominal head feature of the HP and to mark the clause nominal. For this argument, I assume that the clitic onto the verb is either an agreement marker or a subject and the number feature is weak in MSA and strong in SA. In addition, before the syntax, the noun is split into an expletive and a noun or a clitic and the TP in SA has two specs. The two supporting pieces of evidence for my argument are the subject of the embedded clauses and the structure of the wh-questions in MSA. The contribution of the paper lies in the fact that it investigates the agreement issue in a dialect that is, to my knowledge, heretofore uninvestigated; and adds to our understanding of the syntax of the agreement asymmetry in MSA and SA. Full Text: PDF DOI: 10.15640/jflcc.v4n2a8

Named entities recognition (NER) has become, over time, a potentially helpful pre-processing for several NLP tasks. It mainly identifies and classifies entities in the text into predefined categories, such as a person, location, and organization. Most of the work done in Arabic named entity recognition (ANER) has focused on Modern Standard Arabic (MSA). However, most scripts on the internet, especially in social media (which are currently a source for corpus development), are in a dialectal form that does not follow standard writing rules. This paper investigates the possibility of deep learning based on named entity recognition in Algerian dialect script through a comparative study between 5 models: AraBERT, DziriBERT, MARBERT, ARBERT, and mBERT. We chose these five models for two significant reasons; the first one, that these models are already pre-trained on the MSA and Arabic dialect text, the second reason, that they have proved their efficiency for other tasks such as Part of Speech Tagging, sentiment analysis, and ANER.KeywordsNLPDeep learningAlgerian dialectNamed Entity RecognitionSocial media

Arabic natural language processing: An overview

The Impact of Arabic Part of Speech Tagging on Sentiment Analysis: A New Corpus and Deep Learning Approach

Arabic language suffers from the lack of available large datasets for machine learning and sentiment analysis applications. This work adds to the recently reported large dataset BRAD, which is the largest Book Reviews in Arabic Dataset. In this paper, we introduce HARD (Hotel Arabic-Reviews Dataset), the largest Book Reviews in Arabic Dataset for subjective sentiment analysis and machine language applications. HARD comprises of 490587 hotel reviews collected from the Booking.com website. Each record contains the review text in the Arabic language, the reviewer’s rating on a scale of 1 to 10 stars, and other attributes about the hotel/reviewer. We make available the full unbalanced dataset as well as a balanced subset. To examine the datasets, we implement six popular classifiers using Modern Standard Arabic (MSA) as well as Dialectal Arabic (DA). We test the sentiment analyzers for polarity and rating classifications. Furthermore, we implement a polarity lexicon-based sentiment analyzer. The findings confirm the effectiveness of the classifiers and the datasets. Our core contribution is to make this benchmark-dataset available and accessible to the research community on Arabic language.

Parallel corpora available for building machine translation (MT) models for dialectal Arabic (DA) are rather limited. The scarcity of resources has prompted the use of Modern Standard Arabic (MSA) abundant resources to complement the limited dialectal resource. However, dialectal clitics often differ between MSA and DA. This paper compares morphology-aware DA word segmentation to other word segmentation approaches like Byte Pair Encoding (BPE) and Sub-word Regularization (SR). A set of experiments conducted on Egyptian Arabic (EA), Levantine Arabic (LA), and Gulf Arabic (GA) show that a sufficiently accurate morphology-aware segmentation used in conjunction with BPE outperforms the other word segmentation approaches.

We present in this chapter a practical approach in building Arabic automatic speech recognition (ASR) system for mobile telecommunication service applications. We also present a procedure in conducting acoustic modelling adaptation to better take into account the pronunciation variation across the Arabic speaking countries. Modern Standard Arabic (MSA) is the common spoken and written language for all the Arab countries, ranging from Morocco in the west to Syria in the East, including Egypt, and Tunisia. However, the pronunciation varies significantly from one country to another to a degree that two persons from different countries may not be able understand each other. This is because Arabic speaking countries are characterized by a large number of dialects that differ to an extent that they are no longer mutually intelligible and could almost be described as different languages. Arabic dialects are often spoken rather than written varieties. MSA is common across the Arab countries, but it is often influenced by the dialect of the speaker. This particularity of the Arabic countries constitutes a practical problem in the development of a speech-based application in this region; suppose a speech application system is built for one country influenced by one dialect, what does it take to adapt the system to serve another country with a different dialect region? This is particularly challenging since resource to build accurate speaker independent Arabic ASR system for mobile telecommunication service applications are limited for most of the Arabic dialects and countries. Recent advances in speaker independent automatic speech recognition (SI-ASR) have demonstrated that highly accurate recognition can be achieved, if enough training data is available. However, the amount of available speech data that take into account the dialectal variation of each Arabic country is limited, making it challenging to build a high performance SI-ASR system, especially when we target specific applications. Another big challenge when building an SI-ASR is to handle speaker variations in spoken language. These variations can be due to age, gender, educational level as well as the dialectical variants of Arabic language. Usually an ASR system trained in one regional variation exhibits poorer performance when applied to another regional variation. Three problems may arise when a SI-ASR system built for one dialect but applied to target users with a different dialect: (1) Acoustic model mismatch, (2) Pronunciation lexicon mismatch and (3) Language model mismatch.

This paper argues that Arabic function words (FWs) vary in usage between old and modern Arabic, thus prompting an experimental investigation into their changeability. This investigation is carried out by testing classical Arabic (CA) in Arabic heritage language (AHL) texts – those labeled as archistratum – and the modern standard Arabic (MSA) of Arabic newspaper texts (ANT), each group of which contains randomly collected 5 million (M) word texts. The linguistic theory of the grammar of Arabic FWs is explained through the differences between CA and MSA, despite Arabic FW changes and the unlearnability and/or unusability of some FW constructions between in these two eras of Arabic usage. The dispersion/distribution of the construction grammar (CxG) of FWs and the number (n) of word attractions/repulsions between the two distinct eras is explored using the very latest and most sophisticated Arabic corpus processing tools, and Sketch Engine’sSkeEn gramrelsoperators. The analysis of a 5 M word corpus from each era of Arabic serves to prove the non-existence of rigorous Arabic CxG. The approach in this study adopts a technique which, by contrasting AHL with ANT, relies on analyzing the frequency distributions of FWs, the co-occurrences of FWs in a span of 2n-grams collocational patterning, and some cases of FW usage changes in terms of lexical cognition (FW grammatical relationships). The results show that the frequencies of FWs, in addition to the case studies, are not the same, and this implies that FWs and their associations with the main part of speech class in a fusion language like Arabic have grammatically changed in MSA. Their constructional changes are neglected in Arabic grammar.

The present chapter deals with the coarticulatory effects of the contrast of pharyngealisation in Modern Standard Arabic (MSA) and Dialectal Arabic (DA). The first part of the study reports acoustic data, based on locus equation (LE) parameters extracted from the production of sixteen subjects from Yemen, Kuwait, Jordan and Morocco who spoke 24 words in MSA and 24 words in DA, with a symmetrical VCV context [iCi, uCu, aCa] where C was either pharyngealised /tˁ, dˁ, sˁ, ðˁ/ or non-pharyngealised /t, d, s, ð/. LE parameters were found to accurately reflect pharyngealisation patterns of CV coarticulatory effects according to speech variety and geographical clustering. The second part of the study deals with EMA data from one Tunisian speaker. It provides some observations of the articulatory mechanism of pharyngealisation. This section provides also a comprehensive interpretation of the coarticulatory effects of pharyngealisation as revealed by locus equation parameters.

The rise of social media such as blogs and social networks has fueled interest in sentiment analysis. With the proliferation of reviews, ratings, recommendations and other forms of online expression, online opinion has turned into a kind of virtual currency for businesses looking to market their products, identify new opportunities and manage their reputations, therefore many are now looking to the field of sentiment analysis. In this paper, we present a feature-based sentence level approach for Arabic sentiment analysis. Our approach is using Arabic idioms/saying phrases lexicon as a key importance for improving the detection of the sentiment polarity in Arabic sentences as well as a number of novels and rich set of linguistically motivated features contextual Intensifiers, contextual Shifter and negation handling), syntactic features for conflicting phrases which enhance the sentiment classification accuracy. Furthermore, we introduce an automatic expandable wide coverage polarity lexicon of Arabic sentiment words. The lexicon is built with gold-standard sentiment words as a seed which is manually collected and annotated and it expands and detects the sentiment orientation automatically of new sentiment words using synset aggregation technique and free online Arabic lexicons and thesauruses. Our data focus on modern standard Arabic (MSA) and Egyptian dialectal Arabic tweets and microblogs (hotel reservation, product reviews, etc.). The experimental results using our resources and techniques with SVM classifier indicate high performance levels, with accuracies of over 95%.

The study investigated the acquisition of Modern Standard Arabic (MSA) by second language (L2) learners and by heritage speakers of the colloquial varieties of Arabic. The study focused on three questions: (1) whether heritage speakers who enroll in college‐level elementary MSA classes have an advantage over their L2 counterparts, (2) whether any potential advantage that heritage speakers may bring into the classroom carries on to a later stage of their MSA learning, and (3) the role of positive or negative transfer from the colloquial variety in the acquisition of MSA in a formal setting. Thirty‐five heritage speakers (19 elementary and 16 advanced), 28 advanced L2 learners, and 16 native‐speaker controls completed five negation tasks. The results indicated that, whereas elementary heritage speakers entered the MSA classroom with an advantage over their L2 counterparts in the area under study, the advanced heritage speakers exhibited patterns that were comparable to those of the L2 learners. Unlike their L2 counterparts, however, the advanced heritage speakers displayed positive and negative transfer effects from colloquial Arabic. The findings are discussed in the framework of three models/hypotheses of first language transfer to a third language, and their pedagogical implications for Arabic language teaching and learning are explicated.

Globalization has had an impact on the education system in the UAE, where increased use of bilingual curriculum (Arabic-English) is held in high regard. Nevertheless, literacy among Emirati children and teenagers remains low. This study uses a 15-item, open-ended questionnaire completed by Emirati parents and an 8-item, open-ended questionnaire completed by Emirati primary school children from 8 to 11 years of age, and compares the translation of The Little prince into Modern Standard Arabic (MSA) and into the Emirati dialect. The results suggest that not only the Emirati dialect, but any Arabic dialect (in its respective Arab country) could be used in primary schools to motivate children to read in Arabic and bridge the gap between their spoken language (dialect) and formal written Arabic. Not only must an Arab child learn how to read, but also, they have to understand a very formal language system that they are not used to speaking at home, i.e., Modern Standard Arabic (MSA). Having books in their own dialects may make children more interested in reading, as they can understand them more easily.

A reasonable amount of annotated data is required for fine-tuning pre-trained language models (PLM) on down-stream tasks. However, obtaining labeled examples for different language varieties can be costly. In this paper, we investigate the zero-shot performance on Dialectal Arabic (DA) when fine-tuning a PLM on modern standard Arabic (MSA) data only— identifying a significant performance drop when evaluating such models on DA. To remedy such performance drop, we propose self-training with unlabeled DA data and apply it in the context of named entity recognition (NER), part-of-speech (POS) tagging, and sarcasm detection (SRD) on several DA varieties. Our results demonstrate the effectiveness of self-training with unlabeled DA data: improving zero-shot MSA-to-DA transfer by as large as ~10% F₁ (NER), 2% accuracy (POS tagging), and 4.5% F₁ (SRD). We conduct an ablation experiment and show that the performance boost observed directly results from the unlabeled DA examples used for self-training. Our work opens up opportunities for leveraging the relatively abundant labeled MSA datasets to develop DA models for zero and low-resource dialects. We also report new state-of-the-art performance on all three tasks and open-source our fine-tuned models for the research community.