Annotated Slovak Datasets for Toxicity, Hate Speech, and Sentiment Analysis

Online hate has emerged as a rapidly growing issue worldwide, often stemming from differences in opinion. It is crucial to use appropriate language and words on social media platforms, as inappropriate communication can negatively impact others. Consequently, detecting hate speech is of significant importance. While manual methods are commonly employed to identify hate and offensive content on social media, they are time-consuming, labor-intensive, and prone to errors. Therefore, AI-based approaches are increasingly being adopted for the effective classification of hate and offensive speech. The proposed model incorporates various text preprocessing techniques, such as removing extraneous elements like URLs, emojis, and blank spaces. Following preprocessing, tokenization is applied to break down the text into smaller components or tokens. The tokenization technique utilized in this study is TF-IDF (Term Frequency–Inverse Document Frequency). After tokenization, the model performs the classification of hate and offensive speech using the proposed BiLSTM-based SM-CJ (Scalable Multi-Channel Joint) framework. The BiLSTM-based SM-CJ model is particularly effective in detecting hate, offensive, and neutral tweets due to its ability to capture both forward and backward contexts within a given text. Detecting hate speech requires a comprehensive understanding of the text and the identification of patterns spanning across multiple words or phrases. To achieve this, the LSTM component of the BiLSTM model is designed to capture long-term dependencies by utilizing information from earlier parts of the text. The proposed SM-CJ framework further aligns the input sequence lengths fetched from the input layer, enabling the model to focus on specific segments of the input sequence that are most relevant for hate speech detection. This approach allows the model to accurately capture derogatory language, and subtle nuances present in hate speech. Finally, the performance of the proposed framework is evaluated using various metrics, including accuracy, recall, F1-score, and precision. The results are compared with state-of-the-art approaches, demonstrating the effectiveness of the proposed model. Doi: 10.28991/ESJ-2025-09-01-03 Full Text: PDF

Social media sites facilitate users’ discussions, expression of opinions, sharing of information and news, and promotion of ideas and products, thus rapidly increasing the volume of hate speech and offensive content on online platforms. Consequently, hate speech and offensive content have turned out to be a widespread issue that negatively affects both individuals and society and needs to be controlled through detection and removal. This paper aims to provide a further understanding of the meaning of hate speech and offensive language and to provide a comprehensive discussion of the various techniques to detect hate speech and offensive language. A systematic literature review (SLR) of 90 research papers published between 2018 and 2024 was conducted to discover gaps within the literature. This review revealed challenges and possibilities for further development and improvement of previous findings. The results show that most of these works classified hate speech and offensive language using techniques from machine learning (ML) and deep learning (DL), and the most common performance metrics were Accuracy, Precision, Recall, and F‐measure. The benchmark datasets are also described. Twitter was the most commonly utilized social network for obtaining datasets, while Facebook is sometimes used. Moreover, the findings of this review offer insight into research trends in Arabic hate speech and offensive language, as well as new research directions. The most interesting finding is that until now most Social Media Network Developers have not included autodetection of hate speech or offensive language plugins. Finally, this study presents a guideline for choosing the best strategies and techniques to detect and predict Arabic offensive language and hate speech.

This article investigates the identification of hate speech on social media using machine learning and deep learning techniques. The research uses metrics such as F-measure, AUC-ROC, precision, accuracy, and recall assessing the effectiveness of various tactics. The findings indicate that deep learning models, particularly the bidirectional long short-term memory (BiLSTM) architecture, consistently outperform other methods in categorization tasks. The research emphasizes the importance of sophisticated neural network designs in identifying the intricacies of hostile and offensive content online. The study offers insights for promoting early identification and prevention of cyberbullying, improving secure and inclusive online environments. Future research may explore real-time detection systems, hybrid approaches, or the integration of complementary components to enhance and improve innovative technology in tackling this significant social issue. A sample tweet was annotated by specialists who categorize tweets as hate speech, offensive language, or neutral. The researchers applied shallow learning methodologies and integrated word embeddings like Word2Vec and GloVe to enhance the efficacy of deep learning models. The results indicate that BiLSTM surpasses shallow learning methods in detecting hate speech on Twitter, highlighting the efficacy of deep learning approaches in recognizing and tracking hate speech on social media platforms. When comparing different deep learning and machine learning models on different datasets, the results reveal that deep learning techniques are usually more effective. A reasonably high level of accuracy is achieved by KNN and SVM among classical algorithms, whereas Naïve Bayes performs the poorest. While deep learning approaches provide better results, tree-based models such as Random Forest and Decision Trees offer more consistent accuracy. Models based on neural networks, such as LSTM, CNN, and BI-LSTM, perform well, with LSTM-based methods excelling in particular. The most successful strategy for classification problems is the model presented, which obtains the greatest accuracy, precision, recall, F1-score of 95%. The research aids in the development of advanced tools and methodologies to mitigate hate speech on social media and foster positive online interactions. Future research may investigate alternative deep learning architectures, such as transformers, to enhance hate speech detection efficacy. The advancement of interpretable AI methodologies for identifying hate speech and delivering transparent forecasts might enhance user confidence and facilitate better content moderation decisions.

Hate speech towards a group or an individual based on their perceived identity, such as ethnicity, religion, or nationality, is widely and rapidly spreading on social media platforms. This causes harmful impacts on users of these platforms and the quality of online shared content. Fortunately, researchers have developed different machine learning algorithms to automatically detect hate speech on social media platforms. However, most of these algorithms focus on the detection of hate speech that appears in English. There is a lack of studies on the detection of hate speech in Arabic due to the language’s complex nature. This paper aims to address this issue by proposing an effective approach for detecting Arabic hate speech on social media platforms, namely Twitter. Therefore, this paper introduces the Arabic BERT-Mini Model (ABMM) to identify hate speech on social media. More specifically, the bidirectional encoder representations from transformers (BERT) model was employed to analyze data collected from Twitter and classify the results into three categories: normal, abuse, and hate speech. In order to evaluate our model and state-of-the-art approaches, we conducted a series of experiments on Twitter data. In comparison with previous works on Arabic hate-speech detection, the ABMM model shows very promising results with an accuracy score of 0.986 compared to the other models.

In recent times, the Internet and social media are very well-known and popular among people. Usage of social media is increased exponentially during the last few years globally, and it allows people to engage with one another and share ideas, thoughts, opinions, etc. Every day, massive amounts of data are disseminated at breakneck speed via social media platforms, reaching a massive audience. Furthermore, the ability to write anonymous posts and comments makes expressing and spreading hate speech even easier. To improve the users’ experience, social media sites are attempting to remove hateful remarks. In this research work, the main focus is on developing automated hate speech and offensive language detection models. Started with traditional hate speech and offensive language identification approaches, than reaches to advanced hate speech recognition methods for social media. This brings a need for integrated datasets and the hate speech prediction method. This paper describes the study on hate speech and offensive content identification in English language by using the various approaches based on machine learning algorithms (Support vector machine, decision tree, and so on) and NLP, along with the features used for the classification problem. The models were tested on the HASOC (2021) datasets and concluded that the ensemble model perform better than other algorithms with the test dataset for different tasks. Results and analysis part of this paper offers researchers a comprehensive picture of approaches.

Social media has changed the world and play an important role in people lives. Social media platforms like Twitter, Facebook and YouTube create a new dimension of communication by providing channels to express and exchange ideas freely. Although the evolution brings numerous benefits, the dynamic environment and the allowable of anonymous posts could expose the uglier side of humanity. Irresponsible people would abuse the freedom of speech by aggressively express opinion or idea that incites hatred. This study performs hate speech and offensive language detection. The problem of this task is there is no clear boundary between hate speech and offensive language. In this study, a selected new features set is proposed for detecting hate speech and offensive language. Using Twitter dataset, the experiments are performed by considering the combination of word n-gram and enhanced syntactic n-gram. To reduce the feature set, filter-embedded combining feature selection is used. The experimental results indicate that the combination of word n-gram and enhanced syntactic n-gram with feature selection to classify the data into three classes: hate speech, offensive language or neither could give good performance. The result reaches 91% for accuracy and the averages of precision, recall and F1.

Hate speech is a form of negative communication intended to harm people and communities. Hate speech is quite common in the real world, and it has reached alarming proportions on social media as well. These days our lives have become increasingly reliant on social media platforms, such as Facebook. This is due to the rapid advancement of technology and communication. In Bangladesh, the number of people using social media platforms is also rapidly increasing. In English, detecting hate speech on social media is a difficult task. Comparatively, Bengali is a complicated language with few datasets available. As a result, detection of Bengali hate speech becomes even more challenging. In this paper, we present a new dataset of 10,133 user comments. We have collected them from the comment section of various public Facebook pages. We explore the performance of various machine learning and deep learning models in detecting hate speech. Bengali pre-trained word embeddings from fastText are used to train the models. We are especially interested in Convolutional Neural Network (CNN). To our knowledge it was never used for hate speech detection in binary classification. Another goal of this research is to create a new and large dataset, which will facilitate further research of Bengali Hate Speech Detection. All machine learning and deep learning models performed very well from our experiments. But, Support Vector Machine (SVM) is the one that performed the best among them.

Over the last two decades, social media platforms have grown dramatically. Twitter and Facebook are the two most popular social media platforms, with millions of active users posting billions of messages daily. These platforms allow users to have freedom of expression. However, some users exploit this facility by disseminating hate speeches. Manual detection and censorship of such hate speeches are impractical; thus, an automatic detection mechanism is required to detect and counter hate speeches in a real-time environment. Most research in hate speech detection has been carried out in the English language. Still, minimal work has been explored in other languages, mainly Urdu written in Roman Urdu script. A few research have attempted machine learning, and deep learning models for Roman Urdu hate speech detection; however, due to a scarcity of Roman Urdu resources, and a large corpus with defined annotation rules, a robust hate speech detection model is still required. With this motivation, this study contributes in the following manner: we developed annotation guidelines for Roman Urdu Hate Speech. Second, we constructed a new Roman Urdu Hate Speech Dataset (RU-HSD-30K) that was annotated by a team of experts using the annotation rules. To the best of our knowledge, the Bi-LSTM model with an attention layer for Roman-Urdu Hate Speech Detection has not been explored. Therefore, we developed a context-aware Roman Urdu Hate Speech detection model based on Bi-LSTM with an attention layer and used custom word2vec for word embeddings. Finally, we examined the effect of lexical normalization of Roman Urdu words on the performance of the proposed model. Different traditional as well as deep learning models, including LSTM and CNN models, were used as baseline models. The performance of the models was assessed in terms of evaluation matrices like accuracy, precision, recall, and F1-score. The generalization of each model is also evaluated on a cross-domain dataset. Experimental results revealed that Bi-LSTM with attention outperformed the traditional machine learning models and other deep learning models with an accuracy score of 0.875 and an F-Score of 0.885. In addition, the results demonstrated that our suggested model (Bi-LSTM with Attention Layer) is more general than previous models when applied to unseen data. The results confirmed that lexical normalization of Roman Urdu words enhanced the performance of the suggested model.

Social media has been a real-world sensor to observe the pulse of society. Although it provides unique communication opportunities, it also brings along vital challenges. One of them is hate speech, which attacks a single individual or targeted groups. Previous researchers claim that among social networks, the Twitter platform is mostly used to spread hate speech. However, data on a larger scale makes it hard to capture and understand the nature of hate speech on Twitter within specific food brands. In this study, sentiment analysis techniques were used to filter hate speech on Twitter and on three popular food brands in Malaysia. This study was conducted in five phases, namely raw data collection, pre-processing, sentiment analysis, visualization, and performance evaluation. This corpus consists of 28,898 data samples based on user tweet searches. A Twitter API was created and SQLite was used to store all the sample data. VADER sentiment analysis is used to classify tweet sentiment into positive, negative, and neutral. In the visualization phase, these three food brands are visualized using a histogram to gain sentiment analysis insights. Then, three machine learning methods were implemented to predict the best model for sentiment analysis. The Decision Tree classifier outperforms the average accuracy in Support Vector Machine and Logistic Regression with 99.99% for the training data set. This study provides insights to assist humans in making decisions. With the growth of opinions expressed in multimedia on social media, such as spoken feedback on Twitter, sentiment analysis has the potential to become a more news aggregation and low-cost endeavour.

Finding hate speech with auxiliary emotion detection from self-training multi-label learning perspective

Hate speech has long been a prevalent issue offline, but with the rise of the internet and social media, its spread has accelerated. The anonymity afforded by these platforms enables individuals to engage in hate speech often without facing substantial consequences. As technology continues to evolve, new opportunities arise to tackle this problem, particularly through the use of natural language processing (NLP). NLP technology can help automate processes that have traditionally been done manually, such as flagging hate speech in online content. Yet, many issues remain unresolved before we can achieve efficient hate speech detection systems. These include foundational challenges, such as defining hate speech, as well as issues that arise at the end of an NLP pipeline, such as evaluating whether a model can generalize. Additionally, the issue of bias in models poses a significant challenge, as biased training data can lead to inaccurate or unfair results. In this thesis, I will focus on addressing these issues. First, I examine the definitions of hate speech and related concepts like toxicity and abusive language, and their impact on re-annotated datasets. I then compare the original and re-annotated labels in terms of robustness and generalization using a BERT-based classifier. Next, I explore the use of hate speech legislation from three countries for annotation, expanding the task of hate speech detection to prosecutable hate speech detection. The results show even law interpretation can be subjective, which has a consequent effect on model training and evaluation. To address this issue, I introduce a semantic componential analysis of hate speech definitions, leading to the creation of the HateDefCon corpus. Moreover, I present a pipeline for generating parallel multilingual hate speech corpora and discuss the associated challenges. The thesis concludes with an examination of textual biases based on the psycholinguistic aspects of harmful language.

This paper examined the phenomenon of hate speech and foul language on social media platforms in Nigeria, and assessed their moral and legal consequences in the society and to journalism practice. It used both quantitative and qualitative methodology to investigate the phenomenon. In the first place, the paper employed the survey research methodology to sample 384 respondents using questionnaire and focus group discussion as instruments for data collection. Findings from the research indicate that promoting hate speech and foul language on social media have moral and legal consequences in the society and to journalism practice. Findings also show that although, the respondents understand that hate speech and foul language attract legal consequences, they do not know what obligations are created by law against perpetrators of hate speech and foul language in Nigeria. The paper therefore, adopted the qualitative, doctrinal and analytical methodology to discuss the legal consequences and obligations created against perpetrators of hate speech and foul language in Nigeria. The paper concluded based on the findings that hate speech and foul language is prevalent on social media platforms in Nigeria and that there are adequate legal provisions to curb the phenomenon in Nigeria. It recommends among others things that the Nigerian government and NGOs should sponsor monitoring projects like the UMATI in Kenya to better understand the use of hate speech and that monitoring agencies set up under the legal regime should adopt mechanisms to identify and remove hate speech content on social media platforms in Nigeria.

Hate speech detection is an important issue in sentiment analysis and natural language processing. This study aims to improve the effectiveness of hate speech detection in English text using the BERT model, along with modified preprocessing techniques to enhance the F1-score. The dataset, sourced from Kaggle, contains English text with hate speech content. Evaluation results show a significant improvement in the model's accuracy and overall text classification performance. The BERT model achieved 89.11% accuracy on test data, correctly predicting 85 out of 95 samples. While the model excels at classifying offensive text with around 95% accuracy, it struggles to distinguish between hate and offensive text, with some confusion between neither and offensive categories. The classification report shows F1-scores of 0.43 for the hate class, 0.94 for the offensive class, and 0.84 for the neither class, with a weighted average F1-score of 0.89 and a macro average of 0.73. These results indicate that the BERT model delivers solid performance in detecting hate speech, though there is room for improvement, particularly in distinguishing certain classes.

The paper introduces a very current topic in the field of natural language processing oriented to the automatic detection of hate speech and offensive language performed in the Slovak language. In this work, we describe the creation and processing database of short texts composed of posts and comments written in Slovak and published on social media. The proposed approach is based on sentiment analysis and implementing a tool for detecting hate speech using a convolutional neural network with elements of a recursive neural network, applied to a created database of comments. We achieved 61.32% detection accuracy only on a small set of training data balanced in the number of positive, neutral, and negative sentiments.

Homecoming, more traditionally known as Mudik, has become a trending topic on several social media platforms as soon as the 11-day homecoming ritual ban was announced on 7 April 2021. Opinions, varying from those in favor of and against the ban, start to rapidly appear. Twitter, a social media platform which is now considered to be an extension of oneself and often used to express ones’ opinion, has become flooded with comments on the homecoming ritual ban. The swarm of opinions in the form of tweets were then used as a dataset for sentiment analysis in order to understand how people perceive the ban. The algorithm used in this research is the classification algorithm using the Support Vector Machine method. The dataset was classified into three sentiments: positive, negative, and neutral. The use of the Support Vector Machine algorithm yielded a 62% accuracy with this dataset. The sentiment analysis showed that the keyword "mudik" had a neutral sentiment for the most part. Meanwhile, results of engagement analysis show that the largest forms of engagements were retweets and liking tweets that had a neutral sentiment. When the neutral sentiment was removed, we found that the largest sentiment on the homecoming ritual ban was negative. This is likely due to the release of an addendum to the Covid-19 Handling Task Force Circular Number 13 of 2021 on 22 April 2021 that imposes more restrictions on and extends the effective dates of the restrictions related to the homecoming ritual ban; exactly one day before the data scraping of 5000 datasets on tweets from 23 April 2021 was carried out. The researcher had already sampled the tweets with the most engagements (those with the most retweets and likes). It was found that some tweets had a negative sentiment, but the model classified it as having a neutral sentiment. This may be affected by inaccuracies of dataset training as some of the tweets were in Malay rather than Indonesian. A challenge that needs to be overcome is the limited number of datasets for NLP training or sentiment analysis for the Indonesian language in comparison to that of the English language. On the other hand, this has become an opportunity for the researcher to develop a more appropriate training model.