Penta Classification of Landslide Via Deep Learning Based SegNet and RegNet

In the rapidly evolving Internet of Things (IoT) landscape, accurate indoor positioning is increasingly vital. The proposed algorithm synergizes an ultrawideband (UWB) sensor with an inertial measurement unit (IMU) and artificial intelligence to obtain precise positioning in nonline-of-sight (NLOS) scenarios. In the proposed UWB module, a large language model (LLM) such as bidirectional encoder representations from transformers (BERT) algorithm is designed to utilize the channel impulse response (CIR) for effective NLOS identification and UWB ranging trustworthiness evaluation. Concurrently, the IMU module is also designed with BERT to recognize various pedestrian activity states, thereby optimizing positioning. BERT’s self-attention mechanism and deep learning (DL) bidirectional training efficiently extract essential features from sequential data, capturing both local and global information. The integration of both UWB and IMU through a proposed tightly coupled algorithm significantly boosts positioning performance. Experimental campaigns demonstrate an average NLOS identification accuracy, line-of-sight (LOS), and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">${F}2$ </tex-math></inline-formula> of 98.8%, 99.4%, and 0.9926, respectively. These performances surpass the state-of-the-art least-squares support vector machine (LS-SVM), convolutional neural network (CNN), and CNN with long short-term memory (CNN-LSTM) up to 17.66% in NLOS identification. In terms of pedestrian activity recognition using BERT, the BERT algorithm achieves a precision (recall) of 99.3% (99.4%), notably outperforming the CNN and CNN-LSTM by 17.9% (16.2%) and 11.9% (10.9%), respectively. Finally, the UWB-IMU algorithm significantly enhances positioning accuracy by 80.5%, outperforming Kalman, LSTM-EKF, and particle filter (PF) methods by 68.1%, 48.3%, and 45.0%, respectively. The proposed approach presents a robust solution for indoor positioning for IoT applications, particularly in challenging NLOS environments.

As the online shopping industry continues to grow at a staggering rate, e-commerce channels have also begun the use of automated systems for sentiment analysis so as to be able to assess comprehension of user reviews. Over the past few years, machine learning (ML) has come of age with the introduction of deep learning models such as Convolutional Neural Networks (CNNs) and Bidirectional Encoder Representations from Transformers (BERT) which have greatly improved the levels of accuracy achieved in sentiment analysis. This paper aims to address this problem by proposing a hybrid model, based on networks of convolutional neural networks (CNNs) and BERT, for the purposes of e-commerce sentiment classification. Posting more plausible situations, academic findings, and perspectives for future research, we focus more on practical issues. When we compared traditional approaches with our hybrid, it became clear that the model performs better in many dimensions: accuracy, precision, and progressively recall

Fenomena bullying di media sosial, khususnya di Twitter, telah menjadi isu yang semakin memprihatinkan dengan dampak signifikan terhadap kesehatan mental pengguna. Dalam rangka mengatasi masalah ini, deteksi otomatis tweet yang mengandung konten bullying menjadi sangat penting. Penelitian ini bertujuan untuk membandingkan performa dua model pemrosesan bahasa alami terbaru, yaitu BERT (Bidirectional Encoder Representations from Transformers) dan XLNet, dalam klasifikasi tweet yang mengandung bullying. Metodologi penelitian ini melibatkan pengumpulan dataset tweet yang telah dilabeli sebagai bullying atau non-bullying. Proses preprocessing teks dilakukan untuk membersihkan dan menyiapkan data sebelum digunakan dalam pelatihan model. Kedua model, BERT dan XLNet, dilatih dan diuji menggunakan dataset yang sama. Evaluasi performa dilakukan dengan menggunakan metrik akurasi, presisi, recall, dan F1-score. Hasil penelitian menunjukkan bahwa kedua model memiliki kemampuan yang baik dalam mengidentifikasi tweet bullying, akan tetapi XLNet menunjukkan performa yang lebih unggul dibandingkan BERT dengan tingkat akurasi sebesar 95%. Dengan nilai presisi = 100%, recall = 0,87%, dan F1-score = 0,88%. XLNet mampu menangkap konteks dan nuansa bahasa yang lebih kompleks dalam tweet, yang berkontribusi pada akurasi klasifikasi yang lebih tinggi. Penelitian ini memberikan kontribusi penting dalam bidang deteksi bullying di media sosial dengan menunjukkan bahwa penggunaan model XLNet lebih efektif dibandingkan BERT. Temuan ini dapat membantu platform seperti Twitter dalam mengidentifikasi dan mencegah konten bullying, sehingga menciptakan lingkungan online yang lebih aman bagi pengguna, serta dapat digunakan sebagai dasar untuk pengembangan sistem deteksi bullying yang lebih canggih dan efisien di masa depan. Abstract The phenomenon of bullying on social media, particularly on Twitter, has become an increasingly concerning issue with significant impacts on users' mental health. In order to address this issue, automatic detection of tweets containing bullying content is crucial. This study aims to compare the performance of two recent natural language processing models, namely BERT (Bidirectional Encoder Representations from Transformers) and XLNet, in the classification of tweets containing bullying. The research methodology involves collecting a dataset of tweets that have been labelled as bullying or non-bullying. Text preprocessing is done to clean and prepare the data before it is used in model training. Both models, BERT and XLNet, were trained and tested using the same dataset. Performance evaluation was conducted using accuracy, precision, recall, and F1-score metrics. The results show that both models have a good ability to identify bullying tweets, but XLNet shows superior performance compared to BERT with an accuracy rate of 95%. With precision = 100%, recall = 0.87%, and F1-score = 0.88%. XLNet is able to capture more complex context and language nuances in tweets, which contributes to higher classification accuracy. This research makes an important contribution to the field of bullying detection on social media by showing that the use of the XLNet model is more effective than BERT. These findings can help platforms like Twitter identify and prevent bullying content, thereby creating a safer online environment for users, and can be used as a basis for the development of more sophisticated and efficient bullying detection systems in the future.

Emotion detection plays a vital role in understanding human sentiments and behaviors across various applications, including customer feedback analysis and mental health monitoring. This research assesses the efficiency of different algorithms for machine learning in detecting emotions in text data. A meticulously curated dataset is utilized for the study. The research compares conventional models like Logistic Regression (LR), Random Forest (RF), Support Vector Machines (SVM), and Naive Bayes (NB) with deep learning models like Long Short-Term Memory (LSTM), Convolutional Neural Networks (CNN), and Bidirectional Encoder Representations from Transformers (BERT). The performance of each algorithm is assessed using accuracy, precision, recall, and F1 score. BERT exhibits superiority over other models, achieving the maximum accuracy of 0.8867 and F1 Score of 0.8871. CNN and SVM also display commendable performance. While the traditional models perform adequately, they are surpassed by deep learning models, with Naive Bayes showing the lowest metrics. This study underscores the significance of selecting models based on specific application requirements, taking into account factors like interpretability and efficiency. Future research endeavors may explore multimodal approaches, model interpretability, bias reduction, and real-time applications, thereby contributing to the advancement of emotion detection in text.

This study aims to explore and develop tools for early identification of depression concerns among cancer patients by leveraging the novel data source of messages sent through a secure patient portal. We developed classifiers based on logistic regression (LR), support vector machines (SVMs), and 2 Bidirectional Encoder Representations from Transformers (BERT) models (original and Reddit-pretrained) on 6600 patient messages from a cancer center (2009-2022), annotated by a panel of healthcare professionals. Performance was compared using AUROC scores, and model fairness and explainability were examined. We also examined correlations between model predictions and depression diagnosis and treatment. BERT and RedditBERT attained AUROC scores of 0.88 and 0.86, respectively, compared to 0.79 for LR and 0.83 for SVM. BERT showed bigger differences in performance across sex, race, and ethnicity than RedditBERT. Patients who sent messages classified as concerning had a higher chance of receiving a depression diagnosis, a prescription for antidepressants, or a referral to the psycho-oncologist. Explanations from BERT and RedditBERT differed, with no clear preference from annotators. We show the potential of BERT and RedditBERT in identifying depression concerns in messages from cancer patients. Performance disparities across demographic groups highlight the need for careful consideration of potential biases. Further research is needed to address biases, evaluate real-world impacts, and ensure responsible integration into clinical settings. This work represents a significant methodological advancement in the early identification of depression concerns among cancer patients. Our work contributes to a route to reduce clinical burden while enhancing overall patient care, leveraging BERT-based models.

Emails are the simplest way to communicate between users. But increasing rate of cybercrime activities require shrewd use of email. Phishing email is one of the most perilous threats nowadays. Through phishing email, attackers want to steal the sensitive information of the email users. This sensitive information includes username, password, credit card details etc. These may cause excessive financial loss. Many anti-phishing methods exist in literature to detect phishing emails. But criminals use new methods day by day, which causes challenge to build anti-phishing methods to prevent phishing emails. In this paper the performance of two deep learning models for detection of phishing email have been compared. The first model uses convolution neural network with Global Vector (GloVe) word embedding where Bidirectional Encoder Representations from Transformers (BERT) model with fine tuning has been used in second model. The proposed method detects the phishing email by analyzing the content of the email. Some widely used datasets (lingSpam, enronSpamSubset, completeSpamAssassin, Jose Nazario's phishing dataset and Enron email dataset) are merged together and applied to measure the performance of the model. It is found that the GloVe word embedding achieves better accuracy (98%) than BERT model (96%) in detection of phishing emails in the present context.

Recently, people used social media to convey their emotions in the text. Text classification is required for group a text containing emotion or not and identify the type of emotion. Most of the research in emotion classification use machine learning methods. So this research offers the use of deep learning for the classification of emotions in text. One of the algorithms that is widely used for deep learning is BERT (Bidirectional Encoder Representations from Transformers). For the experiment, classification is divided into two phases, namely the first phase groups emotion or not emotion tweets, and the second phase groups emotion tweets into five types of emotions, namely happy, angry, sad, scared and surprised. The experiment uses three data scenarios, namely the commuter line, transjakarta and combination data. The BERT method was compared with the BiLSTM and LSTM methods for emotion classification. The result of this study is BERT generates the highest accuracy compared to other methods for the three data scenarios with values of 87.24%, 86.27%, and 86.99% in the first phase. Then, the next accuracy value followed by obtained BiLSTM, and the last is LSTM. In the second phase, BERT also obtained the highest accuracy values for the three data scenarios with values of 94.26%, 93.71 %, and 93.98%. Then, the evaluation using Precision, Recall and F1-Measure results that BERT provides the best overall method. In this study, BERT has good performance for emotion classification.

We study the problem of incorporating prior knowledge into a deep Transformer-based model,i.e.,Bidirectional Encoder Representations from Transformers (BERT), to enhance its performance on semantic textual matching tasks. By probing and analyzing what BERT has already known when solving this task, we obtain better understanding of what task-specific knowledge BERT needs the most and where it is most needed. The analysis further motivates us to take a different approach than most existing works. Instead of using prior knowledge to create a new training task for fine-tuning BERT, we directly inject knowledge into BERT's multi-head attention mechanism. This leads us to a simple yet effective approach that enjoys fast training stage as it saves the model from training on additional data or tasks other than the main task. Extensive experiments demonstrate that the proposed knowledge-enhanced BERT is able to consistently improve semantic textual matching performance over the original BERT model, and the performance benefit is most salient when training data is scarce.

With the continuous evolution of natural language processing (NLP), conversational agents have become pivotal in enhancing user engagement and satisfaction in the e-commerce domain. This paper presents a comprehensive study and implementation of intent recognition and slot filling techniques tailored for an e-commerce chatbot. Leveraging a dataset of 8,000 samples, we employed a Convolutional Neural Network (CNN) combined with Long Short-Term Memory (LSTM) networks for intent recognition, achieving an accuracy of 87%. For slot filling, we utilized the Bidirectional Encoder Representations from Transformers (BERT) model, also attaining an accuracy of 87%. The chatbot seamlessly integrates with an e-commerce database, using OpenAI's GPT-3 to generate natural language responses from query results. Our system demonstrates significant advancements in processing user queries, generating precise database queries, and providing coherent and relevant responses, thereby enhancing the overall user experience. This work aims to equip researchers and practitioners with insights into the methodologies and challenges in developing sophisticated e-commerce chatbots, fostering further innovation in this field. Key Words: Intent Recognition, Slot Filling, Convolutional Neural Network (CNN), Long Short-Term Memory (LSTM), Bidirectional Encoder Representations from Transformers (BERT), GPT-3, Natural Language Understanding (NLU), Chatbot

For the traditional model based on the deep learning method most used CNN(convolutional neural networks) or RNN(Recurrent neural Network) model and is based on the dynamic character-level embedding or word-level embedding as input, so there is a problem that the text feature extraction is not comprehensive. In the development environment of the Internet of Things, A method of Automatic text classification based on BERT(Bidirectional Encoder Representations from Transformers) and Feature Fusion was proposed in this paper. Firstly, the text-to-dynamic character-level embedding is transformed by the BERT model, and the BiLSTM(Bi-directional Long-Short Term Memory) and CNN output features are combined and merged to make full use of CNN to extract the advantages of local features and to use BiLSTM to have the advantage of memory to link the extracted context features to better represent the text, so as to improve the accuracy of text classification task. A comparative study with state-of-the-art approaches manifests the proposed method outperforms the state-of-the-art methods in accuracy. It can effectively improve the accuracy of tag prediction for text data with sequence features and obvious local features.

Mood disorders are ubiquitous mental disorders with familial aggregation. Extracting family history of psychiatric disorders from large electronic hospitalization records is helpful for further study of onset characteristics among patients with a mood disorder. This study uses an observational clinical data set of in-patients of Nanjing Brain Hospital, affiliated with Nanjing Medical University, from the past 10 years. This paper proposes a pretrained language model: Bidirectional Encoder Representations from Transformers (BERT)–Convolutional Neural Network (CNN). We first project the electronic hospitalization records into a low-dimensional dense matrix via the pretrained Chinese BERT model, then feed the dense matrix into the stacked CNN layer to capture high-level features of texts; finally, we use the fully connected layer to extract family history based on high-level features. The accuracy of our BERT–CNN model was 97.12 ± 0.37% in the real-world data set from Nanjing Brain Hospital. We further studied the correlation between mood disorders and family history of psychiatric disorder.

With the development of Internet technology and the transformation of news media to informatization, news images, texts, and other information have exploded. The news image and text classification can effectively solve the disorder problem of news information. The early news image text classification is to establish artificial classifiers according to specific classification rules, but the classification error rate is high and the classification speed is slow. Later, machine learning technology replaced manual classification of news image texts. Although the classification efficiency has been greatly improved, the classification time is still a bit long. The bidirectional encoder representations from transformers (BERT) model uses transformer and encoder to pretrain news image text to improve classification efficiency. By comparing the differences between machine learning and BERT models in news image text classification, the experiments showed that the average precision, recall, and F1 values of the news image text classification algorithm using the BERT model were 96.6%, 95.7%, and 96.1%, respectively. All three evaluation criteria were about 5% more than the classification algorithm of the machine learning model. The classification speed of the news image text classification algorithm using the BERT model was 1.8 times that of the news image text classification algorithm based on support vector machine. Therefore, the news image text classification algorithm using the BERT model can improve the classification accuracy and efficiency of news image text.

Sentiment analysis of news media articles is essential for understanding the dynamics of conflict and cooperation in transboundary rivers. However, it is not known which machine learning model(s) can best meet the requirement of sentiment analysis for transboundary rivers. This study presents a comparative examination of ten machine learning models commonly used in the field of text sentiment analysis, including K-Nearest Neighbors, Naive Bayes, Support Vector Machine, Decision Tree, Random Forest, Gradient Boosting Decision Tree, Extreme Gradient Boosting, Multilayer Perceptron, Long Short-Term Memory and Bidirectional Encoder Representations from Transformers, for five-class sentiment classification of 9382 news articles (1977–2022) attending to transboundary water conflict and cooperation. By evaluating their performance in terms of accuracy, precision, recall and F1-score, the Bidirectional Encoder Representations from Transformers (BERT) model demonstrated good overall performance and prediction capabilities for news articles with conflictive sentiments. By comparing with the AFINN sentiment dictionary, BERT showed superior performance in the prediction and identification of conflictive sentiment labels. And by validating against historical water events in the three river basins, BERT performed best in the Indus River basin. The findings of this study hold significant implications for government agencies in transboundary rivers, allowing them to promptly assess and respond to public sentiment, thereby preventing water conflict and promoting water cooperation.

In this paper we focus on an Natural Language Inference task. Being given two sentences, we classify their relation as NEUTRAL, ENTAILMENT or CONTRADICTION. Considering the achievements of BERT (Bidirectional Encoder Representations from Transformers) in many Natural Language Processing tasks, we use BERT features to create our base model for this task. However, several questions arise: can other features improve the performance obtained with BERT? If we are able to predict the situations in which BERT will fail, can we improve the performance by providing alternative models for these situations? We test several strategies and models, as alternatives to the standalone BERT model in the possible failure situations, and we take advantage of semantic features extracted from Discourse Representation Structures.

Abstract—This paper presents a hybrid sentiment analysis system that integrates lexicon-based, traditional machine learning, and deep learning techniques to classify textual data into positive, negative, or neutral sentiments. The system leverages Python-based libraries such as Scikit-learn, NLTK, and Transformers to pre- process text, extract features, and apply models in- cluding Support Vector Machine (SVM), Bidirectional Encoder Representations from Transformers (BERT), Linear Regression, and Valence Aware Dictionary and sEntiment Reasoner (VADER). The framework pro- cesses customer reviews from e-commerce platforms (Amazon, Flipkart) and social media (Instagram) using web scraping and provides actionable insights through sentiment summarization and visualizations (bar and pie charts). Experimental results demonstrate BERT’s superior performance with 92.3% accuracy, followed by SVM (85.6%), Linear Regression (81.2%), and VADER (76.8%). The system addresses challenges like sarcasm, class imbalance, and scalability, offering a scalable, user-friendly solution for real-world applications in e- commerce, social media analytics, and brand reputation management. Index Terms—Sentiment Analysis, Machine Learning, Deep Learning, Natural Language Processing, BERT, SVM, VADER, Text Classification, Data Visualization.