Predicting determinants influencing user satisfaction with mental health app: An explainable machine learning approach based on unstructured data

This paper aims to objectively compare the use of mental health apps between the pre-COVID-19 and during COVID-19 periods and to study differences amongst the users of these apps based on age and gender. The study utilizes a dataset collected through a smartphone app that objectively records the users’ sessions. The dataset was analyzed to identify users of mental health apps (38 users of mental health apps pre-COVID-19 and 81 users during COVID-19) and to calculate the following usage metrics; the daily average use time, the average session time, the average number of launches, and the number of usage days. The mental health apps were classified into two categories: guidance-based and tracking-based apps. The results include the increased number of users of mental health apps during the COVID-19 period as compared to pre-COVID-19. Adults (aged 24 and above), compared to emerging adults (aged 15–24 years), were found to have a higher usage of overall mental health apps and guidance-based mental health apps. Furthermore, during the COVID-19 pandemic, males were found to be more likely to launch overall mental health apps and guidance-based mental health apps compared to females. The findings from this paper suggest that despite the increased usage of mental health apps amongst males and adults, user engagement with mental health apps remained minimal. This suggests the need for these apps to work towards improved user engagement and retention.

This letter builds upon McCarthy and Horwitz’s study published in Journal of American College Health, which examined first-year college students’ attitudes and barriers to mental health apps. The letter highlights key areas for further research, including considering the role of international student status in mental health app use, incorporating pharmacological treatment history into mental health treatment measures, and expanding the application of the Health Belief Model to capture additional constructs pertinent to college students’ attitudes toward mental health apps. It advocates for longitudinal studies to assess causal relationships between prior mental health care use and mental health app engagement. Finally, the letter proposes utilizing the Consolidated Framework for Implementation Research to explore the multi-level factors influencing attitudes toward and use of mental health apps among college students. Insights provided in this letter have the potential to inform strategies to expand the reach of mental health apps among college students.

In this study, we explore the potential of machine learning (ML) models after different text representation methods on the balanced IMDB dataset, which is widely regarded as a gold standard in sentiment analysis, one of the Natural Language processing (NLP) tasks. On the open source IMDB movie reviews dataset, we first undertake data cleaning and text representation with data preprocessing steps. Then, we apply sentiment classification using different ML models. In order to evaluate the models, we used precision (P), recall (R), F1-score (F1), and area under curve (AUC), as well as receiver operating characteristic (ROC). It is worth noting that text feature extraction with Bidirectional Encoder Representations from Transformers (BERT) provided the highest performance in all models, with the SVM model offering particularly promising results. In this model, we observed the following results: ACC 0.9033, F1 0.9308, R 0.9015, R 0.9015, P 0.9072, AUC 0.9638, and ROC 0.96. These findings suggest that NLP techniques and, in particular, machine learning models that employ BERT may offer high levels of accuracy and reliability in text classification problems. It would be beneficial for future studies to validate these findings using BERT on different NLP tasks. This would help to evaluate the effectiveness and applicability of the models in practice.

Sentiment analysis of news media articles is essential for understanding the dynamics of conflict and cooperation in transboundary rivers. However, it is not known which machine learning model(s) can best meet the requirement of sentiment analysis for transboundary rivers. This study presents a comparative examination of ten machine learning models commonly used in the field of text sentiment analysis, including K-Nearest Neighbors, Naive Bayes, Support Vector Machine, Decision Tree, Random Forest, Gradient Boosting Decision Tree, Extreme Gradient Boosting, Multilayer Perceptron, Long Short-Term Memory and Bidirectional Encoder Representations from Transformers, for five-class sentiment classification of 9382 news articles (1977–2022) attending to transboundary water conflict and cooperation. By evaluating their performance in terms of accuracy, precision, recall and F1-score, the Bidirectional Encoder Representations from Transformers (BERT) model demonstrated good overall performance and prediction capabilities for news articles with conflictive sentiments. By comparing with the AFINN sentiment dictionary, BERT showed superior performance in the prediction and identification of conflictive sentiment labels. And by validating against historical water events in the three river basins, BERT performed best in the Indus River basin. The findings of this study hold significant implications for government agencies in transboundary rivers, allowing them to promptly assess and respond to public sentiment, thereby preventing water conflict and promoting water cooperation.

BackgroundRapid triage reduces the patients' stay time at an emergency department (ED). The Korean Triage Acuity Scale (KTAS) is mandatorily applied at EDs in South Korea. For rapid triage, we studied machine learning-based triage systems composed of a speech recognition model and natural language processing-based classification.MethodsWe simulated 762 triage cases that consisted of 18 classes with six types of the main symptom (chest pain, dyspnea, fever, stroke, abdominal pain, and headache) and three levels of KTAS. In addition, we recorded conversations between emergency patients and clinicians during the simulation. We used speech recognition models to transcribe the conversation. Bidirectional Encoder Representation from Transformers (BERT), support vector machine (SVM), random forest (RF), and k-nearest neighbors (KNN) were used for KTAS and symptom classification. Additionally, we evaluated the Shapley Additive exPlanations (SHAP) values of features to interpret the classifiers.ResultsThe character error rate of the speech recognition model was reduced to 25.21% through transfer learning. With auto-transcribed scripts, support vector machine (area under the receiver operating characteristic curve [AUROC], 0.86; 95% confidence interval [CI], 0.81–0.9), KNN (AUROC, 0.89; 95% CI, 0.85–0.93), RF (AUROC, 0.86; 95% CI, 0.82–0.9) and BERT (AUROC, 0.82; 95% CI, 0.75–0.87) achieved excellent classification performance. Based on SHAP, we found “stress”, “pain score point”, “fever”, “breath”, “head” and “chest” were the important vocabularies for determining KTAS and symptoms.ConclusionWe demonstrated the potential of an automatic KTAS classification system using speech recognition models, machine learning and BERT-based classifiers.

With the development of Internet technology and the transformation of news media to informatization, news images, texts, and other information have exploded. The news image and text classification can effectively solve the disorder problem of news information. The early news image text classification is to establish artificial classifiers according to specific classification rules, but the classification error rate is high and the classification speed is slow. Later, machine learning technology replaced manual classification of news image texts. Although the classification efficiency has been greatly improved, the classification time is still a bit long. The bidirectional encoder representations from transformers (BERT) model uses transformer and encoder to pretrain news image text to improve classification efficiency. By comparing the differences between machine learning and BERT models in news image text classification, the experiments showed that the average precision, recall, and F1 values of the news image text classification algorithm using the BERT model were 96.6%, 95.7%, and 96.1%, respectively. All three evaluation criteria were about 5% more than the classification algorithm of the machine learning model. The classification speed of the news image text classification algorithm using the BERT model was 1.8 times that of the news image text classification algorithm based on support vector machine. Therefore, the news image text classification algorithm using the BERT model can improve the classification accuracy and efficiency of news image text.

As the online shopping industry continues to grow at a staggering rate, e-commerce channels have also begun the use of automated systems for sentiment analysis so as to be able to assess comprehension of user reviews. Over the past few years, machine learning (ML) has come of age with the introduction of deep learning models such as Convolutional Neural Networks (CNNs) and Bidirectional Encoder Representations from Transformers (BERT) which have greatly improved the levels of accuracy achieved in sentiment analysis. This paper aims to address this problem by proposing a hybrid model, based on networks of convolutional neural networks (CNNs) and BERT, for the purposes of e-commerce sentiment classification. Posting more plausible situations, academic findings, and perspectives for future research, we focus more on practical issues. When we compared traditional approaches with our hybrid, it became clear that the model performs better in many dimensions: accuracy, precision, and progressively recall

Artificial Intelligence (AI) technology has undergone significant advancements, driving its adoption across various fields. AI, especially Machine Learning (ML) adoption has also grown widely in the medical field, which naturally requires actions with high precision. Challenges arise when AI-based predictive models prove to be vulnerable to adversarial attacks. These attacks use perturbed data (modified data), which is unnoticeable to humans but can significantly alter prediction results. This paper aims to implement the TextFooler technique to deceive predictive models on medical text against adversarial attacks. The experiment reveals that three ML models developed using popular approachs i.e., transformer-based model based on Bidirectional Encoder Representations from Transformers (BERT), Stack Classifier methods and traditional machine learning algorithms achieved the same classification accuracy of 99.98%. Among the three models, BERT is the most affected by adversarial attacks, with an attack success rate of 76.8%, followed by traditional machine learning methods and the stack classifier, with success rates of 28.73% and 5.21%, respectively. These findings indicate that although the advanced ML model BERT demonstrates good performance, it is highly vulnerable to adversarial attacks. Therefore, there is an urgency to develop predictive models that are robust and secure against potential attacks.

Bali is one of the provinces with the most abundant natural and cultural wealth in Indonesia. One commodity that supports it is coffee. Bali Coffee is not only a gastronomic identity, but also a cultural identity which makes it have added value to be developed into various business lines. One business derivative that is quite promising is a coffee shop. However, these favorable conditions also need to be maintained to ensure good quality reaches consumers. One thing that can do is analyze reviews from customers. One of the most popular methods is Sentiment Analysis. This technique allows business to analyze customer reviews on social media. It can be a feedback to maintaining and improving quality and good relationships with customers. This research aims to create a machine learning model to analyze customer reviews at several coffee shops in Bali which are divided into three labels, namely: positive, negative and neutral. The methods used are: scraping, cleaning, stopword removal, embedding, undersampling, and modeling. The algorithms used are Bidirectional Encoder Representation from Transformer (BERT) and Generative Pre-trained Transformers (GPT). The performance metrics used in this research are precision, recall, accuracy and loss. This research succeeded in creating a sentiment analysis model for coffee shop customers in Bali. The BERT model obtained an accuracy value of 78% without undersampling with a loss in the 10th iteration of 0.27. Meanwhile, the BERT model with undersampling obtained an accuracy value of 32.85% with a loss in the 10th iteration of 0.16. The GPT2 model without undersampling gets an accuracy of 78% with a loss in the 10th iteration of 0.25. Meanwhile, the GPT model with undersampling obtained an accuracy value of 32.85% with a loss in the 10th iteration of 0.15.

Abstract—This paper presents a hybrid sentiment analysis system that integrates lexicon-based, traditional machine learning, and deep learning techniques to classify textual data into positive, negative, or neutral sentiments. The system leverages Python-based libraries such as Scikit-learn, NLTK, and Transformers to pre- process text, extract features, and apply models in- cluding Support Vector Machine (SVM), Bidirectional Encoder Representations from Transformers (BERT), Linear Regression, and Valence Aware Dictionary and sEntiment Reasoner (VADER). The framework pro- cesses customer reviews from e-commerce platforms (Amazon, Flipkart) and social media (Instagram) using web scraping and provides actionable insights through sentiment summarization and visualizations (bar and pie charts). Experimental results demonstrate BERT’s superior performance with 92.3% accuracy, followed by SVM (85.6%), Linear Regression (81.2%), and VADER (76.8%). The system addresses challenges like sarcasm, class imbalance, and scalability, offering a scalable, user-friendly solution for real-world applications in e- commerce, social media analytics, and brand reputation management. Index Terms—Sentiment Analysis, Machine Learning, Deep Learning, Natural Language Processing, BERT, SVM, VADER, Text Classification, Data Visualization.

Misinformation of COVID-19 is prevalent on social media as the pandemic unfolds, and the associated risks are extremely high. Thus, it is critical to detect and combat such misinformation. Recently, deep learning models using natural language processing techniques, such as BERT (Bidirectional Encoder Representations from Transformers), have achieved great successes in detecting misinformation. In this paper, we proposed an explainable natural language processing model based on DistilBERT and SHAP (Shapley Additive exPlanations) to combat misinformation about COVID-19 due to their efficiency and effectiveness. First, we collected a dataset of 984 claims about COVID-19 with fact checking. By augmenting the data using back-translation, we doubled the sample size of the dataset and the DistilBERT model was able to obtain good performance (accuracy: 0.972; areas under the curve: 0.993) in detecting misinformation about COVID-19. Our model was also tested on a larger dataset for AAAI2021 - COVID-19 Fake News Detection Shared Task and obtained good performance (accuracy: 0.938; areas under the curve: 0.985). The performance on both datasets was better than traditional machine learning models. Second, in order to boost public trust in model prediction, we employed SHAP to improve model explainability, which was further evaluated using a between-subjects experiment with three conditions, i.e., text (T), text+SHAP explanation (TSE), and text+SHAP explanation+source and evidence (TSESE). The participants were significantly more likely to trust and share information related to COVID-19 in the TSE and TSESE conditions than in the T condition. Our results provided good implications in detecting misinformation about COVID-19 and improving public trust.

Fenomena bullying di media sosial, khususnya di Twitter, telah menjadi isu yang semakin memprihatinkan dengan dampak signifikan terhadap kesehatan mental pengguna. Dalam rangka mengatasi masalah ini, deteksi otomatis tweet yang mengandung konten bullying menjadi sangat penting. Penelitian ini bertujuan untuk membandingkan performa dua model pemrosesan bahasa alami terbaru, yaitu BERT (Bidirectional Encoder Representations from Transformers) dan XLNet, dalam klasifikasi tweet yang mengandung bullying. Metodologi penelitian ini melibatkan pengumpulan dataset tweet yang telah dilabeli sebagai bullying atau non-bullying. Proses preprocessing teks dilakukan untuk membersihkan dan menyiapkan data sebelum digunakan dalam pelatihan model. Kedua model, BERT dan XLNet, dilatih dan diuji menggunakan dataset yang sama. Evaluasi performa dilakukan dengan menggunakan metrik akurasi, presisi, recall, dan F1-score. Hasil penelitian menunjukkan bahwa kedua model memiliki kemampuan yang baik dalam mengidentifikasi tweet bullying, akan tetapi XLNet menunjukkan performa yang lebih unggul dibandingkan BERT dengan tingkat akurasi sebesar 95%. Dengan nilai presisi = 100%, recall = 0,87%, dan F1-score = 0,88%. XLNet mampu menangkap konteks dan nuansa bahasa yang lebih kompleks dalam tweet, yang berkontribusi pada akurasi klasifikasi yang lebih tinggi. Penelitian ini memberikan kontribusi penting dalam bidang deteksi bullying di media sosial dengan menunjukkan bahwa penggunaan model XLNet lebih efektif dibandingkan BERT. Temuan ini dapat membantu platform seperti Twitter dalam mengidentifikasi dan mencegah konten bullying, sehingga menciptakan lingkungan online yang lebih aman bagi pengguna, serta dapat digunakan sebagai dasar untuk pengembangan sistem deteksi bullying yang lebih canggih dan efisien di masa depan. Abstract The phenomenon of bullying on social media, particularly on Twitter, has become an increasingly concerning issue with significant impacts on users' mental health. In order to address this issue, automatic detection of tweets containing bullying content is crucial. This study aims to compare the performance of two recent natural language processing models, namely BERT (Bidirectional Encoder Representations from Transformers) and XLNet, in the classification of tweets containing bullying. The research methodology involves collecting a dataset of tweets that have been labelled as bullying or non-bullying. Text preprocessing is done to clean and prepare the data before it is used in model training. Both models, BERT and XLNet, were trained and tested using the same dataset. Performance evaluation was conducted using accuracy, precision, recall, and F1-score metrics. The results show that both models have a good ability to identify bullying tweets, but XLNet shows superior performance compared to BERT with an accuracy rate of 95%. With precision = 100%, recall = 0.87%, and F1-score = 0.88%. XLNet is able to capture more complex context and language nuances in tweets, which contributes to higher classification accuracy. This research makes an important contribution to the field of bullying detection on social media by showing that the use of the XLNet model is more effective than BERT. These findings can help platforms like Twitter identify and prevent bullying content, thereby creating a safer online environment for users, and can be used as a basis for the development of more sophisticated and efficient bullying detection systems in the future.

The omnipresence of smartphones helps people to declare an emergency in realtime. In times of crisis, Twitter has become a big communication platform. As a result, more organizations are interested in tracking Twitter programmatically. Although governments and emergency management agencies work together through their respective national system for response to disasters, the sentiments of the people affected during and after the catastrophe decide the effectiveness of the disaster response and the recovery process. In recent years, sentiment analysis via Twitter-based machine learning has become a common subject. However, the detection of such tweets was often difficult due to the tweets’ language structure’s uncertainty. Thus, it is not always apparent if the words of a person announce a catastrophe. BERT (Bidirectional Encoder Representations by Transformers) is a profound learning model developed by Google. Since Google opened it, several scientists and companies have embraced it and have applied it to many text classification tasks. Therefore, we use BERT in this paper to some dataset disaster tweets. This research will help rescue and emergency responders establish effective knowledge management techniques for a rapidly evolving disaster environment.

As social media and online social networking sites have emerged as essential channels for everyday social interactions, researchers are increasingly turning to conversational social media as a valuable alternative source of insights into public sentiment and behavior during times of crisis. In a similar vein, our study investigates the utility of using conversational social media data for analyzing public sentiment related to the COVID-19 pandemic. Our research question is, to what extent can machine learning models based on conversational social media data help gauge public sentiment during a pandemic? To evaluate the performance of various machine learning algorithms in sentiment analysis tasks, we used a Twitter dataset and compared their effectiveness using eight different models (i.e., Decision Tree (DT), Random Forest (RF), Naïve Bayes (NB), Extreme Gradient Boosting (XGBoost), Logistic Regression, Support Vector Machine (SVM), Long Short-Term Memory (LSTM), and Bidirectional Encoder Representations from Transformers (BERT)). The results show that among the eight models, the top three models are BERT, LSTM, and SVM, achieving overall accuracies of 87.44%, 81.94%, and 80.02%, respectively. Employing sentiment analysis models on pandemic-related social media posts allows us to gain insights into of human emotions and sentiments during public health crises like a pandemic.

For assessing customer sentiment in Amazon product reviews, this article compares two machine learning algorithms and a deep learning method, BERT (Bidirectional Encoder Representations from Transformer). Machine learning is the most practical approach in the current era of artificial intelligence for training a neural network to handle the majority of real-world issues. In this paper, the real-world scenario of sentiment analysis is considered, ideally the classification problem. Firstly, the data is provided into a model, which evaluates the feature that uses the Term Frequency (TF) and Inverse Document Frequency (IDF) pre-processing methods. Secondly, the algorithms, Naive Bayes classifier and Support Vector Machine are used to analyze the sentiment of the consumer comments and compute metrics like F1 score. Finally, the input data is fed for BERT to process and compute the F1 score. To summarize, this study is to provide a detailed comparative analysis of machine learning techniques and deep learning algorithms.