Distributed Keyword-guided Topic Model with Lexical Knowledge Supervision

Traditional topic modeling approaches generally rely on document-term co-occurrence statistics to find latent topics in a collection of documents. However, relying only on such statistics can yield incoherent or hard to interpret results for the end-users in many applications where the interest lies in interpreting the resulting topics (e.g. labeling documents, comparing corpora, guiding content exploration, etc.). In this work, we propose to leverage external common sense knowledge, i.e. information from the real world beyond word co-occurrence, to find topics that are more coherent and more easily interpretable by humans. We introduce the Common Sense Topic Model (CSTM), a novel and efficient approach that augments clustering with knowledge extracted from the ConceptNet knowledge graph. We evaluate this approach on several datasets alongside commonly used models using both automatic and human evaluation, and we show how it shows superior affinity to human judgement. The code for the experiments as well as the training data and human evaluation are available at https://github.com/D2KLab/CSTM.

Decision support tools have emerged as key technologies in data analysis enabling companies to make personalized offers to their customers with the help of data processing and data visualization tools. Embedded agents with conversational interface allow expressing queries in natural language in order to accelerate the report navigation. However the coverage, complexity and scale of such agents are much larger than simple query understanding. Moreover, most of them are often restricted to specific domain and explicit rules. In this paper, we present an approach which uses topic modeling and anomaly detection to point out the information that may be of particular interest to a domain expert. The strength of our approach is that it shows both domain and common sense knowledge and does not require the model to be re-trained for different experts. The proposed model can suggest decision options based on the analyzed data and user's activity. In particular, we propose an unsupervised method for automatic topic labeling, and show how it might be helpful in decision making. We also implement a framework, called MacNabbs, and give some results of our approach.

A short text topic modeling method based on integrating Gaussian and Logistic coding networks with pre-trained word embeddings

This paper describes Contexteller, a collaborative educational system for storytelling that supports teachers in configuring and adapting stories to match children's cultural background, by means of exploiting common sense knowledge. This game is meant to support teachers to interact with students with different social and cultural backgrounds, allowing her/him to create stories according to the students´ cultural reality, expressed in their common sense knowledge, and consequently enabling them to get interested in helping the development of the story. This cultural sensitive storytelling environment intends to promote a closer contact between teacher and students and among students themselves giving them a computer tool to help children to freely express their thoughts, desires and to support them to cooperative work with teachers what is desirable for their intellectual and cognitive development. Contextualization and the narrative are supported by the OMCS-Br common sense knowledge base, which gives to the teacher some resources to perform an activity according to the pedagogical goals

Despite the richness and vastness of the common-sense knowledge bases, we argue that common-sense knowledge has to be integrated into the target applications (Text Classification, Dialogue systems, Information Extraction systems, etc.) more effectively. In order to consider this common-sense knowledge in target applications, we propose a deep learning model of common-sense knowledge in Portuguese language, which can be easily coupled in Natural Language Understanding (NLU) systems in order to leverage their performance. More specifically, the model is composed by a neural network LSTM (Long Short Term Memory) that receives a text from the target application, for example, an user message in a dialog, a response to a user tweet, a news text; and selects and learns what is the best set of common-sense relations to return to the target application, which should be considered in the target learning model or system. We implemented the common-sense learning module in two target applications - a Stance Classification system and an End-to-End Dialogue system. In both cases, incorporating the deep learning model improved the results.

With the emerging of massive short texts, e.g., social media posts and question titles from Q&A systems, discovering valuable information from them is increasingly significant for many real-world applications of content analysis. The family of topic modeling can effectively explore the hidden structures of documents through the assumptions of latent topics. However, due to the sparseness of short texts, the existing topic models, e.g., latent Dirichlet allocation, lose effectiveness on them. To this end, an effective solution, namely Dirichlet multinomial mixture (DMM), supposing that each short text is only associated with a single topic, indirectly enriches document-level word co-occurrences. However, DMM is sensitive to noisy words, where it often learns inaccurate topic representations at the document level. To address this problem, we extend DMM to a novel Laplacian Dirichlet Multinomial Mixture (LapDMM) topic model for short texts. The basic idea of LapDMM is to preserve local neighborhood structures of short texts, enabling to spread topical signals among neighboring documents, so as to modify the inaccurate topic representations. This is achieved by incorporating the variational manifold regularization into the variational objective of DMM, constraining the close short texts with similar variational topic representations. To find nearest neighbors of short texts, before model inference, we construct an offline document graph, where the distances of short texts can be computed by the word mover’s distance. We further develop an online version of LapDMM, namely Online LapDMM, to achieve inference speedup on massive short texts. Carrying this implications, we exploit the spirit of stochastic optimization with mini-batches and an up-to-date document graph that can efficiently find approximate nearest neighbors instead. To evaluate our models, we compare against the state-of-the-art short text topic models on several traditional tasks, i.e., topic quality, document clustering and classification. The empirical results demonstrate that our models achieve very significant performance gains over the baseline models.

Common sense is essential for building intelligent machines. While some commonsense knowledge is explicitly stated in human-generated text and can be learnt by mining the web, much of it is unwritten. It is often unnecessary and even unnatural to write about commonsense facts. While unwritten, this commonsense knowledge is not unseen! The visual world around us is full of structure modeled by commonsense knowledge. Can machines learn common sense simply by observing our visual world? Unfortunately, this requires automatic and accurate detection of objects, their attributes, poses, and interactions between objects, which remain challenging problems. Our key insight is that while visual common sense is depicted in visual content, it is the semantic features that are relevant and not low-level pixel information. In other words, photorealism is not necessary to learn common sense. We explore the use of human-generated abstract scenes made from clipart for learning common sense. In particular, we reason about the plausibility of an interaction or relation between a pair of nouns by measuring the similarity of the relation and nouns with other relations and nouns we have seen in abstract scenes. We show that the commonsense knowledge we learn is complementary to what can be learnt from sources of text.

Combining paper cooperative network and topic model for expert topic analysis and extraction

This study aims to assess the effectiveness of topic modeling in the English translation of the Holy Quran. Topic modeling is a popular text mining technique for uncovering latent semantic patterns in the collection of textual documents and helps to annotate the documents based on these topics. This study identifies the most significant topics in each document as well as grasping an understanding of the topic distribution throughout the document sets. Different steps are performed to acquire the dominant topics in each document and identify the distribution of topics across documents. In this context, the present research work chose to employ Latent Dirichlet Allocation as an unsupervised approach for topic modeling since there is no requirement for a training phase as hidden topics can be discovered throughout the topic modeling process. For this, the word cloud is generated to understand and interpret the results after pre-processing. A dictionary and corpus are created to extract the features from the dataset using the Bag of Words approach. The results are evaluated by calculating the perplexity and coherence score, where high coherence indicates the goodness of well-structured topic models and low perplexity score indicates the correctness of prediction made by the topic models. Lastly, the visualization step is performed.

Cyberbullying (harassment on social networks) is widely recognized as a serious social problem, especially for adolescents. It is as much a threat to the viability of online social networks for youth today as spam once was to email in the early days of the Internet. Current work to tackle this problem has involved social and psychological studies on its prevalence as well as its negative effects on adolescents. While true solutions rest on teaching youth to have healthy personal relationships, few have considered innovative design of social network software as a tool for mitigating this problem. Mitigating cyberbullying involves two key components: robust techniques for effective detection and reflective user interfaces that encourage users to reflect upon their behavior and their choices. Spam filters have been successful by applying statistical approaches like Bayesian networks and hidden Markov models. They can, like Google’s GMail, aggregate human spam judgments because spam is sent nearly identically to many people. Bullying is more personalized, varied, and contextual. In this work, we present an approach for bullying detection based on state-of-the-art natural language processing and a common sense knowledge base, which permits recognition over a broad spectrum of topics in everyday life. We analyze a more narrow range of particular subject matter associated with bullying (e.g. appearance, intelligence, racial and ethnic slurs, social acceptance, and rejection), and construct BullySpace , a common sense knowledge base that encodes particular knowledge about bullying situations. We then perform joint reasoning with common sense knowledge about a wide range of everyday life topics. We analyze messages using our novel AnalogySpace common sense reasoning technique. We also take into account social network analysis and other factors. We evaluate the model on real-world instances that have been reported by users on Formspring, a social networking website that is popular with teenagers. On the intervention side, we explore a set of reflective user-interaction paradigms with the goal of promoting empathy among social network participants. We propose an “air traffic control”-like dashboard, which alerts moderators to large-scale outbreaks that appear to be escalating or spreading and helps them prioritize the current deluge of user complaints. For potential victims, we provide educational material that informs them about how to cope with the situation, and connects them with emotional support from others. A user evaluation shows that in-context, targeted, and dynamic help during cyberbullying situations fosters end-user reflection that promotes better coping strategies.

Background: Public health challenges in Indonesia continue to expand across areas such as mental health, chronic diseases, vaccination debates, and environmental issues. Specific background: The rapid use of platform X provides large-scale public discourse that can be analyzed to understand real-time health discussions. Knowledge gap: Limited studies integrate advanced sentiment and topic modeling tailored to informal Indonesian social media language. Aim: This study analyzes public health conversations on platform X using IndoBERTweet for sentiment classification and BERTopic for topic extraction. Results: From 6,740 processed tweets, neutral sentiment dominated public discussions, while topic modeling produced 44 themes, with mental well-being, vaccination debates, chronic disease concerns, and regional disease reports emerging as key issues. IndoBERTweet demonstrated reliable performance (Weighted F1-Score 0.7822), and BERTopic produced coherent and diverse topics. Novelty: This research combines IndoBERTweet and BERTopic to generate a contextual, adaptive, and real-time mapping of public health discourse in Indonesia. Implications: The findings support data-driven health policymaking, enabling authorities to monitor public perceptions, strengthen communication strategies, and design region-specific interventions. Highlights • Public conversations emphasize mental health and lifestyle-related issues. • Topic modeling identifies diverse clusters, including vaccination debates and endemic diseases. • Integrated sentiment–topic analysis enables real-time mapping of health discussions in Indonesia. Keywords Public Health, Social Media Analysis, IndoBERTweet, BERTopic, Sentiment Classification

In recent years, the popularity of the microblog platform has grown rapidly and through which many users are able to communicate. In this paper, we concentrate on automatic user interest detection, which is an important issue in personal information recommendation. Firstly, a dynamic topic model for user interest detection is presented. In this model, topics can be detected in different timestamps and the words with highest probability in each topic can be regarded as the user interests. Secondly, an algorithm for detecting user interest is proposed. The key ideas of our algorithm lie in that topics with the highest probability should be obtained in advance, and then, the words with highest probability in each topic can be extracted as user interests. Particularly, to promote performance of the topic model we present a density-based optimal topic number selection algorithm. Finally, performance evaluation experiments are conducted where a dataset is crawled from Sina Weibo from 6 months. The experimental results demonstrate that (1) our proposed algorithm can dynamically detect user interest with time changing and (2) user interest detection accuracy of our proposed method is higher than other methods.

Each generation imbues family life with myths about the golden era of the past and the breakdown of norms in the present. The explanation for this continued sense of social decay is probably a very basic confusion between processes of history and those of personal biography. But whatever the cause, this strong sense that people have of the changes that are taking place to the family, irrespective of any social reality, makes it especially hard for researchers to assemble and interpret the facts about social change in family composition and structure, and in values and attitudes towards the family. The ease with which explanations can slip into ‘common sense knowledge’ makes it particularly important to support statements about change with empirical evidence collected across time rather than with data collected in the present and contrasted with an assumption about the past.KeywordsLabour MarketMarried CoupleDivorce RateMarriage RateLone ParentThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Optimising topic coherence with Weighted P[formula omitted]lya Urn scheme

This study undertakes a comprehensive analysis of topic modeling techniques-N on-negative Matrix Factorization (NMF), Latent Dirichlet Allocation (LDA), and Correlation Explanation (Corex) enhanced by BERT's depth on Quora and Twitter data. It evaluates their effectiveness in topic extraction and the impact of BERT's capabilities on topic categorization ac-curacy and interpretability. By applying NMF, LDA, and Corex, distinct topics were identified on both platforms, showcasing unique social media thematic structures. BERT's integration significantly improved contextual understanding, especially in sentiment detection and knowledge depth across topics. The findings highlight the strengths and limitations of traditional topic modeling methods for social media, and the benefits of incorporating BERT for deeper analysis. This research advances Natural Language Processing (NLP) by showing the value of hybrid approaches for extracting insights from complex social media content, suggesting directions for future research to enhance NLP methodologies' applicability and quality of being granular in social media analysis.