Topic Modeling based Consumer behavior analysis using Latent Dirichlet Allocation

This study aims to develop high-performing Machine Learning and Deep Learning models in predicting hospital length of stay (LOS) while enhancing interpretability. We compare performance and interpretability of models trained only on structured tabular data with models trained only on unstructured clinical text data, and on mixed data. The structured data was used to train fourteen classical Machine Learning models including advanced ensemble trees, neural networks and k-nearest neighbors. The unstructured data was used to fine-tune a pre-trained Bio Clinical BERT Transformer Deep Learning model. The structured and unstructured data were then merged into a tabular dataset after vectorization of the clinical text and a dimensional reduction through Latent Dirichlet Allocation. The study used the free and publicly available Medical Information Mart for Intensive Care (MIMIC) III database, on the open AutoML Library AutoGluon. Performance is evaluated with respect to two types of random classifiers, used as baselines. The best model from structured data demonstrates high performance (ROC AUC = 0.944, PRC AUC = 0.655) with limited interpretability, where the most important predictors of prolonged LOS are the level of blood urea nitrogen and of platelets. The Transformer model displays a good but lower performance (ROC AUC = 0.842, PRC AUC = 0.375) with a richer array of interpretability by providing more specific in-hospital factors including procedures, conditions, and medical history. The best model trained on mixed data satisfies both a high level of performance (ROC AUC = 0.963, PRC AUC = 0.746) and a much larger scope in interpretability including pathologies of the intestine, the colon, and the blood; infectious diseases, respiratory problems, procedures involving sedation and intubation, and vascular surgery. Our results outperform most of the state-of-the-art models in LOS prediction both in terms of performance and of interpretability. Data fusion between structured and unstructured text data may significantly improve performance and interpretability.

Topic modelling is an important technique for extracting meaningful insights from large volumes of unstructured text data. The paper presents a federated technique for topic modelling that is based on a novel approach of the Latent Dirichlet Allocation (LDA) method for topic model generation. The proposed approach enables a topic model to be developed in a distributed environment using data continually generated from multiple sources without the need for sharing actual data. The first iteration of the topic modelling uses unsupervised LDA at each device generating the data. The results of each device are aggregated at a central server to generate a set of seed words that are used for guided LDA by the subsequent iterations of topic modelling. The proposed work, Federated LDA (F-LDA) has been evaluated using two datasets: a text dataset of dialogues between patients and doctors based on factual conversations and another comprising tweets related to depression. Comparing the performance of F-LDA with that of a centralized LDA, it was observed that F-LDA results in improved coherence score as well as diversity score in comparison to centralized LDA. This indicates that F-LDA achieves better interpretable topics covering a wide range of themes without redundancy.

In an era dominated by an unprecedented deluge of textual information, the need for effective methods to make sense of large datasets is more pressing than ever. This article takes a pragmatic approach to unraveling the intricacies of topic modeling, with a specific focus on the widely used Latent Dirichlet Allocation (LDA) algorithm. The initial segment of the article lays the groundwork by exploring the practical relevance of topic modeling in real-world scenarios. It addresses the everyday challenges faced by researchers and professionals dealing with vast amounts of unstructured text, emphasizing the potential of topic modeling to distill meaningful insights from seemingly chaotic data. Moving beyond theoretical abstraction, the article then delves into the mechanics of Latent Dirichlet Allocation. Developed in 2003 by Blei, Ng, and Jordan, LDA provides a probabilistic framework to identify latent topics within documents. The article takes a step-by-step approach to demystify LDA, offering a practical understanding of its components and the Bayesian principles governing its operation. A significant portion of the article is dedicated to the practical implementation of LDA. It provides insights into preprocessing steps, parameter tuning, and model evaluation, offering readers a hands-on guide to applying LDA in their own projects. Real-world examples and case studies showcase how LDA can be a valuable tool for tasks such as document clustering, topic summarization, and sentiment analysis. However, the journey through LDA is not without challenges, and the article candidly addresses these hurdles. Topics such as determining the optimal number of topics, the sensitivity of results to parameter settings, and the interpretability of outcomes are discussed. This realistic appraisal adds depth to the article, helping readers navigate the nuances and potential pitfalls of employing LDA in practice. Beyond the technical intricacies, the article explores the broad spectrum of applications where LDA has proven its efficacy. From text mining and information retrieval to social network analysis and healthcare informatics, LDA has left an indelible mark on diverse domains. Through practical examples, the article illustrates how LDA can be adapted to different contexts, showcasing its versatility as a tool for uncovering latent patterns. Keywords: Topic Modeling, Latent Dirichlet Allocation, Text Mining, Natural Language Processing, Document Clustering, Bayesian Inference.

Latent Dirichlet allocation (LDA) is a statistical model that is often used to discover topics or themes in a large collection of documents. In the LDA model, topics are modeled as discrete distributions over a finite vocabulary of words. The LDA is also a popular choice to model other datasets spanning a discrete domain, such as population genetics and social networks. However, in order to model data spanning a continuous domain with the LDA, discrete approximations of the data need to be made. These discrete approximations to continuous data can lead to loss of information and may not represent the true structure of the underlying data. We present an augmented version of the LDA topic model, where topics are represented using Gaussian mixture models (GMMs), which are multi-modal distributions spanning a continuous domain. This augmentation of the LDA topic model with Gaussian mixture topics is denoted by the GMM-LDA model. We use Gibbs sampling to infer model parameters. We demonstrate the utility of the GMM-LDA model by applying it to the problem of clustering sleep states in electroencephalography (EEG) data. Results are presented demonstrating superior clustering performance with our GMM-LDA algorithm compared to the standard LDA and other clustering algorithms.

Given that unstructured data is increasing exponentially everyday, extracting and understanding the information, themes, and relationships from large collections of documents is increasingly important to researchers in many disciplines including biomedicine. Latent Dirichlet Allocation (LDA) is an unsupervised topic modeling technique based on the "bag-of-words" assumption that has been applied extensively to unveil hidden semantic themes within large sets of textual documents. Recently, it was extended using the "bag-of-n-grams" paradigm to account for word order. In this paper, we present an alternative phrase based LDA model to move from a bag of words or n-grams paradigm to a "bag-of-key-phrases" setting by applying a key phrase extraction technique, the C-value method, to further explore latent themes. We evaluate our approach by using a phrase intrusion user study and demonstrate that our model can help LDA generate better and more interpretable topics than those generated using the bag-of-n-grams approach. Given topic models essentially are statistical tools, an important problem in topic modeling is that of visualizing and interacting with the models to understand and extract new information from a collection. To evaluate our phrase based modeling approach in this context, we incorporate it in an open source interactive topic browser. Qualitative evaluations of this browser with biomedical experts demonstrate that our approach can aid biomedical researchers gain better and faster understanding of their document collections.

This paper examines the performance of topic modelling techniques on a newly compiled dataset of Malaysian business news, aiming to address the challenge of extracting meaningful insights from unstructured data. We compare traditional methods, such as Latent Dirichlet Allocation (LDA) and Non-Negative Matrix Factorization (NMF), with emerging methods or models, including <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$T$</tex> op2Vec and BERTopic. Each model is trained or fine-tuned on five yearly datasets (2019–2023) to account for temporal variations. Our goal is to identify the most effective technique for topic discovery in this domain. The dataset includes 35,667 articles collected through web scraping from New Straits Times (Malaysia) and other Malaysian news portals. The results show that NMF consistently achieves the highest coherence scores, ranging between 0.65 until 0.80, which indicate strong topic quality and interpretability. On the other hand, <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$T$</tex> op2Vec performs the worst, while LDA and BERTopic produce inconsistent results. These findings provide useful guidance, particularly the strengths and limitations of each technique, for researchers and NLP practitioners applying topic modelling to similar textual corpora.

Topic models have recently shown to be really useful tools for the analysis of microarray experiments. In particular they have been successfully applied to gene clustering and, very recently, also to samples classification. In this latter case, nevertheless, the basic assumption of functional independence between genes is limiting, since many other a priori information about genes’ interactions may be available (co-regulation, spatial proximity or other a priori knowledge). In this paper a novel topic model is proposed, which enriches and extends the Latent Dirichlet Allocation (LDA) model by integrating such dependencies, encoded in a categorization of genes. The proposed topic model is used to derive a highly informative and discriminant representation for microarray experiments. Its usefulness, in comparison with standard topic models, has been demonstrated in two different classification tests.KeywordsTopic ModelLatent Dirichlet AllocationMultinomial DistributionProbabilistic Latent Semantic AnalysisLatent Dirichlet Allocation ModelThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Topic modeling is the statistical model for discovering hidden topics or keywords in a collection of documents. Topic modeling is also considered a probabilistic model for learning, analyzing, and discovering topics from the document collection. The most popular techniques for topic modeling are latent semantic analysis (LSA), probabilistic latent semantic analysis (pLSA), latent Dirichlet allocation (LDA), and the recent deep learning-based lda2vec. LDA is most commonly used in extractive multi-document summarization to determine whether the extracted sentence reflects the concept of the input document. In this paper, we will try to explore various multi-document summarization techniques that use LDA as a topic modeling method for improving final summary coverage and to reduce redundancy. Finally, we compared LDA and LSA using the Genism toolkit, and our experiment results show that LDA outperforms LSA if we increase the number of features considered for sentence selection.KeywordsTopic modelingText summarizationLDALSA

As text data continues to grow quickly, it is increasingly important to develop intelligent systems to help people manage and make use of vast amounts of text data (big text data''). As a new family of effective general approaches to text data retrieval and analysis, probabilistic topic models---notably Probabilistic Latent Semantic Analysis (PLSA), Latent Dirichlet Allocations (LDA), and their many extensions---have been studied actively in the past decade with widespread applications. These topic models are powerful tools for extracting and analyzing latent topics contained in text data; they also provide a general and robust latent semantic representation of text data, thus improving many applications in information retrieval and text mining. Since they are general and robust, they can be applied to text data in any natural language and about any topics. This tutorial systematically reviews the major research progress in probabilistic topic models and discuss their applications in text retrieval and text mining. The tutorial provides (1) an in-depth explanation of the basic concepts, underlying principles, and the two basic topic models (i.e., PLSA and LDA) that have widespread applications, (2) an introduction to EM algorithms and Bayesian inference algorithms for topic models, (3) a hands-on exercise to allow the tutorial attendants to learn how to use the topic models implemented in the MeTA Open Source Toolkit and experiment with provided data sets, (4) a broad overview of all the major representative topic models that extend PLSA or LDA, and (5) a discussion of major challenges and future research directions.

Latent Dirichlet Allocation (LDA), as the most popular topic model, models documents as mixtures of topics and topics as mixtures of words. Topic mixture well represents documents while words mixture extracts meaningful topics from a corpus. The nature of LDA makes it a powerful tool in documents organizing and corpus summarizing. One limitation of LDA is that its performance depends heavily on the priors. Researchers show priors matters in LDA and propose methods to learn the priors for better modeling, regardless of using symmetric priors. However, LDA modeling ability does not necessarily consent with the performance of LDA in documents representation and topic extraction. In this paper, we propose a novel prior setting for LDA. The setting improves LDA in both documents representation and topic extraction performance. We experiment to compare our setting with symmetric priors and previously proposed priors that enhances modeling ability. Experiments on the topic quality show that LDA with our prior setting extracts better topics than LDA with other kinds of prior settings. We compare LDA document representation ability through tasks such as document clustering and document classification. These experiments demonstrate LDA with our proposed priors represents document better. Moreover, our analyses also reveal that better modeling does not necessarily lead to better performance in documents representation and topic extraction.

In this article, I introduce the ldagibbs command, which implements latent Dirichlet allocation in Stata. Latent Dirichlet allocation is the most popular machine-learning topic model. Topic models automatically cluster text documents into a user-chosen number of topics. Latent Dirichlet allocation represents each document as a probability distribution over topics and represents each topic as a probability distribution over words. Therefore, latent Dirichlet allocation provides a way to analyze the content of large unclassified text data and an alternative to predefined document classifications.

Integrating Big Data Into Evaluation: R Code for Topic Identification and Modeling

A massive of text that is generated every minute is increasing dramatically. Therefore, it is more and more important to find an effective model to automatic classify the amount of text. Topic models is the most powerful techniques in text classification. There are many research results in the field of topic model have been published in scholarly journals. The Latent Dirichlet Allocation (LDA) is one of the most popular topic models in text classification. Researchers have proposed many topic evolution models based on LDA to solve some specific problems in applications of text classification. And some joint models which based on topic models combined other algorithms have been studied to enhance the performance of text classification. In this paper, we investigated three categories topic models for text classification and briefly introduced their advantages and disadvantages in the applications of text mining. Also, we introduce the generated process of documents and illustrate the graphical model for each topic models.

The construction industry is the backbone of a nation’s economy. It is a matter of great concern that such an industry suffers from time and cost overruns, especially in these challenging times. Coupled with the overrun issues, the sector is often criticized for lacking adequate quality and quantity of structured secondary data. The emerging technologies in data science and machine intelligence present a unique opportunity to understand the sector better and aid in effective decision-making. To better understand the utility of such technologies, the Management Discussion and Analysis ssections of the annual reports of publicly listed top Indian construction contracting firms are analyzed to identify the presence of ‘strategy themes’ and further map them to the organizations considered. Natural Language Processing (NLP)-based topic modeling algorithms, namely Latent Dirichlet Allocation (LDA) and Non-negative Matrix Factorization (NMF), are used in this study to perform a qualitative content analysis to identify the latent themes. From a methodological standpoint, considering the context of this study, the NMF results are better in accuracy, precision, and recall compared with the LDA. The results show that while most construction contracting firms prioritized a ‘revenue-focused’ strategy to expand their order books, a smaller set of large-sized firms seem to prioritize process improvement to improve their execution productivity and therefore are ‘profit margin improvement focused’ or ‘lean-focussed’ in their approach. Although a proof-of-concept, this study unlocks the immense potential of unsupervised NLP-based topic-modeling tools to understand and infer from unstructured and freely available text data in the public domain to aid sectoral analysis and policymaking.

Topic models are widely used to discover the latent representation of a set of documents. The two canonical models are latent Dirichlet allocation, and Gaussian latent Dirichlet allocation, where the former uses multinomial distributions over words, and the latter uses multivariate Gaussian distributions over pre-trained word embedding vectors as the latent topic representations, respectively. Compared with latent Dirichlet allocation, Gaussian latent Dirichlet allocation is limited in the sense that it does not capture the polysemy of a word such as "bank." In this paper, we show that Gaussian latent Dirichlet allocation could recover the ability to capture polysemy by introducing a hierarchical structure in the set of topics that the model can use to represent a given document. Our Gaussian hierarchical latent Dirichlet allocation significantly improves polysemy detection compared with Gaussian-based models and provides more parsimonious topic representations compared with hierarchical latent Dirichlet allocation. Our extensive quantitative experiments show that our model also achieves better topic coherence and held-out document predictive accuracy over a wide range of corpus and word embedding vectors which significantly improves the capture of polysemy compared with GLDA and CGTM. Our model learns the underlying topic distribution and hierarchical structure among topics simultaneously, which can be further used to understand the correlation among topics. Moreover, the added flexibility of our model does not necessarily increase the time complexity compared with GLDA and CGTM, which makes our model a good competitor to GLDA.