Topmodpy: A Simple Python Script for Topic Modeling

Funding agencies such as the U.S. National Science Foundation (NSF), U.S. National Institutes of Health (NIH), and the Transportation Research Board (TRB) of The National Academies make their online grant databases publicly available which document a variety of information on grants that have been funded over the past few decades. In this paper, based on a quantitative analysis of the TRB’s Research In Progress (RIP) online database, we explore the feasibility of automatically estimating the appropriate funding level, given the textual description of a transportation research project. We use statistical Text Mining (TM) and Machine Learning (ML) technologies to build this model using the 14,000 or more records of the TRB’s RIP research grants big data. Several Natural Language Processing (NLP) based text representation models such as the Latent Dirichlet Allocation (LDA), Latent Semantic Indexing (LSI) and the Doc2Vec Machine Learning (ML) approach are used to vectorize the project descriptions and generate semantic vectors. Each of these representations is then used to train supervised regression models such as Random Forest (RF) regression. Out of the three latent feature generation models, we found LDA gives the least Mean Absolute Error (MAE) using 300 feature dimensions and RF regression model. However, based on the correlation coefficients, it was found that it is not very feasible to accurately predict the funding level directly from the unstructured project abstract, given the large variations in source agencies, subject areas, and funding levels. By using separate prediction models for different types of funding agencies, funding levels were better correlated with the project abstract.

In an era dominated by an unprecedented deluge of textual information, the need for effective methods to make sense of large datasets is more pressing than ever. This article takes a pragmatic approach to unraveling the intricacies of topic modeling, with a specific focus on the widely used Latent Dirichlet Allocation (LDA) algorithm. The initial segment of the article lays the groundwork by exploring the practical relevance of topic modeling in real-world scenarios. It addresses the everyday challenges faced by researchers and professionals dealing with vast amounts of unstructured text, emphasizing the potential of topic modeling to distill meaningful insights from seemingly chaotic data. Moving beyond theoretical abstraction, the article then delves into the mechanics of Latent Dirichlet Allocation. Developed in 2003 by Blei, Ng, and Jordan, LDA provides a probabilistic framework to identify latent topics within documents. The article takes a step-by-step approach to demystify LDA, offering a practical understanding of its components and the Bayesian principles governing its operation. A significant portion of the article is dedicated to the practical implementation of LDA. It provides insights into preprocessing steps, parameter tuning, and model evaluation, offering readers a hands-on guide to applying LDA in their own projects. Real-world examples and case studies showcase how LDA can be a valuable tool for tasks such as document clustering, topic summarization, and sentiment analysis. However, the journey through LDA is not without challenges, and the article candidly addresses these hurdles. Topics such as determining the optimal number of topics, the sensitivity of results to parameter settings, and the interpretability of outcomes are discussed. This realistic appraisal adds depth to the article, helping readers navigate the nuances and potential pitfalls of employing LDA in practice. Beyond the technical intricacies, the article explores the broad spectrum of applications where LDA has proven its efficacy. From text mining and information retrieval to social network analysis and healthcare informatics, LDA has left an indelible mark on diverse domains. Through practical examples, the article illustrates how LDA can be adapted to different contexts, showcasing its versatility as a tool for uncovering latent patterns. Keywords: Topic Modeling, Latent Dirichlet Allocation, Text Mining, Natural Language Processing, Document Clustering, Bayesian Inference.

Topic modeling is the statistical model for discovering hidden topics or keywords in a collection of documents. Topic modeling is also considered a probabilistic model for learning, analyzing, and discovering topics from the document collection. The most popular techniques for topic modeling are latent semantic analysis (LSA), probabilistic latent semantic analysis (pLSA), latent Dirichlet allocation (LDA), and the recent deep learning-based lda2vec. LDA is most commonly used in extractive multi-document summarization to determine whether the extracted sentence reflects the concept of the input document. In this paper, we will try to explore various multi-document summarization techniques that use LDA as a topic modeling method for improving final summary coverage and to reduce redundancy. Finally, we compared LDA and LSA using the Genism toolkit, and our experiment results show that LDA outperforms LSA if we increase the number of features considered for sentence selection.KeywordsTopic modelingText summarizationLDALSA

Information extraction (IE) is a technique for extracting structured data or knowledge from unstructured data by determining the references to words as well as the relationships between them. IE systems can be used to directly extract knowledge from a text corpus, or to extract useful data from a pile of documents which moreover, can be further analyzed with deep learning algorithms to discover more patterns. In today's era, the internet exhibits several characteristics including unstructured data and noisy text information. Unstructured data may contain audio, image, text, video, etc. Information extraction can be useful for topic detection (Twitter, Digital libraries), Breast Cancer detection and many more applications. The proposed system can exploit deep learning for information extraction using image as well as text and can significantly improve its performance with the inclusion of structured and unstructured data. This paper focuses on extraction of information from unstructured data using various deep learning models and concepts of Natural Language Processing (NLP), Long Short Term Networks (LSTM) and for topic extraction used Latent Dirichlet Allocation (LDA) etc.

Paraphrase detection allows determining how original and suspect documents convey the same meaning. It has attracted attention from researchers in many Natural Language Processing (NLP) tasks such as plagiarism detection, question answering, information retrieval, etc., Traditional methods (e.g., Term Frequency-Inverse Document Frequency (TF-IDF), Latent Dirichlet Allocation (LDA), and Latent Semantic Analysis (LSA)) cannot capture efficiently hidden semantic relations when sentences may not contain any common words or the co-occurrence of words is rarely present. Therefore, we proposed a deep learning model based on Global Word embedding (GloVe) and Recurrent Convolutional Neural Network (RCNN). It was efficient for capturing more contextual dependencies between words vectors with precise semantic meanings. Seeing the lack of resources in Arabic language publicly available, we developed a paraphrased corpus automatically. It preserved syntactic and semantic structures of Arabic sentences using word2vec model and Part-Of-Speech (POS) annotation. Overall experiments shown that our proposed model outperformed the state-of-the-art methods in terms of precision and recall

Topic modelling is an approach in data mining, use machine learning methods to discover patterns in large amount of unstructured text. It takes a collection of documents and group the words into clusters of words that we call Bag of words, and identify topics by using process of similarity. Topic modelling provides us with methods to organize, understand and summarize large collections of textual information. There are a lot of approaches have been exposed for Topic modelling, the most in use are Latent Semantic Analysis (LSA), Latent Dirichlet Allocation (LDA) and explicit semantic analysis (ESA). In our study we describing an approach to refine Topic detection based on 2d vector space model VSM by using Apriori algorithm along with Natural language processing, to form a better connected terms in vector space for clean engagement with the query.

A questionnaire text mining model is built based on latent semantic analysis (LSA). Take open questionnaire as the research object and improve Latent Dirichlet Allocation (LDA) model according to the text characteristics, a BM25-LDA model is built. By comparing the LDA model and BM25-LDA model on the open questionnaire text, we can verify that the performance of BM25-LDA mode is better than that of original LDA model.

The progressive advancement in technology has become easy to present the language expression of someone else as one’s own with similar words semantically. This phenomenon increased the potential source of plagiarism. Its detection is a challenge especially in the case of Arabic paraphrase because of the semantic ambiguity of this language. In recent decades, researches have been hindered by the very limited availability of well-structured datasets. In this context, our main objectives are focused on constructing a corpus for Arabic and presenting thereafter its impact for identifying paraphrase. Indeed, we generated the suspect documents from the Open Source Arabic Corpora (OSAC). Distributed word representation (word2vec) and part-of-speech methods were useful for replacing each original word by its most similar one that had the same grammatical class. Moreover, we captured the structure of Arabic sentences with different window sizes and vector dimensions. Then, we studied how this corpus could be used efficiently in the evaluation of Natural Language Processing (NLP) methods (i.e. Term Frequency-Inverse Document Frequency (TF-IDF), Latent Semantic Analysis (LSA), Latent Dirichlet Allocation (LDA), word2vec, Global Vector Representation (GloVe), and Convolutional Neural Network (CNN)) for paraphrase detection. Experiments revealed which one could outperformed significantly for preserving semantic properties of Arabic words with various linear regularities, alleviating data sparseness and increasing the degree of semantic similarity, in terms of precision and recall.

The way we communicate has been revolutionised by the widespread use of social networks. Any kind of online message can reach anyone in the world almost instantly. The speed with which information spreads is undoubtedly the strength of social networks, but at the same time, any user of these platforms can see how toxic messages spread in parallel with likes, comments and ratings about any person or entity. In such cases, the victim feels even more helpless and defenceless as a result of the rapid spread. For this reason, we have implemented an automatic detector of toxic messages on social media. This allows us to stop toxicity in its tracks and protect victims. In particular, the aim of the survey is to demonstrate how traditional Machine Learning methods of Natural Language Processing (NLP) work on equal terms with Deep Learning methods represented by a Transformer architecture and characterised by a higher computational cost. In particular, the paper describes the results obtained by testing different supervised Machine Learning classifiers (Logistic Regression, Random Forest and Support Vector Machine) combined with two topic-modelling techniques of NLP, (Latent Semantic Analysis and Latent Dirichlet Allocation). A pre-trained Transformer named BERTweet was also tested. All models performed well in this task, so much so that values close to or above 90% were achieved in terms of the F1 score evaluation metric. The best result achieved by Transformer BERTweet, 91.40%, was therefore not impressive in this context, as the performance gains are too small compared to the computational overhead.

The popularity of online review sites such as Amazon and TripAdvisor have created a window into rich insights of what matters to reviewers, and potentially customers. For the business sector, this provides opportunities to improve their businesses and to attract more customers. The unstructured nature of user reviews, however, makes it difficult to analyze, and natural language processing techniques are instrumental in extracting useful information from these resources. To digest the free form review texts, it is desirable to abstract the reviews into high level themes. However, research works in this area is still scarce. In this paper, the process taken to perform this high-level abstraction is detailed, with an example derived from the hotel industry (identity concealed for confidentiality purpose). More than 3000 sets of customer reviews were processed using the Extract, Transform and Load process, supported by topic modelling algorithms including Latent Dirichlet Allocation (LDA), Latent Semantic Analysis (LSA) and K-means. The study aims to identify significant business aspects or features based on the customer reviews. Preliminary results from these algorithms lack usability, which can be mainly attributed to unknown number of hidden topics and the brief nature of the review texts. To address this issue, a cosine similarity-based algorithm was introduced to improve the thematic clustering into one that is more intuitive and actionable. Ten themes emerged, and using cosine similarity, the review texts in the corpus were intuitively clustered for business planning insights. With this method, the review texts can be classified in a matter of a few minutes and demonstrates the strength of natural language processing for insights mining.

The demand of scoring natural language responses has created a need for new computational tools that can be applied to automatically grade student essays. Systems for automatic essay assessment have been commercially available since 1990's. However, the progress in the field was obstructed by a lack of qualitative information regarding the effectiveness of such systems. Most of the research in automatic essay grading has been associated with English writing due to its widespread use and the availability of more learner collection and language processing software for the language. In addition, there is large number of commercial software for grading programming assignments automatically. In this work, we investigate document semantic similarity based on Latent Semantic Analysis (LSA) and on Latent Dirichlet Allocation (LDA). We use an open-source Python software, Gensim, to develop and implement an essay grading system able to compare an essay to an answer-key and assign to it a grade based on semantic similarity between the two. We test our tool on variable-size essays and conduct experimental tests to compare the results obtained from human grader (professor) and those obtained from the automatic grading system. Results show high correlation between the professor grades and the grades assigned by both modeling techniques. However, LSA-based modeling showed more promising results than the LDA-based method.

Analysis of Consumer buying patterns is crucial for vendors to understand the necessities and favorites of their audience. With the increasing availability of digital data, collecting large amounts of consumer feedback and opinions has become easier. However, analyzing such data is challenging due to the presence of unstructured text data. Latent Dirichlet Allocation (LDA) is a prevalent topic modeling procedure that can be used to analyze unstructured text data. LDA allows for the identification of latent topics that are present in the data, enabling businesses to gain insights into consumer behavior. By applying LDA to consumer feedback data, businesses can identify the key topics that are most important to their customers. Additionally, LDA can be used to track changes in consumer preferences over time, allowing businesses to stay up-to-date with evolving market trends. Overall, LDA is a powerful tool for analyzing consumer behavior and can provide valuable insights for businesses looking to improve their products and services. In this work, LDA is employed to analyze consumer tweets using topic modeling. LDA has successfully generated topic and

This study aims to develop high-performing Machine Learning and Deep Learning models in predicting hospital length of stay (LOS) while enhancing interpretability. We compare performance and interpretability of models trained only on structured tabular data with models trained only on unstructured clinical text data, and on mixed data. The structured data was used to train fourteen classical Machine Learning models including advanced ensemble trees, neural networks and k-nearest neighbors. The unstructured data was used to fine-tune a pre-trained Bio Clinical BERT Transformer Deep Learning model. The structured and unstructured data were then merged into a tabular dataset after vectorization of the clinical text and a dimensional reduction through Latent Dirichlet Allocation. The study used the free and publicly available Medical Information Mart for Intensive Care (MIMIC) III database, on the open AutoML Library AutoGluon. Performance is evaluated with respect to two types of random classifiers, used as baselines. The best model from structured data demonstrates high performance (ROC AUC = 0.944, PRC AUC = 0.655) with limited interpretability, where the most important predictors of prolonged LOS are the level of blood urea nitrogen and of platelets. The Transformer model displays a good but lower performance (ROC AUC = 0.842, PRC AUC = 0.375) with a richer array of interpretability by providing more specific in-hospital factors including procedures, conditions, and medical history. The best model trained on mixed data satisfies both a high level of performance (ROC AUC = 0.963, PRC AUC = 0.746) and a much larger scope in interpretability including pathologies of the intestine, the colon, and the blood; infectious diseases, respiratory problems, procedures involving sedation and intubation, and vascular surgery. Our results outperform most of the state-of-the-art models in LOS prediction both in terms of performance and of interpretability. Data fusion between structured and unstructured text data may significantly improve performance and interpretability.

In order to enhance the user’s service experience and help the robot to make more intimate service decisions, this paper proposes a service task cognition and decision model based on structured and unstructured data processing. Firstly, all kinds of information mentioned in user instructions are extracted through natural language processing, including object information, namely structured data and environment information, namely unstructured data. For structured data, it is mapped to the predefined ontology knowledge base to obtain its location attributes and state attributes, and then obtain service instructions. Adaptive fuzzy Petri net (AFPN) is constructed based on fuzzy rules of temperature, humidity and other environmental information. The unstructured data is taken as the input parameter of AFPN, and the service instruction deduced as the output. Then, according to the user’s needs, the network weight can be continuously adjusted. If the user does not mention the environmental information, the environment information is periodically detected by the sensor, and the service instruction reasoning of the unstructured data is performed. Finally, back propagation neural network (BPNN) is introduced to combine the service inference of two kinds of data to eliminate the heterogeneity of different service instructions. Experimental results show that the model can provide different personalized services for users’ preferences.

The construction industry is the backbone of a nation’s economy. It is a matter of great concern that such an industry suffers from time and cost overruns, especially in these challenging times. Coupled with the overrun issues, the sector is often criticized for lacking adequate quality and quantity of structured secondary data. The emerging technologies in data science and machine intelligence present a unique opportunity to understand the sector better and aid in effective decision-making. To better understand the utility of such technologies, the Management Discussion and Analysis ssections of the annual reports of publicly listed top Indian construction contracting firms are analyzed to identify the presence of ‘strategy themes’ and further map them to the organizations considered. Natural Language Processing (NLP)-based topic modeling algorithms, namely Latent Dirichlet Allocation (LDA) and Non-negative Matrix Factorization (NMF), are used in this study to perform a qualitative content analysis to identify the latent themes. From a methodological standpoint, considering the context of this study, the NMF results are better in accuracy, precision, and recall compared with the LDA. The results show that while most construction contracting firms prioritized a ‘revenue-focused’ strategy to expand their order books, a smaller set of large-sized firms seem to prioritize process improvement to improve their execution productivity and therefore are ‘profit margin improvement focused’ or ‘lean-focussed’ in their approach. Although a proof-of-concept, this study unlocks the immense potential of unsupervised NLP-based topic-modeling tools to understand and infer from unstructured and freely available text data in the public domain to aid sectoral analysis and policymaking.