Summarizing large text collection using topic modeling and clustering based on MapReduce framework

ABSTRACTQuantitative text analysis tools have become increasingly popular methods for the operationalization of various types of discourse analysis. However, their application usually remains fairly simple and superficial, and fails to exploit the resources which the digital era holds for discourse analysis to their full extent. This paper discusses the discourse-analytic potential of a more complex and advanced text analysis tool, which is already frequently employed in other approaches to textual analysis, notably topic modelling. We argue that topic modelling promises advances in areas where discourse analysis has traditionally struggled, such as scaling, repetition, and systematization, which go beyond the contributions of simpler frequency and collocation counts. At the same time, it does not violate the epistemological premises and methodological ethos of even the more radical theories of discourse, we will demonstrate. Finally, we present two small case studies to show how topic modelling – when used with appropriate parameters – can straightforwardly enhance our ability to systematically investigate and interpret discourses in large collections of text. Abbreviations: CDA: Critical Discourse Analysis; LDA: Latent Dirichlet Allocation

Probabilistic topic models are a group of unsupervised generative machine learning models that can be effectively trained on large text collections. They model document content as a two-step generation process, i.e., documents are observed as mixtures of latent topics, while topics are probability distributions over vocabulary words. Recently, a significant research effort has been invested into transferring the probabilistic topic modeling concept from monolingual to multilingual settings. Novel topic models have been designed to work with parallel and comparable multilingual data (e.g., Wikipedia or news data discussing the same events). Probabilistic topics models offer an elegant way to represent content across different languages. Their probabilistic framework allows for their easy integration into a language modeling framework for monolingual and cross-lingual information retrieval. Moreover, we present how to use the knowledge from the topic models in the tasks of cross-lingual event clustering, cross-lingual document classification and the detection of cross-lingual semantic similarity of words. The tutorial also demonstrates how semantically similar words across languages are integrated as useful additional evidences in cross-lingual information retrieval models.KeywordsProbabilistic topic modelsCross-lingual retrievalRanking modelsCross-lingual text mining

Topic modeling is the statistical model for discovering hidden topics or keywords in a collection of documents. Topic modeling is also considered a probabilistic model for learning, analyzing, and discovering topics from the document collection. The most popular techniques for topic modeling are latent semantic analysis (LSA), probabilistic latent semantic analysis (pLSA), latent Dirichlet allocation (LDA), and the recent deep learning-based lda2vec. LDA is most commonly used in extractive multi-document summarization to determine whether the extracted sentence reflects the concept of the input document. In this paper, we will try to explore various multi-document summarization techniques that use LDA as a topic modeling method for improving final summary coverage and to reduce redundancy. Finally, we compared LDA and LSA using the Genism toolkit, and our experiment results show that LDA outperforms LSA if we increase the number of features considered for sentence selection.KeywordsTopic modelingText summarizationLDALSA

With the rapidly increasing volume of information, it has become essential to categorize data for analysis and management. One technique that enables this is the use of “topics”: semantic shortcuts that let us label texts, group them by label, find relevant texts, or simply convey information about their content. Existing topic-detection methods, however, often fail to provide both reliable content categorization and human-friendly labels. Recent progress in topic detection, including improvements to methods based on Latent Dirichlet Allocation (LDA) with neural embeddings, semantic clustering with assigned top keywords, and transformer-based zero-shot classifiers shows promise but also has significant limitations. A key limitation is their limited world knowledge, which includes not only an understanding of real-world entities and facts, but also a grasp of the categories people commonly use to categorize information. The topic labels these methods produce are not always interpretable. In addition, they are often hard to adopt in dynamic environments because of their heavy computation and rigid frameworks. To overcome these challenges, this research introduces a novel dual approach that integrates predefined categorization with dynamic topic generation. The first approach leverages established taxonomies or internal taxonomies, utilizing a shared embedding space to transform both texts and known topics into a single semantic space. This enables unsupervised topical classification based on semantic similarity, eliminating the need for continuous model retraining as taxonomies evolve. It also allows for multi-label assignments through label-specific thresholding, ensuring that texts can belong to multiple relevant categories seamlessly. In addition to predefined categorization, the second approach employs Large Language Models (LLMs) to generate context-specific topic labels for individual texts. This research demonstrates that LLMs can generate topic labels comparable to those created by humans. This method easily identifies and names topics that may not fit into existing taxonomies. Based on the recognition of the subjective and contextual nature of how humans perceive and name topics, this research introduces a novel topic evaluation method based on information reconstruction, in which we assess the degree to which the assigned labels allow accurate reconstruction of the original text.

People sourcing for a particular job role in any corporate venture is a painstaking, progress-impeding task, especially given the changing job market trends, quality of candidate profile and the sheer number of applicants. This paper elucidates an improvised topic modelling approach, where the semantic analysis stage is prepended with text summarization, for the procedure of people sourcing for job roles using topic modelling and machine learning techniques. The distinction between choosing the perfect candidate for the job and choosing a good candidate who is adept in certain domains but not relevant for the job and getting the candidate up to speed by providing on job training, and the downtime involved in the process is certainly a deciding factor in hiring a potential employee. The following paper aims to alleviate this issue by describing the algorithm which identifies the most suitable candidate in the applicants’ pool through a novel, robust approach which uses topic modelling techniques like Latent Semantic Indexing (LSA) and Latent Dirichlet Allocation (LDA). It is empirically found that the precision of profile matching task was enhanced by using text summarization (through TextRank model) for both LSA and LDA by 21.4% and 50% respectively, and LSA outperforming LDA with regard to precision, in both with and without text summarization cases by 23.8% and 51.57%, respectively.

Semantic Similarity is one of the highest levels of NLP. STS has significant advantages in NLP applications like information retrieval, information extraction, text summarization, data mining, machine translation, and other tasks. This research aims to present a deep learning approach for capturing semantic textual similarity (STS) in the Guragigna. The methodology involves collecting a Guragigna language corpus and preprocessing the text data and text representation is done using the Universal Sentence Encoder (USE), along with word embedding techniques including Word2Vec and GloVe and mean Square Error (MSE) is used to measure the performance. In the experimentation phase, models like LSTM, Bidirectional RNN, GRU, and Stacked RNN are trained and evaluated using different embedding techniques. The results demonstrate the efficacy of the developed models in capturing semantic textual similarity in the Guragigna language. Across different embedding techniques, including Word2Vec, GloVe, and USE, the Bidirectional RNN model with USE embedding achieves the lowest MSE of 0.0950 and the highest accuracy of 0.9244. GloVe and Word2Vec embedding also show competitive performance with slightly higher MSE and lower accuracy. The Universal Sentence Encoder consistently emerges as the top-performing embedding across all RNN architectures. The research results demonstrate the effectiveness of LSTM, GRU, Bi RNN, and Stacked RNN models in measuring semantic textual similarity in the Guragigna language.

Text segmentation is a fundamental problem in natural language processing, which has application in information retrieval, question answering, and text summarization. Almost previous works on unsupervised text segmentation are based on the assumption of lexical cohesion, which is indicated by relations between words in the two units of text. However, they only take into account the reiteration, which is a category of lexical cohesion, such as word repetition, synonym or superordinate. In this research, we investigate the non-systematic semantic relation, which is classified as collocation in lexical cohesion. This relation holds between two words or phrases in a discourse when they pertain to a particular theme or topic. This relation has been recognized via a topic model, which is, in turn, acquired from a large collection of texts. The experimental results on the public dataset show the advantages of our approach in comparison to the available unsupervised approaches.

A massive of text that is generated every minute is increasing dramatically. Therefore, it is more and more important to find an effective model to automatic classify the amount of text. Topic models is the most powerful techniques in text classification. There are many research results in the field of topic model have been published in scholarly journals. The Latent Dirichlet Allocation (LDA) is one of the most popular topic models in text classification. Researchers have proposed many topic evolution models based on LDA to solve some specific problems in applications of text classification. And some joint models which based on topic models combined other algorithms have been studied to enhance the performance of text classification. In this paper, we investigated three categories topic models for text classification and briefly introduced their advantages and disadvantages in the applications of text mining. Also, we introduce the generated process of documents and illustrate the graphical model for each topic models.

Topic modeling can help better understand the content of large collections of text. The objective of the research is to develop an approach to identify latent topics in the Technology Innovation Management Review journal (TIM Review) and how topics have evolved using the LDA (Latent Dirichlet Allocation) and DTM (Dynamic Topic Model) algorithms. We applied the two approaches to a collection of TIM Review articles published between 2007 and 2017. We also examined extracted topics, most associated articles, topic labeling, topic/theme trends, and word trends produced by both approaches and identified the value of each approach. According to the results obtained, we identified 47 topics and categorized them into ten themes: open source, entrepreneurship, innovations, living labs, social technology innovation, growth, co-creation, cybersecurity, research, and ecosystem. While some topic trends became prominent over time, others disappeared. The distribution of the articles across topics in the LDA approach has been made more decisively so that of 597 articles, 503 most associated articles were identified, while this number is 299 articles in DTM. Furthermore, we discussed weaknesses and strengths of the algorithms to compare the performance of the two approaches based on defined criteria. We conclude that DTM provides more accurate word and topic trends over time, although it requires time slice settings and has a longer run-time compared to LDA. Finally, we document a set the steps of a process to carry out topic modeling analysis.

Integrating Big Data Into Evaluation: R Code for Topic Identification and Modeling

The sharing and interoperation of geographical information has always been a hot topic in the field of geographical information science, and the semantic similarity of the concept of geographic information is also an important factor affecting geographic information sharing and interoperability. The semantic similarity of concept calculation is a natural language processing, an important component of the study, but also applications of artificial intelligence problems to be solved. Ontology, which is an explicit and formal specification of sharing information, becoming an effective method for solve the problem of semantic similarity. Considering the special nature of the concept of geographic information, this paper takes an integrated multi-point semantic similarity calculation method in this paper. We made an investigation on the distance-based semantic similarity calculation model, the attribute-based semantic similarity calculation model, the relationship between concepts-based semantic similarity calculation models. And we analyzed all aspects of the concept of geographic information. Then, considering the impact of the distance, the attributes, the relationship between concepts for the semantic similarity of the concept of geographic information, this paper established an integrated and multi-angle model for the semantic similarity calculation of geographic information concept. And we added some adjustable parameters to adapt the flexible application. Make sure the calculation results more accurate and objective. By precise experimental analysis, this integrated model can improve the calculation accuracy.

Topic modelling is an important technique for extracting meaningful insights from large volumes of unstructured text data. The paper presents a federated technique for topic modelling that is based on a novel approach of the Latent Dirichlet Allocation (LDA) method for topic model generation. The proposed approach enables a topic model to be developed in a distributed environment using data continually generated from multiple sources without the need for sharing actual data. The first iteration of the topic modelling uses unsupervised LDA at each device generating the data. The results of each device are aggregated at a central server to generate a set of seed words that are used for guided LDA by the subsequent iterations of topic modelling. The proposed work, Federated LDA (F-LDA) has been evaluated using two datasets: a text dataset of dialogues between patients and doctors based on factual conversations and another comprising tweets related to depression. Comparing the performance of F-LDA with that of a centralized LDA, it was observed that F-LDA results in improved coherence score as well as diversity score in comparison to centralized LDA. This indicates that F-LDA achieves better interpretable topics covering a wide range of themes without redundancy.

Latent Dirichlet allocation (LDA) is a statistical model that is often used to discover topics or themes in a large collection of documents. In the LDA model, topics are modeled as discrete distributions over a finite vocabulary of words. The LDA is also a popular choice to model other datasets spanning a discrete domain, such as population genetics and social networks. However, in order to model data spanning a continuous domain with the LDA, discrete approximations of the data need to be made. These discrete approximations to continuous data can lead to loss of information and may not represent the true structure of the underlying data. We present an augmented version of the LDA topic model, where topics are represented using Gaussian mixture models (GMMs), which are multi-modal distributions spanning a continuous domain. This augmentation of the LDA topic model with Gaussian mixture topics is denoted by the GMM-LDA model. We use Gibbs sampling to infer model parameters. We demonstrate the utility of the GMM-LDA model by applying it to the problem of clustering sleep states in electroencephalography (EEG) data. Results are presented demonstrating superior clustering performance with our GMM-LDA algorithm compared to the standard LDA and other clustering algorithms.

As text data continues to grow quickly, it is increasingly important to develop intelligent systems to help people manage and make use of vast amounts of text data (big text data''). As a new family of effective general approaches to text data retrieval and analysis, probabilistic topic models---notably Probabilistic Latent Semantic Analysis (PLSA), Latent Dirichlet Allocations (LDA), and their many extensions---have been studied actively in the past decade with widespread applications. These topic models are powerful tools for extracting and analyzing latent topics contained in text data; they also provide a general and robust latent semantic representation of text data, thus improving many applications in information retrieval and text mining. Since they are general and robust, they can be applied to text data in any natural language and about any topics. This tutorial systematically reviews the major research progress in probabilistic topic models and discuss their applications in text retrieval and text mining. The tutorial provides (1) an in-depth explanation of the basic concepts, underlying principles, and the two basic topic models (i.e., PLSA and LDA) that have widespread applications, (2) an introduction to EM algorithms and Bayesian inference algorithms for topic models, (3) a hands-on exercise to allow the tutorial attendants to learn how to use the topic models implemented in the MeTA Open Source Toolkit and experiment with provided data sets, (4) a broad overview of all the major representative topic models that extend PLSA or LDA, and (5) a discussion of major challenges and future research directions.

Topic modeling extracts useful potential topics that reflect market information from massive financial news and is widely used in data mining and economic research. Traditional topic modeling approaches such as Latent Dirichlet Allocation (LDA) and Non-Negative Matrix Factorization (NMF) lack semantic information, and short texts have feature sparse problems. We develop a topic clustering model based on BERT-LDA joint embedding that takes both contextual semantics and thematic narrative into account. We cluster document embeddings with the HDBSCAN algorithm and utilize a class-based TF-IDF (c-TF-IDF) method to create topic representations. Empirical results show that the BERT-LDA model is competitive compared with traditional and single topic models. It generates coherent topic words that are dissimilar to each other.